Road driving environments are complex, unstructured and highly changeable. A safe driving is, thus, becoming quite challenging task, in particular from the view point of development and deployment of autonomous vehicles-based urban transport systems. And, in that context, the reliable perception appears as a one of the main enabling strategies in developing safe autonomous driving. Currently, many autonomous vehicles are being tested on public roads with the objective of demonstrating the capability of operating in real world situations. A big effort has been focused towards creating fault-free autonomous vehicles. Nevertheless, fault tolerant perception for autonomous vehicles still needs to be further developed in order to create autonomous vehicles capable of driving under real road traffic conditions since on-board vehicle sensors may fail due to bad calibration, erroneous readings, physical or electrical failures, etc. A multi-sensor based vehicle architecture is a logical response to this issue. While the multi- sensor concept often relates to the strategy of using a variety of sensor types, this research has been focused on to the case when all sensors are vision sensors, either identical or different from each other. This thesis proposes a Fault Tolerant Perception paradigm that deals with possible sensor faults by defining the Federated Data Fusion Architecture designed to detect a faulty sensor and reduce its impact on to the safe autonomous driving. The proposed architecture minimises the influence of faulty data allowing the system to enter in a tolerated error state, where a recovery action can be performed to avoid failures. The developed architecture was then adapted towards meeting requirements of the KITTI Vision Benchmark Suite. Experimental results demonstrated the feasibility of the developed fault tolerant perception paradigm to successfully detect early faulty data from a singular sensor and to minimise the influence of that faulty sensor in the fusion process.