As the number and sophistication of vehicle cameras increase, so does the compute power needed to translate the high throughput of image data into outputs that meet the varying requirements for machine and human vision. To enable new capabilities in ADAS and autonomous driving, the industry will need a new approach to image processing. To address this, chip manufacturer Arm has added the Mali-C78AE ISP to its portfolio of IPs specifically developed for automotive applications.
ADAS features use multiple cameras to enable a variety of human and machine vision applications. For example, surround-view systems use data from cameras around the vehicle to visually display information to drivers to help them make decisions while parking. Adaptive cruise control, on the other hand, directly uses camera data to interpret the environment and make decisions independent of the driver about vehicle control, such as applying the throttle or brake. According to Arm, Mali-C78AE is designed specifically to address both human and machine vision safety applications and can process data from up to four real-time or 16 virtual cameras.
The company says that Mali-C78AE was developed from the ground up with hardware safety mechanisms and diagnostic software features, enabling system designers to meet ISO 26262 ASIL B functional safety requirements. It aims to prevent or detect faults in a single camera frame that may result in incorrectly processed frame data. To do this, the ISP features over 380 fault detection circuits and continuous built-in self-testing. It can also detect sensor and hardware faults in connected cameras.
Equally as important to safety and user experience, says Arm, is processing speed, which is a key element of the Mali-C78AE. It should take 150ms to acquire an image at the sensor, process it through the ISP then GPU, and display it on a screen for the driver; anything longer is noticeable to the driver when using parking assist, for example. In a machine vision application, a vehicle should not travel more than 250mm between a camera image being acquired and it being presented to the decision-making processing, and anything longer means the machine vision system is too slow to react in driving situations where accurate and timely decisions are critical.
To enable drivers and machines to make the best-possible decision, ADAS cameras must collect the most relevant information possible from each frame. To this end, the Mali-C78AE employs advanced noise reduction technology and dynamic range management to ensure each frame is clear and properly exposed by adjusting overly dark or bright areas of a frame. It is also able to perform real-time processing of camera data from up to four high-resolution/high-frame-rate cameras, significantly reducing the memory, communications, and processing requirements and making for a more efficient system.
Arm also highlights that to implement multiple ADAS functions, several dedicated cameras are required, because units used for machine vision applications, such as lane departure warnings, do not produce images that are suitable for human vision, such as surround view. To reduce the cost of implementing multiple ADAS functions, Arm claims that the Mali-C78AE enables camera sensors to be dual-purpose by downscaling and color-translating the outputs of sensors optimized for machine vision to create images adapted to the human eye. By avoiding duplication in cameras and their associated electronics and wiring, OEMs save on cost and complexity. It therefore enables wider deployment of camera-based ADAS features across a diverse range of car models, providing a safer, better user experience for drivers.