UK software specialist rFpro has added Solectrix‘s SXIVE (Simplified eXtensive Image and Video Engine) image-processing technology to its AV Elevate platform, enabling engineers to design, model and optimise the entire camera pipeline in simulation. By facilitating development before physical prototypes are available, the solution allows sensor optimization and training dataset production to shift earlier in the development cycle, significantly reducing costs.
The integration will support the tuning of image signal processing (ISP) and camera parameters against edge cases, the training of perception models with SXIVE-processed synthetic data and testing complete vision pipelines under highly variable conditions.
The update brings Solectrix’s software-based ISP into rFpro’s camera model, enabling engineers to optimize key processing steps directly in simulation, such as debayering, color correction, tone mapping, denoising, lens distortion and lens shading correction.
“With camera-based perception being central to most ADAS and autonomous vehicle performance, ISP optimisztion is a key competitive differentiator,” said Matt Daley, technical director at rFpro.
“Advantages can be gained by enhancing the images for machine rather than human vision and ultimately improving the perception model performance and overall safety of the system. Our combined simulation environment and ISP model create a far faster and safer way for engineers to experience tuning set options in dynamic driving conditions, rather than heading out onto public roads and finding these edge cases conditions.”
“Providing sufficient amounts of high-quality video data is a real challenge for the development of any modern AD or ADAS system,” said Dr Roman Tzschoppe, R&D manager at Solectrix. “Our collaboration with rFpro enables engineers to design and execute hundreds of thousands of test scenarios and generate a wide range of highly-specific training data for camera-based perception models. It enables critical image quality ISP parameters to be varied through the dataset production, this simply isn’t feasible when relying solely on physical hardware image collection.”
Outside of ADAS and automated driving there are also a wide range of camera-based human vision applications with the rapid uptake of digital mirrors and surround-view systems in many passenger vehicles, buses and heavy goods vehicles. Critical light conditions from low sun, highly reflective roads, bright vehicle headlights or covered and underground parking areas all create difficult ISP optimization challenges when tuning these systems.
AV Elevate is a fully integrated simulation solution that accelerates the development of ADAS and automated driving systems. It enables sensor tuning, perception and control system training, and testing of the full AV technology stack in a single environment. The platform supports both closed-loop perception testing and the generation of engineering-grade synthetic training data.
SXIVE is a comprehensive image processing ecosystem consisting of the actual image processing software, hardware accelerators, and a variety of apps and plugins. It enables image processing professionals to practice rapid prototyping as well as real-time processing and analysis of images and video streams. With its flexible architecture, SXIVE can be tailored to the requirements of any imaging project, such as ADAS/AD camera systems, driver monitoring systems or even the video feed for remote driver operating.
The integration of AV Elevate and SXIVE gives users flexibility on both the input and output side of camera development. Driving scenarios, the weather and location can be easily varied to continually create new test data, and the image processing modules or apps can be modified, or sensor versions swapped out in real-time with a click of a button. For example, this enables engineers to assess how color-balancing decisions influence pedestrian detection in a dark, wet urban environment versus a sunny day.
“OEMs now increasingly expect Tier 1 suppliers to deliver a highly accurate digital twin of their sensor system alongside a new product, except they want this long before the hardware is available,” said Daley. “Developing a high-quality camera model isn’t just a benefit for development but it is now becoming a commercial necessity. By integrating SXIVE into AV Elevate we are giving the industry a complete environment to tune, train and test the entire camera system.”
Related news, Wayve and Uber receive private hire vehicle licences from Transport for London