One obstacle to launching a fully autonomous car is achieving a high level of confidence in image analysis from the cars’ sensors. This is especially challenging in foggy conditions, which can confuse visual cameras, lidar and other conventional sensor technologies.
Thermal imaging represents a potential solution, helping sensors get a much more defined view via infrared technology. But how can automotive developers, using thermal imaging – and other sensor systems – accurately assess their products’ performance in fog?
Fog modeling provides a clear view
To answer that question, Ansys collaborated with FLIR Systems, a specialist in thermal imaging, to evaluate the fog modeling capabilities of our Ansys SPEOS solution. The tests verified that fog modeling via SPEOS predicts and matches the physical fog tests that FLIR produced in the lab.
Engineers created a fog model that virtually replicates the effects of fog in a real-world setting. By obtaining a static view of a physical scenario under foggy conditions, engineers developing ADAS and AV systems can use Ansys SPEOS to simulate not only a physics-based thermal camera but also that camera combined with other sensors such as visible cameras.
Correlating the new fog model
In the first phase of the study, a simulation scene was defined to recreate the FLIR laboratory setup that was based on physical testing of its Boson thermal camera at Sandia National Laboratories, using a closed system for artificial fog generation.1 This setup included two targets – a black one and a white one – at different distances with different temperatures, placed in front of the camera. The targets had been characterized following FLIR’s specifications for optical properties, temperature, size and position.
The next step was to recreate the FLIR Boson camera model using SPEOS’s camera sensor simulation capabilities. Finally, the SPEOS fog model was created based on the information provided by FLIR about the fog generated during physical testing in the lab. The engineering team performed the simulation with fog and without fog, then adapted the density in the fog model to achieve the same results (50%) as the real results obtained by FLIR.
Comparing the number of particles propagated in the laboratory by FLIR and the number of particles propagated in the simulation resulted in two close curves (see graph below). The 20% difference might be explained by the fact that FLIR used salt water in its lab tests and Ansys performed the simulation with only water particles. Further investigation is needed to completely confirm the results.
Verifying the simulation
In phase two, a real-world driving environment scene was recreated in SPEOS. The key inputs in assessing the efficiency of thermal imaging in fog are the optical properties of the object and the temperature of the object. Based on these inputs, SPEOS software calculated the light emission from the object in the far infrared range. All objects emit light energy; humans and animals will typically be some of the hottest objects in a scene, therefore infrared is an ideal sensor to include in an automotive safety system.
Measured optical properties were applied to all the components of the Ansys SPEOS virtual driving scenario, including infrastructure components, a pedestrian and a car made of different materials. Each of these objects also had a temperature value associated with it.
In the final step, the thermal camera model produced in phase one was recreated and positioned on the roof of a virtual vehicle. Realistic simulations were performed using the virtual driving scene, camera details and weather conditions with the introduction of the SPEOS fog model.
The results shown in the figure below indicate a vast improvement in perception when a thermal camera is used in foggy conditions, compared with a visible (RGB) camera.
The difference is clear: simulated thermal imaging capabilities make it far easier for sensors to ‘see’ a far-away pedestrian in medium or dense fog – in view of triggering an appropriate response from the autonomous vehicle’s steering and braking systems.
“Optimizing algorithm behaviors and overall system performance is especially challenging for environments such as foggy conditions, because collecting data and performing tests is completely reliant on actually having foggy weather,” according to Chris Posch, FLIR’s engineering director for automotive. “Because the Ansys SPEOS Fog model simulation so closely matches the results of our physical lab tests, FLIR and other developers can more reliably test and tune their systems using simulation. The results of this work will help us improve our own systems and share those benefits with the world’s auto makers and OEMs.”