Clearing the Fog: How Ansys Virtually Verifies Autonomous Perception Systems

One obstacle to launching a fully autonomous car is achieving a high level of confidence in image analysis from cars’ sensors certainty. This is especially challenging in foggy conditions, which can confuse visual cameras, lidar and other conventional sensor technologies.

Thermal imaging represents a potential solution, helping sensors get a much more defined view via infrared technology. But how can automotive developers, using thermal imaging ― and other sensor systems ― accurately assess their products’ performance in fog?

Fog Modeling Provides a Clear View

To answer that question, Ansys collaborated with FLIR Systems, a leader in thermal imaging, to evaluate the fog modeling capabilities in our Ansys Speos solution. The tests verified that fog modeling via Speos predicts and matches physical fog tests FLIR® produced in the lab.

Ansys Speos engineers created a fog model that virtually replicates the effects of fog in a real-world setting. By obtaining a static view of a physical scenario under foggy conditions, engineers developing ADAS and AV systems can use Ansys Speos to simulate not only a physics-based thermal camera, but also that camera combined with other sensors such as visible cameras.

Download the full white paper to learn how Ansys created the simulations in two phases: correlating a new Ansys fog model and verifying the thermal camera in a virtual environment with fog.

Correlating the New Fog Model

In the first phase of the study, a simulation scene was defined to recreate the FLIR laboratory setup that was based on physical testing of its Boson® thermal camera at Sandia National Laboratory, using a closed system for artificial fog generation.¹ This setup included two targets at different distances ― a black one and white one ― with different temperatures, placed in front of the camera. The targets had been characterized following FLIR’s specifications for optical properties, temperature, size and position.

A simulated scene in Ansys Speos of a FLIR physical test in the laboratory

The next step was to recreate the FLIR Boson camera model using Speos’ camera sensor simulation capabilities. Finally, the Speos fog model was created based on the information provided by FLIR about the fog generated during physical testing in the lab. The Ansys engineering team performed the simulation with fog and without fog, then adapted the density in the fog model to reach the same results (50%) as the real results obtained by FLIR.

Comparing the number of particles propagated in the laboratory by FLIR and the number of particles propagated in the simulation resulted in two close curves (see graph below). The 20% difference might be explained by the fact that FLIR used salt water in its lab tests and Ansys performed the simulation with only water particles. Further investigation is needed to completely confirm the results.

Number of particles for fog, measured versus modeled distribution. (Results were presented by FLIR during SPIE conference, April 2019.)

Verifying the Simulation

In phase two, a real-world driving environment scene was recreated in Speos. The key inputs in assessing the efficiency of thermal imaging in fog are the optical properties of the object and the temperature of the object. Based on these inputs, Speos software calculated the light emission from the object in the far infrared range. All objects emit light energy, humans and animals will typically be one of the hottest objects in a scene, therefore infrared is an ideal sensor to include in an automotive safety system.

Measured optical properties were applied to all the components of the Ansys Speos virtual driving scenario, including infrastructure components, a pedestrian and a car made of different materials. Each of these objects also had a temperature value associated with it.

During Phase 2, optical properties were applied to all components in a realistic driving scene

In the final step, the Boson thermal camera model produced in phase one was recreated and positioned on the roof of a virtual vehicle. Realistic simulations were performed using the virtual driving scene, camera details and weather conditions with the introduction of the Speos fog model.

The results shown in the figure below indicate a vast improvement in perception when a thermal camera is used in foggy conditions, compared to a visible (RGB) camera.

The difference is clear: Thermal imaging capabilities simulated in Ansys Speos make it far easier for sensors to “see” a far-away pedestrian in medium or dense fog ― in view of triggering an appropriate response from the autonomous vehicle’s steering and braking systems. 

“Optimizing algorithm behaviors and overall system performance is especially challenging for environments such as foggy conditions, because collecting data and performing tests is completely reliant on actually having foggy weather,” according to Chris Posch, FLIR, Engineering Director for Automotive. “Because the Ansys Speos Fog model simulation so closely matches the results of our physical lab tests, FLIR and other developers can more reliably test and tune their systems using simulation. The results of this work will help us improve our own systems and share those benefits with the world’s automakers and OEMs.”

Download the full white paper to learn more.

1. “Multispectral short-range imaging through artificial fog,” Michael P. Thornton, Kelsey M. Judd, Austin A. Richards, Brian J. Redman, FLIR Systems, Inc., Wilsonville, OR, USA; Sandia National Laboratories, Albuquerque, NM, USA.