Why Autonomous Vehicles Need Thermal Cameras

The first step to fully autonomous vehicles is getting a clear picture of the world around us. The challenge is that no sensor is perfect. Some sensors are better suited for clear sunny days; others are better suited for dark foggy nights.

If autonomous vehicles have a blind spot for a specific road condition, it means there is a sensor gap in the car’s design. According to Ansys Speos simulations run by FLIR Systems Inc. (FLIR) — the thermal imaging company — one of the best ways to fill in the autonomous vehicle sensor gap is with thermal cameras.

ANSYS Speos simulations comparing a black and white camera to FLIR’s thermal imaging camera during nighttime conditions (Courtesy of FLIR Systems Inc.)

“If you look at many of the accidents involving assisted and autonomous driving systems, you see a gap in what the sensor can observe — like a white truck on a bright day or a person crossing a dark, unlit roadway,” says Ezra Merrill, director of marketing at FLIR. “Thermal sensors will fill in that gap and improve the safety of these vehicles.”
 

Why Thermal Cameras Fill in the Autonomous Vehicle Sensor Gap

Some electromagnetic (EM) based sensors — like visible-light cameras — have a hard time seeing through a dark or foggy night.

Speos simulations comparing a black and white camera to FLIR’s thermal imaging camera during heavy fog (Courtesy of FLIR Systems Inc.)

Merrill explains that cameras rely on illumination — visible light bouncing off objects. Therefore, cameras cannot see beyond a car’s headlights on dark nights. In foggy conditions, the light from headlights is scattered, which can limit the distance a camera can see.

Thermal sensors detect objects from the infrared energy they emit. This gives them several advantages over visible-light cameras in low or challenging light conditions.

First, all objects above zero Kelvin emit infrared energy, so visibly dark areas have little effect on thermal sensor performance. As a result, thermal cameras are well-suited for nighttime driving and seeing through glare and most fog. In fact, driver-warning systems on the road today, made by Veoneer, use over 600,000 thermal cameras from FLIR.

Second, longwave infrared (LWIR) energy, the wavelength seen by thermal cameras, travels through fog better than the shorter-wavelength of visible light energy.

Furthermore, since LWIR is emitted by objects, it doesn’t require an illumination source or headlights, which scatters significantly in foggy conditions.
 

Simulating How Thermal Cameras Fill the Autonomous Vehicle Sensor Gap

FLIR’s Speos simulations demonstrate the utility of thermal cameras to the autonomous vehicle market, because they help show how these sensors would perform under various road conditions. 

These simulations can also speed up development because they can be adjusted faster than finding an appropriate location and driving through it in different road conditions.

Speos simulations comparing a black and white camera to FLIR’s thermal imaging camera in mild fog (Courtesy of FLIR Systems Inc.)

The simulations prove that thermal sensors can see further than a camera under dark and foggy conditions.

“Our infrared sensors can even see beyond headlamps during nighttime driving,” adds Merrill. “In fact, our tests show that our sensor can see about four times farther than a headlamp can illuminate.

“Additionally, since a thermal sensor sees heat, it’s especially good at seeing living things — like pedestrians in difficult lighting or cluttered conditions,” says Merrill. “It is really good at detecting people, which complements the driver-warning or autonomous vehicle sensor suite. This makes smarter cars and safer roads.”

Merrill has a point about thermal cameras helping to make cars smarter. The images they produce can also be used to help engineers train artificial intelligence systems to distinguish living things from inanimate objects. Bringing this logic into autonomous cars will boost their safety.

“We have a free starter dataset that engineers can use to start building the logic of their autonomous systems,” says Merrill. “The dataset consists of over 14,000 annotated thermal images that classify cars, people, bicycles and even dogs.”

Clearly, thermal cameras offer the autonomous vehicle industry a way to fill its sensor gap and boost car intelligence. This brings the industry one step closer to fully autonomous vehicles.

To learn more, visit FLIR and Ansys at CES. Or read the blog: Top 3 Challenges to Produce Level 5 Autonomous Vehicles.