For drivers, reacting quickly to hazards can mean the difference between life and death – for themselves and others. As a rule, the reaction time is less than half a second. If it’s also a matter of diverting and/or braking, another second or so is added. Especially at intersections, where many cars, cyclists, buses, streetcars and even pedestrians meet, it quickly becomes confusing.
Researchers at the Fraunhofer Institutes for High Frequency Physics and Radar Techniques FHR, for Integrated Circuits IIS and the IVI Application Center “Networked Mobility and Infrastructure” have now invented a radar sensor system as part of the HORIS project that will use artificial intelligence to warn drivers and autonomous vehicles in the future if a pedestrian is walking towards the road and car.
Speed instead of distance
Distance sensors on modern cars can warn drivers when they are getting too close to an object – as when parking – but if, for example, a child suddenly runs across the road, this warning comes too late. “Instead of relying solely on a distance warning in the car, as has been the case in the past, the focus here is on reliable behavior prediction,” explains Dr. Reinhold Herschel, group manager at Fraunhofer FHR. If a person runs onto the road, the sensors should detect the impending danger at the very beginning of the movement. This would save valuable, sometimes even life-saving time.
The new sensors are to be used first at critical points such as bus and streetcar stops. There, they monitor the movements of people waiting and transmit the data to passing traffic. Since these are radar sensors that do not record optical images, data protection and privacy would not be a problem, the scientists emphasize. In the long run, the algorithms could also be used for evaluation in radar sensors in cars.
Division of tasks
The three Fraunhofer institutes have divided the tasks among themselves. Fraunhofer FHR is developing algorithms that recognize an object as a person. They set a marker and determine the speed at which the person is moving. “Are they running towards the radar sensor and therefore towards the road? The key here is to find the fine line between false alarms and alarms that are too late,” the researchers explain. The radar sensor takes about a hundred measurements per second. The alarm is triggered “when the person consistently moves towards the road at a certain minimum speed over several measurements.”
At Fraunhofer IIS, they are handling reference measurement technology and motion detection. “With our L.I.N.K. Test and Application Center, we have a unique measurement capability: The hall is so large that you can set up a bus stop as well as put a car in it,” says Nicolas Witt, group manager at Fraunhofer IIS. In several measurement campaigns, the scientists here use 30 motion-capture cameras that recognize people tagged with small markers in an area measuring 20 by 30 meters.
Finally, the Fraunhofer IVI designs suitable test scenarios. “For example, we installed two radar sensors in a test bus stop on the campus of the Ingolstadt University of Applied Sciences that observe the scene from different angles,” says Prof. Gordon Elger, head of the application center. Depending on the number of people waiting at the bus stop, networked or autonomous vehicles could be told to slow down in the future. If no one is standing at the stop, the car could drive by at normal speed.
Spring 2021 demonstration planned
The demonstrator currently works with up to eight people. It can detect them all at the same time and determine whether they are moving towards the road. According to the Fraunhofer Gesellschaft, a demonstration as well as a presentation for interested customers – depending on the coronavirus situation – is planned for the second quarter of 2021.
For their next project, the researchers from the three institutes want to build on the results from HORIS and further optimize the sensor. There it would be a matter of “how” instead of just “if.” With the help of AI and an additional infrared camera, the system should then not only see that someone is approaching the car, but also understand the entire scenario. For example, if the system detects that a ball is rolling onto the road, it understands that a few seconds later a child could follow. This is similar to a situation in which a bus stops. Here, someone could run across the road. This could improve the system’s reaction time and gain valuable seconds to warn the driver.
Cover photo: Test measurement at Fraunhofer IIS: A person is detected by radar just when he starts running. The detected position is verified via an optically monitored marker on his helmet. © Fraunhofer FHR
Also interesting: New early warning system for self-driving cars warns of overly complex situations
