Blue Jay Aeden, that’s the name of the latest drone that students from the Blue Jay student team at the Eindhoven University of Technology (TU/e) presented today. The public at the Evoluon was introduced to what the students believe is the world’s first drone capable of conveying emotions.
It works like this: Via a built-in camera and an algorithm designed by the students, Blue Jay Aeden is able to recognize someone’s state of mind and respond accordingly. Aeden has a screen with eyes and eyebrows with which it can show different emotions. For example, if someone looks sad, the drone tries to cheer or comfort that person with its ‘face.’
Normally, the student team builds a drone for a different application every year. For instance, there was already a drone for use as a waiter, help in the home and one for use in hospitals. But the students decided to do things differently this year. They went on where their predecessors left off. Team manager Paul Kuppevelt explains: “In principle, it is a year-long project. But by building on the knowledge of the previous group, you end up with a better drone. If we had started something new, the standard would have gone from 70 percent down to 20 percent. You have to rebuild all that knowledge. The same goes for the relationship with external parties.”
Help in the event of a fire
Like the previous drone from Blue Jay, Aeden helps elderly people in nursing homes evacuate outside in case of an emergency. Along with the social aspect, the drone has a number of new features. For example, the device is connected to fire alarms. In case of fire, Aeden knows exactly where to go. The drone can also send a live stream to the fire department via the built-in camera. This allows them to estimate the seriousness of the situation en route.
According to Van Kuppevelt, it still happens too often that only one emergency response officer (ERO) is present at a nursing home. “If a fire breaks out, then this emergency response officer must escort the elderly out of the building as quickly as possible. But the elderly are often no longer that mobile or need some time to grasp the situation. This makes an evacuation more time-consuming. Our drone can help with this and save valuable time in the process.”
On-site testing?
To test and improve the drone, Blue Jay worked with, among others, the Joris Zorg nursing home in Oirschot and the Brabant-Southeast Fire Department (in the Netherlands). The students wanted to run on-site tests and speak to residents so that they could improve the drone. Yet there was one major problem: The BlueJay team members were not allowed into the nursing home because of corona. Rik Schutte, responsible for human-drone interaction, found it a challenging time. “Of course, you want to know how elderly people react to the drone. You want to test and try out all kinds of things. What do they like? What don’t they like? There is not very much research on this yet, so on-site testing was really important.”
All tests were still conducted with the the help of the staff from Joris Zorg, but Schutte notes that this still caused inevitable delays. “It was far from ideal because there was always a lot of time between testing and the transfer of information. It is also difficult to have to do your tests via someone else. You then have to explain very thoroughly what the exact intention is. Still, we were able to gather all the information that we needed. Despite the situation, we made the best of it.”
The elderly reacted to the drone in different ways, the students say. Schutte: “According to the nurses, the first reaction was that they liked the look of the drone. That was also a focal point for us. We don’t want our drone to look scary. We want to give it human traits. But there were also older people who were more reserved and mainly wondered what such a drone can do and what it does.”
Good mood around the drone
Because the students could not imitate a real fire, they talked to the elderly through the staff to ask them how a drone could help them. Schutte: “Again, there is not much research on this, so our own tests and conversations were important. We assumed that the elderly know themselves how they react to panic and what they need in situations like these. By doing this, we created several scenarios that we still need to flesh out and explore further.” In the process, the students also looked at what kind of mood the drone evoked and are still researching whether sound can contribute. ” We can create a good mood around the drone using LED lights on the side. But maybe this can also be done with sound,” Schutte said.
The drone still needs to be controlled at the moment, but the intention is that Aeden will be able to track elderly people autonomously in the nursing home in the future. Van Kuppevelt: ” Using Lidar sensors, the drone can map out the environment; that’s going well. But the obstacle detection needed for flying indoors still has to be improved. The stability for autonomous flight is also something that needs to be improved.”
To improve emotion detection, the students removed ‘Fearful’ and ‘Disgusted’ from the model. These two emotions were difficult for the algorithm to differentiate between and provided a precision of only 66 percent. After leaving them out, this shot up to 94 percent. But the aim is for the drone to recognize fearful faces in the future. Schutte: “For now, we have decided to first make an accurate model that better recognizes the emotions it contains. But panic or fear is an essential emotion, especially if a fire breaks out. So these definitely still have to be in the model. We have to figure out how this works, it involves very subtle differences that are hard to detect.”
Maybe something for the next team?
Related Posts
