Carefully making one’s way along the street and in unfamiliar surroundings with the help of a cane could one day be a thing of the past for blind and severely visually impaired people. This is possible thanks to an intelligent shoe that scans its surroundings from the foot’s perspective and detects obstacles. The “InnoMake” shoe, recently been approved as a medical device, was developed by the Austrian company Tec-Innovation. It is intended to make the personal mobility of blind and visually impaired people safer. “Ultrasonic sensors on the toe of the shoe detect obstacles up to four meters away,” says Markus Raffer, one of the founders of Tec-Innovation who is himself visually impaired. “The wearer is then warned by vibrations and/or acoustic signals. This works very well and is also already a great help to me personally.”
Even in the development phase, Raffer, his founding partner Kevin Pajestka and their team found that simply detecting an obstacle was not enough, however. In particular, the type of obstacle and its directional course are crucial, especially if it leads downhill, such as stairs or even holes. “Not only the warning that I am standing in front of an obstacle but also the information about which obstacle I am standing in front of is relevant. It makes a big difference whether it’s a wall, car or staircase,” Raffer said.
AI recognizes walkable areas
Since 2016, Tec-Innovation has been working with Graz University of Technology, Austria, as a collaborative partner. Researchers there from the Institute for Machine Vision and Representation developed a camera-based addition to the first version of the product. “We have developed state-of-the-art, deep-learning algorithms modeled on neural networks which can essentially do two things after recognizing and interpreting the image content: They determine an area that is free of obstacles and thus safe to walk on from camera images taken from the foot perspective. And they can recognize and distinguish objects,” explains computer scientist Friedrich Fraundorfer.
According to the scientists, the algorithms trained using machine learning can already be operated on a specially designed mobile system. In addition, thanks to the latest, powerful special processors, mobile use of complex AI algorithms is now also possible, they say. “This can be attributed to the enormous processor development of recent years,” emphasizes Fraundorfer’s colleague David Schinagl.
Navigation map for bundling data
Currently, Tec-Innovation is working on integrating the system – camera plus processor – into a prototype shoe that is both robust and comfortable. At the Graz University of Technology, they have already gone one step further. Friedrich Fraundorfer and his team want to combine all the information that the shoe collects while its wearer is out and about into a kind of Streetview navigation map for visually impaired people. “As it stands now, only the wearer benefits from the data the shoe collects while they are walking,” says Fraundorfer. “It would be much more sustainable if this data could also be made available to other people as a navigation aid.”
An application for funding is currently being submitted to the Austrian Research Promotion Agency FFG for the design and prototypical implementation of such a Streetview map for blind and visually impaired people. When such navigation support will become reality, however, is still uncertain. According to Fraundorfer, the biggest hurdles are to continuously expand and update the map. Beyond that the challenge will be to “link it with previous data and the IT connection of the shoe system.” But he is certain that the walking stick will eventually become obsolete. “In any case, we will keep working on this topic. After all, in our highly innovative world, there has to be a better alternative to the over 90-year-old walking stick.”
Cover photo: The Innomake shoe as it is already available on the market. An ultrasonic sensor is attached to the toe of the shoe. In the future, a camera plus a processor running the algorithm will be integrated there. © Lunghammer – Graz University of Technology, Austria
