HART wants to stretch the limits of our senses with a sleeve that speaks all languages

“So much happened last year!” Lisa Overdevest enthuses. She is head of Marketing, Awareness and Business within the TU/e student team HART. And she is not joking. Besides a place in the finals of the largest student entrepreneurship competition in the Netherlands – the Philips Innovation Award (PHIA), the team has expanded from about eight to twenty students, they presented their first proof of concept of their language application last November and they are now also looking into converting non-verbal communication.

Developing new senses

When IO spoke with HART (Human Augmentation Research & Technology) last year, the student team was putting the finishing touches on their proof of concept of a type of sleeve. Not just any sleeve; anyone who puts on this item of clothing can feel words through vibrations. “Software converts the foreign language into English and then into phonetic sounds, each of which has its own vibrational pattern. That information is then sent to the sleeve via a Bluetooth connection. In the future, we hope the sleeve will have a microphone that can pick up and convert language directly, without the need for a computer.”

The sleeve is a first step toward the student team’s ultimate goal: to develop new senses by means of human augmentation. “Humans are limited by their genetic make-up. As a result, there’s a lot we don’t pick up on. For instance, a snake can see infrared and pigeons can sense magnetic fields. We want to use AI to stretch the limits of what people are able to perceive. Ultimately this should result in a kind of app store where you can download certain ‘extra’ senses,” Overdevest explains.

Patient zero

The fact that their proof of concept was received well, is what Overdevest sees as the biggest milestone so far in the past year. She herself was patient zero and mastered 39 vibrational patterns – which resonate with the 39 phonetic sounds of the English language.

“Learning those patterns went amazingly fast,” Overdevest says. “At some point it became unconscious and I began to understand what I was feeling progressively faster. In other words, I was actually able to understand words through my arm.” A test will be held two weeks from now with fifteen test subjects, who will learn the vibrational patterns with the help of specially developed software.

Non-verbal communication

Since November, the team has also been working on converting non-verbal communication. They are doing this together with someone who became blind during his lifetime. “He misses non-verbal communication. You can hear emotions in a voice, but things like a raised eyebrow or a smile are just as expressive. We’re now looking to see if we can use AI to detect those kinds of gestures and translate them into those same vibrational patterns.”

Final place

And then there’s the final place in the largest entrepreneurship prize for students in the Netherlands. That she, along with another team member, will have to convince the jury tonight that HART deserves first place (and fifty thousand euros in prize money), is something that Overdevest never expected. “We took part because it seemed like such a great educational process to us. We are students from a technical university after all, so being able to look at our team more from the business side of things is really useful. We had to write a business plan, learn how to pitch and are being pushed to think about what we want in the future.”

In that future, Overdevest hopes that the team will continue to grow and that they will connect with companies. ” This would allow our technology to solve specific problems and we could really put our application into practice,” he says. Theoretically, it is possible to convert any information that reaches the sleeve into vibrational patterns. The possibilities are endless.”

This story was produced in conjunction with the Eindhoven University of Technology.