Project SignON is receiving 5.6 million euros in funding from the European Commission. The three-year project is developing an app that simplifies communication between deaf people, those who are hearing-impaired, and those who can hear.
Goal
The SignON project aims to simplify communication between people with hearing loss and those who can hear throughout Europe. Each spoken language has a unique sign language which makes communication extremely complicated. The project is focused on Irish, British, Dutch, Flemish, and Spanish sign languages. In terms of spoken languages, the consortium is concentrating on Irish, English, Dutch, and Spanish.
The application
One of the researchers involved in the project is Dr. Dimitar Shterionov from Tilburg University in the Netherlands where he is the scientific project leader. “The project is very wide-ranging. From a technological research perspective, it is mainly about recognition, analysis, and an understanding of text, speech, and sign language. The app automatically translates into the chosen language using a model that is a hybrid of a deep learning and rule-based model.”
“When something is translated into sign language, it is represented by a 3D avatar. When sign language is translated into a spoken language, it is articulated using artificial speech.” Imagine thereby a computer voice like Siri.
Co-creation
There is a high level of cooperation with deaf and hearing-impaired people during the project. “Communication with the deaf and hard of hearing will steer the development of the app. We will evaluate the app in the interim so that we can incorporate the feedback from this group.” According to Shterionov, this form of co-creation makes the project unique. This approach ensures that the future app will be well-suited to the needs of the user target group.
Partners
Four Dutch partners are involved in the project. Radboud University, the Nederlandse TaalUnie (Dutch Language Association), the Dutch Language Institute, and Tilburg University. A total of 17 European partners are working on the project. The consortium includes various experts in the field of Natural Language Processing (NLP), Machine Learning (ML), Deep Learning (DL), Machine Translation (MT), Linguistics, Deaf Studies, education, and 3D graphics.
Also interesting:
KitMe provides a signal to help the hard-of-hearing
Gerard & Anton Award-winner Hable: super fast braille on your smartphone
Related Posts
