It doesn’t just depend on what someone says, but how they say it. This is not only true for our daily dealings with our fellow human beings; the intelligent speech recognition of the Munich-based audEERING knows all about the how. “Different nuances can have many different meanings. This is also the crux in intelligent man-machine communication that this ‘how’ was neglected for a very long time in favor of the ‘what’,” says Dagmar Schuller, CEO and co-founder of audEERING. “Of course it was important to recognize the what first, but the way one speaks says much more, whether I mean something sarcastic or ironic, whether I sound happy, whether I sound sad or negative or positive.”
This video by audEERING, right below the ad, illustrates this point with a wink.
https://www.youtube.com/watch?v=EXW8xMGFqsU&feature=youtu.be
Exactly these small but subtle differences are often of decisive importance – in various areas of life. In a small series, we break down how intelligent speech recognition can be used in different areas of application. Today we are starting with the field of medicine, in which the software is used for the early detection of diseases such as Alzheimer’s, Parkinson’s or burnout and depression.
“Many neurocognitive diseases manifest themselves first in language at an early stage; how to articulate, how to speak and also how to behave,” explains Dagmar Schuller. “In the long-term perspective, there are classic indicators such as uncontrolled emotional outbursts in Alzheimer’s patients. If these people are suddenly particularly aroused or particularly unhappy and you don’t really know why, because it has been taken out of context, this is a classic Alzheimer’s indicator that you can recognize”.
In addition to emotional changes, language also plays a decisive role in early detection – and that’s where audEERING comes into play. “What you can tell from the language or the voice is that the rhythm of speech changes. That the tonality changes and sometimes also the semantics if you go in the direction of understanding. Words are inserted that are completely taken out of context because you can no longer remember the correct word.” And here, too, the how is more important than the what. “Even if you pronounce certain vowels differently, this might by an indication. In Parkinson’s disease, for example, a classic test is to let patients say “A”, then one examines this different vowel tonality in order to get an early indication.”
Using previous comparison data is good, but not necessary
These comparisons are possible without prior individual comparison values, as long as one has a “sufficiently large database, out of which one knows what a healthy patient sounds like and what the sound is of someone who has the corresponding disease. You can independently determine the first indicators from the person, based on what the system has already learned from a certain database.”
In a joint study with the British company EMTEQ, audEERING recorded speech data from Parkinson’s patients and speech data from healthy patients in order to obtain an automatic recognizer. Nevertheless, Schuller admits, it is always better to have long-term data of an individual patient. “For example in a doctor-patient relationship, in which a patient always gives these speech examples over a certain period of time. This is of course much better because I can compare it individually with the profile of this person. Then I can see exactly if and what has changed.”
If these indicators are combined – changes in behaviour and language – it is an indication for the doctor in early diagnosis to take a closer look and, if necessary, to consult a neurologist. “Ideally, the neurologist will also carry out appropriate tests, but in principle, it is important to have indicators as early as possible and to recognise the disease, because then the therapy is much more effective.”
Burnout and depression
The main field of application for intelligent speech recognition in medicine is burnout and depression. “There is a very large number of unreported cases of people who, on the one hand, are not registered at all and, on the other hand, are registered far too late and where burnout has already turned into a real depression,” stresses Dagmar Schuller. “There is also a risk that this depression will develop into chronic depression. That’s why it makes sense to start therapy early so that it doesn’t get that far in the first place. Our systems and the classical audio analysis are a very good possibility.”
Main target group: Teenagers
Modern young people are largely afraid of not being cool, of being bullied on Facebook, Twitter and the like and of losing so-called friends if they admit their weaknesses. So the problem increases over time. “They are becoming more and more introverted and entrusting themselves to far fewer people because they are afraid that they will lose their status. With the app, the young people are supposed to play around and have a kind of self-diagnostic tool. The app is neutral and does not evaluate, so you can trust it much more risk-free”, explains Dagmar Schuller.
The app then proposes, if necessary, to seek professional help and “the young people do not have to worry that anyone will blame them. The aim is to combine the app with therapy centers, which will provide the opportunity for therapists to intervene at an early stage, who can also actively contact the young people.
“Together we want to train the system through sufficient numbers of stitches and cases and patient data that we receive so far that it automatically recognizes the early indicators that can then be used directly for clinical operation.” However, since patient data is subject to data protection, this hurdle must still be overcome before an operation. “This is the job of the universities to get done. Our job is to deliver the intelligent technology.”
This possibility does not yet exist in medical practices, as the software is still being tested as part of a joint project with psychological institutes of the Ludwig-Maximilians-University Munich and the University of Exeter. “They do the entire clinical studies, which are not only about the early diagnosis but also about early interventions”, Schuller says. The goal is an intelligent health app especially for young people who are most frequently affected by burnout and depression.
The app, which should be available as a beta version within the next two to three years, may even prevent suicides and prevent chronic depression that can otherwise only be treated with drugs. “This is the main objective of the project and it is, therefore, also supported by the EU.”
Photo: Pixabay
