© Pixabay

Creating a machine learning agent to ease interactions between participants in virtual and augmented reality environments —such as the metaverse— in order to help them achieve their goals and address conflict situations. This is the main objective of GuestXR, the European project under the coordination of professor Mel Slater, from the Faculty of Phycology and the Institute of Neurosciences of the University of Barcelona – UB – (UBNeuro).

As part of the GuestXR project (A Machine Learning Agent for Social Harmony in eXtended Reality), coordinated by the Eurecat technological centre, a machine learning system called guest will be designed, and it will work as an agent able to examine the individual and group behaviour of the participants by drawing on existing models from the neuroscience and social psychology standpoint. Moreover, Deep Reinforcement Learning will train the Guest to learn over time —both in simulations and real meetings— and it will be more efficient in social interaction, informs UB in a press release.

The agent will engage in the conversation

In these lines, the agent “will take part in the conversation in different ways, performing a range of multisensory actions through music or changes in space, among others”, notes the director of Eurecat Audiovisual Technologies, Adan Garriga.

“Each meeting in virtual and augmented reality has a purpose, be it explicit (for instance, reaching an agreement on a topic) or implicit (such as enjoyment). The basic idea of the guest is to help the group of people to accomplish this purpose. This involves an exciting research with a strong interdisciplinary element, which opens new paths, and hopefully, contributes to the success of virtual and augmented reality”, notes Professor Mel Slater.

In order to modulate this social interaction, the guest will carry out a series of multisensory actions, “through, for instance, visual and auditory features to create specific states of mind and stimulate a relaxed environment when it identifies a conflict”, highlights researcher Umut Saying, from Eurecat Audiovisual Technologies.

Offering a solution for online harassment

This project “is designed to help offering solutions to the existing challenges in online environments of social interaction such as online harassment, for which there isn’t a specific European legislation”, says the coordinator of the Eurecat GuestXR project, Aurora Sesé.

The intervention of the machine learning system will initially be tested in conflict resolution situations, interactions with participants with hearing difficulties and contexts that lead to polarized debates such as climate change.

Two open calls to include other innovative use cases to test the effectiveness of the system are expected. One case will be aimed at society in general —including companies and associations, among others— while the other will aimed at the field of arts.

Ethical approach

The GuestXR technology will be carried out under the Ethics by design approach, which entails considering the potential ethical problems derived from the use of artificial intelligence (respect for human autonomy, prevention of harm, fairness, etc.).

The GuestXR consortium, coordinated by the UB and the Eurecat technology centre, is formed by eight organizations from six countries with multidisciplinary teams of the field of the expanded reality, machine learning, artificial intelligence, social psychology, neuroscience of emotions, multisensory integration, ethics of research and technology transfer. Among the collaborators are the University of Maastricht (the Netherlands), the University of Warsaw (Poland), the University of Reichman (Israel), the National Institute for Research in Digital Science and Technology (INRIA, France), and the companies Virtual Bodyworks and G.tec medical engineering GMBH.

Selected for you!

Innovation Origins is the European platform for innovation news. In addition to the many reports from our own editors in 15 European countries, we select the most important press releases from reliable sources. This way you can stay up to date on what is happening in the world of innovation. Are you or do you know an organization that should not be missing from our list of selected sources? Then report to our editorial team.