Why we’re writing about this topic:
Until now, robots have worked almost exclusively on factory floors, separate from humans. Their work area is fenced off to prevent accidents, which cannot be ruled out when automated tools and humans collide. If robots had social skills, things would be different, says an interdisciplinary team of researchers in the EU’s ELSA project, which aims to enable humans and machines to work together safely.
Computer science is reaching its limits in this task, which is why psychological principles are also needed. Specifically, the project refers to the findings of the U.S. psychologist James Gibson, who studied the possibilities of action in the 1960s. He coined the term “affordances” and pointed out that a tennis ball offers dogs different possibilities for action than ants or humans.
The ELSA project (Effective Learning of Social Affordances for Human-Robot Interaction) involves teams from three universities in Austria and France. In Austria, a group of computer scientists and psychologists from the Digital Science Center (DiSC) at the University of Innsbruck are collaborating on this project. In an interview with Innovation Origins, computer scientist and head of the DiSC, Professor Justus Piater, talks about the challenges and goals of the project:
In what situations do robots need to have social skills?
In the future, robots will increasingly collaborate with humans and make themselves useful in human environments, such as hospitals, train stations, museums, stores, or even private homes. In such environments, it is important that robots have a basic understanding of humans. This understanding must have multiple levels such as movements, actions, and speech. In our research, we focus on the level of actions, specifically on the interaction between robots and humans. To correctly interpret an action, a robot needs to know the characteristics of the human performing that action. These include, for example, a person’s abilities and intentions. This allows the robot to collaborate with that person to expand the joint action possibilities. Questions that arise include, for example, what action possibilities (affordances) does the person offer the robot that could help it perform its tasks, such as tying up a package. Beyond that, what can the robot do to help the human perform their task, such as moving a heavy package?
What technical basis can you build on?
The term affordance was picked up by robotics scientists over 20 years ago. It refers to an action option that an object offers to an agent. Until now, however, this term has been applied almost exclusively to objects. In the ELSA project, we extend it to include a social dimension, that is, the action options offered to an agent by other agents.
In the area of algorithms, we build on a number of learning algorithms developed in robotics. Our main tools include reinforcement learning algorithms which learn through trial and error and feedback. But such algorithms are also increasingly used in cognitive-psychological research. Here, attempts are being made to adapt (parameterize) algorithms in such a way that the actual learning behavior of humans can be emulated.
What effect do social skills have on robots?
The goal is to enable robots to engage humans in their actions to complete their tasks and possibly do so more effectively. One means is to use social affordances directly. For example, a small robot can have a tall person hand it an item that is out of its reach high on a shelf. The two important aspects are the recognition of which humans offer which affordances to the robot, and how to effectively trigger such assistance, for example, by pointing and asking—literally, learning appropriate social skills.
A second means is for the robot to accelerate the acquisition of its competence by observing competent humans. If a robot wants to learn from a human how to do something, it is important to understand what that human can and cannot do. A human may not perform an action well because it is generally very difficult, such as getting a golf ball to the target in a single shot over a wide field. But it can also be that a human being does not succeed in an action because they do not have the necessary competence, such as juggling three balls; some people can do it, others cannot. These examples show that a robot should ideally observe different people performing an action to draw a conclusion: What constitutes the actual action, and what is the human factor? Recognizing this is a simple but fundamentally necessary social skill.
What is the biggest challenge?
While object affordances usually arise from the directly observable physical properties of the object, social affordances bring people into play, with all their vagaries and subtleties. Therefore, it is much more difficult to learn how to deal with social affordances than to learn how to deal with object affordances. Take the example of a social affordance, which depends not only on a person’s size but also on their affordances. Thus, the question might be whether they are in possession of the appropriate key. Another relevant factor is social affordances. Thus, the question could arise whether they are the owner of the desired object, or whether it might be a third person.
What would be accomplished by overcoming this challenge?
The research goal is achieved when robots interact with humans as fluidly and effectively as they do now with inanimate objects. As it is now, the degree of fluidity and effectiveness leaves plenty of room for improvement. This goal is a long way off and is not something we can achieve within the ELSA project. Rather, it will be necessary to build a supporting foundation for this goal, step by step. Our research equips robots with an expanded understanding of actions that should make human/robot interactions more fluid and provide a broader range of possibilities.
In what context could robots with social capabilities then be deployed?
Robots that have social capabilities like the ones we are aiming for in the ELSA project will be used in a wide variety of fields. In the manufacturing industry, they will provide flexibility that will also enable a previously uneconomical type of production in high-wage countries such as Austria. Examples include small batches and one-off items as well as manufacturing processes that cannot be fully automated. But socially acculturated robots pay off wherever they can help people, including in care and assisted living, as service and information robots in hospitals, museums, train stations, airports, retail, etc., if they interact just as fluidly and effectively as they do now with inanimate objects. As it stands, the degree of fluidity and effectiveness leaves plenty of room for improvement.
This goal is a long way off, and we cannot achieve it within the ELSA project. Rather, it will be necessary to build a supporting foundation for this goal, step by step. Our research equips robots with an enhanced understanding of actions that should make human/robot interactions more fluid and provide a broader range of possibilities.
About the project:
ELSA is a collaborative effort between Innsbruck, Austria; Sorbonne/CNRS, Paris, France; and LAAS/CNRS, Toulouse, France. The main drivers behind it at DiSC in Innsbruck are computer scientists Dr. Justus Piater and Erwan Renaudo, and psychologist Matthias Schurz.
