As the relationship between humans and technology gets more diverse and complex, scientists and researchers are constantly experimenting with novel ways to bring machines to the same par in understanding human emotions. There is no denying that over the last decade in the AI and generative AI arena, experts and critics have always pointed out that technological advancement is far from understanding or acting like their human counterparts.
Bridging that gap slowly, young researchers at the International Institute of Information Technology-Bangalore (IIIT-Bangalore) have developed a pipeline for modelling engagement in human-robot interactions. This innovative tech will provide deeper insights into a person’s personality, attitude, and emotions including the adoption of the Big Five personality traits, the Interpersonal Circumplex (IPC), and the Triandis Theory of Interpersonal Behavior (TIB).
“Automated systems of today are very static and standard in their responses and the quality of interaction is not up to the mark. However, when a human interacts with another human, we analyse their mood, personality, hand gestures and expressions and alter our responses accordingly. That’s what this pipeline aims to do and be more cognitive and human-like,” says Shrisha Rao, Professor at IIIT-Bangalore, the senior researcher of the study -- ‘A pipeline to model engagement in human-robot interactions’.
The interpretable and ready-to-use pipeline was developed by graduate students Soham Joshi and Arpitha Malavalli, along with Rao. The research was published in the One Journal of the Public Library Of Science (PLOS). The applications of this pipeline or to say approach can be found in domains like online learning platforms, assistive robotics, and intelligent conversational agents. If there is low engagement in customers using chatbots, the pipeline may show that there needs to be a change in the interaction style.
Similarly for online learning, if students are not engaged in the topic the pipeline could predict and introduce gamification and incentives for the students. For control systems, if a pilot or driver falls asleep, the trigger can be sent by the pipeline to put it on auto-mode. Similar applications can be looked at in improving the buying experiences of customers based on their current emotions and even in the medical field. “We wanted to leverage existing psychology theories. We tried to mirror the same in human-robot interaction. Given that if the robot can see your head movement or hear your voice modulation, then it can derive some inferences on your personality and the end goal is to analyse if you’re engaged in the conversation and suggest ways on how a human would go about it,” explains Arpitha.
The work on the possible game-changer tech commenced in 2021 and the pipeline was trained on human-to-human interaction and human-robot interaction, in a total of 36 sessions, 18 for each. Speaking on the accuracy of the model, Arpitha and Rao added that just like any psychology-related study cannot be 100% accurate, as behavioural patterns are involved, however, they were able to derive better correlations and are explainable. “There are so many technologies today that are built on the concept of ‘black box.’ What they do is they take some deep learning model, they train it directly, and they can’t actually say what exactly the model is doing. So I think with this pipeline, we can determine the accuracy, because it directly mirrors a human as it follows the same inference patterns,” adds Arpitha. The team will take the study forward and further establish the use of psychology for robots and also partner with industry and academia to bring the application of the pipeline into products.