Creating emotionally intelligent machines

  • October 30, 2015
Creating emotionally intelligent machines

Andra Adams wins UK ICT Pioneers award for her research

Facial expressions and head motions are important modalities that humans use to communicate their mental states to others. As computers continue to become ubiquitous, their ability to recognise and respond to the emotional displays of humans will become increasingly essential.

Andra Adams

Gates Cambridge Scholar Andra Adams has won the Transforming Society category at the 2015 UK ICT Pioneers competition for her research on emotionally intelligent machines.

The winners of the competition were announced at an exhibition and awards ceremony in London last night.  The awards are a unique partnership between the Engineering and Physical Sciences Research Council (EPSRC) and key stakeholders that aim to recognise “the most exceptional UK doctoral students in ICT-related topics who are able to communicate and demonstrate the excellence and exploitation potential of their research”.

Andra was among 15 finalists who showcased their research at the prestigious event which was compered by science journalist Quentin Cooper. The four category winners were awarded £2,000 each and pitched their research to a ‘Dragon’s Den’ style panel of industry experts and sponsors from the EPSRC, Defence Science and Technology Laboratory (Dstl), Hewlett Packard Enterprise, Facebook, British Computing Society (BCS), Samsung and BT.

Andra researches and develops interfaces that recognise, provide coaching on and synthesise, facial expressions and head motions which convey complex emotions. She says: “Facial expressions and head motions are important modalities that humans use to communicate their mental states to others. As computers continue to become ubiquitous, their ability to recognise and respond to the emotional displays of humans will become increasingly essential.”

Previous research in affective computing has focused either only on basic emotions or on a tiny subset of complex emotions and has largely ignored head motions.

Rather than training on pixel values or obscure facial geometry, Andra [2010], who is doing a PhD in Computer Science, is training on intuitive features such as the intensity and speed of facial muscle movements (such as a strong eyebrow raise or a weak jaw drop). She describes this as “a pioneering effort to create a tool which provides coaching on facial expressions and head motions for humans to better convey complex emotions”.

Andra developed the research herself under the supervision of Professor Peter Robinson and using face tracking software which was developed by her colleagues at the Computer Laboratory.

She is also synthesising the emotions on a realistic android head. The android hed  is currently animated through direct imitation of input videos and Andra says it will soon also be able to express its own novel expressions of complex emotions using various extrapolation techniques built on the emotion classifier. She says it marks a departure for affective robotics which has only investigated the synthesis of basic emotions up to now.

She hopes her research will benefit many industries including education, health and safety, marketing and healthcare. For instance, automated tutors could detect and adjust a student’s lesson according to the emotions they are exhibiting; emotionally ­aware computers could also monitor employees during critical moments, like detecting drowsiness or boredom in pilots to prevent accidents from falling asleep at the wheel; and children with autism could practise with a facial expression coaching system to improve their ability to recognise complex emotions on the playground.

Martin Sadler, Vice President and Director of Hewlett Packard Labs, EMEA told the winners: “We are truly amazed at the quality and excellence  of all the finalists. Industry and universities absolutely need you and your talents.”

*Watch Andra’s video entry for the second stage of the competition here.

Latest News

New app aims to help women through the menopause

A new app which helps women to manage the menopause was soft launched last month in collaboration with Mumsnet. Stella is the first product by Vira Health, a company which was co-founded in 2019 by Gates Cambridge Scholar Rebecca Love. Stella offers women relief from the most common symptoms of menopause, including sleep disturbances, hot […]

A global perspective on gender and health

The middle of a global pandemic may not seem the ideal time to move country with a new baby, but Johanna Riha [2011] took up her new role as a research fellow at the United Nations University International Institute for Global Health (UNU-IIGH) in Malaysia during the pandemic and moved to Kuala Lumpur around a […]

Scholars share 2021 Bill Gates Sr. Prize

Two Gates Cambridge Scholars are sharing the 2021 Bill Gates Sr. Prize in recognition of their outstanding research and social leadership. Emma Soneson and Maša Josipović have been selected for the prize which was established by the Gates Cambridge Trustees in June 2012 in recognition of the late Bill Gates Sr.’s role in establishing the […]

The censoring effect of populist anti-media messages

Populist attacks on the press should be viewed as a form of soft censorship which uses journalistic norms regarding objectivity to undermine the media, according to a new study by a Gates Cambridge Scholar. The study, Covering populist media criticism: When journalists’ professional norms turn against them, by Ayala Panievsky, is published in the International […]