Success for robotic arm controlled by eye gaze

  • July 13, 2020
Success for robotic arm controlled by eye gaze

Project could help those with speech and motor impairments to operate computers with their eyes

A team led by a Gates Cambridge Scholar has designed a robotic arm that can be manipulated by eye movement using a computer interface, aiding many people whose disabilities make it difficult to operate devices such as mouses or joysticks.

A research team at the Centre for Product Design and Manufacturing (CPDM), Indian Institute of Science (IISc), led by Pradipta Biswas, has designed the robotic arm. It aims to help people with Severe Speech and Motor Impairment (SSMI) ‒ a condition caused by disorders like cerebral palsy. The computer interface is non-invasive since it is through a webcam and a computer, unlike other eye gaze-tracking devices that use head-mounted systems.

The team worked with young adult students who have SSMI at Vidya Sagar in Chennai, an institution for individuals with disabilities. “Many of these students [with cerebral palsy] are not able to precisely focus at a single point in their visual field, due to uncontrolled gaze movement. They are also not comfortable to look at all portions of the visual field equally,” says Pradipta, who is Assistant Professor at CPDM. 

Pradipta and his team used computer vision and machine learning algorithms to analyse live feeds of facial video from the users and were able to estimate where the user was looking. They coupled this with an Augmented Reality application to allow the user to use a robotic arm for tasks like picking up and dropping objects and placing them where they want, as long as it was within reach of the robotic arm.

One of the main applications of this eye gaze-controlled robotic arm is rehabilitation of young adults with SSMI, through tasks such as fabric printing. They usually require assistance in doing such tasks, as they can only do a small part of it by themselves. Using the robotic arm designed by the researchers, people with SSMI can use their eye gaze to perform mechanical tasks that can help them work on handicrafts independently.

The eye gaze-controlled interface has been deployed at Vidya Sagar and is in regular use. “This validation and evaluation [of the interface and robotic arm] with end users is a major contribution of the study,” says Pradipta, who did his PhD in Computer Science at Cambridge [2006] with the support of a Gates Cambridge Scholarship. The students with SSMI are able to use the robotic manipulator almost as well as their non-disabled counterparts and the results were the same during repeated trials.

This technology can also be used by younger individuals with SSMI, to move toys like cars. “We are using play as a medium to teach new technologies, which they can use for the rest of their lives,” adds Pradipta. He says further modifications to this tool could also allow young individuals with SSMI to use it for e-learning.

The authors believe that this interface is a step towards designing future technologies which enable those with SSMI to lead more independent lives. Pradipta also says that such a system could be useful for automotive and aeronautical applications, as well as for developing collaborative robots used in smart manufacturing.

Latest News

How might extreme heat contribute to human migration?

Rising temperatures due to climate change are likely influencing human migration patterns, according to a new study co-authored by Gates Cambridge Scholar Dr Kim van Daalen [2018]. The study, led by Rita Issa of University College London, is published today in the open-access journal PLOS Climate. It looks at the role of heat in human […]

Scholar scoops prestigious science innovation fellowship

Freja Ekman has been named one of the 2023 class of Hertz Fellows as the prestigious fellowship celebrates its 60th year. The 15 fellowships in applied science, engineering and mathematics are awarded by Fannie and John Hertz Foundation, a non-profit organisation for innovators in science and technology. Winners will have their graduate studies funded for […]

Scholar hosts first UN communications technology access meeting in India

Gates Cambridge Scholar Pradipta Biswas has hosted a UN meeting on improving access to communications technology – the first ever held in India. The meeting of ITU-T Study Group 9 (SG-9) on “Broadband Cable and Television/Audiovisual content transmission and integrated broadband cable networks”  was held in May at the Indian Institute of Science in Bengaluru […]

The role of storytelling in addressing colonial trauma

When she was growing up, Briseyda Barrientos Ariza [2023] spent her childhood summers in rural Guatemala, visiting her grandparents. In the evenings she would listen to her grandfather, a man who couldn’t read or write, tell intricate stories about his and others’ encounters with female folkloric figures. Two stood out in particular: La Llorona and […]