Are computer algorithms sexist?

  • August 8, 2016
Are computer algorithms sexist?

James Yang Zou co-authors paper which attempts to counter inherent sexism in computer algorithms.

By reducing the bias in today’s computer systems (or at least not amplifying the bias), which is increasingly reliant on word embeddings, in a small way debiased word embeddings can hopefully contribute to reducing gender bias in society.

James Yang Zou and colleagues

Are computer algorithms inherently sexist and if so what can be done about it? A research paper co-authored by a Gates Cambridge Scholar shows how data sets embed sexist assumptions into searches and investigates how to counter this.

The paper, Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings, is published in arXiv.org. Led by Tolga Bolukbasi  from Boston University and co-authored by Gates Cambridge Scholar James Yang Zou [2007], who is currently an assistant professor at Stanford University, it investigates patterns in the way words on the internet appear next to each other based on a powerful data set called Word2vec devised by Google researchers researching Google News.

But the new study finds vector space is blatantly sexist with embedded pairings including she:he :midwife:doctor; sewing:carpentry; registered_nurse:physician; whore:coward; hairdresser:barber; nude:shirtless; boobs:ass; giggling:grinning; and nanny:chauffeur. This occurs because any bias in the Google News articles that make up the Word2vec corpus is captured in the geometry of the vector space.  They are concerned at the role of vector space in web searches, for instance, it could affect searches for potential candidates for jobs in professions deemed more "male" such as computer programming. The researchers says this could have the effect of increasing bias, rather than simply reflecting it.

To counter this, they use standard mathematical tools to manipulate vector space. That involves searching the vector space using Amazon's Mechanical Turk to find whether embedded pairings are appropriate or inappropriate.

After compiling a list of gender biased pairs, the team subjected it to a process of "hard de-biasing", with the sexist bias removed from the vector space. The pairings were then subject to the Mechanical Turk again and both direct and indirect bias was significantly reduced.

“One perspective on bias in word embeddings is that it merely reflects bias in society, and therefore one should attempt to debias society rather than word embeddings,” say the researchers. “However, by reducing the bias in today’s computer systems (or at least not amplifying the bias), which is increasingly reliant on word embeddings, in a small way debiased word embeddings can hopefully contribute to reducing gender bias in society…At the very least, machine learning should not be used to inadvertently amplify these biases.”

*Picture credit: Wikipedia.

James Zou

James Zou

  • Alumni
  • United States
  • 2007 CASM Applied Mathematics
  • Jesus College

I am participating in the Part III program in Applied Mathematics at Cambridge. I'm interested in the quantitative aspects of a wide range of topics--biology, sociology, and AI. I hope to explore the synthesis of these diverse topics at a fundamental level. I look forward to completing a Ph.D. after Part III.

Latest News

$75,000 grant for technology to assist people with disabilities

A Gates Cambridge Scholar and his colleague have been awarded a prestigious grant from Facebook Reality Labs to Dr Pradipta Biswas and Professor Yogesh Simmhan have been awarded a $75,000 grant from Facebook Reality Labs for their proposal on ‘privacy-respecting augmented reality[AR]/virtual reality[VR] to enable differently abled people in multi-cultural societies. The grant was the […]

Gut bacteria links to immune responses in the brain

Bugs in the gut may hold the key to protective immune measures in the brain which could have implications for diseases such as Parkinson’s and multiple sclerosis, according to a new study led by Gates Cambridge Scholar Zachary Fitzpatrick. A paper based on his PhD research has recently been published in Nature and it highlights […]

Exploring the social barriers to take-up of green technology

How can rural communities be encouraged to take up green energy solutions? A new study co-authored by Gates Cambridge Scholar Ramit Debnath investigates the social barriers to uptake of household appliances fuelled by green energy. Based on research on more than 14.5K households in rural communities in Rwanda, the study, published in Renewable Energy, found […]

A new technique to decode the way the nervous system works

How do the billions of neurons in the human brain work together to give rise to thought or certain types of behaviour? A new study led by Gates Cambridge Alumnus Eviatar Yemini [2007] outlines a colouring technique, known as NeuroPAL (a Neuronal Polychromatic Atlas of Landmarks), which makes it possible – at least in experiments […]