Incident: Borrowed word vectors judged people by name

Module 6 · Incident file · About 30 minutes

What happened

In July 2017 Rob Speer, of the ConceptNet project, published "How to make a racist AI without really trying" on the ConceptNet blog. He followed a standard recipe of the time: take pretrained GloVe word vectors, the kind Lesson 2 loaded, and a sentiment word list, and train a classifier that scores how positive a text is.

The classifier worked as a sentiment model. It also rated "Let's go get Mexican food" lower than "Let's go get Italian food", and it scored sentences differently depending on the people's names in them. Speer wrote no rule about food or names. The bias came from the pretrained word vectors.

Tidewell is about to use word vectors near customers, so the question for this module is direct: would the same recipe judge Tidewell's customers by their names, and how would you know?