Reading notes from Monday 10/21.
Mitigating Gender Bias in Natural Language Processing: Literature Review
What is the problem the paper identifies?
Although current NLP practices have very high accuracy, they tend to pick up on and amplify the discriminatory practices held within their text corpora. Methods of mitigation (particularly in the arena of gender bias) are relatively new, and thus lacking.
What is the contribution the paper presents?
Give more detail about the specific problem the paper is addressing.
NLP algorithms display and amplify gender bias present in their training data. The authors sort gender bias into two main categories:
They also identify four main types of gender bias:
The paper aims to summarize recent work in combatting gender bias in NLP, while pointing out its shortcomings and proposing optimizations, gaps in knowledge, and future directions for research.
Adopting Psychological Tests
Recent methods of testing gender bias in NLP has adopted measures used by psychological tests to detect bias in humans.
The Implicit Association Test (IAT) is a psychological test used to measure subconscious bias in people. It functions by making users categorize the members of a pair (men-women, elderly-youth) with descriptive associations (STEM-arts, bad-good). The difference in time and accuracy between the tests is quantified.
Word Embedding Association Test (WEAT) does the same thing but with word embeddings. It confirms that GloVe and Word2Vec have implicit gender bias (using the test men = STEM, women = arts, and vice versa).
This associative bias in word embeddings of a specific occupation has been shown to correspond to the percentage of women in that occupation in the US labor force. Additionally, the reverse has been shown: the strength of such word embedding associations has been shown to predict the percentage of women in that occupation.
Analyzing Gender Sub-space in Embeddings
Another method of detecting gender bias in NLP is by analyzing the relationships between a word embedding's magnitude within a gender subspace. The gender subspace is rendered by classifying words into a set of gender-specific and a set of gender-neutral terms.
However, this does not fully capture the extent of gender bias. Even after the projection of gender-neutral words onto the gender subspace have been removed, words with similar biases continue to cluster together (cluster bias).
Measuring Performance Differences Across Genders
The extent of gender bias is quantified by measuring the difference in performance when gender-specific words are swapped out for their "opposites." This is called gender swapping. For instance, the sentence "He went to the park" would be evaluated, then "She went to the park" would be evaluated, and the difference between their performance results in a metric of the system's bias.
Performance differences are mainly measured by two quantifiers: False Positive Equality Difference (FPED) and False Negative Equality Difference (FNED). These measure differences in false positive and false negative rates of predictions and for their gender-swapped counterparts.
Gender Bias Evaluation Testsets (GBETs)
→ The use of standard data sets do not fully measure gender bias because they
Instead, we use GBETs.
Counterargument: "Some may argue that the artificial design of GBETs does not reflect the true distribution of the data, implying that these evaluations are artificial. We argue that if humans can avoid making mistakes due to gender bias, then machines should as well."
Coreference resolution
Gender-swapped GBETs have been analyzed in the context of coreference resolution, using Winograd Schemas.
For example, sentences contain an occupation, a participant, and a gendered pronoun. If occupation is "doctor" and participant is "patient," we measure the likelihood of having male/female/non-binary pronouns for the occupation/participant.
Having a global difference in accuracies of a model does not take into account individual inputs (different situations can cancel out).
GBETS using GAP have also been utilized (where they are not gender-swapped and text corpora is mined from Wikipedia, so the gender-specific word in question is a name and not a pronoun).
Sentiment analysis
An Equity Evaluation Corpus (EEC) has been designed to measure the emotional intensity and type of emotion associated with gender-swapped sentences.
Debiasing methods can also be categorized as retraining and inference.
Debiasing Training Corpora
Data Augmentation
Debias training data by creating an augmented set consisting of both the original set and its gender-swapped equivalent. This second set is created both through gender-swapping (in similar fashion to GBETs) and name anonymization (to remove bias associated with names; e.g. replace "Mary" with E1).
Drawbacks include the creation of nonsensical sentences and the doubling of the test set (and therefore increased expenses and resources).
Gender Tagging
Machine training models that need to predict the gender of a source of some data overwhelmingly predict their sources are male due to the overrepresentation of male data in training sets. An antidote to this is gender tagging, in which a particular data point is "tagged" with a gender, and then used (with the tag parsed separately from the rest of the data). This is helpful in situations such as translation where the new language requires knowledge of the speaker's gender to be grammatically correct (e.g. French).
Drawbacks include the expenses associated with this process, as knowledge of the gender of each data point requires "meta-information" which is costly to obtain.
Bias Fine-Tuning
Another strategy is to train models on both a biased data set and a related data set known to be unbiased, so as to pinpoint the source of bias. This approach allows predictions to avoid bias while still maintaining accuracy.
Drawbacks include that in the little testing this method has undergone, it has not been as effective as gender-swapping; but further experimentation is needed.
Debiasing Gender in Word Embeddings
Removing Gender Subspace in Word Embeddings
Previous works have removed word embeddings' similarities to the gender subspace by creating a new genderless framework. This may be problematic when we consider whether or not the concept of gender is intrinsic to the semantic understanding of a word. In light of this, more recent work has altered individual word embeddings by only removing gendered components from gender-neutral words (by placing them orthogonal in the gender subspace). Yet even had the same performance results as unaltered embeddings.
Learning Gender-Neutral Word Embeddings
Lastly, another method trains word embeddings by isolating gender information in specific dimensions, which can then me utilized or ignored based on the circumstance, allowing for greater flexibility.
Constraining Predictions
Reducing Bias Amplification (RBA) takes an existing model's optimization function and constrains it to to make sure it fits prediction quotas.