El, La, or Neither? The Problem of Gender Bias in NLP

Handling gender in natural language processing (NLP) introduces a number of technical and ethical challenges. Some of these challenges span multiple years or decades, such as translation from weakly gender-inflected languages to strongly gender-inflected languages (Schiebinger et al., 2018).  Other gender issues have seen academic treatment only more recently, such as the tendency for deep learning-based word embeddings to ingrain stereotypes as part of the semantic meaning (Garg, 2018). We can divide the origin of issues with gender-related bias in NLP systems into two categories: problems with data selection and a lack of timely responses to dynamic sociological trends. NLP practitioners aiming to be ethical scientists must recognize these two classes of issues and continually assess how to mitigate their effect in NLP systems.

With the recent trend of NLP systems toward using statistical-based deep learning approaches, the choice of the data to train the models is crucial to their behavior. While de-biasing techniques exist (Sun et al., 2019), machine learning systems ultimately learn primarily the information and relationships in the data used to train them. Problematically, data can contain embedded biases that seep into NLP. A longstanding example of NLP incorporating stereotypes is in machine translation. NLP applications such as Google Translate attempt to translate between weakly gender-inflected languages (e.g., English) and strongly gender-inflected languages (e.g., Spanish). For some inputs, translation from English to Spanish is non-deterministic. For example, the single English sentence “I like the professor” can translate to “Me gusta la profesora” or “Me gusta el profesor” in Spanish. Early machine-translation systems approached translation from language X to language Y by selecting the string in Y with the highest probability of being the translation of X according to a corpus of examples. This technique, however, revealed stereotypes encoded in the probabilities (Schiebinger et al., 2018). When translating “the doctor,” Translate outputted “el doctor.” For “the nurse,” it gave “la enferma.” More recently, the use of word embeddings has provided state-of-the-art results for natural language tasks. Unfortunately, like the probabilities in machine translation, word embeddings can encode gender bias. Garg et al. (2018) demonstrate that the vector for “honorable” is more similar to “man,” while the vector for “submissive” is more similar to “woman.” While seeing these blatant stereotypes is jarring at first, especially from a progressive company like Google, Translate (or word2vec) is not intentionally sexist. Rather, its training corpus contains sexist biases. More examples of the bigram “el doctor” appeared than the bigram “la doctora” and “submissive” appeared more with “woman” than it did with “man.” These stereotypes are indicative of the reality, albeit an unpleasant one, that the way we use language includes stereotypes. Although tempting to assume that NLP experts can sweep the problem under the rug using data augmentation, downsampling, or other techniques to mitigate bias in data, there is systemic bias that results in the biased data. We need to use technical strategies to prevent our models from perpetuating stereotypes, but new sources of bias will continue to crop up unless we fix the societal effects underlying the data.

In addition to issues with data, another problem can arise even for researchers who find unbiased data or manage to mitigate bias. Without staying abreast of changing sociological research and trends, NLP researchers may not recognize when prior assumptions about gender no longer hold or become insensitive. Falling victim to a changing environment does not exclude researchers who theoretically should be attuned to gender bias issues in NLP. For instance, in an article about the social consequences of NLP, Hovy and Spruit (2016) comment that “we might be…amused when receiving an email addressing us with the wrong gender” but would not be so entertained if an NLP system misidentified our sexual orientation. Even while writing about the ill effects of gender bias, the authors assume that misgendering might be a cause for laughter. As numerous studies demonstrate (McLemore, 2018; Ansara and Hegarty, 2014), misgendering can be traumatizing, not amusing. While difficult to ascertain Hovy and Spruit’s reasoning for this comment, the authors were likely operating in a gender-binary frame of mind. While a cisgender man may chuckle at an AI system referring to him as a woman, this type of misgendering would likely not be so funny to a transgender man. Perhaps influenced by the contemporaneous Obgerfell v. Hodges Supreme Court case that legalized same-sex marriage, the authors consider sexual orientation more important for systems to predict correctly than gender. Hovy and Spruit’s comment highlights why NLP practitioners must be vigilant about all types of gender bias, not just those receiving media coverage at the time.

Solutions to the instances of gender bias are not straightforward. Schiebinger proposes a framework for using coreference to extract information about the gender of subjects in the global context of a document rather than looking just at one sentence at a time. She also suggests integrating gender analysis into computer science curricula. While I concur with this recommendation, I would suggest it is not enough. Those developing NLP products and techniques (like Translate and word2vec) should not stop learning about sources of bias when they conclude their formal education. They should instead continue to interface with researchers in sociology, women and gender studies, and other disciplines to ensure they are aware of the potential vulnerabilities of their systems to bias.Along these lines, Crawford and Kalo (2016) advocate a social-systems approach to thinking about the effects of bias on NLP. A social-systems approach puts considering the “social and political history” of data, and the subsequent potential for bias, ahead of trying to exploit that data to build NLP products. It would also emphasize human-in-the-loop systems where “people affected by AI systems” can ask questions about how the systems work rather than being subjected to black-box solutions. Along with a social-systems approach that makes systems more transparent and gives experts outside of NLP the opportunity to identify bias in NLP systems, NLP practitioners must recognize that they are operating in a dynamic environment. Even issues that at one point in time seem essentially solved, like translation between weakly and strongly gender-inflected languages, can become unsolved due to sociological changes like the increase in non-binary identifying people, and generally well-meaning researchers like Hovy and Spruit can unintentionally trvialize an entire source of gender bias.

References

Ansara, Y. G., & Hegarty, P. (2014). Methodologies of misgendering: Recommendations for reducing cisgenderism in psychological research.  Feminism & Psychology ,  24 (2), 259-270.

Crawford, K., & Calo, R. (2016). There is a blind spot in AI research.  Nature ,  538 (7625), 311-313.

Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018). Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16), E3635-E3644.

Hovy, D., & Spruit, S. L. (2016, August). The social impact of natural language processing. In  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)  (pp. 591-598).

McLemore, K. A. (2018). A minority stress perspective on transgender individuals’ experiences with misgendering.  Stigma and Health ,  3 (1), 53.

Schiebinger,  Londa  et  al.  (2018).  Machine  Translation:  Analyzing  Gender (Stanford case study on gender in Machine Translation). Stanford Gendered Innovations.

Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., ... & Wang, W. Y. (2019). Mitigating gender bias in natural language processing: Literature review.  arXiv preprint arXiv:1906.08976 .