顔認識：機械学習におけるジェンダーと交差性の分析

顔認識システムは、群衆の中にいる人の識別、感情の分析、性別、年齢、人種、性的指向、顔の特徴などを検出することができる。これらのシステムは、採用、支払いの承認、警備、監視、電話のロック解除などに多く採用されている。信頼性と堅固性を向上させようとする学術界と産業界の研究者の努力にもかかわらず、最近の研究では、これらのシステムが、人種や性別などの特徴やそれらの交差性に基づいた差別を行う可能性が実証されている（Buolamwini & Gebru, 2018）。

方法：機械学習におけるジェンダーと交差性の分析

機械学習におけるバイアスは多面的であり、データ収集から生じることもあれば、データ準備やモデル選択から生じることもある。例えば、男性や肌が白い人が多いデータセットでは、肌が黒い女性を誤認することが多い。これは、異なる種類の識別が個人やグループに対して負の反応を増幅させる交差バイアスの一例である。

ジェンダード・イノベーション：

1. 顔認識における識別の理解 ：顔認識システムでは、顔検出、顔属性分類、顔照合、顔識別の各ステップにおいて、バイアスがかかっていないかどうかを確認する必要がある。モデルを有効に活用するためには、ターゲットとなる母集団を代表するデータセットをモデルとして学習させる必要がある。例えば、壮年層で訓練されたモデルは、若年層ではうまく機能しない（Howard et al., 2017）。
2. 交差的な学習データセットの作成：国境を越える人々の身元を確認するようなタスクを適切に行うためには、学習データセットにジェンダーや人種などの交差的な特徴を含める必要がある。マサチューセッツ工科大学を拠点とする「Gender Shades」プロジェクトは、肌が黒い女性、肌が黒い男性、肌が白い女性、肌が白い男性という4つのカテゴリーのデータセットを作成し、検証を行った（Buolamwini & Gebru, 2018）。
3. 多様な顔に対応するパラメータの設定：化粧した顔では、顔認識システムの精度が76％低下する（Dantcheva et al.2012、Chen et al.2015）。トランスジェンダーの人々、特に性別移行期には、正しく識別されないことがある（Keyes, 2018）。公開されている顔データベースは、多様なカテゴリーにわたって認識精度を向上させるパラメータを確立する必要があり（Eckert et al., 2013）、トランスジェンダーなどのコミュニティを危険にさらすことを避ける配慮が必要である（Keyes, 2018）。

ケーススタディの詳細はこちら

The Challenge

Facial recognition systems (FRSs) can identify people in crowds, analyse emotion, and detect gender, age, race, sexual orientation, facial characteristics, etc. These systems are employed in many areas, e.g. hiring, authorizing payments, security, surveillance, and unlocking phones. Despite efforts by both academic and industrial researchers to improve reliability and robustness, recent studies demonstrate that these systems can discriminate based on characteristics such as race and gender and their intersections (Buolamwini & Gebru, 2018). In response, national governments, companies, and academic researchers are debating the ethics and legality of facial recognition. One point is to enhance the accuracy and fairness of the technology itself; another is to evaluate its use and regulate deployment through carefully implemented policies.

Method: Analyzing Gender and Intersectionality in Machine Learning

Bias in machine learning (ML) is multi-faceted and can result from data collection, or from data preparation and model selection. In the first case, the data may contain human bias. For example, a dataset populated with men and lighter-skinned individuals will misidentify darker-skinned females at higher rates. This is an example of intersectional bias, where different types of discrimination amplify negative effects for an individual or group. Intersectionality refers to intersecting categories such as ethnicity, age, socioeconomic status, sexual orientation and geographic location that impact individual or group identities, experiences and opportunities.

Bias may also be introduced during data preparation and model selection, which involves selecting attributes that the algorithm should consider or ignore, such as facial cosmetics or the faces of transgender individuals during transition.

View General Method

Gendered Innovation 1: Understanding Discrimination in Facial Recognition

To understand discrimination in FRSs, it is important to understand the difference between face detection, facial attribute classification, facial recognition, face verification and facial identification.

Face detection can occur without facial recognition; however, facial recognition does not work without face detection. Face detection differentiates a human face from other objects in an image or video. Face detection systems have repeatedly been shown to fail for darker-skinned individuals (Buolamwini, 2017). For example, early versions of the statistical image-labelling algorithm used by Google Photos classified darker-skinned people as gorillas (Garcia, 2016). If this first step fails, FRSs fail.

Facial attribute classification labels facial attributes such as gender, age, ethnicity, the presence of a beard or a hat, or even emotions. For emotions, a review of online recognition APIs showed that over 50% of the systems used facial expressions to define emotions (Doerrfeld, 2015). To work, these models must be trained on datasets that are representative of the target population. Models trained on mature adults, for example, will not perform well on young people (Howard et al., 2017).

Facial recognition identifies a unique individual by comparing an image of that person to their known facial contours. One type of facial recognition is verification, often used to unlock a phone or to validate the identity of someone crossing an international border (see the EU-funded H2020 project, iBorderCtrl). These solutions are designed to decrease processing time. If, however, the FRSs are not trained sufficiently, for example on darker-skinned women (Buolamwini & Gebru, 2018) or transgender faces (Keyes, 2018), the technologies will fail for them—see below. The other type of facial recognition is facial identification, which matches the face of a person of interest to a database of faces. This technology is often used for missing persons or criminal cases.

Gendered Innovation 2: Creating Intersectional Training Datasets

The accuracy of FRSs is determined by the set of images or videos collected for testing. For the FRSs to perform well, training data must be sufficiently broad and diverse to enable the predictive model to accurately identify faces in a variety of contexts.

Zhao et al. (2017) found that when photographs depict a man in a kitchen, automated image captioning algorithms systematically misidentify the individual as a woman, in part because training sets portray women in cooking contexts 33% more frequently than men. The trained model amplified this disparity from 33% to 68% during testing. It is crucial to get the training data right.

In the now well-known Gender Shades study, Buolamwini & Gebru (2018) measured the accuracy of commercial gender classification systems from Microsoft, IBM and Face++, and found that darker-skinned women were often misclassified. Systems performed better on men’s faces than on women’s faces, and all systems performed better on lighter-skin than darker-skin. Error rates were 35% for darker-skinned women, 12% for darker-skinned men, 7% for lighter-skinned women and less than 1% for lighter-skinned men.

To overcome these problems, the team developed and labelled an intersectional dataset to test gender and race classification performance on four subgroups: darker-skinned women, darker-skinned men, lighter-skinned women and lighter-skinned men. Since race and ethnicity labels are culturally specific, the team used skin shade to measure dataset diversity (Cook et al., 2019). Their dataset consisted of 1270 images from three African countries (Rwanda, Senegal and South Africa) and three European countries (Iceland, Finland and Sweden).

An update to Gender Shades retested the three commercial systems previously examined and expanded their review to include Amazon’s Rekognition and a new system from a small AI company called Kairos. They found that IBM, Face++ and Microsoft had all improved their gender classification accuracy for darker-skinned women, with Microsoft reducing its error rate to below 2% (Raji & Buolamwini, 2019). Amazon’s and Kairos’s platforms, however, still had accuracy gaps of 31% and 23%, respectively, between lighter-skinned men and darker-skinned women.

Accuracy in Intersectional Classification

IBM Research has recently released a large dataset called Diversity in Faces (DiF). Their goal is to advance the study of fairness and accuracy in FRSs (IBM, DiF).

Gendered Innovation 3: Establishing Parameters for a Diverse Set of Faces

Cosmetics

Wearing makeup has been a popular practice throughout history and across numerous cultures. Although men’s use of cosmetics is on the rise, women use makeup more. Loegel et al. (2017) report that women use makeup to give themselves an image that is compatible with cultural norms of femininity.

Facial cosmetics reduce the accuracy of both commercial and academic facial recognition methods by up to 76.21% (Dantcheva et al., 2012; Chen et al., 2015). One reason is that makeup has not been established as a parameter in publicly available face databases (Eckert et al., 2013). One proposal for developing FRSs that are robust for facial makeup is to map and correlate multiple images of the same person with and without makeup. These solutions also need to take into account different makeup practices across cultures (Guo & Yan, 2014).

Transgender

One of the emerging challenges in the field of FRSs is transgender faces, especially during transition periods. Keyes (2018) found that Automatic Gender Recognition (AGR) research ignores transgender people, with negative consequences. For instance, a transgender Uber driver was required to travel two hours to a local Uber office after a facial recognition security feature was unable to confirm their identity. The company uses Microsoft Azure’s Cognitive Services technology to reduce fraud by periodically prompting drivers to submit selfies to Uber before beginning a shift (Melendez, 2018).

Gender-affirming hormone therapy redistributes facial fat and changes the overall shape and texture of the face. Depending on the direction of transition (i.e., male to female or female to male), the most significant changes in the transformed face affect fine wrinkles and lines, stretch marks, skin thickening or thinning and texture variations. Androgen hormone therapy, for example, renders the face more angular.

Is the solution to correct bias by ensuring that plenty of trans people are included in training data? Debiasing might sound nice, but collecting data from a community that has reason to feel uncomfortable with data collection is not the best practice (Keyes, 2018). In this case, it may be important to revise algorithmic parameters. Existing methods indicate that the eye (or periocular) region can be used more reliably than the full face to perform recognition of gender-transformed images, as the periocular region is less affected by change than other facial regions (Mahalingam et al., 2014).

Scheuerman et al. (2019) found that four commercial services (Amazon Rekognition, Clarifai, IBM Watson Visual Recognition and Microsoft Azure) performed poorly on transgender individuals and were unable to classify non-binary genders. Using the gender hashtag provided by individuals in their Instagram posts (#woman, #man, #transwoman, #transman, #agender, #genderqueer, #nonbinary), the team calculated the accuracy of gender classification results across 2,450 images. On average, the systems categorized cisgender women (#woman) with 98.3% accuracy and cisgender men (#man) with 97.6% accuracy. Accuracy decreased for transgender individuals, averaging 87.3% for #transwoman and 70.5% for #transman. Those who identified as #agender, #genderqueer or #nonbinary were mischaracterized 100% of the time.

Emotion

The European Union-funded SMART project (Scalable Measures for Automated Recognition Technologies) sought to understand how smart surveillance can protect citizens against security threats while also protecting their privacy. The project presented a multi-modal emotion recognition system exploiting both audio and video information (Dobrišek et al., 2013). The experimental results show that their Gender Recognition (GR) subsystem increased overall emotion recognition accuracy from 77.4% to 81.5%.

Conclusions

FRSs can perpetuate and even amplify social patterns of injustice by consciously or subconsciously encoding human bias. Understanding underlying intersectional discrimination in society can help researchers develop more just and responsible technologies. Policy makers should ensure that biometric programs undergo thorough and transparent civil rights assessment prior to implementation.

The potential misuse of facial recognition has led to several actions: Belgium has declared the use of facial recognition illegal, France and Sweden have expressly prohibited it in schools, and San Francisco, California has banned its use by local agencies, such as the transport authority or law enforcement. Companies, too, are pulling back: IBM has left the facial recognition business entirely, and Amazon has stopped police from using its facial recognition technology in response to worldwide protests against systemic racial injustices in 2020.

Seeking long-term solutions, the US Algorithmic Justice League has called for the creation of a federal office similar to the US FDA to regulate facial recognition (Learned-Miller et al., 2020).

Next Steps

1. Algorithms are only as good as the data they use. Training datasets should be sufficiently large and diverse to support FRSs across diverse populations and in diverse contexts.
2. In the Face Recognition Vendor Test published by the US National Institute of Standards and Technology (NIST, 2019), the 1:N report (most relevant for FRSs used in bodycams) does not include a detailed breakdown of accuracy metrics for factors such as race, age and gender. This should be corrected.
3. Develop fair, transparent and accountable facial analysis algorithms.
4. Many of the studies and systems still adopt a binary view of gender. Research and innovation in this field must acknowledge the existence of a large number of gender-diverse individuals and create technologies that work across all of society.