Study evaluates gender bias in relation extraction systems.
problem Gender bias in relation extraction systems.
method Created WikiGenderBias dataset, evaluated systems for bias, analyzed bias mitigation techniques.
result NRE systems exhibit gender bias in predictions.
Reduces gender classification bias by learning race-invariant face representations.
problem Societal bias in gender recognition systems.
method Adversarially trained autoencoder model to learn race-invariant face representations.
result Achieved a significant drop of over 40% in racial bias surrogate metric with race invariant representations.
Study measures gender bias in machine translation using multiple reference points.
problem Measuring and identifying gender bias in machine translation.
method Used an optimal non-biased translator, reference points from occupational statistics and survey.
result Found bias against both genders, but more against women, and found occupations have a greater effect than adjectives.
Reduces gender bias in patient notes while maintaining medical classification accuracy.
problem Bias in natural language processing of patient notes.
method Identifying and removing gendered language using BERT-based classifiers, then augmenting data to maintain performance.
result Minimal degradation in health condition classification tasks with data augmentation.
We present a large-scale study of gender bias in occupation classification, a task where the use of machine learning may lead to negative outcomes on peoples' lives. We analyze the potential allocation harms that can result from semantic representation bias. To do so, we study the impact on occupation classification of…
Mitigates gender bias amplification in model predictions.
problem Gender bias amplification in model predictions.
method Posterior regularization to mitigate bias.
result Almost removes gender bias amplification in model predictions.
Develops NFCF to reduce gender bias in social media recommendation systems.
problem Reduces gender bias in collaborative filtering systems on social media data.
method Pre-training and fine-tuning neural collaborative filtering with bias correction techniques.
result Achieves better performance and fairness in gender de-biased recommendations.
(Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can inherit gender bias from the data they were trained on. We investigate how this bias affects downstream classification tasks, using the case study of occupation classification (De-Arteaga et al.,2019). We show that traditional techniques for debi…
Contemporary debates on filter bubbles and polarization in public and social media raise the question to what extent news media of the past exhibited biases. This paper specifically examines bias related to gender in six Dutch national newspapers between 1950 and 1990. We measure bias related to gender by comparing loc…
Study quantifies gender bias in language models across 7 languages.
problem Measuring gender bias in language models across multiple languages.
method Curated dataset of politicians, multilingual language models, probing language models.
result Larger language models do not show significant gender bias compared to smaller ones.
Study finds gender bias in human evaluators and shows how machine learning can mitigate it.
problem Gender bias in human decision-making on micro-lending platforms.
method Structural econometric model and machine learning algorithms trained on real-world data.
result Machine learning algorithms can mitigate both preference-based and belief-based biases.
New method neutralizes gender bias in word embeddings without losing semantic information.
problem Gender biases in word embeddings trained on human-generated corpora.
method Latent Disentanglement and Counterfactual Generation with siamese auto-encoder and gradient reversal layer.
result Our method outperforms existing debiasing methods in preserving semantic information and neutralizing gender biases.
Text corpora are widely used resources for measuring societal biases and stereotypes. The common approach to measuring such biases using a corpus is by calculating the similarities between the embedding vector of a word (like nurse) and the vectors of the representative words of the concepts of interest (such as gender…
The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text data as vectors which has been used in many machine learning and natural language processing tasks. We show that even word embeddings traine…
New method reduces gender bias in language models without harming performance.
problem Bias in language models learned from biased data.
method Causal analysis to identify problematic model components, followed by linear projection of weight matrices.
result DAMA significantly decreases bias in language models while maintaining performance.
This paper assesses biases in contextualized word representations.
problem Analyzing biases in contextualized word representations.
method Proposes assessing bias at the contextual word level, capturing contextual effects of bias.
result Demonstrates evidence of bias in contextual word models, including racial bias and exacerbated effects for intersectional minorities.
Paper debiases multiple word embedding biases simultaneously.
problem Reduction of multiple biases in word embeddings.
method Joint multiclass debiasing approach using Word Embeddings Association Test (WEAT).
result Demonstrates reduction or complete elimination of bias in word embeddings.
Many modern Artificial Intelligence (AI) systems make use of data embeddings, particularly in the domain of Natural Language Processing (NLP). These embeddings are learnt from data that has been gathered "from the wild" and have been found to contain unwanted biases. In this paper we make three contributions towards me…
Study reveals bias in machine learning conference reviews.
problem Bias in machine learning conference review process.
method Comprehensive analysis of ICLR papers from 2017-2020.
result Strong institutional bias in accept/reject decisions.
Machine learning algorithms are optimized to model statistical properties of the training data. If the input data reflects stereotypes and biases of the broader society, then the output of the learning algorithm also captures these stereotypes. In this paper, we initiate the study of gender stereotypes in {\em word emb…
New technique reduces gender discrimination in credit lending models.
problem Bias and unfairness in credit lending predictions.
method Subgroup Threshold Optimizer (STO) technique.
result Reduces gender discrimination by over 90%.
fairadapt uses causal inference to mitigate algorithmic bias in data pre-processing.
problem Mitigating algorithmic bias in machine learning predictions.
method Causal graphical model and observed data to address counterfactual questions.
result The method can help eliminate discrimination and justify fair decisions.
Language is increasingly being used to define rich visual recognition problems with supporting image collections sourced from the web. Structured prediction models are used in these tasks to take advantage of correlations between co-occurring labels and visual input but risk inadvertently encoding social biases found i…
Study shows fairness metrics are unreliable for small datasets in NLP tasks.
problem Unreliable fairness metrics for small datasets in NLP tasks.
method Experiments on Bios dataset with varying model sizes.
result Common fairness indices provide unreliable results for small samples.
Pronoun resolution is part of coreference resolution, the task of pairing an expression to its referring entity. This is an important task for natural language understanding and a necessary component of machine translation systems, chat bots and assistants. Neural machine learning systems perform far from ideally in th…
Develops tools to audit ML models for bias and unfairness.
problem Auditing ML models for individual bias and unfairness.
method Formalizes the task as an optimization problem and develops inferential tools for the optimal value.
result Demonstrates the utility of tools in revealing biases in COMPAS recidivism prediction instrument.
Optimal transport strategy reduces gender bias in job recommendation systems.
problem Mitigating gender biases in AI-driven job recommendation systems.
method Model agnostic optimal transport strategy applied to multi-class neural networks.
result Reduced undesirable algorithmic biases in job recommendation tasks.
There is a growing body of work that proposes methods for mitigating bias in machine learning systems. These methods typically rely on access to protected attributes such as race, gender, or age. However, this raises two significant challenges: (1) protected attributes may not be available or it may not be legal to use…
Foundation models improve wage gap decomposition by capturing omitted career history factors.
problem Estimating wage disparities using incomplete career history data.
method Fine-tuning foundation models to mitigate omitted variable bias and estimate wage gaps.
result Foundation models can decompose gender wage gaps more accurately than traditional econometric methods.
Machine learning outperforms crowd investors in predicting loan defaults and investment returns.
problem Determining if machine learning can outperform human decision-making in crowd lending.
method Using data from Prosper.com, a sophisticated ML algorithm was trained to predict loan defaults and investment returns.
result The ML algorithm outperforms crowd investors in predicting loan defaults and investment returns, especially for risky loans.
Method estimates dataset utility via minimal program length proxy.
problem Determining if data labels are generated by useful subroutines.
method Rissanen Data Analysis (RDA) estimates minimum description length (MDL) as a proxy.
result Method reveals dataset characteristics and utility in various NLP settings.
Federated learning can propagate bias from a few parties to all participants.
problem Bias from a few parties in federated learning can spread to all participants.
method Analysis of naturally partitioned real-world datasets.
result Bias in federated learning is higher than in centralized training.
New method detects bias in AI models that generate data.
problem Detecting bias in AI models that generate data.
method Formalized causal fairness in generative AI, derived new decomposition results, established identification conditions, and introduced efficient estimators.
result Demonstrated the value of new methodology in analyzing bias in large language models.
This paper debiases KG embeddings by filtering sensitive attributes.
problem Social and cultural biases in KG representations.
method Exploring and filtering popularity and gender biases in KGE.
result Sensitivity of KG embeddings to sensitive attributes like gender.
Real-world datasets are often biased with respect to key demographic factors such as race and gender. Due to the latent nature of the underlying factors, detecting and mitigating bias is especially challenging for unsupervised machine learning. We present a weakly supervised algorithm for overcoming dataset bias for de…
Stochastic encoding improves gender classification of brain networks from UK Biobank data.
problem Complexity and bias in interpreting deep learning models of brain connectivity.
method Stochastic encoding in ensemble of CNNs, multivariate balancing algorithm.
result AUROC of 0.8459, with resting-state data more accurate than task data.
The power of machine learning systems not only promises great technical progress, but risks societal harm. As a recent example, researchers have shown that popular word embedding algorithms exhibit stereotypical biases, such as gender bias. The widespread use of these algorithms in machine learning systems, from automa…
The study examines how social biases are reinforced in machine learning models used for credit scoring.
problem Reinforcement of societal biases in machine learning algorithms for credit scoring.
method Analysis of machine learning models predicting gender or ethnicity based on loan applications data.
result Machine learning models can reflect and reinforce social biases present in the data.
Proposes a method to enforce fairness in machine learning models without sensitive data.
problem Bias in machine learning models from historical data.
method Infers sensitive attributes from auxiliary features and integrates fairness constraints into model training.
result Mitigates bias while preserving predictive accuracy.
Securely trains fair models using homomorphic encryption.
problem Protecting sensitive features while testing model fairness.
method Fully homomorphic encryption for training and testing.
result Practical application to adult income data set.
Word embedding models have become a fundamental component in a wide range of Natural Language Processing (NLP) applications. However, embeddings trained on human-generated corpora have been demonstrated to inherit strong gender stereotypes that reflect social constructs. To address this concern, in this paper, we propo…
Unified framework for causal inference under sample selection.
problem Causal inference under sample selection with treatment and outcome non-randomness.
method ForestRiesz estimator, Riesz representation framework.
result ForestRiesz estimator yields more stable treatment effect estimates than conventional double machine learning approaches.
Intersectional constraints improve selection outcomes by reducing inequality.
problem Persistent inequality and reduced utility in selection processes due to implicit bias.
method Introducing intersectional constraints to mitigate the adverse effects of implicit bias in selection processes.
result Intersectional constraints can recover almost all the utility achievable in the absence of implicit bias, offering a significant advantage over non-intersectional constraints.
Computer Vision and machine learning methods were previously used to reveal screen presence of genders in TV and movies. In this work, using head pose, gender detection, and skin color estimation techniques, we demonstrate that the gender disparity in TV in a South Asian country such as Bangladesh exhibits unique chara…
Assessing the fairness of a decision making system with respect to a protected class, such as gender or race, is challenging when class membership labels are unavailable. Probabilistic models for predicting the protected class based on observable proxies, such as surname and geolocation for race, are sometimes used to …
Deep learning improves gender classification from handwriting.
problem Classifying gender from handwritten text.
method Convolutional Neural Network (CNN) for feature extraction and gender classification.
result Deep learning approach outperforms human examiners in gender classification accuracy.
New insights into bias mitigation show DRO isn't a complete solution.
problem Bias in machine learning systems across different data subsets.
method Theoretical analysis of Distributionally Robust Optimization (DRO) and data curation.
result Neither DRO nor data curation alone can fully address bias issues.
In this study, we proposed a convolutional neural network model for gender prediction using English Twitter text as input. Ensemble of proposed model achieved an accuracy at 0.8237 on gender prediction and compared favorably with the state-of-the-art performance in a recent author profiling task. We further leveraged t…