Interview study reveals considerations for designing semi-automated bias detection tools.
problem Detecting and mitigating machine learning biases.
method Interview study with 11 machine learning practitioners.
result Four considerations identified for tool design.
The paper detects and identifies bias in data using a counterfactual approach.
problem Detecting and identifying bias in data, especially in medical image classification.
method A global explanation framework using the counterfactual approach to identify bias causing artifacts.
result Black frames significantly influence Convolutional Neural Network's prediction, changing benign to malignant.
Data-driven algorithms play a large role in decision making across a variety of industries. Increasingly, these algorithms are being used to make decisions that have significant ramifications for people's social and economic well-being, e.g. in sentencing, loan approval, and policing. Amid the proliferation of such sys…
fairmodels tool detects and mitigates bias in machine learning models.
problem Bias in machine learning models leading to discrimination.
method Model-agnostic approach to bias detection, visualization, and mitigation.
result Validates fairness and eliminates bias in classification models.
UBM transfers bias mitigation from upstream to downstream tasks efficiently.
problem Bias in fine-tuned language models across various tasks.
method Apply bias mitigation to an upstream model, then fine-tune a downstream model on this mitigated model.
result UBM effects transfer to new downstream tasks, creating less biased models.
SLUG method detects bias and out-of-distribution content in generative models.
problem Generative models can underrepresent certain groups and fail on out-of-distribution data.
method SLUG: A new uncertainty quantification method for VAEs combining Laplace approximations and stochastic trace estimators.
result SLUG's UQ score correlates with bias and out-of-distribution content.
A new watermarking method corrects bias in language models using maximal coupling.
problem Correcting bias in language model token distributions.
method Maximal coupling to balance bias correction and text quality.
result Outperforms prior techniques in preserving text quality and detectability.
Study uncovers bias in image classification models using attribution maps.
problem Data bias in image classification models.
method Created an artificial dataset with known bias, trained CNNs, and used attribution maps to inspect decisions.
result Different attribution map techniques highlight bias better than others, and metrics support bias identification.
Paper detects bias in AI medical models using CART.
problem Ensuring fairness in AI medical decision support systems.
method Uses Classification and Regression Trees (CART) algorithm to identify bias.
result Validated the CART approach in both synthetic and real-world data.
We detect lookahead bias in LLM forecasts using a novel statistical method.
problem Detecting lookahead bias in LLM-generated economic forecasts.
method Developed a statistical procedure using date-only recall queries and estimated Lookahead Propensity (LAP).
result LLM forecasts are contaminated with lookahead bias, as indicated by a positive interaction between LAP and the forecast in accuracy regressions.
CNNs can develop blind spots due to uneven padding in feature maps.
problem Spatial bias in convolutional networks leads to blind spots in certain tasks.
method Identified and analyzed the role of padding in convolutional networks, proposing solutions to mitigate bias.
result Mitigating spatial bias improves model accuracy, especially in tasks like small object detection.
New method detects and mitigates historical bias in data.
problem Detecting and explaining historical bias in data.
method Developed a sample bias criterion and algorithms to measure and counter sample bias.
result Derived bias score provides sample-level attribution and explanation of historical bias.
Ensemble methods for classification and clustering have been effectively used for decades, while ensemble learning for outlier detection has only been studied recently. In this work, we design a new ensemble approach for outlier detection in multi-dimensional point data, which provides improved accuracy by reducing err…
We present a novel subset scan method to detect if a probabilistic binary classifier has statistically significant bias -- over or under predicting the risk -- for some subgroup, and identify the characteristics of this subgroup. This form of model checking and goodness-of-fit test provides a way to interpretably detec…
A new metric MSD detects bias in datasets efficiently.
problem Detecting bias in AI systems and datasets.
method Introduced Maximum Subgroup Discrepancy (MSD) metric and a practical algorithm based on MIO.
result MSD provides a linear sample complexity for practical applications, distinguishing biases effectively.
Study detects and explains positional bias in financial LLMs.
problem Positional bias in financial decision-making using LLMs.
method Unified framework and benchmark for detecting and quantifying bias in Qwen2.5 models.
result Positional bias is pervasive, scale-sensitive, and resurfaces under nuanced prompt designs.
Study data biases to predict algorithmic discrimination, developing a Data Bias Profile.
problem Data biases contribute to algorithmic discrimination, but their impact is understudied.
method Analyzed three common data biases across various datasets and models, developing a Data Bias Profile.
result Combination of proxies and label bias can lead to more significant discrimination than underrepresentation alone.
Real-world datasets are often biased with respect to key demographic factors such as race and gender. Due to the latent nature of the underlying factors, detecting and mitigating bias is especially challenging for unsupervised machine learning. We present a weakly supervised algorithm for overcoming dataset bias for de…
Systematic review of ML models for detecting social media deception.
problem Detecting fake news, spam, and fake accounts on social media.
method 36 studies evaluated using PROBAST tool, identifying biases and limitations.
result Over-reliance on accuracy in imbalanced data settings is a flaw.
Study improves statistical power for detecting algorithmic bias in educational data.
problem Challenges in measuring algorithmic bias using ABROCA due to skewed distribution.
method Investigates ABROCA's distributional properties and proposes nonparametric randomization tests.
result ABROCA-based bias assessments are underpowered in typical EDM sample sizes.
Proposes a new algorithm to estimate invariant subspaces across multilayer networks.
problem Estimating invariant subspaces across heterogeneous multiple networks.
method Bias-corrected joint spectral embedding algorithm that recursively calibrates diagonal bias and iteratively updates the subspace estimator.
result Established entrywise subspace perturbation bound and entrywise eigenvector central limit theorem for the algorithm.
Detect changes in noisy dynamical systems using empirical approximations and finite-sample bounds.
problem Change detection in noisy dynamical systems
method Partition-based empirical approximations and finite-state stationary distribution stability
result Finite-sample bound for empirical stationary density
This work uncovers how model and data biases interact to cause unfairness in fraud detection.
problem Unfairness in fraud detection algorithms due to model and data biases.
method Taxonomy of data bias, hypotheses on fairness-accuracy trade-offs, real-world fraud use case study.
result Data bias affects fairness in expected value and variance, and simple pre-processing can balance group-wise error rates.
This work introduces a bias-variance decomposition for proper scores, improving uncertainty estimation in predictive models.
problem Reliable uncertainty estimation for predictions in safety-critical applications, especially under domain drift.
method Developed a general bias-variance decomposition for proper scores, introducing the Bregman Information as the variance term.
result The decomposition provides novel formulations for different predictive tasks, including classification and model ensembles.
The influence of human judgement is ubiquitous in datasets used across the analytics industry, yet humans are known to be sub-optimal decision makers prone to various biases. Analysing biased datasets then leads to biased outcomes of the analysis. Bias by protected characteristics (e.g. race) is of particular interest …
Edge language models show bias over time, especially on resource-constrained devices.
problem Bias in edge language models on resource-constrained devices.
method Comparative analysis of text-based bias across edge, cloud, and desktop environments; optimized Llama-2 model on Raspberry Pi 4; feedback loop mechanism to correct bias.
result Llama-2 on Raspberry Pi 4 shows 43.23% and 21.89% more bias over time compared to cloud and desktop models.
We identify and analyze selection structure in sequential data.
problem Selection biases in sequential data can distort analysis and hide underlying generation processes.
method Nonparametric identifiability of selection structure without interventional experiments.
result Selection structure is identifiable in sequential data without parametric assumptions.
CARD detects treatment responders with machine learning and adjustment.
problem Identifying responders in non-random treatment settings.
method Conformal prediction, machine learning, propensity score adjustment.
result High power responder detection in various scenarios.
Unified framework for intersectionally fair AI models using MIO.
problem Bias in AI models for high-risk domains.
method Mixed-Integer Optimization (MIO) for fairness and interpretability.
result Improved performance in detecting and mitigating bias at intersections.
CondMTL improves toxicity detection by learning group-specific representations.
problem Algorithmic bias in toxic language detection across demographic groups.
method Conditional Multi-Task Learning (CondMTL) for demographic-specific tasks.
result CondMTL improves predictive recall for minority demographic groups.
New method detects bias in AI models that generate data.
problem Detecting bias in AI models that generate data.
method Formalized causal fairness in generative AI, derived new decomposition results, established identification conditions, and introduced efficient estimators.
result Demonstrated the value of new methodology in analyzing bias in large language models.
Reduces false positives in classifying rare online platforms.
problem Challenges in accurately identifying rare online platforms with ML.
method Calibrated probabilities and ensembles to reduce bias.
result Significantly reduces false positives in rare event detection.
New method reduces gender bias in language models without harming performance.
problem Bias in language models learned from biased data.
method Causal analysis to identify problematic model components, followed by linear projection of weight matrices.
result DAMA significantly decreases bias in language models while maintaining performance.
Detect hidden confounding in observational data using multiple environments.
problem Detect hidden confounding in observational data.
method Theoretical framework and simulation studies to test for hidden confounding.
result The proposed procedure correctly predicts hidden confounding, especially when bias is large.
New method uncovers bias mechanisms in observational studies.
problem Understanding the sources of bias in observational studies.
method Analyzing the relationship between bias magnitude and nuisance function estimators' performance.
result Method can distinguish between common sources of causal bias.
We study the problem of detecting change points (CPs) that are characterized by a subset of dimensions in a multi-dimensional sequence. A method for detecting those CPs can be formulated as a two-stage method: one for selecting relevant dimensions, and another for selecting CPs. It has been difficult to properly contro…
The paper debiases machine learning predictions to correct bias in regression coefficients.
problem Bias in regression coefficients from machine learning predictions.
method Proposes an adversarial machine learning algorithm to de-bias predictions.
result Adversarial predictions recover true coefficients, while naive predictions are biased.
New method preserves privacy while detecting communities in distributed networks.
problem Privacy-preserving community detection in locally distributed multi-layer networks.
method Privacy-preserving Distributed Spectral Clustering (ppDSC) using randomized response mechanism.
result Developed a novel algorithm that maintains community structure while protecting privacy.
Machine learning algorithms can unintentionally discriminate; tools detect and fix this.
problem Unintentional discrimination in machine learning algorithms.
method Statistical tools to detect and eliminate biases.
result Tools can identify and mitigate algorithmic discrimination.
New concept of mixture complexity helps detect gradual clustering changes.
problem Determining the number of clusters in mixture models with overlaps and weight biases.
method Introducing mixture complexity (MC) as a new measure of cluster size, defined from information theory.
result MC can detect gradual clustering changes, allowing earlier detection and finer distinction.
Paper proposes robust method to detect risk heterogeneity across ethnic groups.
problem Detecting risk heterogeneity across ethnic groups in ICU studies.
method Proposes a robust framework using Neyman orthogonality for inference.
result Demonstrates improved inferential stability and reduced bias compared to standard methods.
New method improves OOD detection by integrating diffusion models into discriminator models.
problem Overconfidence in discriminator models leads to poor OOD detection.
method Integrates diffusion models into discriminator and generation models to mitigate overconfidence.
result Demonstrates significant improvement in AUROC scores for challenging datasets.
ABROCA assesses algorithmic bias, revealing skewed distributions that inflate results.
problem Detecting nuanced performance differences in classifier fairness.
method Study of ABROCA metric's statistical properties under various conditions.
result ABROCA distributions are skewed, inflating results by chance in imbalanced classes.
Accurate facial expression analysis is an essential step in various clinical applications that involve physical and mental health assessments of older adults (e.g. diagnosis of pain or depression). Although remarkable progress has been achieved toward developing robust facial landmark detection methods, state-of-the-ar…
5D AI model detects bad loans without biased features, improving consumer protection.
problem Detecting bad loans without biased features and improving consumer protection.
method Machine learning, BiMOPT features, European Banking Authority principles, AI principles, historical and validation datasets.
result 5D correctly detected 1,461 bad loans out of 1,613 (Sensitivity = 0.91, Prevalence = 0.0253, Positive Predictive Value = 0.19).
New methods improve anomaly detection in deep networks by leveraging hierarchical likelihoods and multi-scale features.
problem Challenges in detecting anomalies in high-level features due to model bias and domain prior.
method Two methods: 1) Log likelihood ratios between in-distribution and general distribution models, 2) Multi-scale likelihood contribution.
result Strong anomaly detection performance in unsupervised setting, slightly underperforming supervised methods.
This paper presents an innovative and generic deep learning approach to monitor heart conditions from ECG signals.We focus our attention on both the detection and classification of abnormal heartbeats, known as arrhythmia. We strongly insist on generalization throughout the construction of a deep-learning model that tu…
Methods for unsupervised anomaly detection suffer from the fact that the data is unlabeled, making it difficult to assess the optimality of detection algorithms. Ensemble learning has shown exceptional results in classification and clustering problems, but has not seen as much research in the context of outlier detecti…