Study shows statistical biases can mislead transformer models, impairing their generalization.
problem Statistical biases in transformers affect their ability to generalize.
method Evaluated transformer models on synthetic algorithmic tasks with varying statistical biases.
result Statistical biases lead to overestimation of transformer models' generalization capabilities.
Biased sampling and missing data complicates statistical problems ranging from causal inference to reinforcement learning. We often correct for biased sampling of summary statistics with matching methods and importance weighting. In this paper, we study nearest neighbor matching (NNM), which makes estimates of populati…
Biased mean regression estimates factors exceeding expected loss or radiation release severity.
problem Estimating factors exceeding expected loss or radiation severity levels.
method Biased mean regression using superexpectation error minimization.
result Equivalent to quantile regression and CVaR optimization under specific conditions.
With the deluge of digitized information in the Big Data era, massive datasets are becoming increasingly available for learning predictive models. However, in many practical situations, the poor control of the data acquisition processes may naturally jeopardize the outputs of machine learning algorithms, and selection …
Deep learning's success is puzzling from a statistical perspective.
problem Deep learning's success is puzzling from a statistical perspective.
method Physics-informed investigation of deep learning features and surprises.
result Neural scaling laws and their interplay with inductive biases.
Improved estimator reduces bias in statistical learning models.
problem Asymptotic bias in classic WDRO estimator.
method Adjusted Wasserstein distributionally robust estimator.
result Asymptotic unbiased estimator with smaller MSE.
Paper detects biases in medical imaging ML models using counterfactual analysis.
problem Bias in medical imaging ML models negatively impacts generalization performance.
method Counterfactual invariance framework combining conditional latent diffusion models and statistical hypothesis testing.
result The method identifies and quantifies biases without direct access to counterfactual data.
We study high-dimensional Gaussian mixture classification using statistical physics methods.
problem Classifying high-dimensional Gaussian mixture with general covariance matrices.
method Replica method from statistical physics for asymptotic analysis of convex classifiers.
result Construction and validation of a de-biased estimator for variable selection.
Framework assesses autograders' reliability and biases.
problem Mixed reliability and biases in autograders for LLM evaluation.
method Bayesian GLMs to model evaluation outcomes.
result Explicit quantification of scoring differences and biases.
Estimates peeking effects in p-values to correct bias.
problem Data peeking biases reported p-values downward.
method Develops mechanisms to estimate running extrema of test statistics.
result Corrects bias in p-values due to peeking.
New method for analyzing compositional data, addressing biases in summary statistics.
problem Inadequate effect measures for compositional data, especially in high-dimensionality and sparsity.
method Perturbation-based effect measures, average perturbation effects.
result Proposed estimators efficiently estimate average perturbation effects, outperforming existing techniques.
The paper tackles individual fairness in ML models, developing statistical methods to detect bias.
problem Detecting and measuring violations of individual fairness in machine learning models.
method Formalizing the problem as adversarial attack, developing inference tools for the adversarial cost function.
result Statistical methods to assess and test hypotheses of model fairness with non-coverage error rate control.
Proposes a new fairness definition based on equity for machine learning classification.
problem Machine learning systems can perpetuate societal biases.
method Formalizes a new fairness definition based on equity, operationalizes it for classification, and evaluates its effectiveness.
result Demonstrates the effectiveness of the new fairness definition for equitable classification.
This work uses statistical bootstrapping to provide accurate confidence intervals for policy value in reinforcement learning.
problem Bias in estimating policy value using empirical transitions and rewards.
method Statistical bootstrapping to produce calibrated confidence intervals for the true policy value.
result Statistical bootstrapping can yield correct confidence intervals under certain conditions, and mechanisms are proposed to mitigate these conditions.
LLM evaluation suffers from systematic biases and lacks reliable positive judgments.
problem LLM evaluation suffers from systematic biases and lacks reliable positive judgments.
method Formulate LLM evaluation as a positive-unlabelled learning problem and propose a geometric auditing framework based on Partial Optimal Transport.
result Improved alignment with human preferences, increased robustness to presentation biases, and interpretable confidence estimates.
Proposes methods to learn from biased samples, ensuring robust decision rules.
problem Learning from biased samples can lead to poor performance in real-world applications.
method Modeling sampling bias, using distributionally robust optimization and deep learning.
result Proposes a method to minimize worst-case risk under various test distributions.
Sig-PCA integrates model outputs and observations to correct model biases.
problem Improving model accuracy and reliability by correcting biases and numerical approximations.
method Sig-PCA framework that combines summary statistics from model outputs with localized observations via a neural network.
result Corrects model outputs to align closely with observational data, preserving essential statistical information.
Neural nets learn simple distributions first, then more complex ones.
problem Understanding how neural networks generalize from simple to complex functions.
method Stochastic gradient descent training, synthetic data, CIFAR10, ImageNet pre-training.
result Neural networks initially use lower-order statistics, then higher-order ones.
The paper addresses bias in visual recognition models by reweighting observations.
problem Bias in deep neural networks trained on biased image databases.
method Reweighting observations based on known biasing mechanisms to form a nearly debiased estimator.
result The approach can remedy representativeness issues in visual recognition systems.
This primer tackles biases in machine learning for image analysis, proposing solutions.
problem Causal and statistical biases in machine learning methods for image analysis.
method Introduction of causal and statistical structures that induce failure, highlighting two problems: no fair lunch and subgroup separability.
result Current fair representation learning methods fail to solve these problems, suggesting new paths forward.
New algorithm finds unbiased subnetworks in biased datasets.
problem Finding unbiased subnetworks in biased neural networks.
method Debiased Contrastive Weight Pruning (DCWP) algorithm.
result DCWP significantly outperforms state-of-the-art debiasing methods.
A test detects unfairness in machine learning classifiers.
problem Detecting and mitigating algorithmic biases in machine learning.
method Optimal transport theory to quantify and mitigate bias.
result Proposes a statistical test for detecting unfair classifiers.
Survival trees exhibit end-cut preference, leading to biased splits.
problem End-cut preference in survival trees causes biased splits and poor interpretability.
method Proposed a smooth sigmoid surrogate (SSS) approach to replace hard-threshold indicator function.
result Smooth sigmoid surrogate (SSS) effectively mitigates end-cut preference in survival trees.
LLMs struggle to generate random numbers from statistical distributions, leading to biased results in applications.
problem LLMs' inability to generate random numbers accurately from specified distributions.
method Dual-protocol design: Batch Generation and Independent Requests, benchmarking 11 models across 15 distributions.
result Sampling fidelity degrades with distributional complexity and horizon, leading to systematic biases in downstream applications.
Study contextual online pricing with biased offline data, achieving optimal regret bounds.
problem Contextual online pricing with biased offline data.
method Identify δ2 to measure data bias, use OFU policy and robust variant for unknown bias. result Achieve minimax-optimal regret bounds for contextual pricing.
Study aims to measure and mitigate biases in motor insurance pricing.
problem Ethical biases in motor insurance pricing that affect fairness and regulatory compliance.
method Statistical methodologies and data analysis to measure and mitigate biases.
result Developed tools to measure and mitigate ethical biases in motor insurance pricing.
The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family of estimates based on the order statistics over the path ensemble, which allows one to flexibly drive the learning process, towards or agains…
The paper introduces Relative Bias to quantify LLM bias systematically.
problem Quantifying bias in LLMs is challenging due to ambiguity and rapid model emergence.
method Relative Bias framework using Embedding Transformation and LLM-as-a-Judge methodologies.
result The two scoring methods show strong alignment, providing a systematic approach.
Fair GLASSO estimates fair GGMs by balancing statistical dependencies across groups.
problem Fairness in graphical models with biased data.
method Regularized graphical lasso with bias metrics, proximal gradient algorithm.
result Preserves statistical accuracy while promoting fairness across groups.
Machine learning algorithms are now frequently used in sensitive contexts that substantially affect the course of human lives, such as credit lending or criminal justice. This is driven by the idea that `objective' machines base their decisions solely on facts and remain unaffected by human cognitive biases, discrimina…
New methods for CI testing under model misspecification.
problem Challenges in CI testing with misspecified models.
method Proposes new approximations and upper bounds for testing errors of regression-based CI tests.
result Introduces the Rao-Blackwellized Predictor Test (RBPT) robust against misspecified inductive biases.
New method uses imperfect LLM annotations for valid statistical inference in social science.
problem Inaccurate large language model annotations in social science research.
method Design-based supervised learning (DSL) combining imperfect LLM surrogates with gold-standard labels.
result DSL provides valid statistical inference with comparable predictive accuracy to existing methods.
Develops methods for statistical inference in high-dimensional linear mixed models.
problem Statistical inference for high-dimensional linear mixed models with large fixed effects and small random effects.
method Inspired by de-biasing penalized estimators, corrects a `naive' ridge estimator to build asymptotically valid confidence intervals.
result Demonstrates that the proposed method outperforms those that ignore correlation induced by random effects.
The observation of power laws in the time to extrema of volatility, volume and intertrade times, from milliseconds to years, are shown to result straightforwardly from the selection of biased statistical subsets of realizations in otherwise featureless processes such as random walks. The bias stems from the selection o…
Resampling outperforms reweighting for correcting biased data in machine learning models.
problem Correcting sampling bias in machine learning models trained on biased data sets.
method Compared resampling and reweighting techniques, focusing on their performance with stochastic gradient algorithms.
result Resampling outperforms reweighting when combined with stochastic gradient algorithms.
Models for which the likelihood function can be evaluated only up to a parameter-dependent unknown normalising constant, such as Markov random field models, are used widely in computer science, statistical physics, spatial statistics, and network analysis. However, Bayesian analysis of these models using standard Monte…
Fairmetrics evaluates fairness in ML models for specific groups.
problem Ensuring models do not produce biased outcomes for specific groups.
method User-friendly R package for evaluating group-based fairness criteria.
result Rigorous evaluation of multiple fairness metrics.
Post-ADC inference corrects bias in statistical inference after active data collection.
problem Bias in inference after active data collection.
method Post-ADC inference framework that corrects bias from both ADC process and data-driven target construction.
result Valid inference for data collected by SMBO methods like GP-UCB and TPE.
New online method for statistical inference with matrix context in decision-making.
problem Statistical inference in decision-making with matrix context.
method Proposes a fully online procedure to conduct statistical inference with adaptive data collection, handling low-rank structure.
result Establishes asymptotic normality of debiased estimators and proves validity of confidence intervals.
This paper corrects climate model biases using a factor model approach.
problem Systematic biases in GCM outputs due to unobserved confounders.
method Factor model approach to learn latent confounders from historical data and apply them to enhance bias correction.
result Significant improvements in the accuracy of precipitation outputs.
Develops methods to correct bias in AI feedback for more accurate alignment.
problem Systematic bias in AI feedback compared to human labels.
method Two debiased alignment methods: DDPO and DIPO.
result Methods improve alignment efficiency and performance close to human-labeled data.
Machine fairness is impossible to achieve fully due to historical biases.
problem Machine learning models inherit biases from historical data, making it impossible to satisfy fairness metrics simultaneously.
method Presented a causal perspective to the impossibility theorem of fairness.
result It is impossible to satisfy fairness metrics like demographic parity, equal opportunity, and equalized odds simultaneously.
New model corrects bias in crowdsourced ratings for diverse items.
problem Bias and noise in crowdsourced ratings for training data.
method Bayesian rating model with item-level effects for difficulty, discriminativeness, and guessability.
result New model avoids bias in training data, improving model goodness of fit.
Study finds no significant difference in neural network weights with quantum random numbers.
problem Effects of biased quantum random numbers on neural network initialization.
method Empirical study using quantum hardware and classical pseudo-random numbers.
result No statistically significant difference found between quantum random numbers and other types.
Study measures gender bias in machine translation using multiple reference points.
problem Measuring and identifying gender bias in machine translation.
method Used an optimal non-biased translator, reference points from occupational statistics and survey.
result Found bias against both genders, but more against women, and found occupations have a greater effect than adjectives.
Study examines biases in clinical word embeddings, revealing performance gaps across groups.
problem Biases in clinical word embeddings leading to performance differences across groups.
method Pretrained BERT models on MIMIC-III, fill-in-the-blank method, fairness evaluation on clinical tasks.
result Classifiers trained from BERT representations exhibit statistically significant differences in performance across groups.
Transform non-private e-values into differentially private ones.
problem Leaking sensitive data through non-private e-values.
method Developed a novel biased multiplicative noise mechanism.
result Differentially private e-values maintain strong statistical power and asymptotic equivalence to non-private ones.
We consider the occurrence of record-breaking events in random walks with asymmetric jump distributions. The statistics of records in symmetric random walks was previously analyzed by Majumdar and Ziff and is well understood. Unlike the case of symmetric jump distributions, in the asymmetric case the statistics of reco…