The paper develops a theory explaining how machine learning models can amplify biases.
problem Understanding and mitigating bias in machine learning models.
method Analytical theory of ridge regression with and without random projections.
result Observations and predictions align with empirical data on machine learning bias.
Modern neural networks show no bias-variance tradeoff with increased parameters.
problem The traditional bias-variance tradeoff does not hold in over-parameterized neural networks.
method Empirical measurements and theoretical analysis of bias and variance in modern neural networks.
result Bias and variance can decrease as the number of parameters grows in over-parameterized neural networks.
Mitigates spurious correlations without bias labels.
problem Spurious correlations bias model performance.
method Introduces a novel training objective and debiasing method DPR.
result DPR achieves state-of-the-art performance.
Study reveals biases and generalization patterns in deep image models.
problem Understanding the inductive bias of deep generative models in high dimensions.
method Proposed a framework to empirically investigate bias and generalization using designed training datasets.
result Identified similarities to human psychology in model behavior and patterns.
We analyze bias in post-bandit inference for stable index algorithms.
problem Bias in post-bandit inference for stable index algorithms.
method Empirical fluid approximation of sampling dynamics.
result Sharp leading-order expressions for bias and expected Z-statistic.
We rebias estimates to improve interval calibration and prediction accuracy.
problem Constructing accurate intervals for noisy and biased estimates.
method Empirical Bayes rebiasing strategy that learns bias distribution from data.
result Substantial precision gains in prediction-powered inference.
Machine can learn its own bias from related tasks.
problem Machine learning bias through hand-crafted features.
method Introduces two models: PAC-based and hierarchical Bayes.
result Machine can learn bias from multiple tasks.
Raising statistical hurdles may not be justified due to data bias.
problem Data bias leads to unobserved results that weaken identification of revised hurdles.
method Theoretical and empirical analysis of statistical hurdles and data bias.
result Statistics targeting only published findings can be strongly identified.
Ensembles improve classifier performance by reducing bias, not variance.
problem Improving classifier performance through ensemble methods.
method Extended bias-variance decomposition for classification tasks, introducing dual reparameterization.
result Ensembling reduces bias in classifiers, contrary to the traditional view.
SGD and weight decay encourage neural networks to learn low-rank weight matrices.
problem The bias of SGD towards low-rank weight matrices in neural networks.
method The study investigates the effect of SGD and weight decay on the rank of weight matrices in neural networks, both theoretically and empirically.
result Training with SGD and weight decay induces a bias towards rank minimization in weight matrices, which becomes more pronounced with smaller batch sizes and stronger weight decay.
Theory and methods to mitigate omitted variable bias in causal machine learning.
problem Mitigating omitted variable bias in causal machine learning models.
method Developed a general theory and flexible statistical inference methods for bounding and testing the magnitude of omitted variable bias.
result Simple plausibility judgments can bound the magnitude of omitted variable bias in complex, nonlinear models.
Publication bias skews asset pricing research findings.
problem Bias in sharing and publishing research findings.
method Meta-studies and empirical Bayes corrections.
result Publication bias effects are minimal and not dominant.
EBQL reduces bias in Q-learning for improved performance.
problem Over- and under-estimation biases in Q-learning degrade performance.
method Ensemble Bootstrapping to reduce both over- and under-estimation biases.
result EBQL outperforms other Q-learning methods in Atari games.
Algorithm corrects bias in classification data.
problem Underrepresentation and intersectional bias in classification data.
method Estimate group-wise drop-out rates with small unbiased data, construct reweighting scheme, and present algorithm.
result Efficiently approximate loss of any hypothesis on true distribution.
Corrects sample selection bias in empirical risk minimization using importance sampling.
problem Statistical learning with biased training data.
method Weighted empirical risk minimization using importance sampling.
result Generalization capacity preserved with estimated importance weights.
WeMix improves data augmentation by correcting bias in deep learning.
problem Data augmentation's effectiveness is limited by data bias.
method Developed AugDrop and MixLoss algorithms to correct data bias.
result WeMix improves data augmentation performance through bias correction.
The paper analyzes the tradeoff between bias and overfitting in reinforcement learning with partial observability.
problem Analyzing the tradeoff between asymptotic bias and overfitting in reinforcement learning with partial observability.
method Theoretical analysis and empirical illustration using truncated history of observations and function approximators.
result A smaller state representation decreases the risk of overfitting, but potentially increases asymptotic bias.
The paper evaluates biased methods for alpha-divergence minimization.
problem The impact of bias on solutions found for alpha-divergence minimization.
method Empirical evaluation of biased methods for alpha-divergence minimization, focusing on bias effects and dimensionality.
result Solutions are biased towards KL-divergence minimizers and require impractical computation in high dimensions to minimize alpha-divergence.
Active learning introduces bias; this paper fixes it.
problem Bias in active learning due to non-representative training data.
method Formalized bias, identified situations where it's harmful/helpful, introduced corrective weights.
result Corrective weights can improve active learning, especially with overparameterized models.
New estimator reduces bias and variance issues in mutual information estimation.
problem Difficulty in using variational MI estimators due to bias/variance tradeoffs and self-consistency issues.
method Developed a new estimator based on a unified perspective of variational approaches, focusing on variance reduction.
result Empirical results show improved bias-variance trade-offs compared to existing estimators.
Study shows how bias in optimization affects robustness in adversarial settings.
problem Understanding and mitigating implicit bias in adversarially robust models.
method Analyzes the implicit bias in robust empirical risk minimization and its impact on generalization.
result Implicit bias in optimization can significantly affect robust generalization.
New method corrects bias in estimating entropic risk for better decision-making.
problem Underestimation of entropic risk when data are limited.
method Parametric bootstrap procedure to overestimate entropic risk.
result Corrected method provides better risk estimates, leading to improved decision-making.
Differential privacy reduces bias in adaptive data gathering.
problem Bias in adaptive data gathering, including numeric and complex data.
method Apply differential privacy to reduce bias and correct p-values.
result Near optimal regret bounds for differentially private bandit algorithms.
The paper improves machine learning regression methods by reducing bias.
problem Reducing bias in machine learning regression methods.
method Residual bootstrap for bias correction in ensembles of trees.
result The proposed bootstrap bias correction leads to up to 70% improvement in test-set accuracy.
Study improves confidence measures in medical imaging pipelines by addressing bias.
problem Bias in metric-based imaging pipelines compromises the efficiency of prediction intervals.
method Formalized symmetric and asymmetric CP formulations, analyzed bias effects, and validated empirically.
result Symmetric intervals are inflated by bias, while asymmetric intervals remain unaffected.
New method improves model risk prediction using cross-audit projection.
problem Over-optimism in K-fold CV for binary classification. method Cross-audit projection (CAP) procedure combining resampling and asymptotic bias correction.
result CAP estimator achieves second-order asymptotic unbiasedness.
SGD noise has no bias advantage in online learning, contrary to offline learning.
problem The role of SGD noise in online learning.
method Extensive empirical analysis of image and language data.
result Small batch sizes do not confer any implicit bias advantages in online learning.
ARFF reduces spectral bias in SGD-trained neural networks.
problem Spectral bias in two-layer neural networks.
method Comparison of SGD and ARFF on spectral bias and robustness.
result ARFF yields a closer to zero spectral bias compared to SGD.
Parameters defined via General Estimating Equations (GEE) can be estimated by maximizing the Empirical Likelihood (EL). Newey and Smith (2004) have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its O(n^-1) bias is small and that bias-corrected EL is higher-ord…
Parameters defined via general estimating equations (GEE) can be estimated by maximizing the empirical likelihood (EL). Newey and Smith [Econometrica 72 (2004) 219--255] have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its O(n−1) bias is small and that …
Detect changes in noisy dynamical systems using empirical approximations and finite-sample bounds.
problem Change detection in noisy dynamical systems
method Partition-based empirical approximations and finite-state stationary distribution stability
result Finite-sample bound for empirical stationary density
The paper finds a pervasive and severe bias in accounting semi-identity models.
problem Bias in investment-cash flow sensitivity models.
method Augmented specification with a bias-capturing variable tested across multiple databases.
result The Accounting Semi-Identity (ASI) distortion is universal and severe, affecting 100% of databases and explaining more than 83% of total explained variance.
Paper proposes synthetic data generator to study and mitigate bias in machine learning.
problem Bias in machine learning data can lead to unfair outcomes.
method Developed a synthetic data generator to introduce and analyze various types of bias.
result Demonstrated how synthetic data can be used to study and mitigate bias in machine learning models.
Weight Decay induces low-rank weight matrices in neural networks, improving generalization.
problem Improving generalization in neural networks.
method Training ReLU NN with Weight Decay and Stochastic Gradient Descent.
result The weight matrix of a trained NN is approximately rank-two.
New methods reduce bias in estimating calibration error.
problem Reducing bias in estimating calibration error.
method Synthesizing model outputs and using equal-mass bins.
result Two reliable calibration-error estimators found: debiased estimator and ECE_sweep.
EB improves asset pricing by mining large strategies without lookahead bias.
problem Lack of unbiased asset pricing models with out-of-sample performance.
method Empirical Bayes applied to 136,000 long-short strategies.
result EB provides unbiased predictions with transparent intuition.
Machine learning forecasts show bias at long horizons, contrary to standard tests.
problem Forecast efficiency tests misinterpret machine learning performance.
method Theoretical and empirical analysis of regularization and measurement noise.
result Machine learning forecasts exhibit overreaction at longer horizons, not bias.
Deep learning models show bias and variance are aligned, not in trade-off.
problem The classical bias-variance trade-off in deep learning models.
method Empirical evidence and theoretical analysis of bias and variance in deep learning models.
result Squared bias is approximately equal to variance for correctly classified sample points in deep learning models.
Entropy asymmetry affects regularization in ERM, leading to biased solutions.
problem Analyzing the impact of relative entropy asymmetry in ERM regularization.
method Examined Type-I and Type-II ERM-RER, comparing their solutions and properties.
result Type-II ERM-RER regularization introduces a strong bias against training data.
Improved optimization methods for discrete distributions reduce bias in gradient estimation.
problem Estimating gradients for discrete distribution parameters is challenging.
method Analyzed and proposed methods to reduce bias in gradient estimation, including Gumbel-Softmax and piece-wise linear continuous relaxation.
result Reduced bias leads to better performance in variational inference and binary optimization tasks.
SSMs have a built-in bias towards low-frequency components, which can be adjusted.
problem Frequency bias in SSMs affects their performance on long-range sequences.
method Proposed two mechanisms to tune frequency bias: scaling initialization or applying a Sobolev-norm-based filter.
result Tuning frequency bias improves SSMs' performance on long-range sequence learning tasks.
Improves cross-validation for biased data by adjusting risk estimator variance.
problem Cross-validation under sample selection bias produces suboptimal results.
method Introduces control variate to reduce variance of importance-weighted risk estimator.
result Control variate increases robustness to problematic weights.
An empirical investigation of the interaction of sample size and discretization - in this case the entropy-based method CAIM (Class-Attribute Interdependence Maximization) - was undertaken to evaluate the impact and potential bias introduced into data mining performance metrics due to variation in sample size as it imp…
Reduces gender bias in patient notes while maintaining medical classification accuracy.
problem Bias in natural language processing of patient notes.
method Identifying and removing gendered language using BERT-based classifiers, then augmenting data to maintain performance.
result Minimal degradation in health condition classification tasks with data augmentation.
Proposes a three-stage debiasing framework to improve out-of-distribution accuracy.
problem Inaccurate uncertainty estimations in bias-only models damage ensemble-based debiasing performance.
method Calibrates the bias-only model to improve its uncertainty estimations, creating a three-stage ensemble-based debiasing framework.
result The three-stage debiasing framework consistently outperforms traditional methods in out-of-distribution accuracy.
The paper introduces a new bias measure, infra-marginality, to quantify unfairness in group fairness.
problem The trade-off between group fairness and individual-level bias in decision-making.
method Proposes a new notion of η-infra-marginality, proves its independence from accuracy, and provides practical methods to measure and avoid it. result High accuracy does not lead to high infra-marginality, but maximizing group fairness often increases infra-marginality.
New bias found in cluster validity indices when ground truth distribution changes.
problem Bias in external cluster validity indices when ground truth distribution changes.
method Identified new type of bias (GT bias) and studied its empirical and theoretical implications.
result GT bias can change the bias status of cluster validity indices.
Bayesian adaptive designs can be biased by active learning, especially with misspecified models.
problem Active learning bias in Bayesian adaptive experimental designs.
method Analysis of linear and preference learning models, empirical testing.
result Model misspecification and noise influence active learning bias in Bayesian designs.