The study examines methods to correct measurement error in nutritional epidemiology studies.
problem Measurement error in nutritional studies leads to biased and underconfident estimates.
method The article reviews various bias-correction models for exposure variables in nutritional epidemiology.
result Bias-correction methods are essential for accurate inference in nutritional studies.
Paper analyzes LSA algorithm bias and error bounds with RR extrapolation.
problem Analyzing bias and high-order error bounds of LSA with Markovian noise.
method Polyak-Ruppert averaging, linearization, Richardson-Romberg extrapolation.
result RR extrapolation effectively cancels the leading bias term.
Bias correction needed after deep learning regression training.
problem Systematic error accumulation in deep learning regression models.
method Adjust bias of the machine learning model post-training.
result Bias correction efficiently solves error accumulation.
Study shows exponential error reduction in multiclass classification without bias-variance trade-off.
problem Multiclass classification with margin conditions.
method Analysis of classification error under hard-margin conditions.
result Exponential decrease in classification error without bias-variance trade-off.
New insights into bias and variance in over-parameterized models.
problem Understanding bias and variance in over-parameterized models.
method Analytic expressions derived from statistical physics for two minimal models.
result Over-parameterized models can overfit even in noiseless conditions.
Paper analyzes ECE bias and provides bounds for its estimation.
problem Understanding the estimation bias in ECE for machine learning models.
method Information-theoretic approach to analyze bias in uniform mass and uniform width binning strategies.
result Established upper bounds on ECE estimation bias and optimal number of bins.
ADB framework improves OOD generalization by increasing ID bias during training.
problem Machine learning models degrade on new data distributions.
method ADB framework introduces controlled statistical diversity during training.
result Higher in-distribution bias leads to better out-of-distribution generalization.
The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve. However, recent empirical results with over-parameterized neural networks are marked by a striking absence of the classic U-shaped test error curve: test error keeps decrea…
New methods reduce bias in estimating calibration error.
problem Reducing bias in estimating calibration error.
method Synthesizing model outputs and using equal-mass bins.
result Two reliable calibration-error estimators found: debiased estimator and ECE_sweep.
Generalizes bias-variance decomposition for Bregman divergences.
problem No specific problem stated; generalization of bias-variance for Bregman divergences.
method Provided a generalization of the bias-variance decomposition for Bregman divergences.
result A clear, standalone derivation of the bias-variance decomposition for Bregman divergences.
Machine learning algorithms can misrepresent training data, study finds.
problem Misrepresentation of training data in machine learning algorithms.
method Demonstrated through underestimation of training data due to irreducible error, regularization, and class imbalance.
result Careful management of synthetic counterfactuals can mitigate underestimation bias.
Gradient descent learns ReLU functions with non-zero bias efficiently.
problem Learning ReLU functions with non-zero bias under Gaussian distributions.
method Gradient descent starting from random initialization.
result Gradient descent achieves near-optimal error with high probability.
Paper proposes methods to reduce bias and variance in recommender systems.
problem Bias in recommender systems due to users' preferences.
method Proposes a principled approach to reduce bias and variance in DR methods, and a novel semi-parametric collaborative learning approach.
result The proposed methods outperform existing debiasing methods in both theory and experiments.
We find the optimal error for a constrained regression model under a linear model.
problem Minimizing error while adhering to demographic parity constraints.
method Proposed a minimax optimal error analysis for a demographic parity-constrained regression problem within a linear model.
result The minimax optimal error is characterized by $Θ( rac{dM}{n})$ .
LatentNN corrects neural network attenuation bias in astronomical data.
problem Neural networks underestimate extreme values due to measurement errors.
method Jointly optimizes network parameters and latent input values.
result LatentNN reduces attenuation bias across various signal-to-noise ratios.
Bias - variance decomposition of the expected error defined for regression and classification problems is an important tool to study and compare different algorithms, to find the best areas for their application. Here the decomposition is introduced for the survival analysis problem. In our experiments, we study bias -…
ASGD outperforms SGD in overparameterized linear regression, especially in subspaces of small eigenvalues.
problem Generalization of ASGD for overparameterized linear regression.
method Established instance-dependent excess risk bound for ASGD in each eigen-subspace of the data covariance matrix.
result ASGD outperforms SGD in subspaces of small eigenvalues, exhibiting faster decay of bias error.
Paper unifies bias and variance models for classification.
problem Different frameworks for bias and variance in classification.
method Unified Tumer & Ghosh and James approaches.
result Closed form relationships between 0/1 loss and squared error loss.
New method reduces bias in estimating causal effects from discretized variables.
problem Bias in estimating causal effects from discretized continuous variables.
method Proposes a bias-reduced functional that evaluates outcome regression at within-bin conditional means.
result Demonstrates substantial bias reduction and near-nominal confidence interval coverage.
Deep neural networks can generalize by reducing high-frequency noise over time, not always following a monotonic learning bias.
problem Understanding the learning dynamics and generalization of over-parameterized DNNs.
method Experimental analysis of deep double descent, focusing on the spectral bias of DNNs.
result The high-frequency components of DNNs diminish over training, leading to a second descent in test error.
New EiV models correct bias in operator learning with noisy data.
problem Bias in operator learning due to noisy independent variables.
method Developed EiV models for MOR-Physics and DeepONet.
result EiV models reduce bias in noisy operator learning.
We consider the off-policy evaluation problem in Markov decision processes with function approximation. We propose a generalization of the recently introduced \emph{emphatic temporal differences} (ETD) algorithm \citep{SuttonMW15}, which encompasses the original ETD( λ λ λ ), as well as several other off-policy evaluation …
Analyzes bias-variance in overparameterized linear models using random features.
problem Understanding bias-variance trade-off in overparameterized models.
method Zero-temperature cavity method and random matrix theory.
result Three phase transitions in the linear random features model.
New theory shows how learning algorithms can create a bias towards negative outcomes.
problem Negativity bias in adaptive learning algorithms.
method Generalization of the Hot Stove Effect to settings with negative estimates leading to smaller sample sizes.
result Negativity bias persists even when negative estimates do not lead to avoidance.
Corporate bond factor research is flawed due to measurement errors and ex-post filtering.
problem Replication crisis in corporate bond factor research.
method Analysis of 108 signals across nine thematic clusters, correction of transaction prices and return filtering.
result Majority of previously documented factors do not produce statistically significant alphas after correction.
Depth uncertainty networks don't improve with bias correction, contrary to expectations.
problem Improving performance in active learning with overparameterised models like NNs.
method Depth uncertainty networks, compared to underparameterised models, show no improvement in performance with bias correction.
result Depth uncertainty networks do not improve with bias correction, unlike underparameterised models.
This paper analyzes errors in Shapley value-based model explanations.
problem Biased or unreliable explanations from existing SVA methods.
method Error theoretical analysis framework decomposing errors into observation and structural biases.
result Trade-off between observation and structural biases in SVA explanations.
Deep networks generalize well even when they fit training data perfectly, thanks to overparametrization.
problem Understanding generalization in overparametrized deep networks.
method Random features regression, asymptotic analysis, ensemble averaging.
result Bias remains constant beyond the interpolation threshold, while variance components decay with overparametrization.
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
A number of applications (e.g., AI bot tournaments, sports, peer grading, crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL) model to evaluate a given collection of items (e.g., bots, teams, students, search results). Past work has shown that under the BTL model, the widely-used maximum-likeli…
New insights into bias mitigation show DRO isn't a complete solution.
problem Bias in machine learning systems across different data subsets.
method Theoretical analysis of Distributionally Robust Optimization (DRO) and data curation.
result Neither DRO nor data curation alone can fully address bias issues.
Paper proposes a new method to separate low rank and sparse matrices without bias.
problem Recovering low rank and sparse matrices from measurements.
method Uses nonconvex regularizers and alternating proximal gradient descent.
result Error bounds for the algorithm applied to sparse optimization, matrix completion, and robust PCA.
We use tools from geometric statistics to analyze the usual estimation procedure of a template shape. This applies to shapes from landmarks, curves, surfaces, images etc. We demonstrate the asymptotic bias of the template shape estimation using the stratified geometry of the shape space. We give a Taylor expansion of t…
This paper analyzes error bounds for biased SMC samplers in conditional sampling.
problem Analyzing error bounds for biased SMC samplers in conditional sampling.
method Develops a non-asymptotic error analysis for SMC samplers with biased mutation kernels.
result Derives the first non-asymptotic error bound for conditional sampling with score-based diffusion models.
Develops tools to audit ML models for bias and unfairness.
problem Auditing ML models for individual bias and unfairness.
method Formalizes the task as an optimization problem and develops inferential tools for the optimal value.
result Demonstrates the utility of tools in revealing biases in COMPAS recidivism prediction instrument.
New characterization limits sampling with inexact scores.
problem Limiting sampling with inexact scores for unbiased results.
method Characterized types of inexact score oracle access.
result Weaker error assumptions rule out tractability of unbiased sampling.
This lecture presents recent advances in the theory of errors propagation. We first explain in which cases the propagation of errors may be performed with a first order differential calculus or needs a second order differential calculus. Then we point out the link between error propagation and the concept of second ord…
Extended Gauss-Markov theorem for linear estimation with bounded bias.
problem Linear estimation with bounded bias operator.
method Derive optimal estimator formulas for Nuclear and Spectral norms, analyze generalization error.
result Cross-validated Nuclear and Spectral regressors outperform Ridge regression in simulations.
Overparameterized models can worsen minority group errors even when overall test error improves.
problem Overparameterization exacerbates spurious correlations, harming minority groups.
method Simulations and experiments on image datasets, theoretical analysis of linear models.
result Subsampling the majority group can achieve low minority error in overparameterized models.
We analyze Gibbs-based transfer learning algorithms using information theory.
problem Understanding the generalization error of transfer learning.
method Information-theoretic analysis focusing on α α α -weighted-ERM and two-stage-ERM. result Exact characterization of generalization behavior using conditional symmetrized KL information.
CB-SLICE identifies concept-based error slices in deep learning models.
problem Systematic errors in deep learning models on specific groups.
method Concept Bottleneck Models (CBMs) and concept representations.
result CB-SLICE outperforms state-of-the-art methods in error slice identification.
In this paper, we study the accuracy of values aggregated over classes predicted by a classification algorithm. The problem is that the resulting aggregates (e.g., sums of a variable) are known to be biased. The bias can be large even for highly accurate classification algorithms, in particular when dealing with class-…
The accuracy of deep neural networks is significantly affected by how well mini-batches are constructed during the training step. In this paper, we propose a novel adaptive batch selection algorithm called Recency Bias that exploits the uncertain samples predicted inconsistently in recent iterations. The historical lab…
Deep learning models show bias and variance are aligned, not in trade-off.
problem The classical bias-variance trade-off in deep learning models.
method Empirical evidence and theoretical analysis of bias and variance in deep learning models.
result Squared bias is approximately equal to variance for correctly classified sample points in deep learning models.
Ensembles improve classifier performance by reducing bias, not variance.
problem Improving classifier performance through ensemble methods.
method Extended bias-variance decomposition for classification tasks, introducing dual reparameterization.
result Ensembling reduces bias in classifiers, contrary to the traditional view.
This paper proposes a novel multiscale estimator for the integrated volatility of an Ito process, in the presence of market microstructure noise (observation error). The multiscale structure of the observed process is represented frequency-by-frequency and the concept of the multiscale ratio is introduced to quantify t…
Paper proposes a new method to stabilize noisy gradient algorithms.
problem Stochastic-gradient Langevin algorithms can introduce bias when taming denominators depend on stochastic-gradient realizations.
method Proposes a structure-preserving framework for designing tamed denominators that avoid unnecessary taming and maintain the stabilizing effect of taming.
result The method avoids stationary bias and explains the stationary error split into bias and remaining error.
Proposes a method to learn policies from offline data with reduced bias.
problem Learning policies from offline data with reduced bias and complexity constraints.
method Cross-fitted debiasing device for policy learning from offline data.
result Achieves N \sqrt N N regret for complex policy classes with a product-of-errors nuisance remainder.