Paper compares Bayesian and de-biased estimators for low-rank matrix completion.
problem Predict missing entries in partially observed matrices.
method Bayesian and de-biased estimators comparison.
result De-biased estimator performs similarly to Bayesian estimators but is more stable and can outperform in small samples.
New oracles improve stochastic optimization with noisy or biased measurements.
problem Optimizing functions with noisy or biased measurements.
method Introduced biased gradient oracles for stochastic optimization, analyzed RSG and SGD algorithms with these oracles.
result Derived non-asymptotic bounds for convergence rates of algorithms with biased gradient oracles.
Paper proposes a new optimizer for faster nonconvex optimization.
problem Optimizing nonconvex objectives efficiently and quickly.
method Integrates stochastic and biased gradient estimation with a hyper-parameter.
result The hyper-parameter can be configured to improve convergence rate.
Biased mean regression estimates factors exceeding expected loss or radiation release severity.
problem Estimating factors exceeding expected loss or radiation severity levels.
method Biased mean regression using superexpectation error minimization.
result Equivalent to quantile regression and CVaR optimization under specific conditions.
The Whittle likelihood is a widely used and computationally efficient pseudo-likelihood. However, it is known to produce biased parameter estimates for large classes of models. We propose a method for de-biasing Whittle estimates for second-order stationary stochastic processes. The de-biased Whittle likelihood can be …
The paper proposes biased advantage estimates for reinforcement learning, improving efficiency and performance.
problem Estimating advantages in reinforcement learning algorithms.
method A family of estimates based on order statistics over the path ensemble.
result Biased estimates can significantly benefit reinforcement learning, especially in environments with sparse rewards or critical actions.
New variance-reduction methods solve stochastic composite inclusions.
problem Solving nonmonotone stochastic composite inclusions.
method Developed unbiased and biased variance-reduced estimators for FRBS method.
result Achieved best oracle complexities for finite-sum and expectation settings.
Analyze SGD with biased gradients, improving convergence rates and accuracy.
problem Analyzing the convergence of SGD with biased gradients.
method Derive convergence results for smooth non-convex functions and quantify the impact of bias magnitude.
result Improved rates under the Polyak-Lojasiewicz condition and insights into how bias magnitude affects accuracy and convergence.
Strong inductive biases prevent harmless interpolation in overparameterized models.
problem Understanding the conditions under which overparameterized models can interpolate noise without overfitting.
method Theoretical analysis of high-dimensional kernel regression and deep neural networks, focusing on the role of inductive biases.
result The strength of an estimator's inductive bias determines whether interpolation is harmless or requires fitting noise for good generalization.
The paper analyzes time-dependent streaming data with biased gradient estimates and proposes improved stochastic optimization methods.
problem Stochastic optimization in a streaming setting with time-dependent and biased gradient estimates.
method Analysis of several first-order methods including SGD, mini-batch SGD, and time-varying mini-batch SGD, along with their Polyak-Ruppert averages.
result Time-varying mini-batch SGD methods can break long- and short-range dependence structures, and biased SGD methods can achieve comparable performance to their unbiased counterparts.
How do we learn from biased data? Historical datasets often reflect historical prejudices; sensitive or protected attributes may affect the observed treatments and outcomes. Classification algorithms tasked with predicting outcomes accurately from these datasets tend to replicate these biases. We advocate a causal mode…
RBMLE learns from reward bias to optimize bandit decisions.
problem Optimizing decisions in stochastic multi-armed bandits.
method Reward-Biased Maximum Likelihood Estimation (RBMLE) for SMABs.
result RBMLE achieves optimal regret bounds for Gaussian and sub-Gaussian bandits.
The paper tackles bandit problems with biased offline data by using causal methods.
problem Improving bandit algorithms with biased offline data that includes confounding and selection biases.
method Formalizes the problem from a causal perspective, categorizes biases, and derives robust bounds for each arm.
result Causal bounds can guide the bandit agent to learn a nearly-optimal decision policy and consistently reduce asymptotic regret.
Estimates social network structure from random walk subgraphs.
problem Recovering population structure from random walk subgraphs in stochastic block models.
method Maximum likelihood estimation, SAEM algorithm, de-biasing techniques.
result New de-biased estimator provides more accurate recovery of network structure.
A new biased gradient descent method for conditional stochastic optimization.
problem Challenges in constructing unbiased gradient estimators for conditional stochastic optimization.
method Proposes a biased stochastic gradient descent (BSGD) algorithm and analyzes its sample complexities.
result Establishes sample complexities of BSGD for various objectives and shows that BSpiderBoost matches the lower bound complexity.
In many applications, different populations are compared using data that are sampled in a biased manner. Under sampling biases, standard methods that estimate the difference between the population means yield unreliable inferences. Here we develop an inference method that is resilient to sampling biases and is able to …
Biased sampling and missing data complicates statistical problems ranging from causal inference to reinforcement learning. We often correct for biased sampling of summary statistics with matching methods and importance weighting. In this paper, we study nearest neighbor matching (NNM), which makes estimates of populati…
Noise in imputed values corrects biases in machine learning models.
problem Systematic biases in imputed values affect downstream analyses.
method Introducing noise to imputed values to correct biases.
result Noise-corrected imputation methods produce unbiased estimates.
This paper analyzes SGD with biased gradients for deep learning models.
problem Analyzing SGD with biased gradients for deep learning models.
method Non-asymptotic analysis of SGD with adaptive steps for non-convex smooth functions.
result Adagrad, RMSProp, and AMSGRAD converge to critical points at a similar rate to unbiased case.
New method estimates nested expectations with biased and antithetic sampling.
problem Estimating nested expectations with biased and antithetic sampling.
method Nested multilevel Monte Carlo with biased and antithetic sampling.
result Estimator achieves order ε^(-2) asymptotic cost.
Improved estimator reduces bias in statistical learning models.
problem Asymptotic bias in classic WDRO estimator.
method Adjusted Wasserstein distributionally robust estimator.
result Asymptotic unbiased estimator with smaller MSE.
New method for estimating treatment effects without complex propensity models.
problem Estimating treatment effects in dynamic treatment regimes.
method Recursive Riesz representer estimation for de-biasing corrections.
result Directly estimates de-biasing corrections without auxiliary models.
The paper proposes methods to estimate MCMC quality with couplings, bounding Wasserstein distance.
problem Improving MCMC efficiency without sacrificing asymptotic consistency.
method Estimators based on couplings of Markov chains to assess quality of asymptotically biased sampling methods.
result Empirical upper bounds of Wasserstein distance for assessing MCMC quality.
NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.
problem Stochastic contextual bandit problem with general bounded reward functions.
method Reward-biased maximum likelihood estimation with neural networks to enforce exploration.
result Both NeuralRBMLE variants achieve O ~ ( T ) \widetilde{\mathcal{O}}(\sqrt{T}) O ( T ) regret. This study examines biases in flow matching samplers using finite-sample estimation.
problem Biases in flow matching samplers when using finite-sample surrogates.
method Finite-sample plug-in estimation and hierarchy of empirical FM models.
result Exact empirical minimizer and smoothed plug-in regime identified for affine conditional flows.
We study high-dimensional Gaussian mixture classification using statistical physics methods.
problem Classifying high-dimensional Gaussian mixture with general covariance matrices.
method Replica method from statistical physics for asymptotic analysis of convex classifiers.
result Construction and validation of a de-biased estimator for variable selection.
We study the convergence rate of stochastic optimization of exact (NP-hard) objectives, for which only biased estimates of the gradient are available. We motivate this problem in the context of learning the structure and parameters of Ising models. We first provide a convergence-rate analysis of deterministic errors fo…
Contrastive Divergence (CD) and Persistent Contrastive Divergence (PCD) are popular methods for training the weights of Restricted Boltzmann Machines. However, both methods use an approximate method for sampling from the model distribution. As a side effect, these approximations yield significantly different biases and…
Helps visually impaired users make better decisions by adjusting their observations.
problem Systematic biases in users' perception and processing of visual information.
method Synthesizes new observations based on true observations to correct user biases.
result Significant improvement in task performance for users with various biases.
The paper corrects biases in estimating intrinsic dimension and differential entropy.
problem Systematic bias in estimating intrinsic dimension and differential entropy.
method A bias-corrected estimator for both measures is proposed, highlighting shared steps and useful consequences.
result Simultaneous estimation of differential entropy and intrinsic dimension provides complementary perspectives on underlying manifolds.
Algorithm identifies best arm with biased proxy and selective ground truth audits.
problem Fixed-confidence best-arm identification with biased proxy and selective ground truth.
method Propensity-weighted estimator and adaptive auditing algorithm.
result Plug-in Neyman rule achieves near-oracle audit efficiency.
DML addresses biases in machine learning by estimating nuisance functions.
problem Bias in machine learning models due to nuisance functions.
method Double/Debiased Machine Learning (DML) approach to reduce biases.
result DML allows flexible estimation of nuisance functions without auxiliary assumptions.
The paper tackles fair sequential decision making with biased linear bandit feedback.
problem Fair sequential decision making with biased linear bandit feedback.
method Phased elimination algorithm to correct unfair evaluations, establishing upper bounds on regret.
result The worst-case regret is smaller than O ( κ ∗ 1 / 3 log ( T ) 1 / 3 T 2 / 3 ) \mathcal{O}(κ_*^{1/3}\log(T)^{1/3}T^{2/3}) O ( κ ∗ 1/3 log ( T ) 1/3 T 2/3 ) . Proposes unbiased estimators for training mixture of experts models.
problem Efficiently training large-scale mixture of experts models on modern hardware.
method Two unbiased estimators based on principled stochastic assignment procedures.
result Both estimators are more effective and robust than biased alternatives.
We rebias estimates to improve interval calibration and prediction accuracy.
problem Constructing accurate intervals for noisy and biased estimates.
method Empirical Bayes rebiasing strategy that learns bias distribution from data.
result Substantial precision gains in prediction-powered inference.
Chinchilla Approach 2 biases neural scaling law estimates, leading to unnecessary compute costs.
problem Systematic biases in Chinchilla Approach 2's parabolic fits of neural scaling laws.
method Analyzes three sources of error: IsoFLOP sampling grid width, uncentered sampling, and loss surface asymmetry.
result Chinchilla Approach 3 largely eliminates these biases, offering a more convenient or scalable alternative.
Algorithm learns fair ranking from biased data.
problem Unfair ranking policies from biased implicit feedback.
method Policy-gradient approach with amortized fairness constraints.
result Efficient algorithm FULTR learns fair policies.
Kernel quadrature improves CRPS estimation for probabilistic time-series forecasting.
problem Intractable integrations in CRPS evaluation metrics lead to improper rankings of forecasting models.
method Introduced kernel quadrature approach for unbiased CRPS estimation and scalable computation.
result Our approach consistently outperforms existing CRPS estimators.
We consider the problem of fitting a linear model to data held by individuals who are concerned about their privacy. Incentivizing most players to truthfully report their data to the analyst constrains our design to mechanisms that provide a privacy guarantee to the participants; we use differential privacy to model in…
Framework assesses autograders' reliability and biases.
problem Mixed reliability and biases in autograders for LLM evaluation.
method Bayesian GLMs to model evaluation outcomes.
result Explicit quantification of scoring differences and biases.
The paper improves sparse Gaussian processes by optimizing predictive loss.
problem Optimizing predictive loss in sparse Gaussian processes.
method Direct loss minimization (DLM) for log-loss and square loss, with product sampling (uPS) and biased Monte Carlo (bMC) for non-conjugate cases.
result DLM shows significant performance improvement in both log-loss and square loss cases.
FAML addresses biased evidence learning in multi-view learning, improving fairness and prediction reliability.
problem Biased evidence learning in multi-view learning, leading to unreliable uncertainty estimation.
method FAML introduces an adaptive prior and fairness constraint to balance evidence allocation across views.
result FAML enhances fairness and improves prediction reliability compared to state-of-the-art methods.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Simple algorithm approximates rare event frequencies.
problem Approximating the frequency of rare events.
method Iterative update of categorical click-distribution, resulting in a random walk on an n-dimensional simplex.
result The random walk corresponds to a biased Bernoulli convolution under certain conditions.
Sample measures of top centile contributions to the total (concentration) are downward biased, unstable estimators, extremely sensitive to sample size and concave in accounting for large deviations. It makes them particularly unfit in domains with power law tails, especially for low values of the exponent. These estima…
LLM evaluation suffers from systematic biases and lacks reliable positive judgments.
problem LLM evaluation suffers from systematic biases and lacks reliable positive judgments.
method Formulate LLM evaluation as a positive-unlabelled learning problem and propose a geometric auditing framework based on Partial Optimal Transport.
result Improved alignment with human preferences, increased robustness to presentation biases, and interpretable confidence estimates.
Estimates causal effect of managed care plans on NYC Medicaid spending.
problem Generalizing causal estimates to a target population not well-represented by randomized studies.
method Conditional cross-design synthesis estimators combining randomized and observational data.
result Estimates causal effect of managed care plans on health care spending.
Algorithm reduces audit costs by identifying best service configurations from biased textual evidence.
problem Designing service systems from textual evidence requires accurate selection despite biased automated scoring.
method Developed PP-LUCB algorithm combining LLM scores and selective audits to minimize costs.
result Correctly identified the best model in 40/40 trials with 90% cost reduction.