Bayesian adaptive designs can be biased by active learning, especially with misspecified models.
problem Active learning bias in Bayesian adaptive experimental designs.
method Analysis of linear and preference learning models, empirical testing.
result Model misspecification and noise influence active learning bias in Bayesian designs.
Adversarial domain adaptation reduces sample bias in high energy physics classifier.
problem Sample bias in high energy physics classifier training.
method Adversarial domain adaptation using neural networks with gradient reversal layer.
result Successful bias removal on simulated events at the LHC.
Truncated backpropagation through time (TBPTT) is a popular method for learning in recurrent neural networks (RNNs) that saves computation and memory at the cost of bias by truncating backpropagation after a fixed number of lags. In practice, choosing the optimal truncation length is difficult: TBPTT will not converge …
BC-ACI corrects time series forecast bias, improving prediction intervals.
problem Persistent bias in time series forecasts leads to overly conservative prediction intervals.
method Augments ACI with an EWM estimate of forecast bias to correct nonconformity scores and re-center intervals.
result Reduces Winkler interval scores by 13-17% under distribution shifts, improving calibration.
New insights on bias in multi-armed bandits under conditional sampling.
problem Understanding bias in multi-armed bandits under conditional sampling.
method Characterized the sign of conditional bias of monotone functions of rewards.
result Sign of conditional bias can differ from marginal bias, depending on conditioning events.
Improves Gaussian process regression without bias.
problem Bias in Gaussian process regression estimates.
method Adaptive computation selection to minimize bias.
result Guaranteed small bias in log marginal likelihood estimates.
ARFF reduces spectral bias in SGD-trained neural networks.
problem Spectral bias in two-layer neural networks.
method Comparison of SGD and ARFF on spectral bias and robustness.
result ARFF yields a closer to zero spectral bias compared to SGD.
This paper balances bias and variance in adaptive importance sampling using mirror descent.
problem Large variance in adaptive importance sampling weights.
method Regularization strategy with power raised importance weights connected to mirror descent.
result The regularization parameter balances bias and variance.
Ant colonies and boosting algorithms both reduce bias and variance through adaptive mechanisms.
problem Understanding the mathematical principles behind ensemble learning and ant colony behavior.
method Developed a formal mapping between AdaBoost's adaptive reweighting and ant recruitment dynamics.
result Proved that the fundamental theorem of weak learnability has a direct analog in colony decision-making.
Fine-tunes LLMs to correct bias in predictions.
problem LLMs exhibit bias in predictions from data.
method Supervised fine-tuning with Low-Rank Adaptation (LoRA).
result Fine-tuning corrects bias in both controlled and real-world settings.
New theory shows how learning algorithms can create a bias towards negative outcomes.
problem Negativity bias in adaptive learning algorithms.
method Generalization of the Hot Stove Effect to settings with negative estimates leading to smaller sample sizes.
result Negativity bias persists even when negative estimates do not lead to avoidance.
Recent years have seen an increased level of interest in pricing equity options under a stochastic volatility model such as the Heston model. Often, simulating a Heston model is difficult, as a standard finite difference scheme may lead to significant bias in the simulation result. Reducing the bias to an acceptable le…
Meta-SAGE improves deep RL scalability for CO tasks by adapting pre-trained models to larger-scale problems.
problem Improving scalability of deep reinforcement learning models for combinatorial optimization tasks.
method Meta-SAGE combines a scale meta-learner and scheduled adaptation with guided exploration to adjust model parameters for larger-scale problems.
result Meta-SAGE outperforms previous methods and significantly improves scalability in CO tasks.
Local adaptive methods in FL can accelerate convergence but introduce bias, which is corrected.
problem The effect of using adaptive optimization methods for local updates in federated learning.
method Proposed correction techniques to overcome the bias introduced by local adaptive methods.
result Correction techniques can achieve faster convergence and higher test accuracy than baseline methods.
Improves transferability of representations from source to target domains with weights and invariant representations.
problem Label shift between source and target domains in unsupervised domain adaptation.
method Integrates weights and invariant representations to bound the target risk, highlighting the role of inductive bias.
result Empirical evidence shows that weak inductive bias makes adaptation more robust.
Study shows how steepest descent algorithms' geometric margin increases during training.
problem Understanding implicit bias in steepest descent algorithms for neural networks.
method Analysis of steepest descent algorithms with infinitesimal learning rates in homogeneous neural networks.
result Limit points of training trajectories correspond to KKT points of margin-maximization problems.
Adaptive Langevin dynamics reduces bias in Bayesian inference with mini-batching.
problem Bias in posterior sampling due to mini-batching in Bayesian inference.
method Adaptive Langevin dynamics with dynamical friction to correct noise.
result Quantified bias in posterior distribution due to mini-batching.
New method reduces gender bias in language models without harming performance.
problem Bias in language models learned from biased data.
method Causal analysis to identify problematic model components, followed by linear projection of weight matrices.
result DAMA significantly decreases bias in language models while maintaining performance.
Corrects bias in LLM-as-a-judge evaluations using adaptive calibration.
problem Bias in LLM evaluations due to imperfect sensitivity and specificity.
method Plug-in framework with confidence intervals accounting for test and calibration dataset uncertainties.
result LML-based evaluation yields more reliable estimates than human-only evaluation.
It is well known that in stochastic multi-armed bandits (MAB), the sample mean of an arm is typically not an unbiased estimator of its true mean. In this paper, we decouple three different sources of this selection bias: adaptive \emph{sampling} of arms, adaptive \emph{stopping} of the experiment, and adaptively \emph{…
New algorithm offers costless model selection in contextual bandits.
problem Minimizing cumulative regret in stochastic contextual bandits.
method Gradually increasing class complexity and adapting to the simplest class with dominant estimation variance.
result Costless model selection is feasible under certain conditions, providing improved regret guarantees.
Paper proposes adaptive parameter selection for KGD algorithms.
problem Improving parameter selection for kernel-based gradient descent.
method Integrates bias-variance analysis with splitting method, introduces empirical effective dimension.
result Adaptive parameter selection strategy achieves optimal generalization error bound.
Most neural-network based speaker-adaptive acoustic models for speech synthesis can be categorized into either layer-based or input-code approaches. Although both approaches have their own pros and cons, most existing works on speaker adaptation focus on improving one or the other. In this paper, after we first systema…
We consider the core reinforcement-learning problem of on-policy value function approximation from a batch of trajectory data, and focus on various issues of Temporal Difference (TD) learning and Monte Carlo (MC) policy evaluation. The two methods are known to achieve complementary bias-variance trade-off properties, w…
TransCal calibrates DA models with lower bias and variance.
problem Calibrating DA models to estimate accurate predictive uncertainty.
method Transferable Calibration (TransCal) in a unified hyperparameter-free optimization framework.
result TransCal achieves more accurate calibration with lower bias and variance.
New SAE algorithm proves feature recovery for LLMs with theoretical guarantees.
problem Achieving interpretable features in large language models (LLMs).
method Proposed a statistical framework and bias adaptation technique for sparse autoencoders (SAEs).
result Proved correct recovery of all monosemantic features under specific data sampling.
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- W-decorrelation -- for transformi…
Data that is gathered adaptively --- via bandit algorithms, for example --- exhibits bias. This is true both when gathering simple numeric valued data --- the empirical means kept track of by stochastic bandit algorithms are biased downwards --- and when gathering more complicated data --- running hypothesis tests on c…
Adaptive optimization methods bias neural network trajectories towards regions of lower local geometry.
problem The success of adaptive optimization methods in neural networks is not fully explained by traditional second-order methods.
method Local trajectory analysis and introduction of a new statistic RextmedextOPT. result Adaptive methods like Adam bias trajectories towards regions of lower local geometry, leading to faster convergence.
From scientific experiments to online A/B testing, the previously observed data often affects how future experiments are performed, which in turn affects which data will be collected. Such adaptivity introduces complex correlations between the data and the collection procedure. In this paper, we prove that when the dat…
Adaptive data analysis is frequently criticized for its pessimistic generalization guarantees. The source of these pessimistic bounds is a model that permits arbitrary, possibly adversarial analysts that optimally use information to bias results. While being a central issue in the field, still lacking are notions of na…
The paper proposes a method for distribution-free prediction sets that adapt to unknown temporal changes.
problem Distribution-free prediction sets require reliable calibration data, which is often unavailable in real-world settings with temporal changes.
method The method selects an adaptive window to construct prediction sets, optimizing a bias-variance tradeoff.
result The method provides sharp coverage guarantees and is shown to be adaptive to temporal drift through numerical experiments.
A new method reduces bias in adaptive Lasso estimates.
problem Bias in adaptive Lasso estimates.
method Proximal gradient approach to learn penalty coefficients as decision variables.
result Reduces bias in estimates and encourages arbitrary sparsity structure.
Self-attention optimizers converge to optimal weights, revealing bias patterns.
problem Understanding the implicit bias in self-attention mechanisms.
method Analysis of gradient-based optimization in self-attention layers.
result Adaptive step-size strategies can accelerate convergence to optimal weights.
We study the informational efficiency of a market with a single traded asset. The price initially differs from the fundamental value, about which the agents have noisy private information (which is, on average, correct). A fraction of traders revise their price expectations in each period. The price at which the asset …
This paper analyzes SGD with biased gradients for deep learning models.
problem Analyzing SGD with biased gradients for deep learning models.
method Non-asymptotic analysis of SGD with adaptive steps for non-convex smooth functions.
result Adagrad, RMSProp, and AMSGRAD converge to critical points at a similar rate to unbiased case.
We study the phenomenon of bias amplification in classifiers, wherein a machine learning model learns to predict classes with a greater disparity than the underlying ground truth. We demonstrate that bias amplification can arise via an inductive bias in gradient descent methods that results in the overestimation of the…
Paper analyzes adaptive Lasso for high-dimensional diffusion processes, improving support recovery and bias.
problem Support recovery for high-dimensional diffusion processes under sparsity constraints.
method Adaptive Lasso estimator for d-dimensional ergodic diffusion process, focusing on linear models.
result Adaptive Lasso achieves support recovery and asymptotic normality for drift parameter under certain conditions.
Paper shows fairness and domain adaptation can work together.
problem Algorithmic bias and distributional shifts in ML models.
method Leveraging fairness and distribution shifts, the paper shows how domain adaptation methods can mitigate bias.
result Enforcing individual fairness can improve out-of-distribution accuracy under covariate shift.
Paper proposes a new method combining random forests and Lasso selection.
problem Improving random forest performance by applying Lasso regression.
method Adaptive Lasso weighting applied to random forest predictions.
result Unified framework strictly outperforms other methods in simulations and real-world datasets.
Various bias-correction methods such as EXTRA, gradient tracking methods, and exact diffusion have been proposed recently to solve distributed {\em deterministic} optimization problems. These methods employ constant step-sizes and converge linearly to the {\em exact} solution under proper conditions. However, their per…
The accuracy of deep neural networks is significantly affected by how well mini-batches are constructed during the training step. In this paper, we propose a novel adaptive batch selection algorithm called Recency Bias that exploits the uncertain samples predicted inconsistently in recent iterations. The historical lab…
Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can adapt to changing training data, altering its outcome based on changes in its input. We measure expressivity by using an information-theoretic …
Machine learning methods adapt the parameters of a model, constrained to lie in a given model class, by using a fixed learning procedure based on data or active observations. Adaptation is done on a per-task basis, and retraining is needed when the system configuration changes. The resulting inefficiency in terms of da…
Forest-guided smoothing uses random forest outputs for interpretable local smoothers.
problem Creating interpretable local smoothers from complex random forest outputs.
method Uses random forest outputs to define spatially adaptive bandwidth matrices for a linear smoother.
result Improves interpretability and applicability of random forest outputs for various analyses.
Paper models market dynamics using bull and bear forces.
problem Complex market dynamics influenced by biases and narratives.
method Bias to Behavior from Bull-Bear Dynamics (B4) model.
result Model predicts market trends with superior performance and interpretable insights.
Adaptive collection of data is commonplace in applications throughout science and engineering. From the point of view of statistical inference however, adaptive data collection induces memory and correlation in the samples, and poses significant challenge. We consider the high-dimensional linear regression, where the s…
Paper measures cognitive bias in positive feedback trading using diffusion process estimates.
problem Measuring cognitive bias in positive feedback trading behavior.
method Conditional estimates of diffusion processes to quantify bias, proving asymptotic properties.
result Bias in positive feedback trading converges to zero over time, leading to adaptive expectations.