arXiv research

New analysis shows bias term affects conditions for benign overfitting in linear classifiers.

Study proposes a new method to estimate bias-correction term for ATE estimation.

Paper analyzes LSA algorithm bias and error bounds with RR extrapolation.

This work shows how penalising bias terms in norm regularisation leads to sparse solutions.

We consider the least-squares regression problem and provide a detailed asymptotic analysis of the performance of averaged constant-step-size stochastic gradient descent (a.k.a. least-mean-squares). In the strongly-convex case, we provide an asymptotic expansion up to explicit exponentially decaying terms. Our analysis…

New bounds show SGD can match deterministic gradient descent's convergence rate.

Gradient descent struggles with learning a single neuron with bias.

Research shows bias in machine learning can be due to algorithmic flaws, not just data.

Study on bias and extrapolation in LSA with Markovian data, showing bias reduction with Richardson-Romberg extrapolation.

Deep convolutional networks often append additive constant ("bias") terms to their convolution operations, enabling a richer repertoire of functional mappings. Biases are also used to facilitate training, by subtracting mean response over batches of training images (a component of "batch normalization"). Recent state-o…

New method reduces bias in incomplete data using deliberate missingness.

Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptron with n input neurons, one output neuron, and no threshold bias term -- we prove that upon random initialisation of weights, the a priori p…

A hybrid strategy forecasts short-term loads using Warm-start Gradient Tree Boosting.

New framework explains neural network bias in solving differential equations.

We show how to convert ICL in linearized transformers into model weights.

A number of applications (e.g., AI bot tournaments, sports, peer grading, crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL) model to evaluate a given collection of items (e.g., bots, teams, students, search results). Past work has shown that under the BTL model, the widely-used maximum-likeli…

Deep learning models show bias and variance are aligned, not in trade-off.

We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or separable linear classification problems. We explore the question of whether the specifi…

Bias correction improves language model training performance.

A new metric MSD detects bias in datasets efficiently.

We analyze bias-variance of margin losses.

The study reveals a persistent bias in the distribution of holonomy on compact hyperbolic 3-manifolds.

Measures of implied volatility roughness corrected for bias.

The paper analyzes the bias-variance tradeoff for Bregman divergences.

The bias potential model explains how generative models can generalize or memorize samples.

Deep learning models can have low bias and variance, contrary to classical theory.

EBQL reduces bias in Q-learning for improved performance.

New method reduces bias in estimating causal effects from discretized variables.

New method quantifies inductive bias for machine learning tasks.

WeMix improves data augmentation by correcting bias in deep learning.

UCBMQ improves Q-learning by adding momentum to correct bias and limit regret.

Improves decision-making by correcting feature selection bias.

Mitigates bias in text classification by weighting instances.

New method controls bias in unadjusted Hamiltonian Monte Carlo and underdamped Langevin.

We consider the off-policy evaluation problem in Markov decision processes with function approximation. We propose a generalization of the recently introduced \emph{emphatic temporal differences} (ETD) algorithm \citep{SuttonMW15}, which encompasses the original ETD($λ$), as well as several other off-policy evaluation …

Develops a method to quantify racial bias in law enforcement systems.

In many applications we seek to maximize an expectation with respect to a distribution over discrete variables. Estimating gradients of such objectives with respect to the distribution parameters is a challenging problem. We analyze existing solutions including finite-difference (FD) estimators and continuous relaxatio…

We study the relationship between the frequency of a function and the speed at which a neural network learns it. We build on recent results that show that the dynamics of overparameterized neural networks trained with gradient descent can be well approximated by a linear system. When normalized training data is uniform…

We revisit fuzzy neural network with a cornerstone notion of generalized hamming distance, which provides a novel and theoretically justified framework to re-interpret many useful neural network techniques in terms of fuzzy logic. In particular, we conjecture and empirically illustrate that, the celebrated batch normal…

Jointly correct bias fields and reconstruct undersampled MRI images.

Fair Adversarial Networks remove bias from data.

Paper analyzes convergence of decentralized algorithms with noise and bias.

Deep convolutional networks can be understood through kernel methods, providing insights into their inductive bias.

Adversarial training leads to large generalization gap, decomposed into bias and variance.

GraphTEE estimates treatment effects on graph-structured targets, mitigating bias.

SGD in linear regression overfits but performs well due to bias-variance trade-off.

The paper analyzes optimal implicit bias in linear regression for over-parameterized models.

Mirror Langevin Algorithm converges with zero bias.