On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “latent gradient bias”

Stochastic Resetting Mitigates Latent Gradient Bias of SGD from Label Noisecs.LG

Resetting from checkpoints improves DNN training with noisy labels.

problem Latent gradient bias induced by noisy labels causes overfitting.

method Stochastic resetting applied to SGD to mitigate latent gradient bias.

result Resetting significantly improves DNN generalization performance.

stochastic resetting SGD label noise latent gradient bias overfitting

2024-06-01abs ↗pdf ↗

Beyond ReinMax: Low-Variance Gradient Estimators for Discrete Latent Variablesstat.ML

New estimators reduce variance in training variational autoencoders with discrete latent variables.

problem Training variational autoencoders with discrete latent variables requires efficient gradient estimation.

method Introduce ReinMax-Rao and ReinMax-CV estimators using Rao-Blackwellisation and control variates.

result Demonstrate superior performance on training variational autoencoders with discrete latent spaces.

gradient estimation discrete latent variables variational autoencoders Rao-Blackwellisation control variates

2026-03-09abs ↗pdf ↗

Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Informationstat.ML

New method learns diverse solutions in reinforcement learning without gradient bias.

problem Lack of diverse solutions in reinforcement learning tasks.

method Maximizes state-action-based mutual information directly, using variational lower bound.

result Successfully learns an infinite set of diverse solutions.

reinforcement learning diverse solutions mutual information gradient bias variational lower bound

2021-03-12abs ↗pdf ↗

Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generationcs.CL

New method neutralizes gender bias in word embeddings without losing semantic information.

problem Gender biases in word embeddings trained on human-generated corpora.

method Latent Disentanglement and Counterfactual Generation with siamese auto-encoder and gradient reversal layer.

result Our method outperforms existing debiasing methods in preserving semantic information and neutralizing gender biases.

word embeddings gender bias latent disentanglement counterfactual generation semantic information

2020-04-07abs ↗pdf ↗

A hitchhiker's guide to Poisson gradient estimationstat.ML

Modified EAT method improves Poisson gradient estimation.

problem Challenging differentiation through Poisson-distributed latent variables.

method Exponential Arrival Time (EAT) simulation with modifications and Gumbel-SoftMax relaxation.

result Modified EAT method provides unbiased first moment and reduced second-moment bias.

Poisson gradient estimation Exponential Arrival Time Gumbel-SoftMax Neural spike trains Latent variable models

2026-02-03abs ↗pdf ↗

Discrete Latent Structure in Neural Networkscs.LG

This text explores strategies for learning discrete latent structures in neural networks.

problem Learning discrete latent structures in neural networks is challenging.

method Continuous relaxation, surrogate gradients, and probabilistic estimation.

result Many latent structure learning strategies use the same fundamental building blocks but apply them differently.

discrete latent structure neural networks latent structure learning continuous relaxation surrogate gradients

2023-01-18abs ↗pdf ↗

Differentiable Sampling of Categorical Distributions Using the CatLog-Derivative Trickcs.LG

A new gradient estimator for categorical distributions reduces bias and variance.

problem Intractability of gradients for categorical distributions in discrete latent variable models.

method CatLog-Derivative trick and IndeCateR gradient estimator.

result IndeCateR reduces bias and variance of gradients for categorical distributions.

gradient estimation categorical distributions discrete latent variables REINFORCE log-derivative trick

2023-11-21abs ↗pdf ↗

Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimatorstat.ML

Paper improves Gumbel-Softmax estimator variance reduction.

problem Challenges in gradient estimation for models with discrete latent variables.

method Rao-Blackwellization applied to straight-through Gumbel-Softmax estimator.

result Reduces mean squared error and variance of Gumbel-Softmax estimator.

gradient estimation discrete latent variables Gumbel-Softmax Rao-Blackwellization straight-through

2020-10-09abs ↗pdf ↗

Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuningstat.ML

GRASP removes spurious correlations in fine-tuned models, improving task performance and reducing bias.

problem Fine-tuned models can latch onto spurious correlations, leading to bias and reduced generalization.

method GRASP identifies and removes spurious correlations from model weights without removing latent factors.

result GRASP significantly reduces bias and improves task performance in various fine-tuning tasks.

fine-tuning bias reduction spurious correlations GRASP model bias

2026-05-26abs ↗pdf ↗

Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Modelscs.LG

Paper presents variational estimates for EBLVMs without structural assumptions.

problem Challenges in learning and evaluating EBLVMs due to intractable true posteriors and partition functions.

method Variational estimates of the score function and its gradient (VaES and VaGES) in a general EBLVM.

result The estimates can be applied to KSD and SM-based methods to learn EBLVMs and estimate Fisher divergence.

energy-based latent variable models variational estimates score function gradient kernelized Stein discrepancy

2020-10-16abs ↗pdf ↗

On the Origins of Linear Representations in Large Language Modelscs.CL

This work explains how linear representations in large language models arise from training objectives and gradient descent.

problem Understanding the origins of linear representations in large language models.

method A latent variable model to abstract and formalize concept dynamics, combined with analysis of the softmax cross-entropy objective and gradient descent.

result Linear representations emerge when learning from data matching the latent variable model, and this simple structure suffices to yield linear representations.

large language models linear representations latent variable model softmax cross-entropy gradient descent

2024-03-06abs ↗pdf ↗

Deconfounding Multi-Cause Latent Confounders: A Factor-Model Approach to Climate Model Bias Correctionstat.ML

This paper corrects climate model biases using a factor model approach.

problem Systematic biases in GCM outputs due to unobserved confounders.

method Factor model approach to learn latent confounders from historical data and apply them to enhance bias correction.

result Significant improvements in the accuracy of precipitation outputs.

climate models bias correction factor model latent confounders time series forecasting

2024-08-22abs ↗pdf ↗

Inference for Regression with Variables Generated by AI or Machine Learningecon.EM

AI-generated variables bias regression estimates; methods correct for invalid inference.

problem Bias in regression estimates due to AI-generated variables.

method Two methods: bias correction and joint estimation.

result Valid inference restored through proposed methods.

AI Machine Learning Regression Latent Variables Bias Correction

2024-02-23abs ↗pdf ↗

Why Machine Learning Models Systematically Underestimate Extreme Values II: How to Fix It with LatentNNastro-ph.IM

LatentNN corrects neural network attenuation bias in astronomical data.

problem Neural networks underestimate extreme values due to measurement errors.

method Jointly optimizes network parameters and latent input values.

result LatentNN reduces attenuation bias across various signal-to-noise ratios.

attenuation bias neural networks astronomy measurement errors latent variables

2025-12-29abs ↗pdf ↗

Revisiting Counterfactual Regression through the Lens of Gromov-Wasserstein Information Bottleneckcs.LG

GWIB improves counterfactual regression by balancing latent distributions and reducing selection bias.

problem Selection bias between control and treatment groups negatively impacts counterfactual regression performance.

method GWIB uses Gromov-Wasserstein information bottleneck to maximize mutual information between covariates and outcomes while penalizing kernelized mutual information between latent representations and covariates.

result GWIB consistently outperforms state-of-the-art CFR methods in ITE estimation tasks.

counterfactual regression information bottleneck selection bias Gromov-Wasserstein individualized treatment effect

2024-05-24abs ↗pdf ↗

Smoothed Gradients for Stochastic Variational Inferencestat.ML

Stochastic variational inference (SVI) lets us scale up Bayesian computation to massive data. It uses stochastic optimization to fit a variational distribution, following easy-to-compute noisy natural gradients. As with most traditional stochastic optimization methods, SVI takes precautions to use unbiased stochastic g…

2014-06-13abs ↗pdf ↗

Foundations of Structural Causal Models with Latent Selectionstat.ME

Develops SCMs for latent selection to simplify causal analysis.

problem Latent selection complicates causal analysis.

method Introduces a conditioning operation for SCMs to encode latent selection.

result Conditioning operation preserves simplicity, acyclicity, and linearity of SCMs.

Structural Causal Models Latent Selection Directed Mixed Graphs Latent Common Causes Selection Bias

2024-01-12abs ↗pdf ↗

Comparing the Value of Labeled and Unlabeled Data in Method-of-Moments Latent Variable Estimationcs.LG

New framework assesses value of labeled vs unlabeled data in latent variable models.

problem Determining the optimal use of labeled and unlabeled data in latent variable models.

method Developed a bias-variance decomposition of the generalization error for method-of-moments latent variable estimation, and introduced a correction for misspecification.

result Labeled data is more valuable than unlabeled data when models are misspecified, but this value can be reduced with correction.

latent variable models method-of-moments labeling cost unlabeled data bias-variance decomposition

2021-03-03abs ↗pdf ↗

To Regularize or Not To Regularize? The Bias Variance Trade-off in Regularized AEscs.LG

FlexAE addresses bias-variance trade-off in RAEs by learning latent priors.

problem Improving generation quality of deterministic AE models.

method Introducing flexibly learnable latent priors in WAEs to optimize the latent distribution.

result FlexAE achieves state-of-the-art performance in AE-based generative models.

Auto-Encoders Regularization Bias-Variance Wasserstein Generative Models

2020-06-10abs ↗pdf ↗

Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Datacs.LG

Gradient descent biases towards stable rank networks for nearly-orthogonal data.

problem Understanding implicit bias in non-smooth neural networks trained by gradient descent.

method Analysis of two-layer ReLU and leaky ReLU networks trained by gradient descent on nearly-orthogonal data.

result Gradient descent biases towards networks with stable rank and uniform margin for nearly-orthogonal data.

gradient descent ReLU networks implicit bias nearly-orthogonal data stable rank

2023-10-29abs ↗pdf ↗

Doubly Reparameterized Gradient Estimators for Monte Carlo Objectivescs.LG

Deep latent variable models have become a popular model choice due to the scalable learning algorithms introduced by (Kingma & Welling, 2013; Rezende et al., 2014). These approaches maximize a variational lower bound on the intractable log likelihood of the observed data. Burda et al. (2015) introduced a multi-sample v…

2018-10-09abs ↗pdf ↗

Agnostic Learning of General ReLU Activation Using Gradient Descentcs.LG

Gradient descent learns ReLU functions with non-zero bias efficiently.

problem Learning ReLU functions with non-zero bias under Gaussian distributions.

method Gradient descent starting from random initialization.

result Gradient descent achieves near-optimal error with high probability.

gradient descent ReLU function non-zero bias Gaussian distributions

2022-08-04abs ↗pdf ↗

Increasing Missingness to Reduce Bias: Richardson-SGD with Missing Datastat.ML

New method reduces bias in incomplete data using deliberate missingness.

problem Systematic gradient biases in incomplete data for stochastic learning.

method Richardson-SGD debiasing procedure with deliberate missingness.

result Reduces gradient bias from

O(\|p\|)

O(\|p\|^2)

missing data gradient bias Richardson extrapolation stochastic gradient descent imputation

2026-05-19abs ↗pdf ↗

Learning Preference from Observed Rankingsstat.ML

The paper develops a method to estimate consumer preferences from observed rankings.

problem Estimating consumer preferences from partial ranking information.

method Interpreting observed rankings as pairwise comparisons, modeling latent utility, and correcting for selection bias.

result The method improves recommendation performance, especially for previously unconsumed products.

consumer preferences partial rankings latent utility pairwise comparisons selection bias

2026-02-18abs ↗pdf ↗

Tab-TRM: Tiny Recursive Model for Insurance Pricing on Tabular Datacs.LG

Tab-TRM uses recursive model for insurance pricing on tabular data.

problem Insurance pricing on tabular data.

method Adapts recursive latent reasoning to insurance modeling using a compact, parameter-efficient network.

result Improves insurance pricing accuracy using iterative refinement of latent tokens.

insurance pricing tabular data recursive model Tiny Recursive Models Gradient Boosting Machines

2026-01-12abs ↗pdf ↗

Iterative Causal Discovery in the Possible Presence of Latent Confounders and Selection Biascs.LG

Algorithm recovers causal graphs in presence of latent confounders and selection bias.

problem Recovering causal graphs in the presence of latent confounders and selection bias.

method Iterative causal discovery (ICD) algorithm that relies on causal Markov and faithfulness assumptions.

result Sound and complete algorithm that recovers the equivalence class of the underlying causal graph.

causal discovery latent confounders selection bias complete graph conditional independence

2021-11-07abs ↗pdf ↗

Causal Calculus in the Presence of Cycles, Latent Confounders and Selection Biasstat.ML

We prove the main rules of causal calculus (also called do-calculus) for i/o structural causal models (ioSCMs), a generalization of a recently proposed general class of non-/linear structural causal models that allow for cycles, latent confounders and arbitrary probability distributions. We also generalize adjustment c…

2019-01-02abs ↗pdf ↗

Distinguishability of causal structures under latent confounding and selectionmath.ST

Paper distinguishes causal structures under latent confounding and selection bias.

problem Distinguishing causal relationships when latent variables and selection bias are present.

method Formulated selected-marginalized directed graphs (smDGs) to distinguish causal structures.

result Two causal structures are indistinguishable if they have the same selected-marginalized directed graph.

causal inference latent confounding selection bias smDG d-separation

2025-09-24abs ↗pdf ↗

Reducing Selection Bias in Counterfactual Reasoning for Individual Treatment Effects Estimationcs.LG

Reduces selection bias in estimating individual treatment effects.

problem Selection bias in counterfactual reasoning.

method Auto-encoder with regularized loss based on Pearson Correlation Coefficient.

result Improves performance in estimating individual treatment effects.

selection bias counterfactual reasoning individual treatment effects auto-encoder Pearson Correlation Coefficient

2019-12-19abs ↗pdf ↗

Session-based Recommender Systems: User Interest as a Stochastic Process in the Latent Spacecs.LG

The paper tackles biases in session-based recommender systems by modeling user interest as a stochastic process.

problem Data uncertainty, popularity bias, and exposure bias in session-based recommender systems.

method The paper proposes treating user interest as a stochastic process in the latent space, debiasing item embeddings, modeling dense user interest, and introducing fake targets to simulate extended exposure.

result The proposed approach mitigates challenges in session-based recommender systems, as shown by computational experiments on various datasets.

session-based recommender systems user interest stochastic process latent space bias mitigation

2025-04-14abs ↗pdf ↗

Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecs.LG

Gradient clipping helps private SGD converge despite potential bias.

problem Gradient clipping in private SGD can bias convergence.

method Theoretical analysis and empirical evaluation of gradient clipping effects.

result Gradient clipping can prevent convergence to stationary points and introduces bias.

private SGD gradient clipping differential privacy convergence bias geometric perspective

2020-06-27abs ↗pdf ↗

A Constraint-Based Algorithm For Causal Discovery with Cycles, Latent Variables and Selection Biasstat.ML

Causal processes in nature may contain cycles, and real datasets may violate causal sufficiency as well as contain selection bias. No constraint-based causal discovery algorithm can currently handle cycles, latent variables and selection bias (CLS) simultaneously. I therefore introduce an algorithm called Cyclic Causal…

2018-05-05abs ↗pdf ↗

Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Datacs.LG

This study investigates how gradient-based methods bias neural networks trained on high-dimensional data.

problem The implicit biases of gradient-based optimization algorithms in neural networks trained on high-dimensional data.

method Investigation of gradient flow and gradient descent in two-layer fully-connected neural networks with leaky ReLU activations.

result Gradient flow and gradient descent lead to neural networks with low-rank solutions and linear decision boundaries.

deep learning neural networks gradient descent leaky ReLU high-dimensional data

2022-10-13abs ↗pdf ↗

On the influence of stochastic roundoff errors and their bias on the convergence of the gradient descent method with low-precision floating-point computationcs.LG

Gradient descent stagnates in low-precision, but unbiased rounding schemes improve convergence.

problem Stagnation of gradient descent in low-precision computation.

method Proposed unbiased stochastic rounding schemes that trade zero bias for larger probability of preserving small gradients.

result Unbiased rounding methods typically improve convergence rate of gradient descent for convex problems.

gradient descent low precision stochastic rounding stagnation convergence

2022-02-24abs ↗pdf ↗

Learning a Single Neuron with Bias Using Gradient Descentcs.LG

Gradient descent struggles with learning a single neuron with bias.

problem Learning a single neuron with a bias term in the realizable setting with ReLU activation.

method Theoretical study using gradient descent, characterizing critical points, and providing convergence guarantees.

result Gradient descent faces significant challenges in learning a single neuron with bias, unlike the bias-less case.

gradient descent single neuron ReLU activation bias term optimization geometry

2021-06-02abs ↗pdf ↗

The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networkscs.LG

Gradient flow in ReLU networks biases towards generalization but makes them vulnerable to adversarial attacks.

problem Generalization vs. Adversarial Robustness in ReLU Networks

method Analysis of gradient flow in two-layer ReLU networks with clustered data.

result Gradient flow biases towards generalization but also makes networks vulnerable to adversarial attacks.

ReLU networks gradient flow generalization adversarial robustness overparameterization

2023-03-02abs ↗pdf ↗

Breaking the Bias Barrier in Concave Multi-Objective Reinforcement Learningcs.LG

This work overcomes bias in concave multi-objective reinforcement learning.

problem Gradient bias in policy gradient methods for concave scalarized multi-objective reinforcement learning.

method Developed a Natural Policy Gradient (NPG) algorithm with a multi-level Monte Carlo (MLMC) estimator.

result Achieved optimal

\widetilde{\mathcal{O}}(ε^{-2})

sample complexity for computing an

ε

-optimal policy.

reinforcement learning multi-objective concave scalarization bias correction policy gradient

2026-03-09abs ↗pdf ↗

Randomised Splitting Methods and Stochastic Gradient Descentmath.OC

New minibatching strategy reduces stochastic gradient bias in optimisation.

problem Reducing bias in stochastic gradient descent.

method Symmetric Minibatching Strategy combined with momentum.

result Reduced stochastic gradient bias from

\mathcal{O}(h^2)

\mathcal{O}(h^4)

stochastic gradient descent minibatching momentum backward analysis splitting methods

2025-04-05abs ↗pdf ↗

Fair Generative Modeling via Weak Supervisioncs.LG

Real-world datasets are often biased with respect to key demographic factors such as race and gender. Due to the latent nature of the underlying factors, detecting and mitigating bias is especially challenging for unsupervised machine learning. We present a weakly supervised algorithm for overcoming dataset bias for de…

2019-10-26abs ↗pdf ↗

Robust and Information-theoretically Safe Bias Classifier against Adversarial Attackscs.LG

A bias classifier is introduced to resist adversarial attacks.

problem Resisting adversarial attacks on deep neural networks (DNNs).

method Introducing the bias part of a DNN with Relu as the activation function as a classifier, and adding a random first-degree part to make it information-theoretically safe.

result The bias classifier is more robust than DNNs of similar size against adversarial attacks.

bias classifier adversarial attacks information-theoretically safe deep neural networks ReLU activation

2021-11-08abs ↗pdf ↗

Semiparametric Efficient Bilevel Gradient Estimationstat.ML

Semiparametric method removes bias in functional bilevel gradient estimation.

problem First-order bias in plug-in hypergradient when lower-level problem is nonparametric.

method Semiparametric debiasing theory based on efficient influence function leads to cross-fitted orthogonal hypergradient estimator.

result Asymptotic normality and uniform control over outer parameter established for the estimator.

bilevel optimization semiparametric hypergradient bias correction efficient influence function

2026-05-20abs ↗pdf ↗

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizerscs.LG

Bias correction improves language model training performance.

problem Stochastic update bias in preconditioned optimizers.

method Cross-fitted preconditioning and variance-corrected inversion.

result Reduces held-out pretraining loss by 0.15 nats.

language models preconditioned optimizers stochastic bias gradient coupling inversion bias

2026-05-20abs ↗pdf ↗

NAS-X: Neural Adaptive Smoothing via Twistingcs.LG

NAS-X improves inference and model learning for SLVMs.

problem Challenges in analytic inference and model learning for flexible SLVMs.

method NAS-X combines reweighted wake-sleep and smoothing sequential Monte Carlo.

result NAS-X provides low-bias and low-variance gradient estimates.

Sequential latent variable models Neural adaptive smoothing Wake-sleep Sequential Monte Carlo Inference

2023-08-28abs ↗pdf ↗

Exploiting Observation Bias to Improve Matrix Completioncs.LG

Improves matrix completion by exploiting biased observation patterns.

problem Matrix completion with biased observation patterns.

method Mask Nearest Neighbor (MNN) algorithm: two-stage process.

result MNN achieves competitive performance with 28x smaller mean squared error.

matrix completion biased observation latent factors supervised learning finite-sample error rates

2023-06-07abs ↗pdf ↗

Unsupervised Learning of Hybrid Latent Dynamics: A Learn-to-Identify Frameworkcs.LG

Paper develops a framework to identify latent dynamics from high-dimensional data.

problem Identifying latent dynamics from high-dimensional time-series data.

method Combines physics inductive bias and learn-to-identify strategy.

result Meta-HyLaD framework effectively identifies hybrid latent dynamics.

unsupervised learning latent dynamics hybrid models physics inductive bias meta-learning

2024-03-13abs ↗pdf ↗

Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysismath.OC

New bounds show SGD can match deterministic gradient descent's convergence rate.

problem Optimizing SGD convergence rate under strong convexity and smoothness.

method Computer-aided Lyapunov analysis, focusing on bias-optimal bounds.

result SGD achieves optimal convergence rate in bias terms for a wide range of step-sizes.

SGD Gradient Descent Optimization Lyapunov Convexity

2025-05-23abs ↗pdf ↗

Deterministic Envelopes for Tamed SGLD: Decoupling Stochastic-Gradient Noise and Localizing Tamingstat.ML

Paper proposes a new method to stabilize noisy gradient algorithms.

problem Stochastic-gradient Langevin algorithms can introduce bias when taming denominators depend on stochastic-gradient realizations.

method Proposes a structure-preserving framework for designing tamed denominators that avoid unnecessary taming and maintain the stabilizing effect of taming.

result The method avoids stationary bias and explains the stationary error split into bias and remaining error.

stochastic-gradient Langevin algorithms taming stability bias

2026-06-03abs ↗pdf ↗

Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applicationsmath.NA

Bayesian sparse learning method improves deep neural network efficiency.

problem Sparse learning in deep neural networks with complex geometry.

method Preconditioned stochastic gradient Langevin Dynamics (PSGLD) for sampling and adaptive optimization of hyperparameters.

result The proposed algorithm achieves asymptotic convergence with controlled bias.

Bayesian learning sparse deep learning stochastic gradient Langevin dynamics preconditioning

2020-06-29abs ↗pdf ↗