On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “gradient interpolation”

On the Oracle Complexity of Interpolation-Based Gradient Descentcs.LG

Piecewise polynomial interpolation-based gradient descent reduces oracle complexity for smooth loss functions.

problem Optimizing empirical risk minimization loss functions

method Piecewise polynomial interpolation-based gradient descent

result Oracle complexity is reduced for smooth loss functions

oracle complexity empirical risk minimization gradient descent piecewise polynomial interpolation

2026-06-18abs ↗pdf ↗

Gradient flow in parameter space is equivalent to linear interpolation in output spacecs.LG

Gradient flow in parameters equals linear interpolation in outputs.

problem Understanding and optimizing training algorithms in deep learning.

method Proving equivalence between gradient flow in parameter space and linear interpolation in output space, and deriving formulas for global minima.

result Gradient flow in parameters can be transformed into linear interpolation in outputs, leading to global minima.

gradient flow parameter space output space linear interpolation deep learning

2024-08-02abs ↗pdf ↗

Improved Complexities for Stochastic Conditional Gradient Methods under Interpolation-like Conditionsmath.OC

Improved complexity for machine learning optimization methods.

problem Optimizing over-parametrized models in machine learning.

method Stochastic conditional gradient methods with interpolation-like conditions.

result Improved oracle complexities for finding optimal solutions.

machine learning optimization stochastic gradient conditional gradient interpolation

2020-06-15abs ↗pdf ↗

When does gradient descent with logistic loss find interpolating two-layer networks?stat.ML

Gradient descent with logistic loss can make two-layer networks interpolate binary classification data.

problem Training two-layer networks for binary classification.

method Gradient descent with logistic loss applied to two-layer networks.

result Gradient descent can drive training loss to zero under certain conditions.

gradient descent logistic loss two-layer networks binary classification interpolating networks

2020-12-04abs ↗pdf ↗

A first-order condition for the independence on

p

of weak gradientsmath.MG

Paper investigates conditions for independence of weak gradients on metric spaces.

problem Dependence of weak gradients on

p

in arbitrary metric measure spaces.

method Investigates the Bounded Interpolation Property to ensure independence of weak gradients.

result Bounded Interpolation Property guarantees independence of weak gradients.

weak gradients metric measure spaces Bounded Interpolation Property curvature dimension conditions

2021-12-23abs ↗pdf ↗

When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations?stat.ML

Gradient descent with logistic loss can interpolate deep networks with smoothed ReLU activations under certain conditions.

problem Conditions for gradient descent to drive logistic loss to zero in deep networks with smoothed ReLU activations.

method Gradient descent applied to fixed-width deep networks with smoothed ReLU approximations (e.g., Swish, Huberized ReLU).

result Gradient descent can drive logistic loss to zero under specific conditions, providing bounds on convergence rate.

gradient descent logistic loss deep networks smoothed ReLU convergence

2021-02-09abs ↗pdf ↗

Interpolation Technique to Speed Up Gradients Propagation in Neural ODEscs.NE

A new interpolation method speeds up neural ODE training.

problem Efficiently approximating gradients in neural ODEs.

method Interpolation-based technique to approximate gradients.

result Our method trains neural ODEs faster than the reverse dynamic method.

neural ODEs gradient approximation neural networks logarithmic norm

2020-03-11abs ↗pdf ↗

Deep Linear Networks can Benignly Overfit when Shallow Ones Docs.LG

Deep linear networks can closely approximate interpolants without improving risk.

problem Understanding the risk bounds of deep linear networks compared to minimum

\ell_2

-norm solutions.

method Bounding excess risk of interpolating deep linear networks trained using gradient flow.

result Deep linear networks can closely approximate or match minimum

\ell_2

-norm solutions in terms of risk.

deep learning linear networks gradient flow interpolation risk bounds

2022-09-19abs ↗pdf ↗

On Optimal Interpolation In Linear Regressionstat.ML

Paper investigates optimal interpolation methods in linear regression.

problem Understanding when interpolating methods generalize well in linear regression.

method Investigates optimal response-linear interpolators using functions linear in the response variable.

result Provides a closed-form expression for the optimal interpolator and shows it can be derived as the limit of gradient descent.

linear regression interpolation optimality gradient descent closed-form

2021-10-21abs ↗pdf ↗

Escaping Saddle-Points Faster under Interpolation-like Conditionsstat.ML

Paper shows faster convergence to local-minimizers in over-parametrized models under interpolation-like conditions.

problem Escaping saddle-points in over-parametrized models.

method Stochastic and deterministic optimization algorithms under interpolation-like conditions.

result Oracle complexity of PSGD and SCRN algorithms to reach

ε

-local-minimizer matches or improves upon deterministic rates.

over-parametrization saddle-points local-minimizers stochastic optimization interpolation-like

2020-09-28abs ↗pdf ↗

Zero Generalization Error Theorem for Random Interpolators via Algebraic Geometrycs.LG

Randomly sampled interpolators achieve zero generalization error with enough data.

problem Understanding the high generalization ability of machine learning models.

method Algebraic geometry tools to prove zero generalization error for random interpolators.

result Generalization error of randomly sampled interpolators becomes zero once the number of training samples exceeds a geometric threshold.

generalization error random interpolators algebraic geometry teacher-student framework zero error

2025-12-06abs ↗pdf ↗

Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)cs.LG

Adaptive gradient methods converge faster with over-parameterization and line-search.

problem Training over-parameterized models using adaptive gradient methods.

method Simplified setting of smooth, convex losses with over-parameterized models, proving convergence rates and demonstrating improvements with line-search techniques.

result Adaptive gradient methods, particularly AMSGrad, converge faster with line-search techniques.

adaptive gradient methods over-parameterization line-search convergence convex optimization

2020-06-11abs ↗pdf ↗

Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Ratescs.LG

Recent works have shown that stochastic gradient descent (SGD) achieves the fast convergence rates of full-batch gradient descent for over-parameterized models satisfying certain interpolation conditions. However, the step-size used in these works depends on unknown quantities and SGD's practical performance heavily re…

2019-05-24abs ↗pdf ↗

Data Interpolants -- That's What Discriminators in Higher-order Gradient-regularized GANs Arestat.ML

Optimized GAN discriminator using polyharmonic interpolation.

problem Optimizing the discriminator in GANs with higher-order gradient regularization.

method Polyharmonic interpolation and variational calculus.

result The optimal discriminator is a polyharmonic radial basis function.

GANs Polyharmonic Interpolation Optimization Discriminator

2023-06-01abs ↗pdf ↗

Fast Convergence for Stochastic and Distributed Gradient Descent in the Interpolation Limitstat.ML

Modern supervised learning techniques, particularly those using deep nets, involve fitting high dimensional labelled data sets with functions containing very large numbers of parameters. Much of this work is empirical. Interesting phenomena have been observed that require theoretical explanations; however the non-conve…

2018-03-08abs ↗pdf ↗

Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Datacs.LG

Neural networks can interpolate noisy data and still generalize well.

problem Generalization of neural networks trained on noisy data.

method Two-layer neural networks trained to interpolation by gradient descent on corrupted labels.

result Neural networks can achieve zero training error and optimal test error.

benign overfitting neural networks gradient descent noisy data logistic loss

2022-02-11abs ↗pdf ↗

Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Biascs.LG

Neural networks learn incrementally from orthogonal data, interpolating with minimal complexity.

problem Understanding the learning dynamics and implicit bias in ReLU networks with orthogonal data.

method Gradient flow analysis of two-layer ReLU networks from small initialization with orthogonal training data.

result The learned interpolator has a squared

\ell_2

-norm scaling as

\sqrt{n}

, close to the minimal interpolator's complexity.

ReLU networks gradient flow orthogonal data incremental learning implicit bias

2026-05-26abs ↗pdf ↗

Aiming towards the minimizers: fast convergence of SGD for overparametrized problemscs.LG

Stochastic gradient method converges as fast as deterministic for overparametrized models.

problem Convergence rate of stochastic gradient methods in overparametrized models.

method Proposes a regularity condition enabling fast convergence of SGD.

result Stochastic gradient method achieves the same convergence rate as deterministic gradient method.

stochastic gradient descent overparametrized models fast convergence regularity condition neural networks

2023-06-05abs ↗pdf ↗

Effectiveness of Distributed Gradient Descent with Local Steps for Overparameterized Modelscs.LG

Local Gradient Descent with local steps converges to the centralized model in the interpolation regime.

problem Understanding the implicit bias of Local Gradient Descent in the interpolation regime.

method Analyzing the implicit bias of Local Gradient Descent for classification tasks with linearly separable data.

result The aggregated global model from Local-GD converges exactly to the centralized model in the interpolation regime.

Local Gradient Descent Federated Learning Implicit Bias Classification Linear Separability

2024-12-10abs ↗pdf ↗

Implicit Regularization Leads to Benign Overfitting for Sparse Linear Regressioncs.LG

New model leads to optimal test loss in sparse linear regression.

problem Sparse linear regression with low test loss despite interpolating training data.

method Developed a new parametrization of the model that combines benefits of ℓ1 and ℓ2 norms.

result Training via gradient descent leads to an interpolator with near-optimal test loss.

sparse linear regression benign overfitting implicit regularization gradient descent test loss

2023-02-01abs ↗pdf ↗

Gradient Dynamics of Shallow Univariate ReLU Networkscs.LG

We present a theoretical and empirical study of the gradient dynamics of overparameterized shallow ReLU networks with one-dimensional input, solving least-squares interpolation. We show that the gradient dynamics of such networks are determined by the gradient flow in a non-redundant parameterization of the network fun…

2019-06-18abs ↗pdf ↗

Continuous Treatment Effect Estimation Using Gradient Interpolation and Kernel Smoothingcs.LG

Estimates individual treatment effects using gradient interpolation and kernel smoothing.

problem Estimating individualized continuous treatment effects in observational data.

method Augment training data with independently sampled treatments and inferred counterfactual outcomes using gradient interpolation and kernel smoothing.

result Our method outperforms state-of-the-art methods on counterfactual estimation error.

continuous treatment effects gradient interpolation kernel smoothing observational data individualized treatment effects

2024-01-27abs ↗pdf ↗

On Acceleration of Gradient-Based Empirical Risk Minimization using Local Polynomial Regressionmath.OC

Accelerates ERM problems with LPI-GD and improved oracle complexity.

problem Empirical Risk Minimization (ERM) problems with strong convexity and smoothness.

method Local Polynomial Interpolation-based Gradient Descent (LPI-GD) and accelerated methods.

result Oracle complexity improved to $ ilde{O}\left(\sqrtσ m^d \log(1/\varepsilon) ight)$.

Empirical Risk Minimization Gradient Descent Local Polynomial Interpolation Strong Convexity Smoothness

2022-04-16abs ↗pdf ↗

Private optimization in the interpolation regime: faster rates and hardness resultscs.LG

Private optimization faster on interpolation problems with quadratic growth.

problem Private optimization in interpolation problems.

method Adaptive algorithm with improved sample complexity.

result Exponential improvement in private sample complexity for quadratic growth.

private optimization interpolation problems quadratic growth adaptive algorithm sample complexity

2022-10-31abs ↗pdf ↗

Fast Generalization after Interpolation via Critically Damped Momentum Optimizationcs.LG

New optimization method helps models generalize better after achieving near-perfect training performance.

problem Models can achieve near-perfect training performance but fail to generalize well to unseen examples.

method GROKtimizer combines rapid convergence to interpolation with post-interpolation norm minimization using Critically Damped Momentum.

result GROKtimizer provides a quadratic speedup over classical gradient descent, offering a natural solution for selecting low-norm interpolating solutions.

optimization machine learning generalization interpolation momentum

2026-06-01abs ↗pdf ↗

Scaling Gaussian Process Regression with Full Derivative Observationscs.LG

DSoftKI scales GP regression with full derivative observations.

problem Efficiently fitting and predicting full derivative observations in Gaussian Processes.

method Extends SoftKI by using local temperature vectors for interpolation, enabling encoding of local directional sensitivity.

result DSoftKI achieves accurate predictions and scales to larger datasets with full derivative observations.

Gaussian Processes Derivative Observations SoftKI Kernel Interpolation Deep Kernel Learning

2025-05-14abs ↗pdf ↗

General Loss Functions Lead to (Approximate) Interpolation in High Dimensionsstat.ML

Unified framework approximates gradient descent's implicit bias in high dimensions.

problem Understanding gradient descent's behavior in overparameterized settings with convex losses.

method Unified framework for convex losses, including sensitivity analysis.

result Approximation of minimum-norm interpolation in high dimensions.

gradient descent overparameterized convex losses implicit bias sensitivity analysis

2023-03-13abs ↗pdf ↗

On the Double Descent of Random Features Models Trained with SGDstat.ML

Study on RF regression with SGD shows double descent phenomenon.

problem Understanding generalization in RF models trained with SGD.

method Precise non-asymptotic error bounds derived for RF regression under constant and polynomial-decay step-size SGD.

result RF regression generalizes well for interpolation learning and exhibits double descent behavior.

Random Features SGD Generalization Double Descent Interpolation Learning

2021-10-13abs ↗pdf ↗

Sharp Generalization for Nonparametric Regression in Interpolation Space by Over-Parameterized Neural Networks Trained with Preconditioned Gradient Descent and Early Stoppingstat.ML

Neural networks trained with PGD achieve sharp regression rates in interpolation spaces.

problem Nonparametric regression using over-parameterized neural networks in interpolation spaces.

method Over-parameterized two-layer neural networks trained with Preconditioned Gradient Descent (PGD) and early stopping.

result Achieves a sharp regression rate of $\cO(n^{-\frac{2αs'}{2αs'+1}})$ in interpolation spaces $\bth{\cH_K}^{s'}$.

Neural Networks Regression Over-parameterization Gradient Descent Interpolation Spaces

2024-07-16abs ↗pdf ↗

Minimizing

f

-Divergences by Interpolating Velocity Fieldsstat.ML

New method estimates velocity fields for minimizing $f$ -divergences without overfitting.

problem Minimizing statistical discrepancies between target and particle distributions.

method Directly estimate velocity fields using interpolation techniques, proving consistency under mild conditions.

result Consistent estimators of velocity fields improve accuracy in applications like domain adaptation and missing data imputation.

Wasserstein Gradient Flow f-divergences density ratio non-parametric curve fitting velocity fields

2023-05-24abs ↗pdf ↗

Gaussian Process Manifold Interpolation for Probabilistic Atrial Activation Maps and Uncertain Conduction Velocitystat.ML

GPMI method interpolates uncertain atrial conduction velocity on non-Euclidean manifolds.

problem Uncertainty in atrial conduction velocity calculations.

method Gaussian Process Manifold Interpolation (GPMI) on human atrial manifolds.

result GPMI accounts for atrial topology and calculates CV uncertainty.

Gaussian process manifold interpolation atrial fibrillation conduction velocity uncertainty

2020-04-22abs ↗pdf ↗

PAC-Chernoff Bounds: Understanding Generalization in the Interpolation Regimecs.LG

This paper introduces a new bound to explain generalization in over-parameterized models.

problem Understanding why some over-parameterized models generalize well while others do not.

method PAC-Chernoff bounds and smoothness measures based on large deviation theory.

result Interpolators with smoother structures generalize better, according to the new theoretical framework.

PAC-Chernoff bounds smoothness generalization over-parameterized models interpolation

2023-06-19abs ↗pdf ↗

The linear trace Harnack quadratic on a steady gradient Ricci soliton satisfies the heat equationmath.DG

We show that the linear trace Harnack quadratic on a steady gradient Ricci soliton satisfies the heat equation. Similar result holds for shrinkers. We also present an interpolation between Perelman's and Cao--Hamilton's Harnacks on a steady soliton.

2011-05-17abs ↗pdf ↗

Convexifying Sparse Interpolation with Infinitely Wide Neural Networks: An Atomic Norm Approachcs.LG

This study uses neural networks to solve interpolation problems with sparse, infinitely wide layers.

problem Exact data interpolation using sparse, infinitely wide neural networks.

method Atomic norm framework to derive convex hulls and equivalent convex formulations.

result Simple characterizations of convex hulls for different constraints on network weights and biases.

sparse neural networks interpolation atomic norm leaky rectified linear units convex hulls

2020-07-15abs ↗pdf ↗

Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regimecs.LG

SGD achieves near optimal convergence rate in smooth interpolation regime.

problem Optimization of smooth convex objectives with zero noise at optimum.

method Stochastic Gradient Descent (SGD) with large stepsize analysis.

result Last iterate of SGD achieves expected excess risk of O(1/T + σ* / √T) with optimal stepsize.

SGD Stochastic Gradient Descent Optimization Smooth Convex Interpolation Regime

2025-07-15abs ↗pdf ↗

Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Timecs.LG

A simple GI loss improves temporal generalization without complex methods.

problem Temporal drift between train and test distributions in evolving data.

method Gradient Interpolation (GI) loss to regularize temporal complexity.

result GI loss outperforms complex methods on real-world datasets.

temporal generalization gradient interpolation machine learning time drift model drift

2021-08-15abs ↗pdf ↗

Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernelstat.ML

New stability bounds for GD in overparameterised shallow nets without NTK assumptions.

problem Generalisation and excess risk bounds for shallow neural networks.

method Oracle inequalities and stability analysis of GD without kernelisation.

result Oracle type bounds reveal GD's generalisation is controlled by an interpolating network with shortest GD path.

Gradient Descent Neural Networks Stability Generalisation Regression

2021-07-27abs ↗pdf ↗

Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networksstat.ML

The paper connects flatness to generalization in learning multi-index models with neural networks.

problem Understanding the generalization of non-convex neural networks using flatness measures.

method Analyzes 2-layer non-convex homogeneous neural networks and their connection to multi-index models.

result Flattest interpolators achieve small population loss and generalize well, establishing a direct link between flatness and generalization.

flatness generalization multi-index models homogeneous neural networks non-convex

2026-06-03abs ↗pdf ↗

Training Neural Networks for and by Interpolationcs.LG

In modern supervised learning, many deep neural networks are able to interpolate the data: the empirical loss can be driven to near zero on all samples simultaneously. In this work, we explicitly exploit this interpolation property for the design of a new optimization algorithm for deep learning, which we term Adaptive…

2019-06-13abs ↗pdf ↗

Merging Deterministic Policy Gradient Estimations with Varied Bias-Variance Tradeoff for Effective Deep Reinforcement Learningcs.LG

Deep reinforcement learning (DRL) on Markov decision processes (MDPs) with continuous action spaces is often approached by directly training parametric policies along the direction of estimated policy gradients (PGs). Previous research revealed that the performance of these PG algorithms depends heavily on the bias-var…

2019-11-24abs ↗pdf ↗

On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernelsmath.ST

We study the risk of minimum-norm interpolants of data in Reproducing Kernel Hilbert Spaces. Our upper bounds on the risk are of a multiple-descent shape for the various scalings of $d = n^α$ , $α\in(0,1)$ , for the input dimension $d$ and sample size $n$ . Empirical evidence supports our finding that minimum-norm interpo…

2019-08-27abs ↗pdf ↗

Interpolating Between Softmax Policy Gradient and Neural Replicator Dynamics with Capped Implicit Explorationcs.LG

Interpolates between SPG and NeuRD with Capped Implicit Exploration.

problem Combining SPG and NeuRD for better performance in non-stationary environments.

method Introduces Capped Implicit Exploration (CIX) to interpolate between SPG and NeuRD.

result NeuRD-CIX performs well more consistently than NeuRD while retaining NeuRD's advantages.

Neural Replicator Dynamics Softmax Policy Gradient Implicit Exploration Monte Carlo Updates Adversarial Bandit

2022-06-04abs ↗pdf ↗

The limitation of neural nets for approximation and optimizationcs.LG

Study assesses neural nets for optimization problems, highlighting SiLU's effectiveness.

problem Using neural nets for optimization problems, especially for accurate approximations.

method Determined best activation function (SiLU) for nonlinear optimization problems. Analyzed function approximations using neural networks and interpolation/regression models.

result Neural nets can deliver competitive zero- and first-order approximations but underperform on second-order approximations.

neural networks optimization activation functions SiLU approximation

2023-11-21abs ↗pdf ↗

A Call to Lagrangian Action: Learning Population Mechanics from Temporal Snapshotscs.LG

New method learns population dynamics from snapshots, outperforming existing models.

problem Capturing periodic and other dynamical properties of population dynamics.

method Wasserstein Lagrangian Mechanics (WLM) for learning second-order dynamics from observed marginals.

result WLM outperforms existing methods across various dynamics, including vortex dynamics, embryonic development, and flocking.

population dynamics Wasserstein Lagrangian Mechanics second-order dynamics learning from marginals forecasting

2026-05-08abs ↗pdf ↗

Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapescs.LG

Monotonic Linear Interpolation property in neural networks persists despite non-convexity.

problem Understanding the geometric properties of neural network loss landscapes.

method Tools from differential geometry to analyze the monotonicity of neural network weights.

result Sufficient conditions for the Monotonic Linear Interpolation property under mean squared error.

neural networks loss landscapes monotonicity SGD differential geometry

2021-04-22abs ↗pdf ↗

Generalization and Stability of Interpolating Neural Networks with Minimal Widthstat.ML

Gradient descent on shallow neural networks achieves near-optimal generalization error.

problem Optimizing shallow neural networks with minimal width for generalization and stability.

method Gradient descent in the interpolating regime with minimal width.

result Gradient descent achieves near-optimal generalization error with minimal width.

Neural Networks Gradient Descent Interpolating Models Generalization Stability

2023-02-18abs ↗pdf ↗

Linear interpolation gives better gradients than Gaussian smoothing in derivative-free optimizationmath.OC

In this paper, we consider derivative free optimization problems, where the objective function is smooth but is computed with some amount of noise, the function evaluations are expensive and no derivative information is available. We are motivated by policy optimization problems in reinforcement learning that have rece…

2019-05-29abs ↗pdf ↗

Optimizing Black-box Metrics with Adaptive Surrogatescs.LG

Optimizes hard-to-optimize metrics using adaptive surrogates.

problem Training models with black-box and hard-to-optimize metrics.

method Expresses metric as a function of surrogates, solves optimization problem over relaxed surrogate space.

result Approach performs on par with known methods and adds value when metric form is unknown.

black-box metrics adaptive surrogates optimization finite differences local linear interpolations

2020-02-20abs ↗pdf ↗