On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,181 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “Lipschitz Regression”

LALR: Theoretical and Experimental validation of Lipschitz Adaptive Learning Rate in Regression and Neural Networkscs.LG

LALR adapts learning rate for faster convergence in regression and neural nets.

problem Finding optimal learning rates for faster convergence in regression and neural networks.

method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.

result Adaptive learning rate policy enables up to 20x faster convergence.

Lipschitz continuity adaptive learning rate Mean Absolute Error Quantile loss regression

2020-05-19abs ↗pdf ↗

Uniform Convergence with Square-Root Lipschitz Lossstat.ML

Guarantees uniform convergence for square-root Lipschitz losses.

problem Uniform convergence guarantees for square-root Lipschitz losses.

method Using Rademacher complexity and square root of scalar loss function Lipschitz constant.

result Generalizes previous results and handles non-smooth loss functions.

uniform convergence Rademacher complexity square-root Lipschitz phase retrieval ReLU regression

2023-06-22abs ↗pdf ↗

Non-asymptotic Oracle Inequalities for the High-Dimensional Cox Regression via Lassomath.ST

We consider the finite sample properties of the regularized high-dimensional Cox regression via lasso. Existing literature focuses on linear models or generalized linear models with Lipschitz loss functions, where the empirical risk functions are the summations of independent and identically distributed (iid) losses. T…

2012-04-09abs ↗pdf ↗

Near-optimal delta-convex estimation of Lipschitz functionsstat.ML

Paper presents an efficient algorithm for estimating Lipschitz functions from noisy data.

problem Estimating unknown Lipschitz functions from noisy observations.

method Extends max-affine methods to Lipschitz setting using nonlinear feature expansion and adaptive partitioning.

result Achieves minimax convergence rate with respect to intrinsic dimension, up to logarithmic factors.

Lipschitz functions estimation noisy data max-affine methods delta-convex functions

2025-11-19abs ↗pdf ↗

Sample Compression for Real-Valued Learnerscs.LG

Efficient algorithm for converting regression to compressed form.

problem Real-valued regression learning and compression.

method Extension of Moran and Yehudayoff's scheme to real-valued hypotheses.

result First general compressed regression result with uniform approximate reconstruction.

regression compression Lipschitz bounded variation weak learners

2018-05-21abs ↗pdf ↗

Improving evidential deep learning via multi-task learningcs.LG

Enhances ENet's prediction accuracy while maintaining uncertainty estimation.

problem Gradient shrinkage problem in ENet's loss function.

method Proposes a multi-task learning framework with a modified MSE loss function.

result Improves ENet's prediction accuracy without losing uncertainty estimation.

deep learning evidential regression multi-task learning gradient shrinkage uncertainty estimation

2021-12-17abs ↗pdf ↗

An Upper Bound of the Bias of Nadaraya-Watson Kernel Regression under Lipschitz Assumptionsstat.ML

Paper provides an upper bound for bias of Nadaraya-Watson kernel regression.

problem Estimating bias of Nadaraya-Watson kernel regression for finite bandwidths.

method Proposes an upper bound for bias under Lipschitz assumptions, extending to discontinuous derivatives and multidimensional domains.

result Upper bound on bias for finite bandwidths, tighter than previous infinitesimal bandwidth analysis.

kernel regression bias estimation Nadaraya-Watson Lipschitz continuity finite bandwidth

2020-01-29abs ↗pdf ↗

Convergence of Two-Layer Regression with Nonlinear Unitscs.LG

A new method for optimizing regression problems with ReLU units converges.

problem Optimizing regression problems involving ReLU units in large language models.

method Introduced a greedy algorithm based on approximate Newton method, proving convergence in terms of the distance to optimal solution.

result The method converges in the sense of the distance to optimal solution under certain assumptions.

ReLUs Regression Optimization Large Language Models Softmax

2023-08-16abs ↗pdf ↗

On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regressioneess.AS

The paper explores MAE as a loss function for DNN vector-to-vector regression, proving its advantages over MSE.

problem Improving loss function for deep neural network based vector-to-vector regression.

method Presenting performance bounds and new properties of MAE, deriving generalized upper bounds, and interpreting MAE as a Laplacian distribution.

result MAE is a more suitable loss function than MSE for DNN based vector-to-vector regression, especially when errors follow a Laplacian distribution.

mean absolute error deep neural networks vector-to-vector regression loss function Lipschitz continuity

2020-08-12abs ↗pdf ↗

Understanding Overfitting in Adversarial Training via Kernel Regressionstat.ML

The paper examines how adversarial training and noise affect neural network performance.

problem Overfitting in adversarial training and data augmentation.

method Adversarial training and data augmentation with noise in the context of regularized regression in RKHS.

result Appropriate regularization can prevent overfitting and improve performance.

Adversarial Training Data Augmentation Regularization Reproducing Kernel Hilbert Space Overfitting

2023-04-13abs ↗pdf ↗

Generalization bounds for regression and classification on adaptive covering input domainsstat.ML

The study examines generalization bounds for regression and classification tasks on adaptive input domains.

problem Understanding the generalization error in adaptive input domains for regression and classification.

method The analysis considers regression and classification separately, using Lipschitz continuity and 2-norm/0/1 loss for measurement. It also highlights the polynomial relationship between generalization bounds and network parameters.

result Generalization bounds for regression and classification are inversely proportional to a polynomial of the number of parameters, emphasizing the advantages of over-parameterized networks.

generalization bounds adaptive input domains regression classification Lipschitz continuity

2024-07-29abs ↗pdf ↗

CLIP: Cheap Lipschitz Training of Neural Networkscs.LG

CLIP controls neural network stability by bounding Lipschitz constants.

problem Neural networks lack mathematical guarantees of stability, especially to adversarial examples.

method Develops a variational regularization method (CLIP) to control the Lipschitz constant of neural networks.

result CLIP provides a tighter bound on the actual Lipschitz constant compared to layer-wise methods.

neural networks Lipschitz constant adversarial examples stability mathematical guarantees

2021-03-23abs ↗pdf ↗

On Lipschitz Continuity and Smoothness of Loss Functions in Learning to Rankcs.LG

In binary classification and regression problems, it is well understood that Lipschitz continuity and smoothness of the loss function play key roles in governing generalization error bounds for empirical risk minimization algorithms. In this paper, we show how these two properties affect generalization error bounds in …

2014-05-03abs ↗pdf ↗

Optimistic bounds for multi-output predictionstat.ML

Optimistic bounds for multi-output learning using self-bounding Lipschitz condition.

problem Learning vector-valued functions from supervised data.

method Introducing self-bounding Lipschitz condition and proving optimistic bounds using local Rademacher complexity and Srebro's inequality.

result Minimax optimal generalization bounds for multi-output learning, up to logarithmic factors.

multi-output learning optimistic bounds self-bounding Lipschitz local Rademacher complexity minoration inequality

2020-02-22abs ↗pdf ↗

Signal Recovery from Pooling Representationsstat.ML

In this work we compute lower Lipschitz bounds of $\ell_p$ pooling operators for $p=1, 2, \infty$ as well as $\ell_p$ pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm tha…

2013-11-16abs ↗pdf ↗

Dual optimization for convex constrained objectives without the gradient-Lipschitz assumptionstat.ML

Improves SDCA convergence for convex objectives with linear constraints.

problem Minimizing convex objectives with linear constraints under gradient-Lipschitz assumption failure.

method Shifted Stochastic Dual Coordinate Ascent (SDCA) under smoothness assumption.

result Obtains linear convergence rate for Poisson regression and Hawkes process objectives.

Convex optimization Stochastic optimization Dual problem SDCA Poisson regression

2018-07-10abs ↗pdf ↗

Fast Kernel Methods for Generic Lipschitz Losses via

p

-Sparsified Sketchesstat.ML

This paper speeds up kernel methods using sparsified Gaussian sketches.

problem Kernel methods' computational limitations.

method Sparsified Gaussian sketches for kernel methods.

result Efficient time and space savings for kernel methods.

kernel methods sketching Gaussian sketch Lipschitz loss decomposition trick

2022-06-08abs ↗pdf ↗

High-dimensional Measurement Error Models for Lipschitz Lossstat.ME

Develops high-dimensional measurement error models for non-linear loss functions.

problem Measurement errors in ultrahigh-dimensional biomedical data.

method Lipschitz loss functions, L1 norm minimization, Lasso analog.

result Improved accuracy in classification and quantile regression problems.

measurement error high-dimensional data Lipschitz loss L1 norm Lasso

2022-10-26abs ↗pdf ↗

Byzantine-Resilient Stochastic Gradient Descent for Distributed Learning: A Lipschitz-Inspired Coordinate-wise Median Approachcs.LG

New algorithm resists up to half of Byzantine workers in distributed learning.

problem Resilience of distributed SGD in the presence of Byzantine attackers.

method Lipschitz-inspired coordinate-wise median approach (LICM-SGD).

result LICM-SGD can resist up to half of Byzantine workers in non-convex settings.

Byzantine attacks distributed learning stochastic gradient descent coordinate-wise median Lipschitz function

2019-09-10abs ↗pdf ↗

Maximum Likelihood Estimation in Gaussian Process Regression is Ill-Posedmath.ST

Maximum likelihood estimation fails to be well-posed in Gaussian process regression.

problem Establishing well-posedness of maximum likelihood estimation in Gaussian process regression.

method Analyzing the conditions under which maximum likelihood estimation is not Lipschitz in the data with respect to the Hellinger distance.

result Maximum likelihood estimation is not well-posed in the noiseless data setting for any Gaussian process with a stationary covariance function whose lengthscale parameter is estimated using maximum likelihood.

Gaussian process regression maximum likelihood estimation well-posedness Lipschitz continuity Hellinger distance

2022-03-17abs ↗pdf ↗

Spatio-temporal model via Locally Adaptive Regression Splinesstat.ME

The paper develops a new method for estimating non-parametric regression functions with spatio-temporal dependencies.

problem Estimating non-parametric regression functions with spatio-temporal dependencies.

method Locally Adaptive Regression Splines (LARS) with ADMM algorithm.

result The method shows superior performance compared to existing techniques.

Locally Adaptive Regression Splines ADMM algorithm spatio-temporal dependencies non-parametric regression minimax optimality

2023-08-30abs ↗pdf ↗

A consistent deterministic regression tree for non-parametric prediction of time seriesmath.ST

We study online prediction of bounded stationary ergodic processes. To do so, we consider the setting of prediction of individual sequences and build a deterministic regression tree that performs asymptotically as well as the best L-Lipschitz constant predictors. Then, we show why the obtained regret bound entails the …

2014-05-07abs ↗pdf ↗

Standard Transformers Achieve the Minimax Rate in Nonparametric Regression with

C^{s,λ}

Targetsstat.ML

Standard Transformers approximate Hölder functions and achieve optimal nonparametric regression rate.

problem Approximating Hölder functions and achieving optimal nonparametric regression rate with Transformers.

method Using the size tuple and dimension vector metrics, the paper characterizes Transformer structures and derives upper bounds for their Lipschitz constant and memorization capacity.

result Standard Transformers achieve the minimax optimal rate in nonparametric regression for Hölder target functions.

Transformers Hölder functions Nonparametric regression Minimax rate Lipschitz constant

2026-02-24abs ↗pdf ↗

Universal consistency and minimax rates for online Mondrian Forestsstat.ML

Improved Mondrian Forests achieve optimal rate for online classification.

problem Consistency and minimax rates for online Mondrian Forests.

method Modified Mondrian Forest algorithm with increasing lifetime parameters and alternative updating rule.

result Achieves minimax rate for Lipschitz regression function estimation.

Mondrian Forests Online Learning Consistency Minimax Rate Lipschitz Regression

2017-11-08abs ↗pdf ↗

Nonparametric Regression with Shallow Overparameterized Neural Networks Trained by GD with Early Stoppingcs.LG

GD with early stopping trains shallow neural nets for nonparametric regression robustly.

problem Learning Lipschitz regression functions with noisy labels.

method Overparameterized shallow neural networks trained by GD with early stopping.

result Optimal rates of convergence for nonparametric regression.

Gradient Descent Early Stopping Lipschitz Functions Label Noise Nonparametric Regression

2021-07-12abs ↗pdf ↗

System Identification Through Lipschitz Regularized Deep Neural Networkscs.LG

The paper uses neural networks to learn system dynamics from data with Lipschitz regularization.

problem Learning governing equations from time-sampled data.

method Lipschitz regularized deep neural networks for ODE system identification.

result Lipschitz regularization improves the smoothness and generalization of the learned function.

neural networks system identification ODE Lipschitz regularization

2020-09-07abs ↗pdf ↗

On the Convergence of Irregular Sampling in Reproducing Kernel Hilbert Spacesstat.ML

Study on how sampling works for complex data functions.

problem Analyzing convergence of sampling algorithms for RKHS functions.

method Minimalistic assumptions on kernel and data, error estimates in RKHS norm, uniform convergence on compact domains.

result New convergence rates for Lipschitz and Hölder continuous kernels.

sampling RKHS kernel regression convergence Hilbert space

2025-04-18abs ↗pdf ↗

Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Modelsstat.ML

Study non-asymptotic bounds for robust estimators under misspecified models.

problem Evaluate performance of robust estimators under adversarial conditions.

method Propose a general approach to adversarial risk analysis, including investigations on generalization and approximation errors.

result Establish non-asymptotic upper bounds for adversarial excess risk under Lipschitz loss functions.

Adversarial Risk Robust Estimation Misspecified Models Generalization Error Approximation Error

2023-09-02abs ↗pdf ↗

SX-GeoTree: Self-eXplaining Geospatial Regression Tree Incorporating the Spatial Similarity of Feature Attributionscs.LG

SX-GeoTree improves spatially coherent explanations in geospatial regression trees.

problem Capturing spatial dependence and producing robust explanations in tabular prediction models.

method Integrates three objectives: impurity reduction, spatial residual control, and explanation robustness via modularity maximization on a consensus similarity network.

result Improves residual spatial evenness and doubles attribution consensus (modularity: Fujian 0.19 vs 0.09; Seattle 0.10 vs 0.05).

geospatial regression spatial similarity explainable AI robustness modularity maximization

2025-11-25abs ↗pdf ↗

Minimax Analysis for Inverse Risk in Nonparametric Planer Invertible Regressionmath.ST

Study on estimating invertible functions with minimax analysis.

problem Minimizing risk of estimating invertible functions on a plane.

method Introduce two types of

L^2

-risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator.

result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.

minimax analysis inverse risk nonparametric regression invertible functions bi-Lipschitz functions

2021-12-01abs ↗pdf ↗

Uniform Error Bounds for Gaussian Process Regression with Application to Safe Controlcs.LG

Paper derives uniform error bounds for Gaussian process regression for safer control applications.

problem Quantifying model error in Gaussian process regression for safety-critical applications.

method Employing Gaussian process distribution and continuity arguments, derive uniform error bounds under weaker assumptions.

result Derives novel uniform error bounds for Gaussian process regression under weaker assumptions.

Gaussian process uniform error bounds safe control continuity arguments asymptotic behavior

2019-06-04abs ↗pdf ↗

Feature Starvation as Geometric Instability in Sparse Autoencoderscs.LG

AEN-SAEs address feature starvation in sparse autoencoders by stabilizing the geometric alignment of sparse coding.

problem Feature starvation in sparse autoencoders, leading to unstable and misaligned representations.

method Adaptive Elastic Net SAEs (AEN-SAEs) combine

\ell_2

and

\ell_1

terms to stabilize the sparse coding map and control feature interactions.

result AEN-SAEs mitigate feature starvation without heuristic resampling, maintaining competitive reconstruction abilities.

sparse autoencoders feature starvation geometric instability sparse regression Lipschitz continuity

2026-05-06abs ↗pdf ↗

Uniform Error and Posterior Variance Bounds for Gaussian Process Regression with Application to Safe Controlcs.LG

This paper addresses error bounds and posterior variance for Gaussian process regression.

problem Deriving performance guarantees for Gaussian process regression without prior knowledge.

method Lipschitz continuity and analysis of posterior variance function.

result Uniform error bounds for Gaussian process regression are derived.

Gaussian processes error bounds posterior variance safe control Lipschitz continuity

2021-01-13abs ↗pdf ↗

Agnostic Learnability of Halfspaces via Logistic Losscs.LG

New research shows logistic regression can achieve optimal error rate for agnostic learning of halfspaces.

problem Agnostic learning of homogeneous halfspaces with logistic loss.

method Constructing a well-behaved distribution and using logistic regression with additional convex optimization steps.

result Logistic regression can achieve

Ω(\sqrt{ extrm{OPT}})

misclassification risk, matching the upper bound.

Logistic Regression Agnostic Learning Halfspaces Optimization Misclassification Risk

2022-01-31abs ↗pdf ↗

Wavelet Scattering Regression of Quantum Chemical Energiesmath.CA

We introduce multiscale invariant dictionaries to estimate quantum chemical energies of organic molecules, from training databases. Molecular energies are invariant to isometric atomic displacements, and are Lipschitz continuous to molecular deformations. Similarly to density functional theory (DFT), the molecule is re…

2016-05-16abs ↗pdf ↗

Learning Prediction Intervals for Regression: Generalization and Calibrationstat.ML

The paper improves methods for generating prediction intervals in regression.

problem Uncertainty quantification in regression models.

method Formalizes prediction interval generation as an optimization problem, studying generalization and calibration.

result Empirical demonstration of improved testing performances compared to existing methods.

prediction intervals regression uncertainty quantification optimization calibration

2021-02-26abs ↗pdf ↗

Piecewise Linear Regression via a Difference of Convex Functionsstat.ML

New method uses DC functions for piecewise linear regression.

problem Regression with piecewise linear constraints.

method Estimates piecewise linear convex functions using a difference of convex functions.

result Method achieves close to minimax statistical risk and comparable performance to existing methods.

piecewise linear regression difference of convex functions DC functions quadratic programming minimax risk

2020-07-05abs ↗pdf ↗

Learning Neural Networks with Two Nonlinear Layers in Polynomial Timecs.DS

Algorithm learns neural networks with two layers in polynomial time.

problem Learning neural networks with two nonlinear layers without assumptions.

method Isotonic regression combined with kernel methods.

result First provably efficient algorithm for two-layer neural networks.

Neural Networks Polynomial Time Isotonic Regression Kernel Methods Lipschitz Functions

2017-09-18abs ↗pdf ↗

Sample Complexity of Scientific Discovery: PAC Learnability of Compositional Function Treescs.LG

This paper shows that scientific discovery can be efficiently learned via compositional function trees, reducing the sample complexity.

problem Statistical and computational intractability of scientific discovery via symbolic regression.

method PAC learning approach focusing on compositional function trees built from a finite vocabulary of smooth operators.

result The Rademacher complexity and excess risk are controlled by depth and Lipschitz constants of the base operators, leading to finite-union bounds and high-probability risk bounds.

scientific discovery PAC learning compositional function trees symbolic regression Rademacher complexity

2026-06-28abs ↗pdf ↗

Representing smooth functions as compositions of near-identity functions with implications for deep network optimizationcs.LG

Deep networks can approximate smooth functions by compositions of nearly identity functions.

problem Optimizing deep networks for smooth function approximation.

method Representing smooth functions as compositions of near-identity functions with decreasing Lipschitz constants.

result Functional gradient methods for residual networks avoid suboptimal critical points in the near-identity region.

deep learning smooth functions composition near-identity functions gradient methods

2018-04-13abs ↗pdf ↗

Fair Regression: Quantitative Definitions and Reduction-based Algorithmscs.LG

The paper introduces fair regression methods to predict real-valued outcomes while ensuring fairness.

problem Predicting real-valued outcomes while ensuring fairness with respect to protected attributes.

method Proposes schemes for fair regression under statistical parity and bounded group loss, applicable to various losses.

result The schemes provide theoretical guarantees on the optimality and fairness of the obtained solutions.

fairness regression protected attributes statistical parity bounded group loss

2019-05-30abs ↗pdf ↗

Bayesian Inference for Consistent Predictions in Overparameterized Nonlinear Regressionstat.ML

Bayesian method improves predictions in overparameterized nonlinear regression.

problem Understanding overparameterization in nonlinear regression models.

method Bayesian framework with adaptive prior considering data spectral structure.

result Posterior contraction established for generalized linear and single-neuron models, demonstrating prediction consistency.

Bayesian inference overparameterization nonlinear regression spectral structure posterior contraction

2024-04-06abs ↗pdf ↗

Towards A Unified Analysis of Random Fourier Featuresstat.ML

Unified analysis improves random Fourier features for kernel methods.

problem Pessimistic theoretical bounds on random Fourier features.

method Unified risk analysis for squared error and Lipschitz loss.

result Improved bounds on number of features for convergence.

Random Fourier Features Kernel Methods Risk Analysis Lipschitz Loss Ridge Regression

2018-06-24abs ↗pdf ↗

Private Isotonic Regressioncs.LG

Paper develops DP algorithms for isotonic regression over posets.

problem Differential privacy in isotonic regression over partially ordered sets.

method Developed pure-DP and near-matching lower bound algorithms for isotonic regression.

result Achieved near-matching bounds for isotonic regression with and without poset structure.

Differential Privacy Isotonic Regression Partially Ordered Sets Lower Bounds Approximate-DP

2022-10-27abs ↗pdf ↗

Concentration inequalities for leave-one-out cross validationmath.ST

The paper proves LOO CV is reliable under estimator stability.

problem Ensuring the reliability of leave-one-out cross validation.

method Using concentration inequalities based on logarithmic Sobolev inequality.

result LOO CV is a valid procedure under estimator stability.

leave-one-out cross validation estimator stability concentration inequalities logarithmic Sobolev inequality

2022-11-04abs ↗pdf ↗

Lipschitz neural networks are dense in the set of all Lipschitz functionsstat.ML

Neural networks are dense among Lipschitz functions with fixed Lipschitz constant.

problem Characterizing neural network approximations to Lipschitz functions.

method Analyzing

L

-Lipschitz neural networks and their density in

L

-Lipschitz functions.

result One layer neural networks are dense in the set of all

L

-Lipschitz functions.

Lipschitz functions neural networks density uniform norm

2020-09-29abs ↗pdf ↗

Lipschitz maps with prescribed local Lipschitz constantsmath.DG

Constructs a map with prescribed local Lipschitz constants on a subset of a manifold.

problem Creating a Lipschitz map with specific local Lipschitz constants on a subset of a manifold.

method Constructs a Lipschitz map that matches a given map on a subset and has a local Lipschitz constant defined by a continuous function.

result A Lipschitz map can be constructed with a local Lipschitz constant prescribed by a continuous function.

Lipschitz map local Lipschitz constant Riemannian manifold subset continuous function

2024-03-12abs ↗pdf ↗

Underdamped Langevin MCMC with third order convergencestat.ML

New Langevin method achieves third order convergence for strongly log-concave distributions.

problem Sampling from complex distributions efficiently.

method Underdamped Langevin diffusion with third order convergence.

result Achieves 2-Wasserstein error of ε in O(√d/ε^1/3) steps under additional Lipschitz condition.

Langevin MCMC Underdamped Langevin Strongly log-concave Third order convergence Wasserstein distance

2025-08-22abs ↗pdf ↗