On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,694 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “Lipschitz loss”

Uniform Convergence with Square-Root Lipschitz Lossstat.ML

Guarantees uniform convergence for square-root Lipschitz losses.

problem Uniform convergence guarantees for square-root Lipschitz losses.

method Using Rademacher complexity and square root of scalar loss function Lipschitz constant.

result Generalizes previous results and handles non-smooth loss functions.

uniform convergence Rademacher complexity square-root Lipschitz phase retrieval ReLU regression

2023-06-22abs ↗pdf ↗

LALR: Theoretical and Experimental validation of Lipschitz Adaptive Learning Rate in Regression and Neural Networkscs.LG

LALR adapts learning rate for faster convergence in regression and neural nets.

problem Finding optimal learning rates for faster convergence in regression and neural networks.

method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.

result Adaptive learning rate policy enables up to 20x faster convergence.

Lipschitz continuity adaptive learning rate Mean Absolute Error Quantile loss regression

2020-05-19abs ↗pdf ↗

Unconstrained Online Learning with Unbounded Lossescs.LG

New algorithms for online learning without boundedness or Lipschitz loss assumptions.

problem Online learning with unbounded domains and non-Lipschitz losses.

method Developed an algorithm with a specific regret bound and used it for saddle-point optimization.

result First algorithm achieving non-trivial dynamic regret in an unbounded domain for non-Lipschitz losses.

online learning unbounded domains non-Lipschitz losses saddle-point optimization dynamic regret

2023-06-08abs ↗pdf ↗

Private Stochastic Optimization With Large Worst-Case Lipschitz Parametercs.LG

Improved DP SO with large Lipschitz parameters, handling outliers and heavy-tailed data.

problem Differential privacy in stochastic optimization with large Lipschitz parameters.

method Assumes bounded k-th order moments, provides linear-time algorithms for smooth convex and non-smooth convex losses.

result Improved risk bounds scaling with k-th moment, not uniform Lipschitz parameter.

Differential Privacy Stochastic Optimization Lipschitz Continuity Outliers Heavy-Tailed Data

2022-09-15abs ↗pdf ↗

Certified Robust Models with Slack Control and Large Lipschitz Constantscs.LG

Improved robustness of machine learning models with controlled Lipschitz constants.

problem Vulnerability of state-of-the-art models to adversarial attacks.

method Proposes a CLL loss that calibrates the margin and Lipschitz constant penalties, improving robustness certificates.

result Consistently outperforms other losses on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets.

Adversarial Attacks Certifiable Robustness Lipschitz Regularization Margin Loss Machine Learning

2023-09-12abs ↗pdf ↗

Improving evidential deep learning via multi-task learningcs.LG

Enhances ENet's prediction accuracy while maintaining uncertainty estimation.

problem Gradient shrinkage problem in ENet's loss function.

method Proposes a multi-task learning framework with a modified MSE loss function.

result Improves ENet's prediction accuracy without losing uncertainty estimation.

deep learning evidential regression multi-task learning gradient shrinkage uncertainty estimation

2021-12-17abs ↗pdf ↗

Algorithmic Chaining and the Role of Partial Feedback in Online Nonparametric Learningstat.ML

We investigate contextual online learning with nonparametric (Lipschitz) comparison classes under different assumptions on losses and feedback information. For full information feedback and Lipschitz losses, we design the first explicit algorithm achieving the minimax regret rate (up to log factors). In a partial feedb…

2017-02-27abs ↗pdf ↗

Risk Bounds for Robust Deep Learningcs.LG

The paper proves deep learning can be robust with certain loss functions.

problem The robustness of deep learning models under flawed data.

method Empirical-risk minimization with unbounded, Lipschitz-continuous loss functions.

result These loss functions provide efficient prediction under minimal data assumptions.

deep learning robustness loss functions empirical-risk minimization Lipschitz continuity

2020-09-14abs ↗pdf ↗

Non-asymptotic Oracle Inequalities for the High-Dimensional Cox Regression via Lassomath.ST

We consider the finite sample properties of the regularized high-dimensional Cox regression via lasso. Existing literature focuses on linear models or generalized linear models with Lipschitz loss functions, where the empirical risk functions are the summations of independent and identically distributed (iid) losses. T…

2012-04-09abs ↗pdf ↗

Lipschitz Bounds and Provably Robust Training by Laplacian Smoothingcs.LG

Graph-based framework for provably robust adversarial training.

problem Adversarial robustness of machine learning models.

method Formulates adversarial robustness as loss minimization with a Lipschitz constraint, using graph-based discretization and primal-dual algorithms.

result Establishes a connection between elliptic operators and adversarial learning, and proves fundamental lower bounds on adversarial sensitivity.

Adversarial robustness Lipschitz constraint Graph-based learning Primal-dual algorithm Elliptic operators

2020-06-05abs ↗pdf ↗

Data-Dependent Bounds for Online Portfolio Selection Without Lipschitzness and Smoothnesscs.LG

New bounds for online portfolio selection without smoothness assumptions.

problem Online portfolio selection with non-Lipschitz, non-smooth losses.

method Data-dependent bounds using novel smoothness characterizations and FTRL with self-concordant regularizers.

result Achieves logarithmic regrets when data is 'easy' and sublinear worst-case regrets.

online portfolio selection non-Lipschitz non-smooth regret bounds FTRL

2023-05-23abs ↗pdf ↗

Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networkscs.LG

1-Lipschitz networks are as accurate as classical networks and offer robustness.

problem Misconceptions about 1-Lipschitz neural networks and their properties.

method Analysis of 1-Lipschitz neural networks' accuracy, robustness, and generalization.

result 1-Lipschitz neural networks are as accurate as classical networks and can fit arbitrarily difficult boundaries.

Lipschitz networks neural networks classification robustness generalization

2021-04-11abs ↗pdf ↗

Corruption-Robust Lipschitz Contextual Searchcs.LG

Study robust learning of Lipschitz functions under corrupted binary signals.

problem Learning a Lipschitz function with corrupted binary signals in a context of unknown corruption rounds.

method Introduced agnostic checking and new analysis techniques to design algorithms for symmetric and pricing losses.

result Achieved small cumulative loss for both symmetric and pricing losses.

Lipschitz function contextual search corruption robustness regret minimization agnostic checking

2023-07-26abs ↗pdf ↗

On Lipschitz Continuity and Smoothness of Loss Functions in Learning to Rankcs.LG

In binary classification and regression problems, it is well understood that Lipschitz continuity and smoothness of the loss function play key roles in governing generalization error bounds for empirical risk minimization algorithms. In this paper, we show how these two properties affect generalization error bounds in …

2014-05-03abs ↗pdf ↗

Optimal Rates for Pure

\varepsilon

-Differentially Private Stochastic Convex Optimization with Heavy Tailscs.LG

Optimizes privacy-preserving optimization for heavy-tailed data.

problem Privacy-preserving optimization with heavy-tailed gradients.

method Pure ε-differential privacy framework for Lipschitz extensions.

result Minimax optimal excess-risk rate for pure ε-DP heavy-tailed SCO.

differential privacy stochastic convex optimization heavy tails Lipschitz extensions privacy-preserving algorithms

2026-04-07abs ↗pdf ↗

Smoothness Analysis of Adversarial Trainingcs.LG

Smoothness analysis of adversarial training reveals $L_\infty$ constraints cause more non-smoothness.

problem Non-smoothness of adversarial training loss function.

method Analyzed the smoothness of adversarial training loss function using optimal attacks for model parameters.

result The

L_\infty

constraint causes more non-smoothness than

L_2

constraint.

adversarial training smoothness non-smoothness Lipschitz continuity EntropySGD

2021-03-02abs ↗pdf ↗

Improved robustness to adversarial examples using Lipschitz regularization of the losscs.LG

We augment adversarial training (AT) with worst case adversarial training (WCAT) which improves adversarial robustness by 11% over the current state-of-the-art result in the $\ell_2$ norm on CIFAR-10. We obtain verifiable average case and worst case robustness guarantees, based on the expected and maximum values of the…

2018-10-01abs ↗pdf ↗

Step by Step: Adaptive Gradient Descent for Training L-Lipschitz Neural Networksstat.ML

Adaptive LR improves neural network Lipschitz regularity without slowing convergence.

problem Training neural networks with high Lipschitz constant.

method Adaptive learning rate (decay) in gradient descent for two-layer networks.

result Decaying learning rate maintains convergence rate and Lipschitz regularity.

neural networks gradient descent Lipschitz regularization learning rate

2025-02-06abs ↗pdf ↗

Achieving robustness in classification using optimal transport with hinge regularizationcs.LG

New framework enhances neural network robustness against adversarial attacks.

problem Vulnerability of deep neural networks to small perturbations.

method Integrates Lipschitz constraint using optimal transport and hinge regularization.

result Proposes a new loss function that certifies adversarial robustness.

adversarial robustness optimal transport hinge regularization Lipschitz constraint certifiable robustness

2020-06-11abs ↗pdf ↗

Fully Unconstrained Online Learningcs.LG

Algorithm achieves optimal regret for unknown Lipschitz convex losses.

problem Online learning with unknown Lipschitz constant and target vector norm.

method Develops an online learning algorithm without knowledge of

G

\|w_\star\|

result Matches optimal regret bound

G\|w_\star\|\sqrt{T}

up to logarithmic factors.

online learning regret minimization convex losses unknown parameters

2024-05-30abs ↗pdf ↗

Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GDstat.ML

Full-batch GD achieves generalization close to any stationary point with fewer assumptions.

problem Generalization and excess risk bounds for smooth losses, including non-Lipschitz and nonconvex cases.

method Path-dependent analysis of GD's generalization error, focusing on optimization error and stability.

result Generalization error is tightly bound in terms of optimization error and iteration count, bypassing common assumptions.

Gradient Descent Generalization Excess Risk Smooth Losses Optimization Error

2022-04-26abs ↗pdf ↗

Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Modelsstat.ML

Study non-asymptotic bounds for robust estimators under misspecified models.

problem Evaluate performance of robust estimators under adversarial conditions.

method Propose a general approach to adversarial risk analysis, including investigations on generalization and approximation errors.

result Establish non-asymptotic upper bounds for adversarial excess risk under Lipschitz loss functions.

Adversarial Risk Robust Estimation Misspecified Models Generalization Error Approximation Error

2023-09-02abs ↗pdf ↗

High-dimensional Measurement Error Models for Lipschitz Lossstat.ME

Develops high-dimensional measurement error models for non-linear loss functions.

problem Measurement errors in ultrahigh-dimensional biomedical data.

method Lipschitz loss functions, L1 norm minimization, Lasso analog.

result Improved accuracy in classification and quantile regression problems.

measurement error high-dimensional data Lipschitz loss L1 norm Lasso

2022-10-26abs ↗pdf ↗

Fast Kernel Methods for Generic Lipschitz Losses via

p

-Sparsified Sketchesstat.ML

This paper speeds up kernel methods using sparsified Gaussian sketches.

problem Kernel methods' computational limitations.

method Sparsified Gaussian sketches for kernel methods.

result Efficient time and space savings for kernel methods.

kernel methods sketching Gaussian sketch Lipschitz loss decomposition trick

2022-06-08abs ↗pdf ↗

On the Generalization of Stochastic Gradient Descent with Momentumcs.LG

New insights into training machine learning models with momentum.

problem Lack of theoretical understanding on the generalization error of momentum-based methods.

method Analyzed modified momentum-based update rule (SGDEM) for smooth Lipschitz loss functions.

result SGDEM admits an upper-bound on the generalization error for smooth Lipschitz loss functions.

Stochastic Gradient Descent Momentum Generalization Error Machine Learning Convex Loss Functions

2021-02-26abs ↗pdf ↗

GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networkscs.LG

GraN-GAN normalizes gradients for better GAN performance.

problem Improving image generation in GANs with piecewise linear discriminators.

method Piecewise Gradient Normalization (GraN) for input-dependent normalization.

result Significant performance gains in image generation across various datasets.

GANs Generative Adversarial Networks Piecewise Linear Gradient Normalization Lipschitz Constraint

2021-11-04abs ↗pdf ↗

Training robust neural networks using Lipschitz boundscs.LG

The paper proposes a method to train NNs with a small Lipschitz constant to improve robustness.

problem Neural networks' susceptibility to adversarial perturbations in safety-critical applications.

method The paper introduces a framework to train multi-layer NNs by minimizing their Lipschitz constant, using an optimization scheme based on the Alternating Direction Method of Multipliers.

result The proposed training procedure successfully increases the robustness of neural networks.

Lipschitz constant neural networks robustness adversarial perturbations optimization

2020-05-06abs ↗pdf ↗

When Does Optimizing a Proper Loss Yield Calibration?cs.LG

Optimizing proper loss yields calibrated models under specific conditions.

problem Understanding when optimizing proper loss functions leads to calibrated predictions.

method Local optimality condition and Lipschitz functions.

result Predictors with local optimality are nearly calibrated and nearly locally optimal.

calibration proper loss local optimality Lipschitz functions DNNs

2023-05-30abs ↗pdf ↗

Near-optimal delta-convex estimation of Lipschitz functionsstat.ML

Paper presents an efficient algorithm for estimating Lipschitz functions from noisy data.

problem Estimating unknown Lipschitz functions from noisy observations.

method Extends max-affine methods to Lipschitz setting using nonlinear feature expansion and adaptive partitioning.

result Achieves minimax convergence rate with respect to intrinsic dimension, up to logarithmic factors.

Lipschitz functions estimation noisy data max-affine methods delta-convex functions

2025-11-19abs ↗pdf ↗

Near-Optimal Algorithms for Omnipredictionstat.ML

Near-optimal algorithms for predicting across multiple loss functions efficiently.

problem Predicting optimally across various loss functions simultaneously.

method Developed near-optimal online and offline learning algorithms for omniprediction.

result Achieved near-optimal complexity for both online and offline settings.

omniprediction online learning offline learning near-optimal algorithms regret minimization

2025-01-28abs ↗pdf ↗

High-probability zeroth-order online convex optimisation beyond Euclidean geometrycs.LG

Unified high-probability regret bounds for online convex optimisation with randomised gradient estimators.

problem Online convex optimisation with randomised gradient estimators for

\ell_q

-Lipschitz losses.

method FTRL with randomised two-point finite-difference gradient estimators based on cone-measure sampling from

\ell_r

-spheres.

result Unified high-probability regret bounds for all

p,q,r \in [1,\infty]

online convex optimisation randomised gradient estimators FTRL ell_q-Lipschitz ell_p-regularised

2025-09-25abs ↗pdf ↗

From Adaptive Query Release to Machine Unlearningcs.LG

The paper tackles machine unlearning by designing efficient algorithms for adaptive query classes.

problem Designing efficient unlearning algorithms for machine learning models.

method Formalizes the problem and gives efficient unlearning algorithms for linear and prefix-sum query classes.

result Improved guarantees for stochastic convex optimization with reduced unlearning query complexity.

machine learning unlearning adaptive queries stochastic convex optimization linear models

2023-07-20abs ↗pdf ↗

Robustly Learning Single-Index Models via Alignment Sharpnesscs.LG

Efficiently learns Single-Index Models with constant factor approximation.

problem Learning Single-Index Models under

L_2^2

loss with unknown link functions.

method An efficient algorithm using alignment sharpness for optimization.

result Achieves constant factor approximation to optimal loss for various distributions and link functions.

Single-Index Models L_2^2 loss agnostic model log-concave distributions Lipschitz link functions

2024-02-27abs ↗pdf ↗

Select without Fear: Almost All Mini-Batch Schedules Generalize Optimallycs.LG

New research shows many batch selection methods for training work just as well as full batch training.

problem Finding optimal batch selection methods for training.

method Analysis of mini-batch Gradient Descent (GD) and Stochastic GD (SGD) with various batch selection rules.

result All mini-batch schedules, including deterministic ones, generalize optimally for smooth Lipschitz-convex/nonconvex/strongly-convex loss functions.

mini-batch GD stochastic GD batch selection generalization error convex optimization

2023-05-03abs ↗pdf ↗

Omnipredicting Single-Index Models with Multi-Index Modelscs.LG

New algorithm reduces sample complexity for omnipredictors of SIMs.

problem Learning optimal predictors for various loss functions.

method Sharp analysis of Isotron algorithm for agnostic learning.

result Improved sample complexity to

\approx \varepsilon^{-2}

for bi-Lipschitz link functions.

Omnipredictors Single-Index Models Multi-Index Models Agnostic Learning Isotron Algorithm

2024-11-20abs ↗pdf ↗

Near-optimal Swap Regret Minimization for Convex Lossescs.LG

Improved online algorithm for convex losses with near-optimal swap regret.

problem Minimizing swap regret for convex losses.

method Randomized online algorithm using multi-scale binning.

result Near-optimal

\widetilde O(\sqrt T)

expected swap regret.

Online Learning Convex Losses Swap Regret Multi-scale Binning Efficiency

2026-02-09abs ↗pdf ↗

ModHiFi: Identifying High Fidelity predictive components for Model Modificationcs.LG

ModHiFi identifies critical components for model modification without gradients or loss function.

problem Modifying open weight models without access to training data or loss function.

method Theoretical analysis of Lipschitz-continuous networks, Subset Fidelity metric, and ModHiFi algorithm.

result ModHiFi-P and ModHiFi-U achieve significant performance improvements in model pruning and unlearning.

model modification open weight models Subset Fidelity Lipschitz continuity pruning

2025-11-24abs ↗pdf ↗

Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscs.LG

Efficient local Lipschitz bounds improve neural network robustness.

problem Certifying robustness of neural networks is challenging and often leads to over-regularization.

method Proposes an efficient trainable local Lipschitz upper bound by considering activation functions and weight matrices.

result Consistently outperforms state-of-the-art methods in clean and certified accuracy on various datasets.

neural networks robustness Lipschitz bound ReLU MaxMin

2021-11-02abs ↗pdf ↗

Low Regret Binary Sampling Method for Efficient Global Optimization of Univariate Functionscs.LG

Efficient binary sampling method for global optimization of univariate functions with low regret.

problem Global optimization of univariate loss functions.

method Binary sampling approach to circumvent hard-to-determine query points in traditional methods.

result At most

L\log (3T)

and

2.25H

regret for

L

-Lipschitz continuous and

H

-Lipschitz smooth functions respectively.

global optimization univariate functions binary sampling cumulative regret Lipschitz continuity

2022-01-18abs ↗pdf ↗

Differentially Private SGD with Non-Smooth Lossesstat.ML

Paper relaxes SGD privacy and generalization guarantees for non-smooth convex losses.

problem Privacy and generalization in SGD for non-smooth convex losses.

method Relaxes Lipschitz and strong smoothness assumptions to Hölder smoothness, proving

(ε,δ)

-DP and optimal excess risk.

result Noisy SGD with

α

-Hölder smooth losses achieves optimal excess risk with linear gradient complexity for

α \geq 1/2

differentially private SGD non-smooth convex losses Hölder smoothness privacy guarantees generalization

2021-01-22abs ↗pdf ↗

Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionalitycs.LG

Paper provides finite-sample guarantees for Wasserstein DRO without dimensionality curse.

problem Tackles empirical success of Wasserstein DRO in operations and ML with performance guarantees.

method Develops non-asymptotic framework for analyzing out-of-sample performance and generalization bound.

result First finite-sample guarantee for generic Wasserstein DRO problems without curse of dimensionality.

Wasserstein DRO finite-sample guarantees generalization bound curse of dimensionality concentration inequality

2020-09-09abs ↗pdf ↗

Convergence Analysis of Optimization Algorithmsstat.ML

The regret bound of an optimization algorithms is one of the basic criteria for evaluating the performance of the given algorithm. By inspecting the differences between the regret bounds of traditional algorithms and adaptive one, we provide a guide for choosing an optimizer with respect to the given data set and the l…

2017-07-06abs ↗pdf ↗

Deep linear neural networks with arbitrary loss: All local minima are globalcs.LG

We consider deep linear networks with arbitrary convex differentiable loss. We provide a short and elementary proof of the fact that all local minima are global minima if the hidden layers are either 1) at least as wide as the input layer, or 2) at least as wide as the output layer. This result is the strongest possibl…

2017-12-05abs ↗pdf ↗

Privacy of the last iterate in cyclically-sampled DP-SGD on nonconvex composite lossescs.LG

New privacy bounds for DP-SGD's last iterate, even with cyclic sampling.

problem Privacy of the last iterate in DP-SGD with cyclic sampling.

method Established new RDP upper bounds for the last iterate under realistic assumptions.

result Privacy bounds for DP-SGD's last iterate with cyclic sampling and clipping, even for nonconvex losses.

Differentially-private SGD last iterate cyclic sampling Lipschitz smoothness optimal transport

2024-07-07abs ↗pdf ↗

On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regressioneess.AS

The paper explores MAE as a loss function for DNN vector-to-vector regression, proving its advantages over MSE.

problem Improving loss function for deep neural network based vector-to-vector regression.

method Presenting performance bounds and new properties of MAE, deriving generalized upper bounds, and interpreting MAE as a Laplacian distribution.

result MAE is a more suitable loss function than MSE for DNN based vector-to-vector regression, especially when errors follow a Laplacian distribution.

mean absolute error deep neural networks vector-to-vector regression loss function Lipschitz continuity

2020-08-12abs ↗pdf ↗

System Identification Through Lipschitz Regularized Deep Neural Networkscs.LG

The paper uses neural networks to learn system dynamics from data with Lipschitz regularization.

problem Learning governing equations from time-sampled data.

method Lipschitz regularized deep neural networks for ODE system identification.

result Lipschitz regularization improves the smoothness and generalization of the learned function.

neural networks system identification ODE Lipschitz regularization

2020-09-07abs ↗pdf ↗

Optimistic bounds for multi-output predictionstat.ML

We investigate the challenge of multi-output learning, where the goal is to learn a vector-valued function based on a supervised data set. This includes a range of important problems in Machine Learning including multi-target regression, multi-class classification and multi-label classification. We begin our analysis b…

2020-02-22abs ↗pdf ↗

Black-Box Generalization: Stability of Zeroth-Order Learningcs.LG

New analysis for black-box learning without gradients, improving generalization bounds.

problem Generalization error analysis for derivative-free optimization.

method Zeroth-order Stochastic Search (ZoSS) algorithm for Lipschitz and smooth losses.

result Generalization bounds independent of model dimension, batch size, and number of perturbed evaluations.

generalization zeroth-order optimization black-box learning stochastic search Lipschitz

2022-02-14abs ↗pdf ↗