In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “nonsmooth loss”
New stability bounds for SGD on nonsmooth convex losses.
problem Understanding stability of SGD on nonsmooth convex losses.
method Sharp upper and lower bounds for SGD and full-batch GD on nonsmooth convex losses.
result SGD can be less stable but still useful for generalization bounds.
Proximal algorithms work well for SQRT-Lasso despite its nonsmooth loss.
problem Tackles the optimization of SQRT-Lasso regression.
method Applies proximal algorithms without concern for nonsmooth loss.
result Proximal algorithms converge fast with high probability.
Fast algorithm tackles nonsmooth optimization problems.
problem Nonsmooth optimization in risk minimization.
method Continuation algorithm for nonsmooth regularized risk minimization.
result Achieves fastest known rates of convergence for strongly convex and general convex problems.
Develops a fast algorithm for high-dimensional LASSO penalized quantile regression.
problem Computational challenges in high-dimensional penalized quantile regression.
method Pathwise coordinate descent algorithm to solve exact coordinatewise minimum of the nonsmooth loss function.
result Algorithm runs faster than existing alternatives and maintains estimation accuracy.
The Strength of Nesterov's Extrapolation in the Individual Convergence of Nonsmooth Optimizationcs.LG
Nesterov's extrapolation improves convergence in nonsmooth optimization.
problem Improving convergence rate in nonsmooth convex optimization.
method Nesterov's extrapolation applied to projected subgradient methods.
result Nesterov's extrapolation optimizes individual convergence for nonsmooth problems.
Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machinestat.ML
Paper tackles efficient SVM classification over decentralized networks.
problem Efficiently classifying high-dimensional data over decentralized networks.
method Convolution-based smoothing technique for nonsmooth hinge loss function, combined with an efficient ADMM algorithm.
result Provable linear convergence of the ADMM algorithm and near-optimal statistical convergence of the sparse estimator.
The flare Package for High Dimensional Linear Regression and Precision Matrix Estimation in Rstat.ML
A new R package for high-dimensional regression and precision matrix estimation.
problem High-dimensional linear regression and precision matrix estimation challenges.
method flare package implements various regression methods and extensions for sparse precision matrix estimation.
result The flare package is efficient and scalable for large problems.
Paper introduces privacy-preserving inventory policy learning for feature-based newsvendor with unknown demand.
problem Privacy-preserving inventory policy learning for feature-based newsvendor with unknown demand distribution and nonsmooth loss function.
method Developed a clipped noisy gradient descent algorithm based on convolution smoothing for optimal inventory estimation within f-differential privacy framework.
result Achieved privacy-preserving optimal inventory policy with provable privacy guarantees and desirable statistical precision.
New solver for MKL-SVM with 0/1 loss function.
problem Optimizing MKL-SVM with -loss function.
method Developed an ADMM solver for nonconvex, nonsmooth optimization.
result Shows promise in simple numerical experiment.
Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.
problem Differentially private optimization of nonconvex, nonsmooth objectives.
method Proposes single-pass and multi-pass DP algorithms with improved sample complexity.
result Sample complexity bounds improved by factors of and .
Nonsmooth High-Order Averaging Theory with Application to Extremum Seeking Optimization and Controlmath.OC
New theory for nonsmooth systems helps optimize and control complex functions.
problem Optimizing and controlling systems with nonsmooth functions.
method Higher-order averaging theory with nonsmooth near-identity transformation and lexicographic differentiation.
result Closed formula for nonsmooth first and second-order averaging.
PPGD solves nonconvex nonsmooth optimization problems without KL property.
problem Nonconvex and nonsmooth optimization problems in statistics and machine learning.
method Projective Proximal Gradient Descent (PPGD) for solving a class of nonconvex and nonsmooth problems.
result PPGD achieves a fast convergence rate of O(1/k^2) for k ≥ k_0.
Two frameworks for fast parameter tuning in high dimensions using ALO of LOOCV risk.
problem Finding optimal regularization parameter in high-dimensional learning problems.
method Two frameworks based on primal and dual formulations of a learning scheme to approximate LOOCV risk.
result Equivalence of two frameworks under smoothness conditions justifies their accuracy.
We study the performance of a family of randomized parallel coordinate descent methods for minimizing the sum of a nonsmooth and separable convex functions. The problem class includes as a special case L1-regularized L1 regression and the minimization of the exponential loss ("AdaBoost problem"). We assume the input da…
Study on Adam-family methods for nonsmooth optimization with convergence guarantees.
problem Training nonsmooth neural networks with convergence guarantees.
method Two-timescale updating scheme and stochastic subgradient methods with gradient clipping.
result Convergence guarantees for various Adam-family methods in training nonsmooth neural networks.
Nonsmoothness in Machine Learning: specific structure, proximal identification, and applicationsmath.OC
This paper tackles nonsmooth optimization in machine learning.
problem Nonsmoothness in machine learning optimization problems.
method Identifying specific structures and leveraging them for practical applications.
result Compression, acceleration, and dimension reduction are possible with nonsmooth optimization.
Paper proposes mini-batch ADMMs for solving nonconvex nonsmooth optimization problems.
problem Solving large-scale nonconvex nonsmooth optimization problems.
method Proposes mini-batch stochastic ADMMs for nonconvex nonsmooth optimization.
result Mini-batch stochastic ADMMs converge to a stationary point with rate O(1/T).
MKL--SVMstat.ML
Paper introduces MKL--SVM for SVM with loss.
problem Optimization of SVM with loss function.
method MKL framework combined with ADMM algorithm for solving the optimization problem.
result Performance of MKL--SVM comparable to SimpleMKL.
Study on nonsmooth contractive SA with constant stepsize and Q-learning.
problem Understanding convergence and bias in nonsmooth contractive SA with different noise types.
method Proposed prelimit coupling technique for steady-state convergence and derived asymptotic bias.
result Asymptotic bias of nonsmooth SA is proportional to the square root of the stepsize.
Researchers found nonsmoothable actions of Z2 * Z2 on spin 4-manifolds.
problem Understanding nonsmoothable actions of Z2 * Z2 on spin 4-manifolds.
method Used an equivariant version of Furuta's 10/8 inequality to construct examples.
result Examples show that any proper subgroup of Z2 * Z2 is smoothable for some smooth structure.
We extend the well-known BFGS quasi-Newton method and its memory-limited variant LBFGS to the optimization of nonsmooth convex objectives. This is done in a rigorous fashion by generalizing three components of BFGS to subdifferentials: the local quadratic model, the identification of a descent direction, and the Wolfe …
Study compares nonsmooth spaces with integrable Ricci bounds.
problem Comparing geometric and functional inequalities on nonsmooth spaces.
method Localization method and one-dimensional comparison estimates.
result Extension of comparison principles to nonsmooth settings.
Improved Frank-Wolfe method tackles nonsmooth functions.
problem Efficiently solving large nonsmooth problems with sparse structures.
method Optimizes for approximation quality over all affine approximations.
result Overcomes issues with existing nonsmooth methods in low-rank matrix estimation.
We introduce a class of quadratic support (QS) functions, many of which play a crucial role in a variety of applications, including machine learning, robust statistical inference, sparsity promotion, and Kalman smoothing. Well known examples include the l2, Huber, l1 and Vapnik losses. We build on a dual representation…
Regularity results for geodesic X-ray transform on nonsmooth manifolds
problem Geodesic X-ray transform on nonsmooth simple manifolds
method Symbol smoothing arguments and pseudodifferential operators with low regularity symbols
result Improved injectivity results for functions
New methods help escape strict saddle points in nonsmooth optimization.
problem Escaping strict saddle points in nonsmooth optimization.
method An inexact stochastically perturbed gradient method applied to the Moreau envelope.
result A variety of algorithms for nonsmooth optimization can efficiently escape strict saddle points of the Moreau envelope.
New algorithms solve nonconvex, nonsmooth optimization problems.
problem Optimizing nonconvex, nonsmooth finite-sum problems with limited existing knowledge.
method Developed fast stochastic algorithms for constant minibatches.
result Proved global linear convergence rate for a specific class of functions.
Proves volume cone implies metric cone in nonsmooth spaces.
problem Generalizing a result from smooth spaces to nonsmooth spaces.
method Proves `volume cone implies metric cone' in RCD spaces.
result Generalizes Cheeger-Colding result to nonsmooth spaces.
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
problem Sampling from distributions with nonsmooth convex composite potentials.
method Leveraging Bregman--Moreau envelopes and proximal operators in mirror descent.
result Efficiency in sampling from nonsmooth distributions, extending existing methods.
New algorithm for large-scale nonsmooth convex optimization with robust convergence.
problem Minimizing the average of many nonsmooth and convex functions in machine learning.
method Developed a new algorithm called Randomized Smoothing SVRG that achieves robust linear convergence.
result Achieves robust linear convergence rate and superior time and gradient complexity compared to state-of-the-art methods.
Sign-RIP improves robust low-rank matrix recovery by preserving norms even with corrupted measurements.
problem Robust low-rank matrix recovery in the presence of corrupted measurements.
method Proposed Sign-RIP, a robust restricted isometry property.
result Sign-RIP guarantees uniform convergence of subdifferentials in robust low-rank matrix recovery.
Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problemsmath.OC
Paper develops accelerated APCD for nonconvex nonsmooth problems with performance guarantees.
problem Efficient methods for nonconvex nonsmooth optimization problems with performance guarantees.
method Asynchronous Accelerated Proximal Coordinate Descent (AAPCD) for nonsmooth and nonconvex problems.
result AAPCD ensures that every limit point is a critical point and achieves linear and sublinear convergence rates.
New method solves nonsmooth low-rank matrix optimization problems efficiently.
problem Nonsmooth and low-rank matrix optimization problems in statistics and machine learning.
method Low-rank Extragradient Method with warm-start initialization.
result The extragradient method converges to an optimal solution with rate and requires only two low-rank SVDs per iteration.
A new method solves complex optimization problems with nonconvex and nonsmooth components.
problem Nonconvex nonsmooth optimization problems with coupled functions.
method Successive difference-of-convex approximation method using Moreau envelopes.
result The method generates bounded sequences with stationary points as accumulation points.
Study on numerical reliability of AD for MaxPool in neural nets.
problem Reliability of automatic differentiation for nonsmooth operations like MaxPool.
method Investigation across precision levels and architectures on various datasets.
result Lower norms of nonsmooth Jacobians help maintain stable learning.
Paper develops privacy-preserving federated learning for nonsmooth objectives.
problem Solving nonsmooth objective functions in a privacy-preserving manner.
method Zero-concentrated differential privacy (zCDP) with Gaussian noise, distributed ADMM, and approximation of augmented Lagrangian.
result The algorithm achieves a competitive privacy-accuracy trade-off and converges to the exact solution.
Analysis of Generalized Bregman Surrogate Algorithms for Nonsmooth Nonconvex Statistical Learningmath.OC
Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.
problem Nonsmooth and nonconvex objectives in statistical learning.
method Bregman-surrogate algorithm framework, including local linear approximation, mirror descent, iterative thresholding, DC programming.
result Global convergence rates for nonconvex and nonsmooth objectives in high dimensions.
The paper develops a convergence framework for inexact nonconvex and nonsmooth algorithms.
problem Tackles convergence of inexact nonconvex and nonsmooth algorithms.
method Promises pseudo sufficient descent and relative error conditions, and assumes continuity and Kurdyka-Lojasiewicz property.
result Proves the convergence of algorithms to critical points under specific conditions.
ProxASAGA solves nonsmooth optimization problems faster than existing methods.
problem Lack of scalable parallel methods for nonsmooth optimization problems.
method ProxASAGA, a fully asynchronous sparse method inspired by SAGA.
result ProxASAGA achieves linear speedup with respect to sequential version under certain assumptions.
We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates each regression coefficient and its corresponding subgradient simultaneousl…
We construct a nonsmoothable Z\times Z-action on the connected sum of an Enriques surface and S^2\times S^2, such that each of generators is smoothable. We also construct a nonsmoothable self-homeomorphism on an Enriques surface.
Paper develops an online covariance estimator for nonsmooth stochastic approximation problems.
problem Estimating covariance in nonsmooth, potentially non-monotone settings.
method Online batch-means covariance matrix estimator.
result Estimator achieves convergence rate of .
New algorithm solves quadratic equations from magnitude measurements efficiently.
problem Solving quadratic system of equations from magnitude measurements.
method Gradient-like algorithm (RWF) minimizing nonconvex nonsmooth loss function.
result RWF converges geometrically to global optimal point with optimal sample complexity.
Unified Lagrangian-based methods for nonsmooth nonconvex optimization.
problem Minimizing nonsmooth nonconvex functions with constraints.
method Developed a unified framework for Lagrangian-based methods using subgradient updates.
result Global convergence guarantees for the proposed framework under mild conditions.
The paper connects dynamical systems to ADMM for solving nonsmooth constrained problems.
problem Solving nonsmooth and constrained optimization problems.
method Developed differential inclusions for accelerated ADMM variants and analyzed their convergence rates.
result Derived rates of convergence for the dynamical systems under different settings, showing tradeoffs between damping strategies.
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
We show that every closed, simply connected, spin topological 4-manifold except and admits a homologically trivial, pseudofree, locally linear action of for any sufficiently large prime number which is nonsmoothable for any possible smooth structure.