arXiv research

This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.

Extends curve theory to non-smooth data with finite curvature and torsion.

The paper explores various stationarity concepts in non-smooth optimization.

In this paper, we develop a novel {\bf ho}moto{\bf p}y {\bf s}moothing (HOPS) algorithm for solving a family of non-smooth problems that is composed of a non-smooth term with an explicit max-structure and a smooth term or a simple non-smooth term whose proximal mapping is easy to compute. The best known iteration compl…

We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for the non-smooth case, which requires different analysis and a different algorithm…

Survey on preserving curvature bounds for non-smooth Ricci flow.

AsylADMM improves gossip-based learning for non-smooth objectives.

Advances smooth over-parameterization for solving non-smooth optimization problems.

New methods improve convergence in non-convex non-smooth learning problems.

Stochastic approximation proves asymptotic normality for non-smooth problems.

Bayesian optimization tackles non-smooth tuning problems.

Paper tackles private optimization for non-smooth objectives efficiently.

We investigate a generalization of the so-called metric splitting of globally hyperbolic space-times to non-smooth Lorentzian manifolds and show the existence of this metric splitting for a class of wave-type space-times. Our approach is based on smooth approximations of non-smooth space-times by families (or sequences…

We provide improved convergence rates for various \emph{non-smooth} optimization problems via higher-order accelerated methods. In the case of $\ell_\infty$ regression, we achieves an $O(ε^{-4/5})$ iteration complexity, breaking the $O(ε^{-1})$ barrier so far present for previous methods. We arrive at a similar rate fo…

Adaptive data fusion boosts efficiency in multi-task optimization.

New SPS variant improves non-smooth optimization without small gradients.

Expanding FCCO to non-smooth weakly-convex problems, improving deep learning performance.

Novel method for shape optimization of non-smooth PDEs.

We consider the problem of finding critical points of functions that are non-convex and non-smooth. Studying a fairly broad class of such problems, we analyze the behavior of three gradient-based methods (gradient descent, proximal update, and Frank-Wolfe update). For each of these methods, we establish rates of conver…

Study solves Gel'fand's inverse problem in non-smooth spaces with Ricci curvature bounds.

Paper proposes ZO-SMD for MERO, achieving optimal convergence rates.

MARINA-P improves non-smooth federated optimization with adaptive stepsizes.

In high dimensional sparse regression, pivotal estimators are estimators for which the optimal regularization parameter is independent of the noise level. The canonical pivotal estimator is the square-root Lasso, formulated along with its derivatives as a "non-smooth + non-smooth" optimization problem. Modern technique…

Given a convex optimization problem and its dual, there are many possible first-order algorithms. In this paper, we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method. This is done through convex duality, and implies notably that for certain problems, such…

New iterative regularization method tackles non-smooth, non-strongly convex functionals.

Optimal private ERM and SCO with subquadratic gradient complexity.

We introduce non-smooth symplectic forms on manifolds and describe corresponding Poisson structures on the algebra of Colombeau generalized functions. This is achieved by establishing an extension of the classical map of smooth functions to Hamiltonian vector fields to the setting of non-smooth geometry. For mildly sin…

Safe-EF improves federated learning for non-smooth, constrained optimization.

We theoretically discuss why deep neural networks (DNNs) performs better than other models in some cases by investigating statistical properties of DNNs for non-smooth functions. While DNNs have empirically shown higher performance than other standard methods, understanding its mechanism is still a challenging problem.…

New algorithm for federated learning with non-smooth regularizers.

Variable projection solves structured optimization problems by completely minimizing over a subset of the variables while iterating over the remaining variables. Over the last 30 years, the technique has been widely used, with empirical and theoretical results demonstrating both greater efficacy and greater stability c…

FedProx algorithm improved for non-smooth and heterogeneous data.

New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.

Smoothness analysis of adversarial training reveals $L_\infty$ constraints cause more non-smoothness.

New sampling algorithm for non-smooth potentials.

We consider the problem of sampling from a density of the form $p(x) \propto \exp(-f(x)- g(x))$, where $f: \mathbb{R}^d \rightarrow \mathbb{R}$ is a smooth and strongly convex function and $g: \mathbb{R}^d \rightarrow \mathbb{R}$ is a convex and Lipschitz function. We propose a new algorithm based on the Metropolis-Has…

The paper proposes a method to model non-smooth functions using clustering, classification, and Gaussian process modeling.

In the framework of Lorentzian warped products, we study the Friedmann-Robertson-Walker cosmological model to investigate non-smooth curvatures associated with multiple discontinuities involved in the evolution of the universe. In particular we analyze non-smooth features of the spatially flat Friedmann-Robertson-Walke…

We analyze convergence rates of stochastic optimization procedures for non-smooth convex optimization problems. By combining randomized smoothing techniques with accelerated gradient methods, we obtain convergence rates of stochastic optimization procedures, both in expectation and with high probability, that have opti…

New method tackles non-smooth tensor data for better recovery.

Sharp ABP estimate on metric spaces via optimal transport.

New algorithms optimize non-smooth, non-convex objectives with improved complexity.

Bayesian Probabilistic Integration uses BART for high-dimensional, non-smooth functions.

In this paper, we discuss the problem of minimizing the sum of two convex functions: a smooth function plus a non-smooth function. Further, the smooth part can be expressed by the average of a large number of smooth component functions, and the non-smooth part is equipped with a simple proximal mapping. We propose a pr…

Positive mass theorem for non-smooth metrics on flat manifolds with corners.

We investigate the theoretical limits of pipeline parallel learning of deep learning architectures, a distributed setup in which the computation is distributed per layer instead of per example. For smooth convex and non-convex objective functions, we provide matching lower and upper complexity bounds and show that a na…

A new optimization method, BPM, converges linearly in non-convex, non-smooth problems.

Deep neural networks improve surrogate models for non-smooth quantities in uncertain geometries.