arXiv research

New methods improve convergence in non-convex non-smooth learning problems.

New iterative regularization method tackles non-smooth, non-strongly convex functionals.

We introduce non-smooth symplectic forms on manifolds and describe corresponding Poisson structures on the algebra of Colombeau generalized functions. This is achieved by establishing an extension of the classical map of smooth functions to Hamiltonian vector fields to the setting of non-smooth geometry. For mildly sin…

Advances smooth over-parameterization for solving non-smooth optimization problems.

Study generalizes Möbius energy to non-smooth sets in arbitrary dimensions.

New algorithm for federated learning with non-smooth regularizers.

Given a convex optimization problem and its dual, there are many possible first-order algorithms. In this paper, we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method. This is done through convex duality, and implies notably that for certain problems, such…

In high dimensional sparse regression, pivotal estimators are estimators for which the optimal regularization parameter is independent of the noise level. The canonical pivotal estimator is the square-root Lasso, formulated along with its derivatives as a "non-smooth + non-smooth" optimization problem. Modern technique…

The paper develops bounds and regularity for minimal boundaries in non-smooth spaces with Ricci curvature.

New Morse theory for shapes at distances.

In this paper, we develop a novel {\bf ho}moto{\bf p}y {\bf s}moothing (HOPS) algorithm for solving a family of non-smooth problems that is composed of a non-smooth term with an explicit max-structure and a smooth term or a simple non-smooth term whose proximal mapping is easy to compute. The best known iteration compl…

We provide improved convergence rates for various \emph{non-smooth} optimization problems via higher-order accelerated methods. In the case of $\ell_\infty$ regression, we achieves an $O(ε^{-4/5})$ iteration complexity, breaking the $O(ε^{-1})$ barrier so far present for previous methods. We arrive at a similar rate fo…

Unified framework for training neural networks with non-smooth, non-convex regularizers.

Study rigidity in low-regularity Riemannian and semi-Riemannian metrics.

New method smooths optimization for sparse regularization.

Study partial derivatives on non-smooth metric measure structures.

New bounds for online portfolio selection without smoothness assumptions.

Using tools and results from geometric measure theory, we give a simple new proof of the main result (Theorem 1.3) in K. Kondo and M. Tanaka, Approximation of Lipschitz Maps via Immersions and Differentiable Exotic Sphere Theorems, \textit{Nonlinear Anal.} \textbf{155} (2017), 219--249, as well as the converse statemen…

Improves robustness of high-dimensional regression with rank objective and group lasso regularization.

For non-smooth surfaces, the measure of Brownian loops is derived using the Polyakov-Alvarez formula.

In high-dimensional and/or non-parametric regression problems, regularization (or penalization) is used to control model complexity and induce desired structure. Each penalty has a weight parameter that indicates how strongly the structure corresponding to that penalty should be enforced. Typically the parameters are c…

Algorithm approximates regularization path for deep neural networks efficiently.

Study geodesics in sub-Riemannian manifolds, resolving open questions.

In this paper we consider the problem of minimizing composite objective functions consisting of a convex differentiable loss function plus a non-smooth regularization term, such as $L_1$ norm or nuclear norm, under Rényi differential privacy (RDP). To solve the problem, we propose two stochastic alternating direction m…

Study solves Gel'fand's inverse problem in non-smooth spaces with Ricci curvature bounds.

Wasserstein distributionally robust optimization (DRO) has recently achieved empirical success for various applications in operations research and machine learning, owing partly to its regularization effect. Although connection between Wasserstein DRO and regularization has been established in several settings, existin…

Stochastic gradient descent's long-term fluctuations are described by a diffusion limit.

The paper proposes a method to model non-smooth functions using clustering, classification, and Gaussian process modeling.

Survey on harmonic maps in non-smooth spaces, focusing on rigidity.

Injectivity of geodesic X-ray transform on low-regularity manifolds.

Generative Adversarial Networks (GANs) are one of the most practical methods for learning data distributions. A popular GAN formulation is based on the use of Wasserstein distance as a metric between probability distributions. Unfortunately, minimizing the Wasserstein distance between the data distribution and the gene…

Smooth bundles with rough data maintain Hodge kernel isomorphism.

We study hyperbolic polyhedral surfaces with faces isometric to regular hyperbolic polygons satisfying that the total angles at vertices are at least $2π.$ The combinatorial information of these surfaces is shown to be identified with that of Euclidean polyhedral surfaces with negative combinatorial curvature everywher…

SAPPHIRE tackles ill-conditioned rERM problems with faster convergence.

Study proves solenoidal injectivity for tensor fields on curved manifolds with low regularity.

MARINA-P improves non-smooth federated optimization with adaptive stepsizes.

RO-TD learns sparse value functions efficiently.

We consider first-order differential operators with locally bounded measurable coefficients on vector bundles with measurable coefficient metrics. Under a mild set of assumptions, we demonstrate the equivalence between the essential self-adjointness of such operators to a negligible boundary property. When the operator…

New algorithms delete user data from machine learning models efficiently.

The paper extends Frobenius-type theorems to non-smooth settings with Hölder estimates.

Extends curve theory to non-smooth data with finite curvature and torsion.

AsylADMM improves gossip-based learning for non-smooth objectives.

This paper presents an asynchronous incremental aggregated gradient algorithm and its implementation in a parameter server framework for solving regularized optimization problems. The algorithm can handle both general convex (possibly non-smooth) regularizers and general convex constraints. When the empirical data loss…

Difference of convex (DC) functions cover a broad family of non-convex and possibly non-smooth and non-differentiable functions, and have wide applications in machine learning and statistics. Although deterministic algorithms for DC functions have been extensively studied, stochastic optimization that is more suitable …

Survey on preserving curvature bounds for non-smooth Ricci flow.

We propose an inference method to estimate sparse interactions and biases according to Boltzmann machine learning. The basis of this method is $L_1$ regularization, which is often used in compressed sensing, a technique for reconstructing sparse input signals from undersampled outputs. $L_1$ regularization impedes the …

The purpose of this work is to develop and study a distributed strategy for Pareto optimization of an aggregate cost consisting of regularized risks. Each risk is modeled as the expectation of some loss function with unknown probability distribution while the regularizers are assumed deterministic, but are not required…

Adaptive NN method improves matrix completion for non-smooth data.