arXiv research

New theory for nonsmooth systems helps optimize and control complex functions.

PPGD solves nonconvex nonsmooth optimization problems without KL property.

Study on Adam-family methods for nonsmooth optimization with convergence guarantees.

This paper tackles nonsmooth optimization in machine learning.

In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…

Study on nonsmooth contractive SA with constant stepsize and Q-learning.

Study compares nonsmooth spaces with integrable Ricci bounds.

New stability bounds for SGD on nonsmooth convex losses.

Regularity results for geodesic X-ray transform on nonsmooth manifolds

New methods help escape strict saddle points in nonsmooth optimization.

New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gra…

New method solves nonsmooth low-rank matrix optimization problems efficiently.

Study on numerical reliability of AD for MaxPool in neural nets.

Paper develops privacy-preserving federated learning for nonsmooth objectives.

Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.

We construct some nonsmoothable actions of Z2 * Z2 on spin four-manifolds by using an equivariant version of Furuta' s 10/8inequality. The examples satisfy following property: any proper subgroup of Z2 * Z2 is smoothable for some smooth structure.

In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth. Existing approaches either have slow or unclear convergence properties, are restricted to limited problem subclasses, or require careful setting of a smoothing parameter…

We construct a nonsmoothable Z\times Z-action on the connected sum of an Enriques surface and S^2\times S^2, such that each of generators is smoothable. We also construct a nonsmoothable self-homeomorphism on an Enriques surface.

Paper develops an online covariance estimator for nonsmooth stochastic approximation problems.

Unified Lagrangian-based methods for nonsmooth nonconvex optimization.

We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…

Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving opti…

We show that every closed, simply connected, spin topological 4-manifold except $S^4$ and $S^2\times S^2$ admits a homologically trivial, pseudofree, locally linear action of $\mathbb{Z}_p$ for any sufficiently large prime number $p$ which is nonsmoothable for any possible smooth structure.

The paper guarantees global stability for stochastic subgradient methods in nonsmooth nonconvex optimization.

We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises …

Nesterov's extrapolation improves convergence in nonsmooth optimization.

Improved shuffling gradient methods converge faster for nonsmooth convex optimization.

Two new methods solve nonsmooth optimization on Riemannian Stiefel manifold.

Study Langevin Monte Carlo for sampling non-log-concave distributions.

Develops a mathematical model for automatic differentiation in machine learning.

Adam's convergence rate in nonsmooth nonconvex optimization is analyzed.

This paper concerns dictionary learning, i.e., sparse coding, a fundamental representation learning problem. We show that a subgradient descent algorithm, with random initialization, can provably recover orthogonal dictionaries on a natural nonsmooth, nonconvex $\ell_1$ minimization formulation of the problem, under mi…

We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …

Study shows limitations of Lie bracket commutation for nonsmooth vector fields.

In this paper, we consider the problem of minimizing the average of a large number of nonsmooth and convex functions. Such problems often arise in typical machine learning problems as empirical risk minimization, but are computationally very challenging. We develop and analyze a new algorithm that achieves robust linea…

New solver SR2 tackles deep neural network training with nonsmooth regularization.

By the Thurston stability theorem, a group of C^1 orientation-preserving diffeomorphisms of the closed unit interval is locally indicable. We show that the local order structure of orbits gives a stronger criterion for nonsmoothability that can be used to produce new examples of locally indicable groups of homeomorphis…

Much of the vast literature on the integral during the last two centuries concerns extending the class of integrable functions. In contrast, our viewpoint is akin to that taken by Hassler Whitney [{\it Geometric integration theory}, Princeton Univ. Press, Princeton, NJ, 1957] and by geometric measure theorists because …

Frank-Wolfe methods (FW) have gained significant interest in the machine learning community due to its ability to efficiently solve large problems that admit a sparse structure (e.g. sparse vectors and low-rank matrices). However the performance of the existing FW method hinges on the quality of the linear approximatio…

Let G be a cyclic group of order 3, 5 or 7, and X=E(n) be the relatively minimal elliptic surface with rational base. In this paper, we prove that under certain conditions on n, there exists a locally linear G-action on X which is nonsmoothable with respect to infinitely many smooth structures on X. This extends the ma…

Let $X$ be a closed, simply-connected, smooth, spin 4-manifold whose intersection form is isomorphic to $n(-E_8)\bigoplus mH$, where $H$ is the hyperbolic form. In this paper, we prove that for $n$ such that $n\equiv 2 ~{\rm mod} ~4$, there exists a locally linear pseudofree $\mathbb{Z}_2$-action on $X$ which is nonsmo…

New algorithm for fast nonsmooth optimization with applications in image processing and machine learning.

Study efficient derivative computation for nondifferentiable maps in machine learning.

We maximize the expected utility of terminal wealth in an incomplete market where there are cone constraints on the investor's portfolio process and the utility function is not assumed to be strictly concave or differentiable. We establish the existence of the optimal solutions to the primal and dual problems and their…

S.Bauer and M.Furuta defined a stable cohomotopy refinement of the Seiberg-Witten invariants. In this paper, we prove a vanishing theorem of Bauer-Furuta invariants for 4-manifolds with smooth Z/2-actions. As an application, we give a constraint on smooth Z/2-actions on homotopy K3#K3, and construct a nonsmoothable loc…

New examples of sub-Riemannian structures satisfying Minimizing Sard conjecture found.

Many machine learning techniques sacrifice convenient computational structures to gain estimation robustness and modeling flexibility. However, by exploring the modeling structures, we find these "sacrifices" do not always require more computational efforts. To shed light on such a "free-lunch" phenomenon, we study the…