arXiv research

Develops wavelet-based neural network approximation theory.

Characterizes neural kernel and NNGP for various activations.

Smooth activations enable optimal error rates in neural networks for Sobolev function classes.

Researchers relax the CVF's smoothness requirement to create more flexible flow models.

We theoretically discuss why deep neural networks (DNNs) performs better than other models in some cases by investigating statistical properties of DNNs for non-smooth functions. While DNNs have empirically shown higher performance than other standard methods, understanding its mechanism is still a challenging problem.…

Deep learning transforms data geometrically, akin to Ricci flow, improving classification accuracy.

We introduce non-smooth symplectic forms on manifolds and describe corresponding Poisson structures on the algebra of Colombeau generalized functions. This is achieved by establishing an extension of the classical map of smooth functions to Hamiltonian vector fields to the setting of non-smooth geometry. For mildly sin…

It is well-known that overparametrized neural networks trained using gradient-based methods quickly achieve small training error with appropriate hyperparameter settings. Recent papers have proved this statement theoretically for highly overparametrized networks under reasonable assumptions. These results either assume…

Smoothness analysis of adversarial training reveals $L_\infty$ constraints cause more non-smoothness.

Extends curve theory to non-smooth data with finite curvature and torsion.

SGD avoids critical points on weakly convex functions.

Algebras of generalized functions offer possibilities beyond the purely distributional approach in modelling singular quantities in non-smooth differential geometry. This article presents an introductory survey of recent developments in this field and highlights some applications in mathematical physics.

We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for the non-smooth case, which requires different analysis and a different algorithm…

Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisi…

Bayesian optimization tackles non-smooth tuning problems.

In this paper, we develop a novel {\bf ho}moto{\bf p}y {\bf s}moothing (HOPS) algorithm for solving a family of non-smooth problems that is composed of a non-smooth term with an explicit max-structure and a smooth term or a simple non-smooth term whose proximal mapping is easy to compute. The best known iteration compl…

One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth. This paper demystifies this surprising phenomenon for two-layer fully connected ReLU activated neural n…

Recent research connects Hörmander's old work to modern boundary Laplacian analysis.

Advances smooth over-parameterization for solving non-smooth optimization problems.

The paper explores various stationarity concepts in non-smooth optimization.

Expanding FCCO to non-smooth weakly-convex problems, improving deep learning performance.

We consider the problem of finding critical points of functions that are non-convex and non-smooth. Studying a fairly broad class of such problems, we analyze the behavior of three gradient-based methods (gradient descent, proximal update, and Frank-Wolfe update). For each of these methods, we establish rates of conver…

New Morse theory for shapes at distances.

Abstracts a theorem for non-smooth maps in infinite dimensions.

In this paper, we discuss the problem of minimizing the sum of two convex functions: a smooth function plus a non-smooth function. Further, the smooth part can be expressed by the average of a large number of smooth component functions, and the non-smooth part is equipped with a simple proximal mapping. We propose a pr…

We consider the problem of sampling from a density of the form $p(x) \propto \exp(-f(x)- g(x))$, where $f: \mathbb{R}^d \rightarrow \mathbb{R}$ is a smooth and strongly convex function and $g: \mathbb{R}^d \rightarrow \mathbb{R}$ is a convex and Lipschitz function. We propose a new algorithm based on the Metropolis-Has…

Injectivity of X-ray transform proven for non-smooth metrics.

Novel method for shape optimization of non-smooth PDEs.

Extending Itô's formula to non-smooth functions is important both in theory and applications. One of the fairly general extensions of the formula, known as Meyer-Itô, applies to one dimensional semimartingales and convex functions. There are also satisfactory generalizations of Itô's formula for diffusion processes whe…

Paper tackles dynamic behavior of variable topology mechanisms, presenting new transition conditions.

The paper proposes a method to model non-smooth functions using clustering, classification, and Gaussian process modeling.

This work improves polynomial approximations for functions with asymmetric behavior.

New iterative regularization method tackles non-smooth, non-strongly convex functionals.

Bayesian Probabilistic Integration uses BART for high-dimensional, non-smooth functions.

Stochastic Gradient Descent (SGD) is one of the simplest and most popular stochastic optimization methods. While it has already been theoretically studied for decades, the classical analysis usually required non-trivial smoothness assumptions, which do not apply to many modern applications of SGD with non-smooth object…

Sharp ABP estimate on metric spaces via optimal transport.

Optimal private ERM and SCO with subquadratic gradient complexity.

SANE improves exploration of noisy, multimodal functions by finding multiple optima.

The study extends curvature bounds to non-smooth spaces and proves stability of mean curvature.

Stochastic gradient descent's long-term fluctuations are described by a diffusion limit.

MARINA-P improves non-smooth federated optimization with adaptive stepsizes.

Gradient descent biases towards stable rank networks for nearly-orthogonal data.

AsylADMM improves gossip-based learning for non-smooth objectives.

New sampling algorithm for non-smooth potentials.

Survey on preserving curvature bounds for non-smooth Ricci flow.

Develops nonparametric regression for non-smooth functions using fractional Laplacian.

Adam achieves optimal convergence in deep ReLU networks via novel Kakeya bounds.

New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.