arXiv research

Develops wavelet-based neural network approximation theory.

Characterizes neural kernel and NNGP for various activations.

Smooth activations enable optimal error rates in neural networks for Sobolev function classes.

Researchers relax the CVF's smoothness requirement to create more flexible flow models.

We theoretically discuss why deep neural networks (DNNs) performs better than other models in some cases by investigating statistical properties of DNNs for non-smooth functions. While DNNs have empirically shown higher performance than other standard methods, understanding its mechanism is still a challenging problem.…

Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisi…

Recent research connects Hörmander's old work to modern boundary Laplacian analysis.

Deep learning transforms data geometrically, akin to Ricci flow, improving classification accuracy.

Paper tackles dynamic behavior of variable topology mechanisms, presenting new transition conditions.

MARINA-P improves non-smooth federated optimization with adaptive stepsizes.

Extends curve theory to non-smooth data with finite curvature and torsion.

AsylADMM improves gossip-based learning for non-smooth objectives.

Survey on preserving curvature bounds for non-smooth Ricci flow.

We introduce non-smooth symplectic forms on manifolds and describe corresponding Poisson structures on the algebra of Colombeau generalized functions. This is achieved by establishing an extension of the classical map of smooth functions to Hamiltonian vector fields to the setting of non-smooth geometry. For mildly sin…

Adam achieves optimal convergence in deep ReLU networks via novel Kakeya bounds.

The paper explores various stationarity concepts in non-smooth optimization.

New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.

We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for the non-smooth case, which requires different analysis and a different algorithm…

Smoothness analysis of adversarial training reveals $L_\infty$ constraints cause more non-smoothness.

It is well-known that overparametrized neural networks trained using gradient-based methods quickly achieve small training error with appropriate hyperparameter settings. Recent papers have proved this statement theoretically for highly overparametrized networks under reasonable assumptions. These results either assume…

In the framework of Lorentzian warped products, we study the Friedmann-Robertson-Walker cosmological model to investigate non-smooth curvatures associated with multiple discontinuities involved in the evolution of the universe. In particular we analyze non-smooth features of the spatially flat Friedmann-Robertson-Walke…

This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.

New algorithms optimize non-smooth, non-convex objectives with improved complexity.

One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth. This paper demystifies this surprising phenomenon for two-layer fully connected ReLU activated neural n…

In this paper, we develop a novel {\bf ho}moto{\bf p}y {\bf s}moothing (HOPS) algorithm for solving a family of non-smooth problems that is composed of a non-smooth term with an explicit max-structure and a smooth term or a simple non-smooth term whose proximal mapping is easy to compute. The best known iteration compl…

Positive mass theorem for non-smooth metrics on flat manifolds with corners.

Paper tackles private optimization for non-smooth objectives efficiently.

We investigate a generalization of the so-called metric splitting of globally hyperbolic space-times to non-smooth Lorentzian manifolds and show the existence of this metric splitting for a class of wave-type space-times. Our approach is based on smooth approximations of non-smooth space-times by families (or sequences…

Advances smooth over-parameterization for solving non-smooth optimization problems.

New methods improve convergence in non-convex non-smooth learning problems.

SGD avoids critical points on weakly convex functions.

Adaptive data fusion boosts efficiency in multi-task optimization.

Algebras of generalized functions offer possibilities beyond the purely distributional approach in modelling singular quantities in non-smooth differential geometry. This article presents an introductory survey of recent developments in this field and highlights some applications in mathematical physics.

Abstracts a theorem for non-smooth maps in infinite dimensions.

New SPS variant improves non-smooth optimization without small gradients.

Annealed Entropic Allocation improves ranking and selection by mitigating hard switching and improving finite-budget discrimination.

Stochastic approximation proves asymptotic normality for non-smooth problems.

Bayesian optimization tackles non-smooth tuning problems.

We provide improved convergence rates for various \emph{non-smooth} optimization problems via higher-order accelerated methods. In the case of $\ell_\infty$ regression, we achieves an $O(ε^{-4/5})$ iteration complexity, breaking the $O(ε^{-1})$ barrier so far present for previous methods. We arrive at a similar rate fo…

In high dimensional sparse regression, pivotal estimators are estimators for which the optimal regularization parameter is independent of the noise level. The canonical pivotal estimator is the square-root Lasso, formulated along with its derivatives as a "non-smooth + non-smooth" optimization problem. Modern technique…

How can local-search methods such as stochastic gradient descent (SGD) avoid bad local minima in training multi-layer neural networks? Why can they fit random labels even given non-convex and non-smooth architectures? Most existing theory only covers networks with one hidden layer, so can we go deeper? In this paper, w…

Safe-EF improves federated learning for non-smooth, constrained optimization.

Modified perturbation method removes non-smoothness in solving Black-Scholes equations.

The study extends curvature bounds to non-smooth spaces and proves stability of mean curvature.

Expanding FCCO to non-smooth weakly-convex problems, improving deep learning performance.

Novel method for shape optimization of non-smooth PDEs.

Example of non-smooth isotopy using K3 surfaces.

Injectivity of X-ray transform proven for non-smooth metrics.