arXiv research

In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared …

Sharp representation theorems show depth benefits for ReLU networks.

Rational neural networks approximate functions more efficiently with less depth.

The paper establishes bounds on the smoothness parameter in Gaussian process interpolation.

Given an associative 3-fold in R^7 which is asymptotically conical with generic rate less than 1, we show that its moduli space of deformations is locally homeomorphic to the kernel of a smooth map between smooth manifolds. Moreover, the virtual dimension of the moduli space is computed and shown to be non-negative for…

This paper studies a Nyström type subsampling approach to large kernel learning methods in the misspecified case, where the target function is not assumed to belong to the reproducing kernel Hilbert space generated by the underlying kernel. This case is less understood, in spite of its practical importance. To model su…

Develops a novel ML smoothing method for incomplete data in state-space models.

We study the iteration complexity of stochastic gradient descent (SGD) for minimizing the gradient norm of smooth, possibly nonconvex functions. We provide several results, implying that the $\mathcal{O}(ε^{-4})$ upper bound of Ghadimi and Lan~\cite{ghadimi2013stochastic} (for making the average gradient norm less than…

Adaptive NN method improves matrix completion for non-smooth data.

We prove that the half-integer valued local index of an isolated umbilic point on a $C^{3+α}$-smooth convex surface in Euclidean 3-space is less than two. The approach is to study the co-kernel of an associated Riemann-Hilbert boundary value problem. The link between the local and global is a semi-local technique that …

Study on the nodal set of Dirac equation solutions on manifolds.

Paper introduces a new invariant for virtual knotoids and proves it's a Vassiliev invariant of order one.

Compactness fails for curvature equations in high dimensions.

Perturbs area-minimizing hypersurfaces to reduce singular set's dimension.

New method combines CATE and CQTE to estimate treatment effects across different quantiles.

We study curvature functionals for immersed 2-spheres in a compact, three-dimensional Riemannian manifold M. Under the assumption that the sectional curvature of M is strictly positive, we prove the existence of a smoothly immersed sphere minimizing the L^{2} integral of the second fundamental form. Assuming instead th…

Whereas recovery of the manifold from data is a well-studied topic, approximation rates for functions defined on manifolds are less known. In this work, we study a regression problem with inputs on a $d^*$-dimensional manifold that is embedded into a space with potentially much larger ambient dimension. It is shown tha…

We are interested in approximation of a multivariate function $f(x_1,\dots,x_d)$ by linear combinations of products $u^1(x_1)\cdots u^d(x_d)$ of univariate functions $u^i(x_i)$, $i=1,\dots,d$. In the case $d=2$ it is a classical problem of bilinear approximation. In the case of approximation in the $L_2$ space the bili…

New shapes enclose less volume than the sphere, surprising in 3D.

We investigate the learning rate of multiple kernel learning (MKL) with $\ell_1$ and elastic-net regularizations. The elastic-net regularization is a composition of an $\ell_1$-regularizer for inducing the sparsity and an $\ell_2$-regularizer for controlling the smoothness. We focus on a sparse setting where the total …

Study evolutes of curves with varying smoothness.

Bayesian optimization (BO) and its batch extensions are successful for optimizing expensive black-box functions. However, these traditional BO approaches are not yet ideal for optimizing less expensive functions when the computational cost of BO can dominate the cost of evaluating the blackbox function. Examples of the…

Calibrating a Lévy process usually requires characterizing its jump distribution. Traditionally this problem can be solved with nonparametric estimation using the empirical characteristic functions (ECF), assuming certain regularity, and results to date are mostly in 1D. For multivariate Lévy processes and less smooth …

Recently, several studies have proven the global convergence and generalization abilities of the gradient descent method for two-layer ReLU networks. Most studies especially focused on the regression problems with the squared loss function, except for a few, and the importance of the positivity of the neural tangent ke…

Kernel-based quadrature rules are becoming important in machine learning and statistics, as they achieve super-$\sqrt{n}$ convergence rates in numerical integration, and thus provide alternatives to Monte Carlo integration in challenging settings where integrands are expensive to evaluate or where integrands are high d…

Learning sparse features can lead to overfitting in neural networks, especially for smooth target functions.

We consider multi-level composite optimization problems where each mapping in the composition is the expectation over a family of random smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient (NPAG) method where the approximate gradients are obtained v…

Advocates against over-smoothing and over-squashing in GNNs, suggesting they are less critical than previously thought.

Smooth surface encloses less volume than a ball.

New nonconvex penalty smooths at origin for deep learning.

Optimal rates for shallow ReLU networks in nonparametric regression.

In this article, we extend the mean curvature flow with surgery to mean convex hypersurfaces with entropy less than $Λ_{n-2}$. In particular, 2-convexity is not assumed. Next we show the surgery flow with just the initial convexity assumption $H - \frac{\langle x, ν\rangle}{2} > 0$ is possible and as an application we …

We establish conditions for a continuous map of nonzero degree between a smooth closed manifold and a negatively curved manifold of dimension greater than four to be homotopic to a smooth cover, and in particular a diffeomorphism when the degree is one. The conditions hold when the volumes or entropy-volumes of the two…

Mini-batch sub-sampling in neural network training is unavoidable, due to growing data demands, memory-limited computational resources such as graphical processing units (GPUs), and the dynamics of on-line learning. In this study we specifically distinguish between static mini-batch sub-sampled loss functions, where mi…

Proves a conjecture about metrics and minimal area enclosures.

Paper optimizes prediction in semi-functional linear models using kernel methods.

Let $f:M^m\to N^n$ be a smooth map between two differential manifolds with $N$ connected, $f(M)$ closed and $f(M)\neq N$. In this short note, we show that either all the points of $M$ are critical points of $f$ or the dimension the collection of all critical points of $f$ is not less than $n-1$. Some consequences of th…

This is the second of a series of three papers which provide proofs of results announced in arXiv:1210.7494. In this paper we consider the Gromov-Hausdorff limits of metrics with cone singularities in the case when the limiting cone angle is less than 2π. We show that these are in a natrual way projective algebraic var…

We prove that a strictly stable constant-mean-curvature hypersurface in a smooth manifold of dimension less than or equal to 7 is uniquely homologically area minimizing for fixed volume in a small L^1 neighborhood.

The paper studies gradings on nilpotent Lie algebras linked to smooth algebraic varieties.

Ever since the proof of asymptotic normality of maximum likelihood estimator by Cramer (1946), it has been understood that a basic technique of the Taylor series expansion suffices for asymptotics of $M$-estimators with smooth/differentiable loss function. Although the Taylor series expansion is a purely deterministic …

Deep neural kernels and Laplace kernel have equivalent RKHS on spheres.

The signature of a surface bundle over a surface is known to be divisible by 4. It is also known that the signature vanishes if the fiber genus is less than or equal to 2 or the base genus is less than or equal to 1. In this article, we construct new smooth 4-manifolds with signature 4 which are surface bundles over su…

We give the diffeomorphism classification of complete intersections with S^1-symmetry in dimension less than or equal to 6. In particular, we show that a 6-dimensional complete intersection admits a smooth non-trivial S^1-action if and only if it is diffeomorphic to the complex projective space or the quadric. We also …

We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the $\ell_2$ norm. This "randomized smoothing" technique has been proposed recently in the literature, but existing guarantees are loose. We prove a tight robu…

Proves existence and uniqueness of solutions to the Lp Gaussian Minkowski problem.

We consider a non-stationary sequential stochastic optimization problem, in which the underlying cost functions change over time under a variation budget constraint. We propose an $L_{p,q}$-variation functional to quantify the change, which yields less variation for dynamic function sequences whose changes are constrai…

New knots found that are 4-genus minimal.