arXiv research

This paper provides a mathematical foundation for deep neural networks solving PDEs.

Estimates neural network error approximating compact sets.

Study shows infoGAN's generalization error bound for two-layer networks.

We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the excess testing loss (squared loss) converges slowly to zero as the number of observations (and thus iterations) goes to infinity, the testing …

New algorithm samples from log-concave distributions with high accuracy in polynomial time.

THEOREM. For every prime $p$ and each $n=2, 3, ... \infty$, there is an action of $G=\prod_{i=1}^{\infty}(Z/ pZ)$ on a two-dimensional compact metric space $X$ with $n$-dimensional orbit space. This theorem was proved in [DW: A.N. Dranishnikov and J.E. West, Compact group actions that raise dimension to infinity, Topol…

Meta-learning extends supervised learning to tasks with varying numbers of examples, revealing conditions for successful learning.

We study the distribution of hard-, soft-, and adaptive soft-thresholding estimators within a linear regression model where the number of parameters k can depend on sample size n and may diverge with n. In addition to the case of known error-variance, we define and study versions of the estimators when the error-varian…

The problem of estimation error in portfolio optimization is discussed, in the limit where the portfolio size N and the sample size T go to infinity such that their ratio is fixed. The estimation error strongly depends on the ratio N/T and diverges for a critical value of this parameter. This divergence is the manifest…

In this paper we study the consistency of an empirical minimum error entropy (MEE) algorithm in a regression setting. We introduce two types of consistency. The error entropy consistency, which requires the error entropy of the learned function to approximate the minimum error entropy, is shown to be always true if the…

Researchers found the maximum number of holes in polyominoes grows proportionally to the dimension.

Paper analyzes singular subspace estimation in noisy matrix models.

We propose a new notion called \emph{infinity-harmonic maps}between Riemannain manifolds. These are natural generalizations of the well known notion of infinity harmonic functions and are also the limiting case of $p$% -harmonic maps as $p\to \infty$. Infinity harmoncity appears in many familiar contexts. For example,…

Researchers improve spectrum reconstruction formula with proof.

SGD fails to converge for deep ReLU networks with limited random initializations.

We consider distributed statistical optimization in one-shot setting, where there are $m$ machines each observing $n$ i.i.d. samples. Based on its observed samples, each machine sends a $B$-bit-long message to a server. The server then collects messages from all machines, and estimates a parameter that minimizes an exp…

We consider distributed statistical optimization in one-shot setting, where there are $m$ machines each observing $n$ i.i.d. samples. Based on its observed samples, each machine then sends an $O(\log(mn))$-length message to a server, at which a parameter minimizing an expected loss is to be estimated. We propose an alg…

We focus on estimating \emph{a priori} generalization error of two-layer ReLU neural networks (NNs) trained by mean squared error, which only depends on initial parameters and the target function, through the following research line. We first estimate \emph{a priori} generalization error of finite-width two-layer ReLU …

Inflating the minimum norm interpolator improves linear regression generalization error.

A new definition of umbilic points at infinity for polynomial surfaces.

Density ratio estimation is a vital tool in both machine learning and statistical community. However, due to the unbounded nature of density ratio, the estimation procedure can be vulnerable to corrupted data points, which often pushes the estimated ratio toward infinity. In this paper, we present a robust estimator wh…

In this work we show that, using the eigen-decomposition of the adjacency matrix, we can consistently estimate latent positions for random dot product graphs provided the latent positions are i.i.d. from some distribution. If class labels are observed for a number of vertices tending to infinity, then we show that the …

The paper proves mapping class groups of closed surfaces are simply connected at infinity.

In this paper we mainly pay attention to the complex hyperbolic triangle groups of type (m, n, infinity) and discuss the discreteness. From the results more explicit conclusions about the triangle groups of type (n, infinity, infinity) will also be given.

New insights into how overfitting affects neural networks' performance.

Motivated by families of formal moduli problems, in this note we generalize the notion of L-infinity space by allowing sheaves of L-infinity algebras over any (reasonable) nilpotent dg manifold. We discuss various examples including those coming from Lie algebroids. Given a Lie algebroid, we show that there is an L-inf…

The Residual Network (ResNet), proposed in He et al. (2015), utilized shortcut connections to significantly reduce the difficulty of training, which resulted in great performance boosts in terms of both training and generalization error. It was empirically observed in He et al. (2015) that stacking more layers of resid…

This is a survey of the author's paper arXiv:1001.0023 on "Algebraic Geometry over C-infinity rings". If X is a smooth manifold then the R-algebra C^\infty(X) of smooth functions c : X --> R is a "C-infinity ring". That is, for each smooth function f : R^n --> R there is an n-fold operation Φ_f : C^\infty(X)^n --> C^\i…

Neural networks can approximate rectifiable measures with small error.

We study connected sum at infinity on smooth, open manifolds. This operation requires a choice of proper ray in each manifold summand. In favorable circumstances, the connected sum at infinity operation is independent of ray choices. For each m at least 3, we construct an infinite family of pairs of m-manifolds on whic…

The study examines the growth of conjugacy classes in negatively curved manifolds.

Efficient algorithms for low-rank bandits using subspace recovery.

The paper examines mass aspects at future null infinity and limits of quasilocal mass.

This paper studies clustering of data sequences using the k-medoids algorithm. All the data sequences are assumed to be generated from \emph{unknown} continuous distributions, which form clusters with each cluster containing a composite set of closely located distributions (based on a certain distance metric between di…

The paper splits manifolds using infinity harmonic functions with linear growth.

We prove a version of the countable union theorem for asymptotic dimension and we apply it to groups acting on asymptotically finite dimensional metric spaces. As a consequence we obtain the following finite dimensionality theorems. A) An amalgamated product of asymptotically finite dimensional groups has finite asympt…

The paper defines curvature at infinity for flat manifolds.

Novel approach to wave equations near null infinity in flat spacetimes.

Derives error formula for convex regression in high dimensions.

Study on linear regression robustness to adversarial attacks.

The paper improves ALO for $\ell_1$-regularized models.

The paper analyzes null infinity's geometry without restrictions.

We study the geometry at infinity of expanding gradient Ricci solitons of dimension greater than two with finite asymptotic curvature ratio without curvature sign assumptions. We mainly prove that they have a cone structure at infinity.

Optimal quantization of measures on Carnot groups

Let F be R or C, d the dimension of F over R. Denote by P(F) either the affine plane A(F) or the hyperbolic plane H(F) over F. An arrangement L of k lines in P(F) (pairwise non-parallel in the hyperbolic case) has a link at infinity K(L) comprising k unknotted (d-1)-spheres in the (2d-1)-sphere, whose topology reflects…

Define mass invariant for asymptotically hyperbolic 3-manifolds with toroidal infinity.

Study asymptotic behavior of Weingarten surfaces at infinity.

We study products of random matrices in the regime where the number of terms and the size of the matrices simultaneously tend to infinity. Our main theorem is that the logarithm of the $\ell_2$ norm of such a product applied to any fixed vector is asymptotically Gaussian. The fluctuations we find can be thought of as a…