arXiv research

Paper establishes bounds for RNN-TPPs, showing four-layer networks can achieve vanishing errors.

The paper provides theoretical guarantees for optimized sampling in compressed sensing, showing error vanishes with more measurements.

Approximate vanishing ideal is a concept from computer algebra that studies the algebraic varieties behind perturbed data points. To capture the nonlinear structure of perturbed points, the introduction of approximation to exact vanishing ideals plays a critical role. However, such an approximation also gives rise to a…

New decision-theoretic calibration error metric improves prediction reliability.

New approach uses Gaussian processes to learn and track complex systems with guaranteed accuracy.

The main aim of this paper is to provide an analysis of gradient descent (GD) algorithms with gradient errors that do not necessarily vanish, asymptotically. In particular, sufficient conditions are presented for both stability (almost sure boundedness of the iterates) and convergence of GD with bounded, (possibly) non…

SS-GEN simulates rare events in heavy and light-tailed data.

We study utility indifference prices and optimal purchasing quantities for a contingent claim, in an incomplete semi-martingale market, in the presence of vanishing hedging errors and/or risk aversion. Assuming that the average indifference price converges to a well defined limit, we prove that optimally taken position…

The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…

Bayesian regression underestimates parameter uncertainties in noisy models.

We consider a problem of learning kernels for use in SVM classification in the multi-task and lifelong scenarios and provide generalization bounds on the error of a large margin classifier. Our results show that, under mild conditions on the family of kernels used for learning, solving several related tasks simultaneou…

Background: Statistical mechanics results (Dauphin et al. (2014); Choromanska et al. (2015)) suggest that local minima with high error are exponentially rare in high dimensions. However, to prove low error guarantees for Multilayer Neural Networks (MNNs), previous works so far required either a heavily modified MNN mod…

We consider the problem of decentralized consensus optimization, where the sum of $n$ smooth and strongly convex functions are minimized over $n$ distributed agents that form a connected network. In particular, we consider the case that the communicated local decision variables among nodes are quantized in order to all…

Diffusion models generate data with Gaussian Universality, matching linear model test errors.

The paper uses non-abelian Hodge theory to generalize Kodaira vanishing theorems.

We present a new PAC-Bayesian generalization bound. Standard bounds contain a $\sqrt{L_n \cdot \KL/n}$ complexity term which dominates unless $L_n$, the empirical error of the learning algorithm's randomized predictions, vanishes. We manage to replace $L_n$ by a term which vanishes in many more situations, essentially …

Develops a method to estimate rare-event probabilities under distributional uncertainty.

We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever and Hinton, 2008; Le Roux and Bengio, 2008, 2010; Montúfar and Ay,…

Training of deep models for classification tasks is hindered by local minima problems and vanishing gradients, while unsupervised layer-wise pretraining does not exploit information from class labels. Here, we propose a new regularization technique, called diversifying regularization (DR), which applies a penalty on hi…

Study linear regression with missing or corrupted data, showing error bounds.

Generalizes Kodaira vanishing theorem to Kahler Lie algebroids.

New characterization limits sampling with inexact scores.

Study compares adversarial regularization to sole supervision in machine learning.

A framework selects GANs for specific applications efficiently.

This paper analyzes the interpolation error of nonlinear Attention compared to linear regression.

Paper tackles identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.

Study on error probabilities of machine learning classification techniques using large deviations theory.

Proposes a new model for mixed membership in Gaussian mixture.

Study on L2-boosting behavior as learning rate approaches zero.

Attention mechanism learns to focus on sparse tokens efficiently.

Gradient descent with biased rounding errors converges faster under certain conditions.

Proves a vanishing property for symplectic manifold cohomology.

Improved multi-class AdaBoost algorithm with stronger weak learnability condition.

Paper proves vanishing homology groups for certain hyperbolic groups.

Proves a generalized vanishing theorem for quasi-smooth stacks, with applications in K-theory and birational geometry.

Study the properties of SGD in non-vanishing learning rate regime.

Modern machine learning classifiers often exhibit vanishing classification error on the training set. They achieve this by learning nonlinear representations of the inputs that maps the data into linearly separable classes. Motivated by these phenomena, we revisit high-dimensional maximum margin classification for line…

We study high-dimensional asymptotic performance limits of binary supervised classification problems where the class conditional densities are Gaussian with unknown means and covariances and the number of signal dimensions scales faster than the number of labeled training samples. We show that the Bayes error, namely t…

We give several generalizations of the Kodaira vanishing and embedding theorems for Kähler manifolds to the case where the relevent line bundle has a small region of negative curvature. To prove the vanishing theorems we adapt techniques of Elworthy-Rosenberg for vanishing theorems in Riemannian geometry. For the embed…

The study analyzes how machine learning classifiers' error rates decrease exponentially based on large deviations theory.

Recurrent Neural Networks (RNNs) are rich models for the processing of sequential data. Recent work on advancing the state of the art has been focused on the optimization or modelling of RNNs, mostly motivated by adressing the problems of the vanishing and exploding gradients. The control of overfitting has seen consid…

Vanishing long-term gradients are a major issue in training standard recurrent neural networks (RNNs), which can be alleviated by long short-term memory (LSTM) models with memory cells. However, the extra parameters associated with the memory cells mean an LSTM layer has four times as many parameters as an RNN with the…

The paper solves linearized Ricci curvature equations on compact manifolds.

Inexact subgradient methods work well for semialgebraic functions with additive errors.

This paper considers the recovery of a low-rank matrix from an observed version that simultaneously contains both (a) erasures: most entries are not observed, and (b) errors: values at a constant fraction of (unknown) locations are arbitrarily corrupted. We provide a new unified performance guarantee on when the natura…

New research shows the maximum ℓ1-margin classifier doesn't adapt to sparse ground truths.

Proposes IPT for modeling complex joint distributions.

A novel hierarchical Bayesian approach to Federated Learning reduces data exposure and improves convergence rates.