arXiv research

Paper introduces new approximations for lognormal sums, matching comonotonicity and moments.

New algorithm finds near-optimal policies efficiently in zero-sum games.

Lognormal random variables appear naturally in many engineering disciplines, including wireless communications, reliability theory, and finance. So, too, does the sum of (correlated) lognormal random variables. Unfortunately, no closed form probability distribution exists for such a sum, and it requires approximation. …

A perturbative approach is used to derive approximations of arbitrary order to estimate high percentiles of sums of positive independent random variables that exhibit heavy tails. Closed-form expressions for the successive approximations are obtained both when the number of terms in the sum is deterministic and when it…

Cumulant expansion is used to derive accurate closed-form approximation for Monthly Sum Options in case of constant volatility model. Payoff of Monthly Sum Option is based on sum of $N$ caped (and probably floored) returns. It is noticed, that $1/\sqrt{N}$ can be used as a small parameter in Edgeworth expansion. First …

New game approximates mean curvature flow evolution.

Algorithm finds a subspace minimizing distances to inliers with outliers.

Proves a formula for a special invariant of 4-manifolds.

Algorithm learns Nash equilibria in stochastic games using entropy-regularized policies.

We propose a new sampling-based approach for approximate inference in filtering problems. Instead of approximating conditional distributions with a finite set of states, as done in particle filters, our approach approximates the distribution with a weighted sum of functions from a set of continuous functions. Central t…

Sharp bounds for Dirichlet sums lead to improved Bayesian algorithm analysis.

This paper certifies cluster assignments from sum-of-norms clustering algorithms.

We propose an explicit recursive method to approximate a power-law with a finite sum of weighted exponentials. Applications to moving averages with long memory are discussed in relationship with stochastic volatility models.

EiGLasso speeds up sparse Kronecker-sum covariance estimation.

Paper studies constrained control games with a novel approximation method.

One of the central goals of Recurrent Neural Networks (RNNs) is to learn long-term dependencies in sequential data. Nevertheless, the most popular training method, Truncated Backpropagation through Time (TBPTT), categorically forbids learning dependencies beyond the truncation horizon. In contrast, the online training …

Designs efficient algorithms to maximize the expectation of Gaussian random variables.

In this paper, we present NESTA, a specialized Neural engine that significantly accelerates the computation of convolution layers in a deep convolutional neural network, while reducing the computational energy. NESTA reformats Convolutions into $3 \times 3$ batches and uses a hierarchy of Hamming Weight Compressors to …

New methods optimize sums of bivariate functions on finite domains.

Study on convergence of Langevin dynamics for zero-sum games in probability distributions.

Sharp bounds for approximating Sobolev functions by ridge functions and networks.

Proposes a new normalization method using convolutional neural networks.

New kernels on symmetric groups enable efficient Gaussian process sampling.

We describe a simple and efficient procedure for approximating the Lévy measure of a $\text{Gamma}(α,1)$ random variable. We use this approximation to derive a finite sum-representation that converges almost surely to Ferguson's representation of the Dirichlet process based on arrivals of a homogeneous Poisson process.…

We study Nesterov's accelerated gradient method with constant step-size and momentum parameters in the stochastic approximation setting (unbiased gradients with bounded variance) and the finite-sum setting (where randomness is due to sampling mini-batches). To build better insight into the behavior of Nesterov's method…

Improves accuracy of SMCI estimators without expanding sum regions.

We consider multi-level composite optimization problems where each mapping in the composition is the expectation over a family of random smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient (NPAG) method where the approximate gradients are obtained v…

Theoretical analysis of entropy approximation for Gaussian mixtures.

This work studies the problem of stochastic dynamic filtering and state propagation with complex beliefs. The main contribution is GP-SUM, a filtering algorithm tailored to dynamic systems and observation models expressed as Gaussian Processes (GP), and to states represented as a weighted sum of Gaussians. The key attr…

We address the structure identification and the uniform approximation of sums of ridge functions $f(x)=\sum_{i=1}^m g_i(a_i\cdot x)$ on ${\mathbb R}^d$, representing a general form of a shallow feed-forward neural network, from a small number of query samples. Higher order differentiation, as used in our constructive a…

The paper develops algorithms for competitive RL in partially observable MGs.

SOS programming verifies MTW tensor non-negativity for optimal transport maps.

Paper introduces a new kernel model for PSD-valued functions with theoretical guarantees and applications.

Paper introduces deterministic EM approximations for non-convex likelihood functions.

New proof shows random neural networks contain sparse subnetworks.

We speed up marginal inference by ignoring factors that do not significantly contribute to overall accuracy. In order to pick a suitable subset of factors to ignore, we propose three schemes: minimizing the number of model factors under a bound on the KL divergence between pruned and full models; minimizing the KL dive…

The Lugannani-Rice formula is a saddlepoint approximation method for estimating the tail probability distribution function, which was originally studied for the sum of independent identically distributed random variables. Because of its tractability, the formula is now widely used in practical financial engineering as …

Algorithm finds ε-equilibrium policies for multi-agent Markov games with hidden low-rank structure.

We derive exponential tail inequalities for sums of random matrices with no dependence on the explicit matrix dimensions. These are similar to the matrix versions of the Chernoff bound and Bernstein inequality except with the explicit matrix dimensions replaced by a trace quantity that can be small even when the dimens…

New method finds global minima using function evaluations and kernel approximations.

Asymptotic error distribution for approximation of a stochastic integral with respect to continuous semimartingale by Riemann sum with general stochastic partition is studied. Effective discretization schemes of which asymptotic conditional mean-squared error attains a lower bound are constructed. Two applications are …

In supervised learning using kernel methods, we often encounter a large-scale finite-sum minimization over a reproducing kernel Hilbert space (RKHS). Large-scale finite-sum problems can be solved using efficient variants of Newton method, where the Hessian is approximated via sub-samples of data. In RKHS, however, the …

New bounds for private matrix approximation using Gaussian noise and Dyson Brownian Motion.

Smooth finite-sum optimization has been widely studied in both convex and nonconvex settings. However, existing lower bounds for finite-sum optimization are mostly limited to the setting where each component function is (strongly) convex, while the lower bounds for nonconvex finite-sum optimization remain largely unsol…

The paper analyzes the variance of different shuffling methods in stochastic gradient descent.

We consdier in dimension four weakly convergent sequences of approximate biharmonic maos into sphere with bi-tension fields bounded in $L^p$ for some $p>1$. We prove an energy identity that accounts for the loss of Hessian energies by the sum of Hessian energies over finitely many nontrivial biharmonic maps on $\mathbb…

For the problem of binary linear classification and feature selection, we propose algorithmic approaches to classifier design based on the generalized approximate message passing (GAMP) algorithm, recently proposed in the context of compressive sensing. We are particularly motivated by problems where the number of feat…

We consider two problems that arise in machine learning applications: the problem of recovering a planted sparse vector in a random linear subspace and the problem of decomposing a random low-rank overcomplete 3-tensor. For both problems, the best known guarantees are based on the sum-of-squares method. We develop new …