arXiv research

This paper shows how to approximate any log-concave distribution using well-conditioned affine coupling flows.

New framework explains normalizing flows' power and limitations.

Lower bounds on MALA and HMC for well-conditioned distributions.

Improved subgradient method tackles ill-conditioned composite optimization problems.

Gradient descent dynamics studied for DEQs in linear and single-index models.

Algorithm learns weight matrix from single trajectory of nonlinear dynamical system.

A well-conditioned Jacobian spectrum has a vital role in preventing exploding or vanishing gradients and speeding up learning of deep neural networks. Free probability theory helps us to understand and handle the Jacobian spectrum. We rigorously show almost sure asymptotic freeness of layer-wise Jacobians of deep neura…

Note on subgaussian bounds for sign-quantized linear maps.

Study shows high-dimensional sparse RL hardness and Lasso Q-iteration's nearly dimension-free regret.

This paper explores conditions for neural networks to extrapolate to new domains.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

Unified complexity bound for sampling logconcave distributions

MESSY estimation recovers symbolic density functions from samples using maximum entropy.

New method uses random features and Tikhonov regularization for operator learning from noisy data.

Semi-supervised learning deals with the problem of how, if possible, to take advantage of a huge amount of not classified data, to perform classification, in situations when, typically, the labelled data are few. Even though this is not always possible (it depends on how useful is to know the distribution of the unlabe…

Reward-free RL in linear MDPs is as hard as reward-aware RL.

We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …

SLR tackles sparse linear regression problems, showing hardness for efficient algorithms.

We provide fast algorithms for overconstrained $\ell_p$ regression and related problems: for an $n\times d$ input matrix $A$ and vector $b\in\mathbb{R}^n$, in $O(nd\log n)$ time we reduce the problem $\min_{x\in\mathbb{R}^d} \|Ax-b\|_p$ to the same problem with input matrix $\tilde A$ of dimension $s \times d$ and corr…

Scattering networks maximize separation on low-dimensional data.

Polynomial-time algorithm learns ReLU networks without assumptions.

The literature provides strong evidence that stock prices can be predicted from past price data. Principal component analysis (PCA) is a widely used mathematical technique for dimensionality reduction and analysis of data by identifying a small number of principal components to explain the variation found in a data set…

We study the problem of learning overcomplete HMMs---those that have many hidden states but a small output alphabet. Despite having significant practical importance, such HMMs are poorly understood with no known positive or negative results for efficient learning. In this paper, we present several new results---both po…

Improved Lasso estimator speeds up variable selection.

Nonnegative matrix factorization (NMF) under the separability assumption can provably be solved efficiently, even in the presence of noise, and has been shown to be a powerful technique in document classification and hyperspectral unmixing. This problem is referred to as near-separable NMF and requires that there exist…

Improved SVRG for quadratic functions achieves better performance and running times.

We propose a decentralized Maximum Likelihood solution for estimating the stochastic renewable power generation and demand in single bus Direct Current (DC) MicroGrids (MGs), with high penetration of droop controlled power electronic converters. The solution relies on the fact that the primary control parameters are se…

The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized co…

Hidden Markov models have successfully been applied as models of discrete time series in many fields. Often, when applied in practice, the parameters of these models have to be estimated. The currently predominating identification methods, such as maximum-likelihood estimation and especially expectation-maximization, a…

This paper delivers improved theoretical guarantees for the convex programming approach in low-rank matrix estimation, in the presence of (1) random noise, (2) gross sparse outliers, and (3) missing data. This problem, often dubbed as robust principal component analysis (robust PCA), finds applications in various domai…

New algorithm reduces pricing error by a factor of T^2/3.

OSA overcomes instability in skipless Transformers.

We develop the first quantum algorithm for the constrained portfolio optimization problem. The algorithm has running time $\widetilde{O} \left( n\sqrt{r} \frac{ζκ}{δ^2} \log \left(1/ε\right) \right)$, where $r$ is the number of positivity and budget constraints, $n$ is the number of assets in the portfolio, $ε$ the des…

In this paper we consider the training of single hidden layer neural networks by pseudoinversion, which, in spite of its popularity, is sometimes affected by numerical instability issues. Regularization is known to be effective in such cases, so that we introduce, in the framework of Tikhonov regularization, a matricia…

New bounds for high-dimensional sparse linear bandits, balancing information and regret.

Parameter reduction has been an important topic in deep learning due to the ever-increasing size of deep neural network models and the need to train and run them on resource limited machines. Despite many efforts in this area, there were no rigorous theoretical guarantees on why existing neural net compression methods …

Many applications in signal processing benefit from the sparsity of signals in a certain transform domain or dictionary. Synthesis sparsifying dictionaries that are directly adapted to data have been popular in applications such as image denoising, inpainting, and medical image reconstruction. In this work, we focus in…

Scalable method completes ill-conditioned matrices from few samples.

We study a noisy tensor completion problem of broad practical interest, namely, the reconstruction of a low-rank tensor from highly incomplete and randomly corrupted observations of its entries. While a variety of prior work has been dedicated to this problem, prior algorithms either are computationally too expensive f…

Sparse Polyak improves high-dimensional statistical estimation.

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

Geometric tempering fails for Langevin dynamics, proving convergence limits.

Gaussian Graphical Models (GGMs) have wide-ranging applications in machine learning and the natural and social sciences. In most of the settings in which they are applied, the number of observed samples is much smaller than the dimension and they are assumed to be sparse. While there are a variety of algorithms (e.g. G…

Algorithm samples composite logconcave densities efficiently.

We consider the problem of sampling from a strongly log-concave density in $\mathbb{R}^d$, and prove an information theoretic lower bound on the number of stochastic gradient queries of the log density needed. Several popular sampling algorithms (including many Markov chain Monte Carlo methods) operate by using stochas…

The inference of deep hierarchical models is problematic due to strong dependencies between the hierarchies. We investigate a specific transformation of the model parameters based on the multivariate distributional transform. This transformation is a special form of the reparametrization trick, flattens the hierarchy a…

Semi-supervised learning deals with the problem of how, if possible, to take advantage of a huge amount of unclassified data, to perform a classification in situations when, typically, there is little labeled data. Even though this is not always possible (it depends on how useful, for inferring the labels, it would be …

KMRCD detects outliers in non-elliptical data using kernel trick.