arXiv research

Algorithm learns weight matrix from single trajectory of nonlinear dynamical system.

Note on subgaussian bounds for sign-quantized linear maps.

This paper explores conditions for neural networks to extrapolate to new domains.

The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized co…

KMRCD detects outliers in non-elliptical data using kernel trick.

Lower bounds on MALA and HMC for well-conditioned distributions.

We introduce a distributionally robust maximum likelihood estimation model with a Wasserstein ambiguity set to infer the inverse covariance matrix of a $p$-dimensional Gaussian random vector from $n$ independent samples. The proposed model minimizes the worst case (maximum) of Stein's loss across all normal reference d…

Reward-free RL in linear MDPs is as hard as reward-aware RL.

This paper shows how to approximate any log-concave distribution using well-conditioned affine coupling flows.

The literature provides strong evidence that stock prices can be predicted from past price data. Principal component analysis (PCA) is a widely used mathematical technique for dimensionality reduction and analysis of data by identifying a small number of principal components to explain the variation found in a data set…

Improved Lasso estimator speeds up variable selection.

Algorithm clusters mixtures with bounded covariances under specific separation conditions.

With the advent of massive data sets much of the computational science and engineering community has moved toward data-intensive approaches in regression and classification. However, these present significant challenges due to increasing size, complexity and dimensionality of the problems. In particular, covariance mat…

Improved subgradient method tackles ill-conditioned composite optimization problems.

Two new tests assess how well conditional models fit data.

A well-conditioned Jacobian spectrum has a vital role in preventing exploding or vanishing gradients and speeding up learning of deep neural networks. Free probability theory helps us to understand and handle the Jacobian spectrum. We rigorously show almost sure asymptotic freeness of layer-wise Jacobians of deep neura…

Study shows high-dimensional sparse RL hardness and Lasso Q-iteration's nearly dimension-free regret.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

Unified complexity bound for sampling logconcave distributions

Sparse feature selection improves batch RL efficiency.

Optimal DP-GD achieves high-dimensional linear regression risk.

Semi-supervised learning deals with the problem of how, if possible, to take advantage of a huge amount of not classified data, to perform classification, in situations when, typically, the labelled data are few. Even though this is not always possible (it depends on how useful is to know the distribution of the unlabe…

We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …

We quantify forgetting in post-training models, distinguishing mass and drift.

SLR tackles sparse linear regression problems, showing hardness for efficient algorithms.

We provide fast algorithms for overconstrained $\ell_p$ regression and related problems: for an $n\times d$ input matrix $A$ and vector $b\in\mathbb{R}^n$, in $O(nd\log n)$ time we reduce the problem $\min_{x\in\mathbb{R}^d} \|Ax-b\|_p$ to the same problem with input matrix $\tilde A$ of dimension $s \times d$ and corr…

Scattering networks maximize separation on low-dimensional data.

Polynomial-time algorithm learns ReLU networks without assumptions.

We study the problem of learning overcomplete HMMs---those that have many hidden states but a small output alphabet. Despite having significant practical importance, such HMMs are poorly understood with no known positive or negative results for efficient learning. In this paper, we present several new results---both po…

The study examines denoising and noisy-input regression under distribution shift, revealing double descent behavior and insights for data augmentation.

New framework explains normalizing flows' power and limitations.

Nonnegative matrix factorization (NMF) under the separability assumption can provably be solved efficiently, even in the presence of noise, and has been shown to be a powerful technique in document classification and hyperspectral unmixing. This problem is referred to as near-separable NMF and requires that there exist…

Improved SVRG for quadratic functions achieves better performance and running times.

We propose a decentralized Maximum Likelihood solution for estimating the stochastic renewable power generation and demand in single bus Direct Current (DC) MicroGrids (MGs), with high penetration of droop controlled power electronic converters. The solution relies on the fact that the primary control parameters are se…

Hidden Markov models have successfully been applied as models of discrete time series in many fields. Often, when applied in practice, the parameters of these models have to be estimated. The currently predominating identification methods, such as maximum-likelihood estimation and especially expectation-maximization, a…

New algorithm reduces pricing error by a factor of T^2/3.

OSA overcomes instability in skipless Transformers.

We develop the first quantum algorithm for the constrained portfolio optimization problem. The algorithm has running time $\widetilde{O} \left( n\sqrt{r} \frac{ζκ}{δ^2} \log \left(1/ε\right) \right)$, where $r$ is the number of positivity and budget constraints, $n$ is the number of assets in the portfolio, $ε$ the des…

In this paper we consider the training of single hidden layer neural networks by pseudoinversion, which, in spite of its popularity, is sometimes affected by numerical instability issues. Regularization is known to be effective in such cases, so that we introduce, in the framework of Tikhonov regularization, a matricia…

New bounds for high-dimensional sparse linear bandits, balancing information and regret.

Parameter reduction has been an important topic in deep learning due to the ever-increasing size of deep neural network models and the need to train and run them on resource limited machines. Despite many efforts in this area, there were no rigorous theoretical guarantees on why existing neural net compression methods …

Many applications in signal processing benefit from the sparsity of signals in a certain transform domain or dictionary. Synthesis sparsifying dictionaries that are directly adapted to data have been popular in applications such as image denoising, inpainting, and medical image reconstruction. In this work, we focus in…

Scalable method completes ill-conditioned matrices from few samples.

Sparse Polyak improves high-dimensional statistical estimation.

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

Gradient descent dynamics studied for DEQs in linear and single-index models.

Geometric tempering fails for Langevin dynamics, proving convergence limits.

Gaussian Graphical Models (GGMs) have wide-ranging applications in machine learning and the natural and social sciences. In most of the settings in which they are applied, the number of observed samples is much smaller than the dimension and they are assumed to be sparse. While there are a variety of algorithms (e.g. G…