arXiv research

Lower bounds on MALA and HMC for well-conditioned distributions.

This paper shows how to approximate any log-concave distribution using well-conditioned affine coupling flows.

This paper explores conditions for neural networks to extrapolate to new domains.

Unified complexity bound for sampling logconcave distributions

A well-conditioned Jacobian spectrum has a vital role in preventing exploding or vanishing gradients and speeding up learning of deep neural networks. Free probability theory helps us to understand and handle the Jacobian spectrum. We rigorously show almost sure asymptotic freeness of layer-wise Jacobians of deep neura…

New framework explains normalizing flows' power and limitations.

Semi-supervised learning deals with the problem of how, if possible, to take advantage of a huge amount of not classified data, to perform classification, in situations when, typically, the labelled data are few. Even though this is not always possible (it depends on how useful is to know the distribution of the unlabe…

Geometric tempering fails for Langevin dynamics, proving convergence limits.

Improved subgradient method tackles ill-conditioned composite optimization problems.

We propose a decentralized Maximum Likelihood solution for estimating the stochastic renewable power generation and demand in single bus Direct Current (DC) MicroGrids (MGs), with high penetration of droop controlled power electronic converters. The solution relies on the fact that the primary control parameters are se…

Polynomial-time algorithm learns ReLU networks without assumptions.

Algorithm learns weight matrix from single trajectory of nonlinear dynamical system.

Improved Lasso estimator speeds up variable selection.

Note on subgaussian bounds for sign-quantized linear maps.

Study shows high-dimensional sparse RL hardness and Lasso Q-iteration's nearly dimension-free regret.

SLR tackles sparse linear regression problems, showing hardness for efficient algorithms.

We provide fast algorithms for overconstrained $\ell_p$ regression and related problems: for an $n\times d$ input matrix $A$ and vector $b\in\mathbb{R}^n$, in $O(nd\log n)$ time we reduce the problem $\min_{x\in\mathbb{R}^d} \|Ax-b\|_p$ to the same problem with input matrix $\tilde A$ of dimension $s \times d$ and corr…

New algorithm reduces pricing error by a factor of T^2/3.

We introduce a distributionally robust maximum likelihood estimation model with a Wasserstein ambiguity set to infer the inverse covariance matrix of a $p$-dimensional Gaussian random vector from $n$ independent samples. The proposed model minimizes the worst case (maximum) of Stein's loss across all normal reference d…

Two new tests assess how well conditional models fit data.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

New bounds for high-dimensional sparse linear bandits, balancing information and regret.

Algorithm samples composite logconcave densities efficiently.

The inference of deep hierarchical models is problematic due to strong dependencies between the hierarchies. We investigate a specific transformation of the model parameters based on the multivariate distributional transform. This transformation is a special form of the reparametrization trick, flattens the hierarchy a…

Semi-supervised learning deals with the problem of how, if possible, to take advantage of a huge amount of unclassified data, to perform a classification in situations when, typically, there is little labeled data. Even though this is not always possible (it depends on how useful, for inferring the labels, it would be …

KMRCD detects outliers in non-elliptical data using kernel trick.

Reward-free RL in linear MDPs is as hard as reward-aware RL.

We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …

The study examines denoising and noisy-input regression under distribution shift, revealing double descent behavior and insights for data augmentation.

Sparse feature selection improves batch RL efficiency.

Lower bounds show many sampling algorithms need many gradient queries.

PMT uses public data moments to make DP feasible for unbounded data.

Algorithm clusters mixtures with bounded covariances under specific separation conditions.

Scattering networks maximize separation on low-dimensional data.

The literature provides strong evidence that stock prices can be predicted from past price data. Principal component analysis (PCA) is a widely used mathematical technique for dimensionality reduction and analysis of data by identifying a small number of principal components to explain the variation found in a data set…

We study the problem of learning overcomplete HMMs---those that have many hidden states but a small output alphabet. Despite having significant practical importance, such HMMs are poorly understood with no known positive or negative results for efficient learning. In this paper, we present several new results---both po…

MESSY estimation recovers symbolic density functions from samples using maximum entropy.

Improved estimator for least squares using random projections achieves smaller error.

New method uses random features and Tikhonov regularization for operator learning from noisy data.

Nonnegative matrix factorization (NMF) under the separability assumption can provably be solved efficiently, even in the presence of noise, and has been shown to be a powerful technique in document classification and hyperspectral unmixing. This problem is referred to as near-separable NMF and requires that there exist…

We quantify forgetting in post-training models, distinguishing mass and drift.

Improved SVRG for quadratic functions achieves better performance and running times.

The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized co…

Hidden Markov models have successfully been applied as models of discrete time series in many fields. Often, when applied in practice, the parameters of these models have to be estimated. The currently predominating identification methods, such as maximum-likelihood estimation and especially expectation-maximization, a…

Optimal DP-GD achieves high-dimensional linear regression risk.

We consider a refinement of differential privacy --- per instance differential privacy (pDP), which captures the privacy of a specific individual with respect to a fixed data set. We show that this is a strict generalization of the standard DP and inherits all its desirable properties, e.g., composition, invariance to …

OSA overcomes instability in skipless Transformers.

We develop the first quantum algorithm for the constrained portfolio optimization problem. The algorithm has running time $\widetilde{O} \left( n\sqrt{r} \frac{ζκ}{δ^2} \log \left(1/ε\right) \right)$, where $r$ is the number of positivity and budget constraints, $n$ is the number of assets in the portfolio, $ε$ the des…