arXiv research

JME continually estimates data moments privately and accurately.

Enhances DP linear regression using public data moments.

PMT uses public data moments to make DP feasible for unbounded data.

The paper tackles matrix completion in ultra-sparse sampling, improving imputation accuracy.

Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.

Consider a random vector with finite second moments. If its precision matrix is an M-matrix, then all partial correlations are non-negative. If that random vector is additionally Gaussian, the corresponding Markov random field (GMRF) is called attractive. We study estimation of M-matrices taking the role of inverse sec…

New estimator tackles multi-task linear regression with outliers, avoiding eigenvalue lower bounds.

A new portfolio optimization method using the Sherman-Morrison identity.

A new method for uncertainty estimation in neural networks using existing optimization steps.

Study resolvent convergence for random matrices with general covariance profiles.

Iteratively reweighted least squares (IRLS) is a widely-used method in machine learning to estimate the parameters in the generalised linear models. In particular, IRLS for L1 minimisation under the linear model provides a closed-form solution in each step, which is a simple multiplication between the inverse of the we…

How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove …

Develops a Gaussian-based message-passing algorithm for noisy matrix completion.

A new memory-efficient Adam variant reduces second moments when feasible.

Estimation of the covariance matrix has attracted a lot of attention of the statistical research community over the years, partially due to important applications such as Principal Component Analysis. However, frequently used empirical covariance estimator (and its modifications) is very sensitive to outliers in the da…

Paper relaxes symmetry conditions for universal feature selection in noisy data.

Extends active subspace analysis to infinite dimensions.

We develop a general framework for applying the Kelly criterion to stock markets. By supplying an arbitrary probability distribution modeling the future price movement of a set of stocks, the Kelly fraction for investing each stock can be calculated by inverting a matrix involving only first and second moments. The fra…

New optimizer Eve uses examplewise gradients for better second-moment estimates.

Paper tackles robust matrix completion with heavy-tailed noise.

Sophisticated volatility models outperform naive portfolio strategies.

New adaptive stepsize method for stochastic approximation converges to target point.

In a recent paper [\textit{M. Cristelli, A. Zaccaria and L. Pietronero, Phys. Rev. E 85, 066108 (2012)}], Cristelli \textit{et al.} analysed relation between skewness and kurtosis for complex dynamical systems and identified two power-law regimes of non-Gaussianity, one of which scales with an exponent of 2 and the oth…

Sharp threshold found for aligning Gaussian-weighted graphs.

Independent component analysis (ICA) is the problem of efficiently recovering a matrix $A \in \mathbb{R}^{n\times n}$ from i.i.d. observations of $X=AS$ where $S \in \mathbb{R}^n$ is a random vector with mutually independent coordinates. This problem has been intensively studied, but all existing efficient algorithms w…

Sketchy reduces memory and compute requirements for adaptive regularization in deep learning.

We consider the problem of extracting a low-dimensional, linear latent variable structure from high-dimensional random variables. Specifically, we show that under mild conditions and when this structure manifests itself as a linear space that spans the conditional means, it is possible to consistently recover the struc…

In several recently proposed stochastic optimization methods (e.g. RMSProp, Adam, Adadelta), parameter updates are scaled by the inverse square roots of exponential moving averages of squared past gradients. Maintaining these per-parameter second-moment estimators requires memory equal to the number of parameters. For …

Develops MENT for interpreting and detecting changes in network trajectories.

Independent Component Analysis (ICA) is the problem of learning a square matrix $A$, given samples of $X=AS$, where $S$ is a random vector with independent coordinates. Most existing algorithms are provably efficient only when each $S_i$ has finite and moderately valued fourth moment. However, there are practical appli…

We consider deep classifying neural networks. We expose a structure in the derivative of the logits with respect to the parameters of the model, which is used to explain the existence of outliers in the spectrum of the Hessian. Previous works decomposed the Hessian into two components, attributing the outliers to one o…

We prove that in metric measure spaces where the entropy functional is K-convex along every Wasserstein geodesic any optimal transport between two absolutely continuous measures with finite second moments lives on a non-branching set of geodesics. As a corollary we obtain that in these spaces there exists only one opti…

We propose a simple method to learn linear causal cyclic models in the presence of latent variables. The method relies on equilibrium data of the model recorded under a specific kind of interventions ("shift interventions"). The location and strength of these interventions do not have to be known and can be estimated f…

ADOPT optimizes Adam to converge with any β2 without bounded noise.

In this paper we introduce an efficient fat-tail measurement framework that is based on the conditional second moments. We construct a goodness-of-fit statistic that has a direct interpretation and can be used to assess the impact of fat-tails on central data conditional dispersion. Next, we show how to use this framew…

Study detects edge correlation between unlabeled random graphs.

Data whitening and second order optimization harm generalization by reducing access to dataset information.

On compact surfaces, a Green-Wasserstein inequality cannot be improved without the sqrt(log n) factor.

Several new estimation methods have been recently proposed for the linear regression model with observation error in the design. Different assumptions on the data generating process have motivated different estimators and analysis. In particular, the literature considered (1) observation errors in the design uniformly …

This paper develops a new portfolio optimization framework that considers network spillovers.

In this paper we introduce a new approach to topic modelling that scales to large datasets by using a compact representation of the data and by leveraging the GPU architecture. In this approach, topics are learned directly from the co-occurrence data of the corpus. In particular, we introduce a novel mixture model whic…

New algorithms minimize regret in both adversarial and stochastic contexts.

The paper studies scaling limits of Wasserstein metrics on Gaussian mixture models.

We study the problem of estimating the mean of a random vector $X$ given a sample of $N$ independent, identically distributed points. We introduce a new estimator that achieves a purely sub-Gaussian performance under the only condition that the second moment of $X$ exists. The estimator is based on a novel concept of a…

New method finds closest martingale to Brownian motion.

While stochastic gradient descent (SGD) and variants have been surprisingly successful for training deep nets, several aspects of the optimization dynamics and generalization are still not well understood. In this paper, we present new empirical observations and theoretical results on both the optimization dynamics and…

Motivated by community detection, we characterise the spectrum of the non-backtracking matrix $B$ in the Degree-Corrected Stochastic Block Model. Specifically, we consider a random graph on $n$ vertices partitioned into two equal-sized clusters. The vertices have i.i.d. weights $\{ φ_u \}_{u=1}^n$ with second moment $Φ…

We adapt to an infinite dimensional ambient space E.R. Reifenberg's epiperimetric inequality and a quantitative version of D. Preiss' second moments computations to establish that the set of regular points of an almost mass minimizing rectifiable $G$ chain in $\ell_2$ is dense in its support, whenever the group $G$ of …