arXiv research

JME continually estimates data moments privately and accurately.

We consider two stage estimation with a non-parametric first stage and a generalized method of moments second stage, in a simpler setting than (Chernozhukov et al. 2016). We give an alternative proof of the theorem given in (Chernozhukov et al. 2016) that orthogonal second stage moments, sample splitting and $n^{1/4}$-…

New optimizer Eve uses examplewise gradients for better second-moment estimates.

PMT uses public data moments to make DP feasible for unbounded data.

New adaptive stepsize method for stochastic approximation converges to target point.

AdamNX improves Adam's stability by adjusting its learning rate.

Several new estimation methods have been recently proposed for the linear regression model with observation error in the design. Different assumptions on the data generating process have motivated different estimators and analysis. In particular, the literature considered (1) observation errors in the design uniformly …

Enhances DP linear regression using public data moments.

A new method for uncertainty estimation in neural networks using existing optimization steps.

Normal distributions ensure asymptotic variance reduction in moment matching Monte Carlo.

MOMENT selects and estimates mixed-effects models using moment identities.

Derives moments of PL networks for robust DNNs.

We propose a method of moments (MoM) algorithm for training large-scale implicit generative models. Moment estimation in this setting encounters two problems: it is often difficult to define the millions of moments needed to learn the model parameters, and it is hard to determine which properties are useful when specif…

Study compares optimal vs. naive diversification in crypto markets, finds time-varying moments improve performance.

Adaptive gradient methods such as Adam have been shown to be very effective for training deep neural networks (DNNs) by tracking the second moment of gradients to compute the individual learning rates. Differently from existing methods, we make use of the most recent first moment of gradients to compute the individual …

In several recently proposed stochastic optimization methods (e.g. RMSProp, Adam, Adadelta), parameter updates are scaled by the inverse square roots of exponential moving averages of squared past gradients. Maintaining these per-parameter second-moment estimators requires memory equal to the number of parameters. For …

Study examines robust regression in high dimensions with heavy-tailed data.

DML-CMR estimator reduces bias in CMR problems using deep neural networks.

Iteratively reweighted least squares (IRLS) is a widely-used method in machine learning to estimate the parameters in the generalised linear models. In particular, IRLS for L1 minimisation under the linear model provides a closed-form solution in each step, which is a simple multiplication between the inverse of the we…

ADOPT optimizes Adam to converge with any β2 without bounded noise.

A new portfolio optimization method using the Sherman-Morrison identity.

In this paper we introduce an efficient fat-tail measurement framework that is based on the conditional second moments. We construct a goodness-of-fit statistic that has a direct interpretation and can be used to assess the impact of fat-tails on central data conditional dispersion. Next, we show how to use this framew…

Method estimates posterior model for boundary value problems with uncertain constraints.

Consider a random vector with finite second moments. If its precision matrix is an M-matrix, then all partial correlations are non-negative. If that random vector is additionally Gaussian, the corresponding Markov random field (GMRF) is called attractive. We study estimation of M-matrices taking the role of inverse sec…

We study the problem of estimating the mean of a random vector $X$ given a sample of $N$ independent, identically distributed points. We introduce a new estimator that achieves a purely sub-Gaussian performance under the only condition that the second moment of $X$ exists. The estimator is based on a novel concept of a…

New estimator tackles multi-task linear regression with outliers, avoiding eigenvalue lower bounds.

Corrected moment-based methods improve inference in topic model regression.

On compact surfaces, a Green-Wasserstein inequality cannot be improved without the sqrt(log n) factor.

Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.

The paper tackles matrix completion in ultra-sparse sampling, improving imputation accuracy.

Develops a new method for estimating models with conditional moment restrictions.

A new memory-efficient Adam variant reduces second moments when feasible.

We relate stability properties (i.e. moment exponents) of a stochastic dynamical system on a compact manifold $M$ to the homotopy and integral homology groups of $M$. In the special case of gradient Brownian systems associated to isometric immersions of $M$ in Euclidean space, these moment exponents can be estimated in…

New robust estimators achieve subgaussian bounds using VC-dimension.

Developed moment estimators for affine stochastic volatility models.

Huber regression assessed for robustness in statistical learning.

Paper proposes ClipSMT algorithm for better ATE estimation.

Second-order estimator improves continuous-time policy evaluation.

New algorithm learns HMM parameters on Riemannian manifolds.

We discuss the probabilistic properties of the variation based third and fourth moments of financial returns as estimators of the actual moments of the return distributions. The moment variations are defined under non-parametric assumptions with quadratic variation method but for the computational tractability, we use …

Double machine learning provides $\sqrt{n}$-consistent estimates of parameters of interest even when high-dimensional or nonparametric nuisance parameters are estimated at an $n^{-1/4}$ rate. The key is to employ Neyman-orthogonal moment equations which are first-order insensitive to perturbations in the nuisance param…

Paper develops methods for statistical inference in SGD with infinite variance.

We tackle causal inference under conditional moment restrictions using importance weighting.

A new method of moments estimator goes beyond data reweighting.

We discuss a certain Riemannian metric, related to the toric Kahler-Einstein equation, that is associated in a linearly-invariant manner with a given log-concave measure in R^n. We use this metric in order to bound the second derivatives of the solution to the toric Kahler-Einstein equation, and in order to obtain spec…

Paper develops robust methods for large-scale testing without tuning parameters.

Nonnegative matrix factorization (NMF) has been widely used in machine learning and signal processing because of its non-subtractive, part-based property which enhances interpretability. It is often assumed that the latent dimensionality (or the number of components) is given. Despite the large amount of algorithms des…

We extend the classical Cox-Ross-Rubinstein binomial model in two ways. We first develop a binomial model with time-dependent parameters that equate all moments of the pricing tree increments with the corresponding moments of the increments of the limiting Itô price process. Second, we introduce a new trinomial model i…