arXiv research

In this paper, we investigate the popular deep learning optimization routine, Adam, from the perspective of statistical moments. While Adam is an adaptive lower-order moment based (of the stochastic gradient) method, we propose an extension namely, HAdam, which uses higher order moments of the stochastic gradient. Our …

Learning rate needs to decrease with higher data moments for effective ICA in high dimensions.

Python package ajdmom simplifies moment formula derivation for jump diffusions.

JME continually estimates data moments privately and accurately.

Spectral features of the empirical moment matrix constitute a resourceful tool for unveiling properties of a cloud of points, among which, density, support and latent structures. It is already well known that the empirical moment matrix encodes a great deal of subtle attributes of the underlying measure. Starting from …

FedIV uses federated GMM for IV analysis in non-i.i.d. data.

Betas are possibly the most frequently applied tool to analyze how securities relate to the market. While in very widespread use, betas only express dynamics derived from second moment statistics. Financial returns data often deviate from normal assumptions in the sense that they have significant third and fourth order…

Analysis of Vlasov plasma dynamics using matched pair Lie-Poisson formulation.

A new method for estimating causal parameters from observables reduces the need for finite moment conditions.

Independent component analysis (ICA) is the problem of efficiently recovering a matrix $A \in \mathbb{R}^{n\times n}$ from i.i.d. observations of $X=AS$ where $S \in \mathbb{R}^n$ is a random vector with mutually independent coordinates. This problem has been intensively studied, but all existing efficient algorithms w…

We present a semi-supervised learning algorithm for learning discrete factor analysis models with arbitrary structure on the latent variables. Our algorithm assumes that every latent variable has an "anchor", an observed variable with only that latent variable as its parent. Given such anchors, we show that it is possi…

We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis …

The third moment variation of a financial asset return process is defined by the quadratic covariation between the return and square return processes. The skew and fat tail risk of an underlying asset can be hedged using a third moment variation swap under which a predetermined fixed leg and the floating leg of the rea…

Polynomial-time algorithm learns high-dimensional halfspaces without labels.

The paper examines how market trade values and volumes affect price autocorrelation.

The learning of domain-invariant representations in the context of domain adaptation with neural networks is considered. We propose a new regularization method that minimizes the discrepancy between domain-specific latent feature representations directly in the hidden activation space. Although some standard distributi…

A new memory-efficient Adam variant reduces second moments when feasible.

The paper analyzes distances and volumes in lens spaces using recursion and formulas.

Adaptive gradient methods such as Adam have been shown to be very effective for training deep neural networks (DNNs) by tracking the second moment of gradients to compute the individual learning rates. Differently from existing methods, we make use of the most recent first moment of gradients to compute the individual …

Factorial moments are convenient tools in nuclear physics to characterize the multiplicity distributions when phase-space resolution ($Δ$) becomes small. For uncorrelated particle production within $Δ$, Gaussian statistics holds and factorial moments $F_q$ are equal to unity for all orders $q$. Correlations between par…

We evaluate the average waiting time between observing the price of financial markets and the next price change, especially in an on-line foreign exchange trading service for individual customers via the internet. Basic technical idea of our present work is dependent on the so-called renewal-reward theorem. Assuming th…

We consider moment matching techniques for estimation in Latent Dirichlet Allocation (LDA). By drawing explicit links between LDA and discrete versions of independent component analysis (ICA), we first derive a new set of cumulant-based tensors, with an improved sample complexity. Moreover, we reuse standard ICA techni…

This work provides a computationally efficient and statistically consistent moment-based estimator for mixtures of spherical Gaussians. Under the condition that component means are in general position, a simple spectral decomposition technique yields consistent parameter estimates from low-order observable moments, wit…

Study finds the minimum number of finite Gaussian mixtures for best approximation.

The paper extends RDPG model to handle weighted graphs, enabling better analysis of network data.

New bounds on machine learning model generalization error moments.

Study on martingale property and moment explosions in signature volatility models.

New SQ lower bounds for NGCA without requiring chi-squared condition.

Study evaluates interpretability of time series foundation models' latent spaces.

Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.

The economic life of an asset is the optimum length of its usefulness, which is the moment that the asset's expenses are minimum. In this paper, the economic life of physical assets, such as industry machine and equipment, can be interpreted as the moment that the minimum is reached by its equivalent property cost func…

We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an associated ordinary differential equation (ODE). We obtain finite-time bounds on t…

Develops a new method for estimating models with conditional moment restrictions.

Paper provides Edgeworth expansions for network moments, improving accuracy of sampling distributions.

We present a detailed analysis of \emph{observable} moments based parameter estimators for the Heston SDEs jointly driving the rate of returns $R_t$ and the squared volatilities $V_t$. Since volatilities are not directly observable, our parameter estimators are constructed from empirical moments of realized volatilitie…

New algorithm learns ReLU networks efficiently using Schur polynomials.

Iteratively reweighted least squares (IRLS) is a widely-used method in machine learning to estimate the parameters in the generalised linear models. In particular, IRLS for L1 minimisation under the linear model provides a closed-form solution in each step, which is a simple multiplication between the inverse of the we…

The paper analyzes extreme risk measures with limited distributional information.

We test for departures from normal and independent and identically distributed (NIID) returns, when returns under the alternative hypothesis are self-affine. Self-affine returns are either fractionally integrated and long-range dependent, or drawn randomly from an L-stable distribution with infinite higher-order moment…

Improved bounds on moments of word measures on unitary groups.

We present and analyze a central cutting surface algorithm for general semi-infinite convex optimization problems, and use it to develop a novel algorithm for distributionally robust optimization problems in which the uncertainty set consists of probability distributions with given bounds on their moments. Moments of a…

Study on SA with heavy-tailed and LRD noise, establishing finite-time bounds.

Many complex systems generate multifractal time series which are long-range cross-correlated. Numerous methods have been proposed to characterize the multifractal nature of these long-range cross correlations. However, several important issues about these methods are not well understood and most methods consider only o…

In the setting of polynomial jump-diffusion dynamics, we provide an explicit formula for computing correlators, namely, cross-moments of the process at different time points along its path. The formula appears as a linear combination of exponentials of the generator matrix, extending the well-known moment formula for p…

Independent Component Analysis (ICA) - one of the basic tools in data analysis - aims to find a coordinate system in which the components of the data are independent. Most popular ICA methods use kurtosis as a metric of non-Gaussianity to maximize, such as FastICA and JADE. However, their assumption of fourth-order mom…

We develop efficient algorithms for estimating low-degree moments of unknown distributions in the presence of adversarial outliers. The guarantees of our algorithms improve in many cases significantly over the best previous ones, obtained in recent works of Diakonikolas et al, Lai et al, and Charikar et al. We also sho…

It is known that Heston's stochastic volatility model exhibits moment explosion, and that the critical moment $s_+$ can be obtained by solving (numerically) a simple equation. This yields a leading order expansion for the implied volatility at large strikes: $σ_{BS}( k,T)^{2}T\sim Ψ(s_+-1) \times k$ (Roger Lee's moment…

New method allocates capital based on tail central moments for financial risk assessment.