Empirical moment matrix reveals properties of point clouds.
problem Uncovering properties of point clouds, especially those with singular support.
method Combining statistics, real algebraic geometry, and approximation theory.
result The empirical moment matrix provides insights into data analysis.
Paper proposes an algorithm for automatically selecting latent dimensions in NMF.
problem Automatic model selection for NMF with theoretical guarantees.
method Empirical second-order moment and support union recovery.
result The algorithm provably detects the true latent dimensionality.
The paper tackles matrix completion in ultra-sparse sampling, improving imputation accuracy.
problem Matrix completion in ultra-sparse sampling, where each row has only a few entries.
method Estimate row span of matrix or averaged second-moment matrix, normalize and impute missing entries.
result Gradient descent method normalizes and imputes missing entries, achieving low variance and unbiased estimation.
JME continually estimates data moments privately and accurately.
problem Private and accurate continual estimation of data moments.
method Uses matrix mechanism and joint sensitivity analysis.
result Improves accuracy in estimating mean and covariance with reduced noise.
Study on Gaussian ensemble of matrix products with mixed moments computed.
problem Understanding the statistical properties of matrix products of Gaussian matrices.
method Analysis of a multi-Wishart ensemble and enumeration of non-crossing pairings.
result Mixed moments of the product matrix are computed and found to be weighted by Fuss-Catalan numbers at large N. Classifies SL(n) covariant matrix-valued valuations on Lp-spaces.
problem Classifying SL(n) covariant matrix-valued valuations on Lp-spaces.
method Established a complete classification for continuous and SL(n) covariant matrix-valued valuations on Lp(Rn,|x|2dx), eliminating matrix symmetry assumption.
result Unique characterization of such valuations by the moment matrix in n>2, rotation matrix in 2D.
Estimates mean of random vector with near-optimal error in all directions.
problem Estimating the mean of a random vector with direction-dependent accuracy.
method Proves existence of an estimator with near-optimal error in all directions under certain conditions.
result The estimator satisfies the error bound for all directions, with probability 1-δ.
Classification of SL(n) covariant valuations on Orlicz spaces.
problem Classifying continuous SL(n) covariant valuations on Orlicz spaces.
method Complete classification without symmetric assumptions, focusing on moment matrix and a new functional in dimension two.
result The moment matrix is the only SL(n) covariant valuation for n≥3, and a new functional appears in dimension two.
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example. In this paper we demonstrate the optimality of Maximum Entropy methods in approxi…
DGMM improves Gaussian mixture modeling efficiency and stability.
problem Efficiently estimating Gaussian mixtures in high dimensions.
method Diagonally-weighted generalized method of moments (DGMM).
result DGMM achieves smaller estimation errors with shorter runtime.
MOMENT selects and estimates mixed-effects models using moment identities.
problem Selecting and estimating random-effects covariance matrix and fixed-effects coefficients in multiresponse linear mixed-effects models.
method MOMENT is a stage-wise moment-based framework that reduces the random-effects selection problem to a smooth constrained convex optimization problem.
result MOMENT performs competitively and can outperform separate univariate analyses for correlated responses.
A new method of moments estimator goes beyond data reweighting.
problem Estimation of moment restrictions and conditional moment restrictions.
method Kernel Method of Moments (KMM) based on maximum mean discrepancy.
result KMM achieves competitive performance on conditional moment restriction tasks.
New method improves estimation of complex models from conditional moment restrictions.
problem Estimation of complex models from conditional moment restrictions.
method Functional Generalized Empirical Likelihood (GEL) with a practical method.
result The method achieves state-of-the-art performance on two problems.
Developed moment estimators for affine stochastic volatility models.
problem Estimating parameters of affine stochastic volatility models.
method Introduced recursive equations for moments and proposed moment estimators.
result Established a central limit theorem and derived asymptotic covariance matrix.
The paper analyzes stability of random matrix products with Markovian noise.
problem Analyzing stability of random matrix products with Markovian noise.
method Using a super-Lyapunov drift condition and controlled growth of matrix-valued functions, the paper provides an exponential stability result for the p-th moment of random matrix product.
result Finite-time p-th moment bounds for linear stochastic approximation and TD learning algorithms.
Many pattern recognition methods rely on statistical information from centered data, with the eigenanalysis of an empirical central moment, such as the covariance matrix in principal component analysis (PCA), as well as partial least squares regression, canonical-correlation analysis and Fisher discriminant analysis. R…
New estimator tackles multi-task linear regression with outliers, avoiding eigenvalue lower bounds.
problem Multi-task linear regression with contaminated tasks and eigenvalue lower bounds failure.
method Matrix-weighted norm regularization and relative balancedness condition.
result Prediction MSE bounds match Duan and Wang (2023) under weaker spectral assumptions.
This paper identifies and bounds ICE central moments using PO marginal central moments.
problem Identifying and characterizing treatment effect heterogeneity.
method Using only marginal central moments of potential outcomes, the paper identifies and bounds central moments of individual causal effects.
result Identification and bounding of central moments of ICE using marginal moments of POs.
The paper develops a mathematical model for strategic shifts.
problem Finding optimal moments for strategy changes in market dynamics.
method Explicit strategy formulation using fluctuation theory.
result Analytical results predict optimal strategy shifts.
Enhances DP linear regression using public data moments.
problem Limited utility of traditional DP methods in linear regression.
method Transform private data using public second-moment matrix for a better OLSE.
result Improved accuracy and robustness of OLSE in DP linear regression.
Study resolvent convergence for random matrices with general covariance profiles.
problem Analyzing resolvent convergence for random matrices with non-identically distributed columns.
method Using moments of quadratic forms and deterministic equivalents, the study provides bounds on the trace of matrix products.
result The trace of matrix products is close to the trace of a deterministic equivalent, controlled by matrix norms.
PMT uses public data moments to make DP feasible for unbounded data.
problem Applying differential privacy to unbounded data distributions.
method Public-moment-guided Truncation (PMT) using second-moments from public data.
result PMT improves the accuracy and stability of DP models.
Corrected proof for C^2UCB contextual combinatorial bandit's regret bound.
problem Error in proof of C^2UCB contextual combinatorial bandit's regret bound.
method Demonstrated and corrected an error in the proof of volumetric expansion of the moment matrix.
result Proved a relaxed inequality that yields the originally-stated regret bound.
Improved portfolio optimization method reduces risk and improves performance.
problem Minimizing risk in large portfolios with limited data.
method Combines Tikhonov regularization and direct shrinkage of portfolio weights.
result Significantly reduces out-of-sample variance and Sharpe ratio compared to existing methods.
Independent component analysis (ICA) is the problem of efficiently recovering a matrix A∈Rn×n from i.i.d. observations of X=AS where S∈Rn is a random vector with mutually independent coordinates. This problem has been intensively studied, but all existing efficient algorithms w…
New method tightens sub-Gaussian concentration inequalities.
problem Estimating variance-type parameters of sub-Gaussian distributions.
method Using sub-Gaussian intrinsic moment norm to maximize normalized moments.
result Provides tighter sub-Gaussian concentration inequalities.
Paper provides unbiased spectral moment estimates from finite data.
problem Challenges in estimating spectral moments from limited data.
method Dynamic programming approach to estimate spectral moments of kernel integral operator.
result Demonstrates consistency with theoretical spectra and practical utility in neural networks.
A new metric assesses latent variable models using data and model moments.
problem Difficulty in assessing the quality of unsupervised learning models.
method A moment-matching metric using matrix norms to compare data and model moments.
result The proposed metric is faster and has less variance than alternative methods.
Independent Component Analysis (ICA) is the problem of learning a square matrix A, given samples of X=AS, where S is a random vector with independent coordinates. Most existing algorithms are provably efficient only when each Si has finite and moderately valued fourth moment. However, there are practical appli…
Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.
problem Eigenvalue distribution of Wishart matrix with temporal correlation.
method Analysis of moments and convergence to deformed Marchenko-Pastur distribution for Gaussian process with temporal correlation.
result Eigenvalue distribution converges to deformed Marchenko-Pastur distribution with longer tail and higher peak.
For a GJR-GARCH specification with a generic innovation distribution we derive analytic expressions for the first four conditional moments of the forward and aggregated returns and variances. Moment for the most commonly used GARCH models are stated as special cases. We also the limits of these moments as the time hori…
Estimation of the covariance matrix has attracted a lot of attention of the statistical research community over the years, partially due to important applications such as Principal Component Analysis. However, frequently used empirical covariance estimator (and its modifications) is very sensitive to outliers in the da…
Proposes robust graph embedding with noisy link weights.
problem Learning feature vectors from noisy link weights.
method β-graph embedding with empirical moment β-score.
result Computational tractability and local minimization of β-score.
Iteratively reweighted least squares (IRLS) is a widely-used method in machine learning to estimate the parameters in the generalised linear models. In particular, IRLS for L1 minimisation under the linear model provides a closed-form solution in each step, which is a simple multiplication between the inverse of the we…
Spectral learning extends matrix methods to tensors for better latent variable modeling.
problem Limitations of matrix-based spectral methods in capturing non-Gaussian data.
method Extend spectral decomposition to tensor-based methods for higher-order moments.
result Tensor decomposition can identify latent effects missed by matrix methods.
Enhanced Adam uses higher-order moments for better performance.
problem Improving the performance of Adam optimization algorithm.
method Proposes HAdam, an extension of Adam using higher-order moments of the stochastic gradient.
result Higher-order moments of the stochastic gradient can lead to better performance than vanilla Adam.
Formula for computing cross-moments of polynomial processes.
problem Computing cross-moments of polynomial jump-diffusion dynamics.
method Explicit formula based on linear combinations of exponentials of the generator matrix.
result Closed and compact formulations for correlators, useful in financial pricing.
The paper explores tail diversification in financial markets using entropy and mutual information.
problem Tail diversification in financial time series.
method Statistical independence through differential entropy and mutual information, using moments as contrast functions.
result Tail covariance matrix is a key driver of tail diversification.
A new method for generating samples without training, using smoothed score matching.
problem Generating samples efficiently and without training.
method Moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD).
result The method enables fast, robust, training-free sampling with competitive sample fidelity and diversity.
We prove optimal subspace embedding conjecture up to sub-polylogarithmic factors.
problem Optimal dimension and sparsity of subspace embeddings.
method Iterative decoupling technique to analyze higher-order trace moment bounds.
result Sub-polylogarithmic factors in dimension and sparsity of subspace embeddings.
In this paper, we study the confounder detection problem in the linear model, where the target variable Y is predicted using its n potential causes Xn=(x1,...,xn)T. Based on an assumption of rotation invariant generating process of the model, recent study shows that the spectral measure induced by the regress…
Proposes a method to identify elements in a skewness matrix for multivariate skew-elliptical distributions.
problem Label switching issue in Bayesian estimation of skewness matrix.
method Imposes a positive lower-triangular constraint and uses Bayesian sparse estimation with horseshoe prior.
result Successfully estimates the true structure of skewness dependency.
We consider two stage estimation with a non-parametric first stage and a generalized method of moments second stage, in a simpler setting than (Chernozhukov et al. 2016). We give an alternative proof of the theorem given in (Chernozhukov et al. 2016) that orthogonal second stage moments, sample splitting and n1/4-…
The paper analyzes the performance of empirical risk minimization for p-norm linear regression.
problem Empirical risk minimization on p-norm linear regression. method Analyzes performance under various conditions and moment assumptions.
result High probability excess risk bounds for empirical risk minimizer, matching asymptotic rates.
Study spectral density of neural networks using resolvent method.
problem Investigate spectral density of neural networks with random feature matrices.
method Use resolvent method from random matrix theory, cumulant expansion.
result Impossible to preserve singular value distribution with additive bias.
Singular values of a data in a matrix form provide insights on the structure of the data, the effective dimensionality, and the choice of hyper-parameters on higher-level data analysis tools. However, in many practical applications such as collaborative filtering and network analysis, we only get a partial observation.…
Corrected whitening restores orthogonality in high-dimensional spherical Gaussian mixtures.
problem In high-dimensional data, standard whitening fails to preserve orthogonality of mixture means.
method Derived exact limits for whitened means dot products using random matrix theory, constructed a corrected whitening matrix.
result Corrected whitening allows for improved estimation of spherical Gaussian mixtures in the large-dimensional regime.
The hidden tail of empirical distributions is analyzed using extreme value theory.
problem Understanding the bias between in-sample mean and true statistical mean for large n. method Extreme value theory applied to empirical distributions and their moments.
result The hidden moment of order 0 for power law distributions follows an exponential distribution with expectation 1/n.