New KSDs control moments in approximations, improving diagnostics and tests.
problem Inability of standard KSDs to control moment convergence.
method Developed alternative diffusion KSDs under sufficient conditions.
result First KSDs to exactly characterize q-Wasserstein convergence.
This study shows the moment-SOS hierarchy converges in polynomial optimization over product of spheres.
problem Minimizing multihomogeneous polynomials over product of spheres.
method Moment-SOS hierarchy, local optimality conditions, differential geometry, Morse theory.
result The moment-SOS hierarchy has finite convergence for generic multihomogeneous objective functions.
Analyzes GJR-GARCH moments for efficient predictive distributions.
problem Estimating moments of GARCH processes for accurate predictions.
method Derives analytic expressions for GJR-GARCH moments and their limits.
result Analytic moments provide excellent approximate predictive distributions.
This work relaxes OT problems with marginal moments constraints, achieving finite discrete measures.
problem Solving Optimal Transport problems with marginal moments constraints.
method Relaxation of OT problems using moment constraints and Tchakaloff's theorem.
result The Moment Constrained Optimal Transport problem (MCOT) is achieved by a finite discrete measure.
RAME adapts learning rates using recent first moment of gradients.
problem Training deep neural networks efficiently and adaptively.
method RAME computes individual learning rates using the most recent first moment of gradients.
result RAME outperforms SHB, Adam, and RMSprop in convergence speed and generalization performance.
The paper examines linking numbers in grid models and finds polynomial moments.
problem Analyzing linking numbers in grid models.
method Examined linking numbers as a random variable on isotopy classes of 2-component links, computed moments and limits.
result The u u u th moment of the linking number is a polynomial in the grid size with degree d ≤ u d\leq u d ≤ u , and all odd moments vanish. Analyzes projections of test configurations to vector fields, proving moment convergence.
problem Analyzing moment convergence in projections of test configurations.
method Analytic approach involving moment convergence and weak geodesic ray.
result Proves moment convergence of weight distributions in projections.
ADOPT optimizes Adam to converge with any β2 without bounded noise.
problem Non-convergence of Adam optimization algorithm.
method ADOPT removes current gradient from second moment estimate and changes momentum update order.
result ADOPT achieves optimal convergence rate of O(1 / √T) with any β2.
COS method convergence conditions expanded for heavy-tailed distributions.
problem Ensuring convergence of the COS method for various densities.
method Analyzing truncation error and providing conditions for convergence.
result Conditions for COS method convergence extended to include heavy-tailed distributions.
Study resolvent convergence for random matrices with general covariance profiles.
problem Analyzing resolvent convergence for random matrices with non-identically distributed columns.
method Using moments of quadratic forms and deterministic equivalents, the study provides bounds on the trace of matrix products.
result The trace of matrix products is close to the trace of a deterministic equivalent, controlled by matrix norms.
Samplets and multiwavelets constructed from scattered data converge to specific densities in the limit.
problem Constructing data-adapted multiresolution analyses and multiwavelets with flexible vanishing moments.
method Probabilistic framework for samplet construction; convergence to multiwavelets with broken polynomial densities.
result Samplet construction converges to multiwavelets in the infinite data limit.
Proposes CMD for learning domain-invariant representations.
problem Learning domain-invariant representations in domain adaptation.
method Minimizes discrepancy between domain-specific latent feature representations using Central Moment Discrepancy (CMD).
result CMD achieves state-of-the-art performance on domain adaptation tasks.
Researchers create a new moduli space for Fano manifolds with special geometric properties.
problem Constructing a new moduli space for Fano manifolds with Kähler-Ricci solitons.
method Developed a moment map picture and used complex analytic charts to construct the moduli space.
result Created a larger moduli space that includes Fano manifolds with Kähler-Einstein metrics.
We associate certain probability measures on R \R R to geodesics in the space $\H_L$ of positively curved metrics on a line bundle L L L , and to geodesics in the finite dimensional symmetric space of hermitian norms on H 0 ( X , k L ) H^0(X, kL) H 0 ( X , k L ) . We prove that the measures associated to the finite dimensional spaces converge weakly to t…
DualAdam improves generalization of Adam by integrating its update mechanisms.
problem Adam's tendency to converge to sharp minima leading to suboptimal generalization.
method DualAdam combines Adam and inverse Adam's update mechanisms to enhance generalization.
result DualAdam outperforms Adam and state-of-the-art variants in generalization performance.
Learning rate needs to decrease with higher data moments for effective ICA in high dimensions.
problem Slower convergence of ICA in high-dimensional data with high-order moments.
method High-dimensional ODE analysis of ICA algorithm under controlled moment structure.
result Critical learning rate threshold for effective ICA when moments are high.
Derives a series expansion for Asian option pricing with polynomial jump-diffusion moments.
problem Pricing Asian options with polynomial jump-diffusion processes.
method Uses Hermite polynomials and moments of the underlying process for closed-form computation.
result Explicit computation of Greeks and accurate series expansion for Asian options.
The paper establishes criteria for approximating processes to match moments, useful for enriching data with simulated data.
problem Ensuring approximating processes match moments for data enrichment.
method Generalizes criteria for approximating processes to match moments, extending to random fields of stochastic processes.
result Uniform integrability is sufficient for matching moments, even for processes under weak stationarity.
Estimates Heston SDE parameters from observable realized volatilities.
problem Estimating parameters of Heston SDEs from observable data.
method Constructs estimators from empirical moments of realized volatilities over sliding windows.
result Explicit bounds for the convergence of realized volatilities to true volatilities.
Let G be a countable group which acts by isometries on a separable, but not necessarily proper, Gromov hyperbolic space X. We say the action of G is weakly hyperbolic if G contains two independent hyperbolic isometries. We show that a random walk on such G converges to the Gromov boundary almost surely. We apply the co…
Nonlinear SGD achieves high-probability rates in non-convex optimization with heavy-tailed noise.
problem Optimization in non-convex problems with heavy-tailed noise.
method General nonlinear framework for SGD, including symmetrization techniques.
result Achieves O ~ ( t − 1 / 2 ) \widetilde{\mathcal{O}}(t^{-1/2}) O ( t − 1/2 ) rate for heavy-tailed noise. Optimizes mixture models without parametrizing distributions using tensor decomposition.
problem Estimating conditionally-independent mixture models in high dimensions.
method Alternating least squares optimization scheme for tensor decomposition.
result Competitive performance and applicability to various models and applications.
Gradient descent on Hadamard manifolds converges to boundary points, solving optimization problems.
problem Optimization on Hadamard manifolds with unbounded convex functions.
method Gradient descent, duality theorem, moment-weight inequality.
result Gradient descent converges to boundary points, solving optimization problems.
New approach for testable learning using moment matching and Rademacher complexity.
problem Replacing hard-to-verify distributional assumptions with testable ones.
method Moment matching and metric distances in probability.
result Improved sample complexity bounds for various concept classes and distributions.
New adaptive stepsize method for stochastic approximation converges to target point.
problem Finding optimal step sizes for stochastic approximation algorithms.
method Adaptive block-coordinate stepsizes using online estimates of second moment.
result New method converges almost surely to a small neighborhood of the target point.
Donaldson defined a parabolic flow on Kahler manifolds which arises from considering the action of a group of symplectomorphisms on the space of smooth maps between manifolds. One can define a moment map for this action, and then consider the gradient flow of the square of its norm. Chen discovered the same flow from a…
Polymom unifies EM and method of moments for mixture models.
problem Estimating mixture models with non-convex likelihood.
method Polymom framework based on method of moments.
result Polymom provides global guarantees for mixture models.
Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.
problem Eigenvalue distribution of Wishart matrix with temporal correlation.
method Analysis of moments and convergence to deformed Marchenko-Pastur distribution for Gaussian process with temporal correlation.
result Eigenvalue distribution converges to deformed Marchenko-Pastur distribution with longer tail and higher peak.
New SGMM algorithm for efficient estimation of moment restriction models.
problem Estimation and inference on overidentified moment restriction models.
method Stochastic Approximation to Generalized Method of Moments (SGMM).
result SGMM offers fast and scalable implementation with streaming dataset handling.
MGD combines maximum entropy and diffusion methods for efficient sampling.
problem Generating samples from limited information in high dimensions.
method Moment Guided Diffusion (MGD) using stochastic differential equations.
result MGD efficiently samples maximum entropy distributions in finite time.
Tensor methods tackle high-dimensional additive index models with discordance and heterogeneity.
problem High-dimensional datasets with sampling problems and heterogeneity.
method Method of moments based procedures for estimating indices of discordant additive index models.
result Rates of convergence of estimators in both high and low-dimensional settings.
Paper proposes an efficient algorithm to handle high-order portfolio moments.
problem Designing portfolios with high-order moments (skewness and kurtosis) is computationally challenging.
method Proposes a SCA algorithm framework for solving high-order portfolios efficiently.
result Demonstrates the efficiency of the proposed algorithm through numerical experiments.
Study improves least squares estimation for heavy-tailed errors.
problem Improving least squares estimation under heteroscedastic and heavy-tailed errors.
method Analyzes the rate of convergence of least squares estimator under bounded conditional variance and finitely many moments of errors.
result Upper bounds on rates of convergence of LSE for heavy-tailed errors are found.
PMT uses public data moments to make DP feasible for unbounded data.
problem Applying differential privacy to unbounded data distributions.
method Public-moment-guided Truncation (PMT) using second-moments from public data.
result PMT improves the accuracy and stability of DP models.
Study shows Euler discretizations preserve exponential integrability of CIR process.
problem Analyzing exponential integrability of CIR process and its discretizations.
method Examined various Euler discretizations with truncation and reflection at 0.
result Preservation of exponential integrability for implicit and explicit Euler-Maruyama discretizations.
Gradient descent with random weights in linear regression analyzed for various noise types.
problem Analyzing the impact of random noise on gradient descent in linear regression.
method Gradient descent with randomly weighted data points, various weighting distributions, geometric moment contraction.
result Characterization of implicit regularization and non-asymptotic convergence bounds.
AdamNX improves Adam's stability by adjusting its learning rate.
problem Adam's tendency to converge to non-flat minima in large-scale models.
method Proposes a novel exponential decay mechanism for Adam's second-order moment estimate.
result AdamNX outperforms Adam and its variants in stability and performance.
A new method for assessing Bayesian sampling quality, PSD, is proposed and shown to be more powerful and efficient.
problem Scalability and convergence assessment of Bayesian sampling algorithms, especially for high-dimensional problems.
method Polynomial Stein Discrepancy (PSD) for measuring discrepancy between samples and posterior distributions.
result PSD detects differences in the first r moments for Gaussian targets and is more powerful and efficient than competitors.
Simplified SGD interpretation as Ito process for broader applicability.
problem Lack of generality in current SGD interpretation.
method Introduced a simplified scheme for discrete-time approximation of Ito process.
result Flexibly interprets SGD and SGLD, providing insights into their asymptotic properties.
Paper proposes a new approach for stochastic gradient descent in probabilistic modeling.
problem Finding optimal predictions in probabilistic models with large step sizes.
method Averaging moment parameters instead of natural parameters for constant-step-size stochastic gradient descent.
result Constant-step-size SGD can lead to better predictions in some cases and always converges in infinite-dimensional models.
For a finite-dimensional (but possibly noncompact) symplectic manifold with a compact group acting with a proper moment map, we show that the square of the moment map is an equivariantly perfect Morse function in the sense of Kirwan, and that the set of critical points of the square of the moment map is a countable dis…
This paper analyzes ASGD using SDEs for a more intuitive convergence rate.
problem Theoretical analysis of ASGD is limited by discrete methods and complex proofs.
method Continuous approximation of ASGD using SDDEs and convergence rate analysis methods.
result Continuous view provides better convergence rates and insights into ASGD.
The study analyzes convergence of adaptive optimizers under low-precision training.
problem Understanding why low-precision training remains effective for large models.
method Developed a theoretical framework for analyzing convergence of adaptive optimizers under floating-point quantization.
result Adaptive optimizers retain convergence rates close to full-precision methods under logarithmic mantissa scaling.
A new method for generating samples without training, using smoothed score matching.
problem Generating samples efficiently and without training.
method Moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD).
result The method enables fast, robust, training-free sampling with competitive sample fidelity and diversity.
AdaShift solves non-convergence issue of Adam by decorrelating gradient and second-moment terms.
problem Non-convergence of adaptive learning rate methods like Adam.
method AdaShift decorrelates gradient and second-moment terms by temporal shifting.
result AdaShift solves non-convergence problem of Adam and maintains competitive performance.
In this paper we investigate the convergence properties of the upwards gradient flow of the norm-square of a moment map on the space of representations of a quiver. The first main result gives a necessary and sufficient algebraic criterion for a complex group orbit to intersect the unstable set of a given critical poin…
The paper strengthens the classical result of MLE convergence to a Gaussian distribution.
problem The classical result of MLE convergence to a Gaussian distribution.
method Sub-Gaussian concentration and entropic normality of the normalized MLE.
result Entropic central limit theorem for a smoothed version of the estimator.
A new method uses LDPC codes to improve gradient descent in distributed computing.
problem Mitigating the effect of straggling processors in distributed computing.
method Encoding the second-moment of data with LDPC codes and iterative decoding.
result The method outperforms existing schemes in real distributed computing setups.