Study quantifies how LLMs capture higher-order statistical structure using cumulant expansion.
problem Understanding how LLMs internalize statistical structure during next-token prediction.
method Cumulant-expansion framework treating softmax entropy as perturbation around center distribution.
result Cumulants reveal distinct signatures for mathematical vs. general text prompts, quantifying feature-learning dynamics.
Neural networks can learn from higher-order cumulants efficiently, requiring quadratic samples.
problem Learning from higher-order cumulants in high-dimensional data.
method Spiked cumulant model, polynomial time algorithms, neural networks, random features.
result Neural networks require quadratic samples to learn from higher-order cumulants efficiently, while random features require more samples.
Paper proposes a new method to identify causal graphs with latent variables using higher-order cumulants.
problem Estimating causal directed acyclic graphs with latent confounders.
method Uses higher-order cumulants to identify causal structures among observed and latent variables.
result Validates the proposed algorithm through simulations and real-world data.
The paper identifies causal effects in latent variable models using higher-order cumulants.
problem Challenges in identifying causal effects in latent variable models with latent confounders.
method Using higher-order cumulants, the paper addresses two challenging setups: a single proxy variable and underspecified instrumental variables.
result Causal effects are identifiable with a single proxy or instrument.
The study examines higher-order modern portfolio theory with complex critical points and feasible portfolio variety.
problem Understanding the complex critical points and feasible portfolio variety in higher-order modern portfolio theory.
method Established genericity conditions for utility functions with higher-order cumulants, analyzed discriminant loci, and determined the dimension and degree of the feasible portfolio variety.
result The utility function has a constant number of complex critical points under genericity conditions, and the feasible portfolio variety has a determined dimension and degree.
Diffusion models learn simple statistics before complex ones, revealing a sample complexity exponent.
problem Understanding the learning dynamics of diffusion models.
method Empirical observations and theoretical analysis of diffusion models and denoisers.
result Diffusion models learn simple statistics (pair-wise correlations) at linear sample complexity, while higher-order statistics (e.g., fourth cumulant) require cubic sample complexity.
New method identifies causal direction with latent confounders.
problem Identifying causal direction in presence of multiple latent variables.
method Use of joint higher-order cumulant matrix properties.
result Causal asymmetry can be seen from rank deficiency properties of cumulant matrices.
New method identifies structural parameters without assuming uncorrelated errors.
problem Identifying structural parameters in simultaneous equation models.
method Exploits higher-order cumulant restrictions, not requiring uncorrelated errors.
result Simple diagonality condition on hth-order cumulants identifies structural parameter matrix. New method identifies latent variables with causal dependencies from observed data.
problem Identify latent variables with causal relationships from observed data.
method Linear causal disentanglement via higher-order cumulants, with perfect and soft interventions.
result Recovery of parameters via coupled tensor decomposition and polynomial equations.
Neural networks learn faster with correlated latent variables.
problem Efficiently learning from higher-order correlations in neural networks.
method Analytical derivation and simulations of two-layer neural networks.
result Correlations between latent variables speed up learning from higher-order correlations.
New methods improve estimation accuracy in noisy settings.
problem Estimating treatment effects in the presence of treatment noise.
method Developed new structure-agnostic cumulant estimators and practical procedures for higher-order robustness.
result Demonstrated that existing DML estimator is suboptimal for non-Gaussian treatment noise and introduced ACE procedures for improved accuracy.
Study optimizes zero-order strongly convex function minimization with higher order smoothness.
problem Optimizing a strongly convex function with noisy evaluations.
method Randomized approximation of projected gradient descent with smoothing kernel.
result Upper bounds and minimax lower bounds for the algorithm, showing near-optimality.
The parametric complexity is the key quantity in the minimum description length (MDL) approach to statistical model selection. Rissanen and others have shown that the parametric complexity of a statistical model approaches a simple function of the Fisher information volume of the model as the sample size n goes to in…
New tensor framework connects Fisher information, hypergraphs, and multi-observable correlations.
problem Missing structure in pairwise Fisher graphs for multi-observable radiation patterns.
method Higher-order Fisher tensors and natural exponential-family coordinates.
result Exact triality of Fisher tensors, cumulants, and hypergraphs.
New method learns graph structure with hidden causes from observational data.
problem Learning the structure of linear non-Gaussian models with hidden causes.
method Augments hidden variable structure by learning multidirected edges and uses higher order cumulants.
result Correct structure recovery for bow-free acyclic mixed graphs with multi-directed edges.
Based on a faithful representation of the heavy tail multivariate distribution of asset returns introduced previously (Sornette et al., 1998, 1999) that we extend to the case of asymmetric return distributions, we generalize the return-risk efficient frontier concept to incorporate the dimensions of large risks embedde…
The paper provides non-asymptotic Edgeworth expansions for neural network outputs.
problem Approximating deviations of finite-width neural networks from their Gaussian limit.
method Multidimensional Edgeworth expansions of arbitrary order for neural network outputs.
result Established a bound on the total variation distance between neural network output and its Edgeworth approximation.
Expectation Propagation (EP) provides a framework for approximate inference. When the model under consideration is over a latent Gaussian field, with the approximation being Gaussian, we show how these approximations can systematically be corrected. A perturbative expansion is made of the exact but intractable correcti…
The paper proves local laws for non-separable sample covariance matrices.
problem Analyzing non-separable sample covariance matrices with dependent or nonlinearly transformed data.
method Tensor network framework for analyzing fluctuation averaging in the presence of higher-order cumulant structure.
result Optimal averaged local law and full anisotropic local law for non-separable sample covariance matrices.
We present a novel algorithm for overcomplete independent components analysis (ICA), where the number of latent sources k exceeds the dimension p of observed variables. Previous algorithms either suffer from high computational complexity or make strong assumptions about the form of the mixing matrix. Our algorithm does…
New algorithm identifies causal effects in latent confounding models.
problem Identifying causal effects in linear non-Gaussian models with latent confounding.
method Recursive algorithm using rank conditions on higher-order cumulants.
result Algorithm achieves comparable performance to overcomplete ICA without knowing the number of latent variables.
New algorithm optimizes smooth functions with Hölder exponent > 1.
problem Optimizing smooth functions with unknown Hölder exponent > 1.
method Two-layer algorithms using misspecified linear/polynomial bandit algorithms in bins.
result Regret bound of O~(Td+2αd+α) for α>1. Derives a general derivative identity for conditional mean in Gaussian noise.
problem Understanding conditional mean in Gaussian noise channels.
method Derives a general derivative identity for the conditional mean of X given Y=y in a Markov chain U↔X↔Y. result Provides a unifying view of conditional mean identities and derives new ones.
In this paper we calibrate chaotic models for interest rates to market data using a polynomial-exponential parametrization for the chaos coefficients. We identify a subclass of one-variable models that allow us to introduce complexity from higher order chaos in a controlled way while retaining considerable analytic tra…
Higher-order tensors arise frequently in applications such as neuroimaging, recommendation system, social network analysis, and psychological studies. We consider the problem of low-rank tensor estimation from possibly incomplete, ordinal-valued observations. Two related problems are studied, one on tensor denoising an…
The CSA-ES is an Evolution Strategy with Cumulative Step size Adaptation, where the step size is adapted measuring the length of a so-called cumulative path. The cumulative path is a combination of the previous steps realized by the algorithm, where the importance of each step decreases with time. This article studies …
Using methods introduced by Scargle in 1978 we derive a cumulative version of the Lomb periodogram that exhibits frequency independent statistics when applied to cumulative noise. We show how this cumulative Lomb periodogram allows us to estimate the significance of log-periodic signatures in the S&P 500 anti-bubble th…
We analyze the semi-hard triplet loss using Edgeworth expansion for better understanding of its behavior.
problem Understanding the behavior of the semi-hard triplet loss function.
method Developed a higher-order asymptotic analysis using the Edgeworth expansion.
result Derived explicit Edgeworth expansions revealing first-order corrections in terms of the third cumulant.
Enhanced Zika spread forecasting using topological data analysis.
problem Challenging prediction of Zika virus spread due to nonlinear spatio-temporal dependency and lack of historical records.
method Integrates topological data analysis, specifically persistent homology, into predictive machine learning models.
result Ensemble forecasting improves Zika spread predictions in Brazil.
For certain classes of knots we define geometric invariants called higher-order genera. Each of these invariants is a refinement of the slice genus of a knot. We find lower bounds for the higher-order genera in terms of certain von Neumann ρ-invariants, which we call higher-order signatures. The higher-order genera o…
Kernelized cumulants improve statistical analysis in high-dimensional spaces.
problem Statistical analysis in high-dimensional spaces with low variance estimators.
method Extending cumulants to RKHS using tensor algebra and kernel trick.
result Kernelized cumulants provide new all-purpose statistics with computational tractability.
A fundamental property of complex networks is the tendency for edges to cluster. The extent of the clustering is typically quantified by the clustering coefficient, which is the probability that a length-2 path is closed, i.e., induces a triangle in the network. However, higher-order cliques beyond triangles are crucia…
Stability of capillary hypersurfaces with higher order mean curvature.
problem Stability of capillary hypersurfaces with constant higher order mean curvature.
method Generalization of classical stability theory for capillary hypersurfaces.
result Results on stability for capillary hypersurfaces with higher order mean curvature.
The paper calculates bounds for risk metrics and entropies under partial information constraints.
problem Analyzing risk metrics and entropies for unimodal, symmetric distributions with limited information.
method Develops lower and upper bounds for worst-case distortion riskmetrics and weighted entropy for unimodal, symmetric distributions with known mean and variance.
result Sharp upper bounds for distortion riskmetrics and weighted entropy for symmetric distributions.
A new GAN loss function based on cumulant generating functions improves stability and robustness.
problem Improving the stability and performance of GANs.
method Cumulant GAN loss function based on variational R{é}nyi divergence.
result Cumulant GAN achieves linear convergence to Nash equilibrium and superior performance in image generation.
A key feature of inductive logic programming (ILP) is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce techniques to learn higher-order programs. Specifically, we extend meta-interpretive learning (MIL) to support learning higher…
The paper improves CR Sobolev inequalities and classifies minimizers.
problem Higher-order CR Sobolev inequalities on the CR sphere.
method Improvement through vanishing higher order moments of the volume element.
result New direct proof of minimizers' classification and existence of minimizers in C2k(N). In this paper we develop a geometric approach to higher order mechanics on graded bundles in both, the Lagrangian and Hamiltonian formalism, via the recently discovered weighted algebroids. We present the corresponding Tulczyjew triple for this higher order situation and derive in this framework the phase equations fro…
This paper presents the first use of graph neural networks (GNNs) for higher-order proof search and demonstrates that GNNs can improve upon state-of-the-art results in this domain. Interactive, higher-order theorem provers allow for the formalization of most mathematical theories and have been shown to pose a significa…
The paper glosses different forms of an introducing of higher order tangent-like functors, especially functors derived from higher order nonholonomic tangent functors. A special attention is devoted to higher order osculating bundles: their identification with higher order tangent bundles is demonstrated as the main re…
A new method predicts higher-order interactions in evolving graphs using simplicial complexes.
problem Predicting higher-order interactions in dynamic graphs with theoretical guarantees.
method Capturing higher-order interactions as simplices, modeling neighborhoods with face-vectors, and developing a nonparametric kernel estimator.
result Our method outperforms existing higher-order prediction methods and is theoretically consistent.
Novel higher-order group synchronization for noisy local measurements on hypergraphs.
problem Synchronizing higher-order local measurements on hyperedges to global estimates on nodes.
method Message passing algorithm for global synchronization of higher-order measurements.
result Higher-order method outperforms standard pairwise synchronization methods in certain applications.
Bayesian methods improve inference for cumulative probit models on large datasets.
problem Challenges in Bayesian inference for large cumulative probit models.
method Proposed scalable algorithms using Variational Bayes and Expectation Propagation.
result Superior computational performance and accuracy compared to MCMC.
Higher-order tangent bundles have geometric structures compatible with their iterated bundle structure.
problem Connection towers and Sasaki metrics on higher-order tangent bundles
method Introduce the notion of a connection tower and study the geometric structures induced by such towers.
result Connection towers determine multiconnections, adapted splittings, and canonical vector bundle structures.
The problem of an arbitrary truncated Levy flight description using the method of cumulant approach has been solved. The set of cumulants of the truncated Levy distribution given the assumption of arbitrary truncation has been found. The influence of truncation shape on the truncated Levy flight properties in the Gauss…
We construct new examples of algebraic curvature tensors so that the Jordan normal form of the higher order Jacobi operator is constant on the Grassmannian of subspaces of type (r,s) in a vector space of signature (p,q). We then use these examples to establish some results concerning higher order Osserman and highe…
We use the Frölicher-Nijenhuis formalism to reformulate the inverse problem of the calculus of variations for a system of differential equations of order 2k in terms of a semi-basic 1-form of order k. Within this general context, we use the homogeneity proposed by Crampin and Saunders in [14] to formulate and discuss t…
After defining cohomologically higher order BRST and anti-BRST operators for a compact simple algebra {\cal G}, the associated higher order Laplacians are introduced and the corresponding supersymmetry algebra Σ is analysed. These operators act on the states generated by a set of fermionic ghost fields transforming u…