arXiv research

Proposes a method to enhance multi-view learning by maximizing higher order correlations.

Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.

Many nonlinear extensions of the Kalman filter, e.g., the extended and the unscented Kalman filter, reduce the state densities to Gaussian densities. This approximation gives sufficient results in many cases. However, this filters only estimate states that are correlated with the observation. Therefore, sequential esti…

TGCCA analyzes higher-order tensors using orthogonal rank-R CP decomposition.

A new method captures higher-order interactions in data clusters.

Diffusion models learn simple statistics before complex ones, revealing a sample complexity exponent.

Neural networks learn faster with correlated latent variables.

Study quantifies how LLMs capture higher-order statistical structure using cumulant expansion.

The statistical dependencies which independent component analysis (ICA) cannot remove often provide rich information beyond the linear independent components. It would thus be very useful to estimate the dependency structure from data. While such models have been proposed, they usually concentrated on higher-order corr…

To date, the instability of prognostic predictors in a sparse high dimensional model, which hinders their clinical adoption, has received little attention. Stable prediction is often overlooked in favour of performance. Yet, stability prevails as key when adopting models in critical areas as healthcare. Our study propo…

This work generalizes transformer attention to capture higher-order correlations efficiently.

Study higher-order spin glass models for social network behavior with peer-group effects.

For joint inference over multiple variables, a variety of structured prediction techniques have been developed to model correlations among variables and thereby improve predictions. However, many classical approaches suffer from one of two primary drawbacks: they either lack the ability to model high-order correlations…

New tensor framework connects Fisher information, hypergraphs, and multi-observable correlations.

First passage models, where corporate assets undergo correlated random walks and a company defaults if its assets fall below a threshold provide an attractive framework for modeling the default process. Typical one year default correlations are small, i.e., of order a few percent, but nonetheless including correlations…

Study neural networks by mapping correlations, revealing essential statistics.

Neural networks can learn from higher-order cumulants efficiently, requiring quadratic samples.

Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.

In an efficient stock market, the log-returns and their time-dependent variances are often jointly modelled by stochastic volatility models (SVMs). Many SVMs assume that errors in log-return and latent volatility process are uncorrelated, which is unrealistic. It turns out that if a non-zero correlation is included in …

New method combines hypergraph structure and node attributes for better community detection.

Weak correlations explain linear dynamics in deep learning models.

The moments of spatial probabilistic systems are often given by an infinite hierarchy of coupled differential equations. Moment closure methods are used to approximate a subset of low order moments by terminating the hierarchy at some order and replacing higher order terms with functions of lower order ones. For a give…

This paper improves multi-label classification by leveraging high-order label correlations.

A two-layer neural network model that systematically includes correlations among input variables to arbitrary order and is designed to implement Bayes inference has been adapted to classify breast cancer tumors as malignant or benign, assigning a probability for either outcome. The inputs to the network represent measu…

We analyze daily prices of 29 commodities and 2449 stocks, each over a period of $\approx 15$ years. We find that the price fluctuations for commodities have a significantly broader multifractal spectrum than for stocks. We also propose that multifractal properties of both stocks and commodities can be attributed mainl…

Paper proposes a new LSTM model for spatio-temporal learning.

Overcomplete latent representations have been very popular for unsupervised feature learning in recent years. In this paper, we specify which overcomplete models can be identified given observable moments of a certain order. We consider probabilistic admixture or topic models in the overcomplete regime, where the numbe…

RFMs transition from linear to nonlinear under specific input-label correlation.

New method shows fully-connected networks can learn convolutional structures from data.

The paper tackles statistical and computational challenges in learning correlated reward models.

Dynamic Vine Copulas detect and quantify time-varying higher-order interactions in multivariate systems.

In this study, we tested the interaction effect of multimodal datasets using a novel method called the kernel method for detecting higher order interactions among biologically relevant mulit-view data. Using a semiparametric method on a reproducing kernel Hilbert space (RKHS), we used a standard mixed-effects linear mo…

New method interprets quantum many-body snapshots for phase detection.

We consider support recovery in the quadratic logistic regression setting - where the target depends on both p linear terms $x_i$ and up to $p^2$ quadratic terms $x_i x_j$. Quadratic terms enable prediction/modeling of higher-order effects between features and the target, but when incorporated naively may involve solvi…

In many real-world applications, data are often unlabeled and comprised of different representations/views which often provide information complementary to each other. Although several multi-view clustering methods have been proposed, most of them routinely assume one weight for one view of features, and thus inter-vie…

New VAE models reveal hierarchical visual cortex computations.

We investigate the relative information content of six measures of dependence between two random variables $X$ and $Y$ for large or extreme events for several models of interest for financial time series. The six measures of dependence are respectively the linear correlation $ρ^+_v$ and Spearman's rho $ρ_s(v)$ conditio…

Proposes BHT-ARIMA for forecasting multiple short time series.

The correlated stochastic volatility models constitute a natural extension of the Black and Scholes-Merton framework: here the volatility is not a constant, but a stochastic process correlated with the price log-return one. At present, several stochastic volatility models are discussed in the literature, differing in t…

Study reveals structural differences in financial networks near and far from crises using balance theory.

For certain classes of knots we define geometric invariants called higher-order genera. Each of these invariants is a refinement of the slice genus of a knot. We find lower bounds for the higher-order genera in terms of certain von Neumann $ρ$-invariants, which we call higher-order signatures. The higher-order genera o…

In topic modeling, many algorithms that guarantee identifiability of the topics have been developed under the premise that there exist anchor words -- i.e., words that only appear (with positive probability) in one topic. Follow-up work has resorted to three or higher-order statistics of the data corpus to relax the an…

A fundamental property of complex networks is the tendency for edges to cluster. The extent of the clustering is typically quantified by the clustering coefficient, which is the probability that a length-2 path is closed, i.e., induces a triangle in the network. However, higher-order cliques beyond triangles are crucia…

Stability of capillary hypersurfaces with higher order mean curvature.

A key feature of inductive logic programming (ILP) is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce techniques to learn higher-order programs. Specifically, we extend meta-interpretive learning (MIL) to support learning higher…

The paper improves CR Sobolev inequalities and classifies minimizers.

We develop a normative framework for hierarchical model-based policy optimization based on applying second-order methods in the space of all possible state-action paths. The resulting natural path gradient performs policy updates in a manner which is sensitive to the long-range correlational structure of the induced st…

In this paper we develop a geometric approach to higher order mechanics on graded bundles in both, the Lagrangian and Hamiltonian formalism, via the recently discovered weighted algebroids. We present the corresponding Tulczyjew triple for this higher order situation and derive in this framework the phase equations fro…