The study reveals how synaptic correlations promote dimension reduction in neural networks.
problem Understanding how synaptic correlations affect neural correlations and dimension reduction in deep neural networks.
method A simplified model of dimension reduction considering pairwise correlations among synapses, using mathematical self-consistency for both binary and continuous synapses.
result Weakly-correlated synapses encourage dimension reduction compared to orthogonal synapses, and they also slow down the decorrelation process.
Paper studies estimating asset correlations across sectors.
problem Estimating correlations between different asset sectors.
method Separates cross-sectional and time dimensions for estimation.
result Developed method for better asset correlation estimation.
Our goal in this paper is to develop an effective estimator of fractal dimension. We survey existing ideas in dimension estimation, with a focus on the currently popular method of Grassberger and Procaccia for the estimation of correlation dimension. There are two major difficulties in estimation based on this method. …
Enhances SDR via Hellinger correlation for better data dependency understanding.
problem Improving sufficient dimension reduction in single-index models.
method Developed a new method using Hellinger correlation for detecting the dimension reduction subspace.
result Significantly enhances and outperforms existing SDR methods through deeper data dependency understanding.
New model analyzes dynamic correlations in stock returns.
problem Analyzing time-varying correlations in high-dimensional data.
method Dynamic factor correlation model with novel parametrization.
result Model accurately captures heterogeneous heavy-tailed distributions and dependent shocks.
Most data is multi-dimensional. Discovering whether any subset of dimensions, or subspaces, of such data is significantly correlated is a core task in data mining. To do so, we require a measure that quantifies how correlated a subspace is. For practical use, such a measure should be universal in the sense that it capt…
Model explains Netflix Prize success with structured correlation.
problem Understanding structured correlation in high-dimensional data.
method Developed a new statistical learning model.
result Characterized learnability in terms of VCNk,k-dimension. CaLoNet integrates spatial and local correlations for multivariate time series classification.
problem Ignoring spatial and local correlations in multivariate time series classification.
method Model spatial correlations using causality modeling, extract local correlations, integrate into graph neural network.
result Competitive performance compared to state-of-the-art methods on UEA datasets.
CCP clusters correlated features and projects them to 1D for efficient dimensionality reduction.
problem Efficiency in handling large datasets with high intrinsic dimensions.
method CCP partitions features into correlated clusters and projects them to 1D based on sample correlations.
result CCP achieves efficient dimensionality reduction without matrix diagonalization.
A new method for real-time CCA on streaming data.
problem Finding correlated features in online data streams.
method Sliding Window Informative Canonical Correlation Analysis (SWICCA) using streaming PCA.
result SWICCA provides real-time CCA components in high dimensions with theoretical guarantees.
Canonical correlation analysis (CCA) is a fundamental statistical tool for exploring the correlation structure between two sets of random variables. In this paper, motivated by recent success of applying CCA to learn low dimensional representations of high dimensional objects, we propose to quantify the estimation loss…
We consider the problem of providing nonparametric confidence guarantees for undirected graphs under weak assumptions. In particular, we do not assume sparsity, incoherence or Normality. We allow the dimension D to increase with the sample size n. First, we prove lower bounds that show that if we want accurate infe…
We introduce a new approach to variable selection, called Predictive Correlation Screening, for predictor design. Predictive Correlation Screening (PCS) implements false positive control on the selected variables, is well suited to small sample sizes, and is scalable to high dimensions. We establish asymptotic bounds f…
It is ubiquitous in natural and social sciences that two variables, recorded temporally or spatially in a complex system, are cross-correlated and possess multifractal features. We propose a new method called multifractal detrended cross-correlation analysis (MF-DXA) to investigate the multifractal behaviors in the pow…
The study uses DCC for financial market analysis, revealing hidden correlations.
problem Identifying hidden nonlinear correlations in financial markets.
method Agglomerative hierarchical clustering with distance correlation coefficient.
result DCC reveals more information than Pearson correlation for financial data.
A new way to describe correlation matrices makes modeling easier.
problem Describing correlation matrices in a flexible and positive-definite way.
method Introduces a novel parametrization that allows unrestricted vectors for correlation matrices.
result The new parametrization ensures positive definiteness without additional constraints.
Sparse GCA finds linear relationships in multiple datasets, using gradient descent.
problem Finding linear relationships across multiple datasets with sparse loading vectors.
method Formulated as generalized eigenvalue problems, used a thresholded gradient descent algorithm.
result Proposed algorithm yields tight estimation error bounds and demonstrates effectiveness on synthetic datasets.
When can reliable inference be drawn in the "Big Data" context? This paper presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large scale inference. In large scale data applications like genomics, connectomics, and eco-informatics the dataset…
The article generalizes Pearson correlation to Riemannian manifolds.
problem Analyzing statistical models on non-linear manifolds.
method Reconstitutes Pearson correlation properties and derives a nonlinear generalization.
result Developed the Riemann-Pearson Correlation for manifold analysis.
Method estimates multivariate counterfactual distributions efficiently and accurately.
problem Estimating multivariate counterfactual distributions in causal models with correlation structures.
method Proposes a method leveraging a one-dimensional subspace to capture correlation structures and efficiently estimate multivariate counterfactual distributions.
result Demonstrates superior performance over existing methods on synthetic and real-world data.
In this work, we contribute a new multi-layer neural network architecture named ONCF to perform collaborative filtering. The idea is to use an outer product to explicitly model the pairwise correlations between the dimensions of the embedding space. In contrast to existing neural recommender models that combine user em…
The paper presents a novel approach to multi-output regression using probabilistic circuits.
problem Capturing correlations between multiple output dimensions in large-scale regression problems.
method Employing a mixture of single-output Gaussian process experts encoded via a probabilistic circuit.
result The method can capture correlations between output dimensions and often outperforms other approaches.
tvGP-VAE models tensor-valued latent variables with Gaussian processes for better data structure representation.
problem Agnostic latent variables in VAEs ignore data structure correlations.
method Proposes tensor-variate Gaussian process prior for variational autoencoder.
result Explicitly modeling correlation structures improves model performance in reconstruction.
Develops log-Euclidean Lie groups for SPD and correlation matrices.
problem Unifies various log-Euclidean constructions for SPD and correlation matrices.
method Theory and explicit isometries linking different log-Euclidean metrics.
result Explicit log-Euclidean metrics on SPD and correlation matrices.
Estimates dimension of subsets from random samples, proving consistency.
problem Estimating the dimension of a compact subset from random samples.
method Consistency proofs for Minkowski, correlation, and pointwise dimensions using empirical volume function.
result Statistical consistency of estimators for various dimension notions.
This paper evaluates fractal dimension and persistent homology for neural network generalization.
problem Bounding and predicting the generalization gap of neural networks.
method Empirical evaluation of fractal dimension and persistent homology as generalization measures.
result Fractal dimension and persistent homology fail to predict generalization of models trained from poor initializations.
VADD enhances discrete diffusion models by capturing inter-dimensional correlations, improving sample quality.
problem Limited modeling of inter-dimensional dependencies in MDMs degrades performance with few denoising steps.
method Introduces an auxiliary recognition model for latent variable modeling, enabling stable training via variational lower bounds maximization and amortized inference.
result VADD consistently outperforms MDM baselines in sample quality with few denoising steps.
This paper investigates the utilization of maximum and average distance correlations for multivariate independence testing. We characterize their consistency properties in high-dimensional settings with respect to the number of marginally dependent dimensions, compare the advantages of each test statistic, examine thei…
We obtain general, exact formulas for the overlaps between the eigenvectors of large correlated random matrices, with additive or multiplicative noise. These results have potential applications in many different contexts, from quantum thermalisation to high dimensional statistics. We find that the overlaps only depend …
We study graph matching with correlated Gaussian features and find thresholds for exact recovery.
problem Graph matching with correlated Gaussian features.
method Information-theoretic thresholds and conditions for exact and almost exact recovery.
result Contextual information introduces a richer structure, with thresholds for exact and almost exact recovery no longer coinciding.
Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction due to its profound theoretical foundation and success in practical applications. In respect of multi-view learning, however, it is limited by its capability of only handling data represented by two-view features, while i…
We introduce a new measure for the capital market efficiency. The measure takes into consideration the correlation structure of the returns (long-term and short-term memory) and local herding behavior (fractal dimension). The efficiency measure is taken as a distance from an ideal efficient market situation. Methodolog…
This paper treats the problem of screening for variables with high correlations in high dimensional data in which there can be many fewer samples than variables. We focus on threshold-based correlation screening methods for three related applications: screening for variables with large correlations within a single trea…
In this paper, a novel learning paradigm is presented to automatically identify groups of informative and correlated features from very high dimensions. Specifically, we explicitly incorporate correlation measures as constraints and then propose an efficient embedded feature selection method using recently developed cu…
Develops methods to learn correlation potentials for time-dependent Kohn-Sham systems.
problem Learning the correlation potential for time-dependent Kohn-Sham systems.
method Optimizing a least-squares objective subject to the TDKS equation using adjoints.
result Learned correlation potential models match ground truth electron densities and can have memory.
We introduce the Randomized Dependence Coefficient (RDC), a measure of non-linear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient. RDC is defined in terms of correlation of random non-linear copula projections; it is invariant with respec…
We present a general method to detect and extract from a finite time sample statistically meaningful correlations between input and output variables of large dimensionality. Our central result is derived from the theory of free random matrices, and gives an explicit expression for the interval where singular values are…
Estimates mean dimension of neural networks to reveal interaction effects.
problem Understanding interaction effects in neural networks.
method Estimation procedure for mean dimension from datasets, analyzing layer-by-layer evolution and impact of activation functions.
result Mean dimension reveals differences in interaction magnitude across neural network architectures.
We show that the Yang-Mills quantum field theory with momentum and spacetime cutoffs in four Euclidean dimensions is equivalent, term by term in an appropriately resummed perturbation theory, to a Fermionic theory with nonlocal interaction terms. When a further momentum cutoff is imposed, this Fermionic theory has a co…
High-dimensional data are ubiquitous in contemporary science and finding methods to compress them is one of the primary goals of machine learning. Given a dataset lying in a high-dimensional space (in principle hundreds to several thousands of dimensions), it is often useful to project it onto a lower-dimensional manif…
Rationality of the Wightman functions is proven to follow from energy positivity, locality and a natural condition of global conformal invariance (GCI) in any number D of space-time dimensions. The GCI condition allows to treat correlation functions as generalized sections of a vector bundle over the compactification o…
During the last decades, we have witnessed a surge of interests of learning a low-dimensional space with discriminative information from one single view. Even though most of them can achieve satisfactory performance in some certain situations, they fail to fully consider the information from multiple views which are hi…
Bayesian method for dynamic correlation matrices improves accuracy and responsiveness.
problem Challenges in estimating time-varying correlation matrices, including slow adaptation, insufficient regularization, and diffuse uncertainty.
method Low-rank factor representation with dynamic shrinkage prior and multivariate factor stochastic volatility model.
result Improved accuracy and responsiveness compared to competing methods in various challenging scenarios.
A new screening method for high-dimensional data reduces computational cost.
problem Challenges in variable selection for ultrahigh-dimensional linear regression.
method Ordering absolute sample ridge partial correlations to screen variables.
result The method provides sure screening property without strong assumptions.
This paper proposes a general adaptive procedure for budget-limited predictor design in high dimensions called two-stage Sampling, Prediction and Adaptive Regression via Correlation Screening (SPARCS). SPARCS can be applied to high dimensional prediction problems in experimental science, medicine, finance, and engineer…
This paper detects Markov violations in RL with noise, improving policy development.
problem Partial observability and sensor/actuator noise invalidate Markovian assumptions in RL.
method Combines PCMCI causal discovery with Markov Violation score (MVS).
result Even substantial noise doesn't always disrupt multi-step dependencies.
How does one find dimensions in multivariate data that are reliably expressed across repetitions? For example, in a brain imaging study one may want to identify combinations of neural signals that are reliably expressed across multiple trials or subjects. For a behavioral assessment with multiple ratings, one may want …
We investigate Ising model description of dynamics of stock price. The model is defined in near 2 dimensions, one dimension is time and another represents ensemble of stocks, and strength of response of investors to price change corresponds to inverse temperature of the system. At critical temperature, infinitely long …