PCA simplifies multivariate extreme data analysis.
problem Analyzing multivariate extreme values with high-dimensional data.
method Principal Component Analysis (PCA) for dimensionality reduction.
result PCA helps preserve essential information for extreme value analysis.
Density destructors simplify complex PDFs to maximize entropy, linking to information theory.
problem Complex multivariate PDFs are hard to analyze.
method Invertible transforms that progressively remove structure from PDFs.
result Density destructors can improve estimates of information theoretic quantities.
Efficiently searches ancestral graphs using multivariate information.
problem Discovering causal relationships in graphs with latent variables.
method Greedy search-and-score algorithm with two-step approach.
result Outperforms existing methods on benchmark datasets.
Data transformation, e.g. feature transformation and selection, is an integral part of any machine learning procedure. In this paper we introduce an information-theoretic model and tools to assess the quality of data transformations in machine learning tasks. In an unsupervised fashion, we analyze the transfer of infor…
New method uses KL-divergence to create non-informative priors for multivariate Gaussian.
problem Handling hyperparameters for non-informative limits in multivariate Gaussian conjugate priors.
method Using scaled KL-divergence between multivariate Gaussians to construct Wishart and normal-Wishart conjugate priors.
result Forming non-informative priors without violating Wishart shape parameter restrictions.
New framework improves multivariate time series forecasting by minimizing redundant information.
problem Improving multivariate time series forecasting with deep learning techniques.
method Cross-variable Decorrelation Aware feature Modeling (CDAM) and Temporal correlation Aware Modeling (TAM) to refine Channel-mixing and exploit temporal correlations.
result Significantly surpasses existing models in comprehensive tests.
Proposes a method to compute information theory measures via Gaussianization.
problem Challenges of computing information from multidimensional data.
method Indirect computation using a multivariate Gaussianization transform.
result Proposed methods outperform existing estimators, especially in high dimensions.
The paper applies Gaussianization to analyze Earth data, simplifying complex multivariate distributions.
problem Challenges in accurately estimating information content in high-dimensional, heterogeneous Earth data.
method Multivariate Gaussianization for robust probability density estimation.
result Validates the method for estimating information-theoretic measures in Earth system data.
Paper describes profiles of multivariate normal distributions and novel estimators for mutual information.
problem Estimating mutual information for complex distributions.
method Analytical description of profiles, introduction of Bend and Mix Models, Monte Carlo estimation.
result Bend and Mix Models accurately estimate mutual information profiles and provide Bayesian estimates.
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of redundancy lattices, which allows separating the information that a set of variables contains about a target variable into nonnegative components interpretable as the unique information of some variables not p…
Paper proposes PMformer for better cryptocurrency price forecasting.
problem Huge volatility and trade-off between univariate and multivariate models.
method Partial-multivariate approach using PMformer.
result PMformer achieves significant statistical accuracy in forecasting.
New metric for disentangling multivariate representations, accounting for more complex entanglements.
problem Current disentanglement metrics fail to detect entanglements involving more than two variables.
method Partial Information Decomposition framework to analyze information sharing and propose a new disentanglement metric.
result The proposed metric correctly identifies entanglements in high-dimensional spaces.
Chronos-2 forecasts multivariate and covariate data without task-specific training.
problem Limited applicability of existing time series forecasting models to real-world multivariate and covariate data.
method Chronos-2 uses a group attention mechanism for in-context learning across multiple time series.
result Chronos-2 achieves state-of-the-art performance across comprehensive benchmarks.
New framework for detecting complex interactions in multivariate data.
problem Insufficient pairwise measures fail to capture multivariate data complexities.
method Lattice theory and operator functions to derive higher-order information-theoretic measures.
result Streitberg Information fully characterizes all interactions among d variables. Confidence intervals are a popular way to visualize and analyze data distributions. Unlike p-values, they can convey information both about statistical significance as well as effect size. However, very little work exists on applying confidence intervals to multivariate data. In this paper we define confidence interval…
CRL framework groups features for multivariate learning with sparse and dense problems.
problem Sparse and dense problems in supervised multivariate learning.
method Clustered reduced-rank learning (CRL) with joint matrix regularizations.
result CRL framework is more interpretable and relaxes sparsity assumption.
The matrix-based Renyi's α-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the matrix-based Renyi's α-order entropy functional only defines the entropy of a single…
The article introduces inferential moments for analyzing uncertain multivariable systems.
problem Handling inference tasks in uncertain multivariable systems.
method Bayesian inference and quantification of inferential moments.
result Quantification of inferential moments and their connection to mutual information.
We consider the problem of universal joint clustering and registration of images and define algorithms using multivariate information functionals. We first study registering two images using maximum mutual information and prove its asymptotic optimality. We then show the shortcomings of pairwise registration in multi-i…
New method interprets multivariate time series for better results.
problem Difficulty in applying traditional methods to multivariate time series.
method Alternative representation of multivariate time series through features.
result Competitive and interpretable results achieved.
Solves utility maximization for delayed informed investors.
problem Maximizing utility in a discrete time framework with delayed information.
method Utilizes theory from [4] and optimal portfolio guessing.
result Solution for exponential utility maximization in a multivariate normal setting with delay.
We consider a multivariate default system where random environmental information is available. We study the dynamics of the system in a general setting and adopt the point of view of change of probability measures. We also make a link with the density approach in the credit risk modelling. In the particular case where …
New framework analyzes pre-stock jump trading behaviors using multivariate time series analysis.
problem Understanding micro-trading behaviors before stock price jumps.
method Multivariate time series analysis considering temporal information.
result Identifies highly informative attributes for predicting price jumps.
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this…
Study shows the corrected Akaike criterion is inadmissible for estimating Kullback-Leibler discrepancy.
problem Inadmissibility of the corrected Akaike information criterion for estimating Kullback-Leibler discrepancy.
method Loss estimation framework to demonstrate inadmissibility and provide improved estimators.
result Improved estimators of Kullback-Leibler discrepancy are provided and perform well in reduced-rank situations.
In this paper, we analyze the behavior of the multivariate symmetric uncertainty (MSU) measure through the use of statistical simulation techniques under various mixes of informative and non-informative randomly generated features. Experiments show how the number of attributes, their cardinalities, and the sample size …
MTSCI uses diffusion models to impute multivariate time series data with consistency.
problem Imputation of missing values in multivariate time series data.
method MTSCI employs a contrastive complementary mask and mixup mechanism to ensure intra-consistency and inter-consistency.
result MTSCI achieves state-of-the-art performance on multivariate time series imputation tasks.
The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are informative about B. The information bottleneck has already been applied to doc…
New method uses information theory to uncover causal relationships in complex systems.
problem Discovering causal relationships in multivariate systems, especially in Bayesian networks and hypergraphs.
method Partial Information Decomposition (PID) to explicitly model higher-order interactions.
result PID components reveal direct causal neighbors and collider relationships in Bayesian networks and multi-tail hyperedges in causal hypergraphs.
A new risk measure framework captures multivariate risk in banking.
problem Scalar risk measures fail to capture the multivariate nature of risk in banking.
method A novel multivariate risk measure framework based on the Magnitude-Propensity approach.
result The proposed framework provides a more comprehensive characterization of extreme events.
Motivation: HIV is difficult to treat because its virus mutates at a high rate and mutated viruses easily develop resistance to existing drugs. If the relationships between mutations and drug resistances can be determined from historical data, patients can be provided personalized treatment according to their own mutat…
Proposes a GNN framework for multivariate time series forecasting.
problem Lack of exploiting latent spatial dependencies in multivariate time series forecasting.
method Automatically extracts graph structures from multivariate time series data, integrates external knowledge, and uses mix-hop and dilated inception layers for capturing dependencies.
result Outperforms state-of-the-art methods on 3 out of 4 benchmark datasets.
Proposes a multivariate regression model for better analysis of multiple datasets.
problem Insufficient performance of single-dataset analysis in integrative studies.
method Sparse estimation for variable and group selection, alternating direction method of multipliers algorithm.
result Demonstrated improved performance through simulations and real data analysis.
Paper presents a framework for learning generative models with structured latent factors.
problem Learning controllable and generalizable representations of multivariate data with desired structural properties.
method The paper introduces a novel generative model framework that uses mask variables to model dependency structure and extends the multivariate information bottleneck theory.
result The framework learns semantically meaningful latent factors that reflect various desired structures and can automatically estimate dependency structure from data.
Generalizes underlap coefficient for multivariate group separation.
problem Quantifying distributional separation across groups in statistical learning.
method Generalizes underlap coefficient (UNL) to multivariate settings, studies its relationship with Bayes risk and mutual information, proposes an efficient importance sampling estimator.
result UNL as a measure of dependence between group labels and variables of interest, interpretable measure of partition-covariate dependence in clustering.
Information theoretic measures (e.g. the Kullback Liebler divergence and Shannon mutual information) have been used for exploring possibly nonlinear multivariate dependencies in high dimension. If these dependencies are assumed to follow a Markov factor graph model, this exploration process is called structure discover…
New method assesses multivariate stochastic dominance using Optimal Transport.
problem Benchmarking models across multiple metrics considering dependencies.
method Characterization of multivariate first stochastic dominance via couplings, entropic regularization, and Optimal Transport.
result Established CLT and consistency for the empirical statistic, enabling hypothesis testing.
In this paper, we consider the multivariate Bernoulli distribution as a model to estimate the structure of graphs with binary nodes. This distribution is discussed in the framework of the exponential family, and its statistical properties regarding independence of the nodes are demonstrated. Importantly the model can e…
Proposes a GNN for multivariate time-series prediction with filtering.
problem Low signal-to-noise ratio in complex systems data.
method Integrates a spatial-temporal GNN with a matrix filtering module to generate filtered graphs.
result Proposed model outperforms baseline approaches in multivariate time-series prediction.
Bayesian model clusters brain activity time series.
problem Heterogeneous multivariate time series in brain imaging.
method Group-based Bayesian mixture of smoothing splines with covariate effects.
result Distinct brain activity patterns identified.
Method introduces topological regularization using information filtering networks.
problem Sparse probabilistic modeling and multicollinear regression.
method Topological regularization via information filtering network.
result Direct application to L0-norm regularized problems. New method discovers causal relationships in complex time series data.
problem Discovering causal relationships in multivariate time series is challenging.
method Temporal Dependency to Causality (TD2C) framework using mutual information.
result TD2C achieves state-of-the-art performance in causal discovery.
Proposes PEID for analyzing synergistic causation in complex systems.
problem Challenges in identifying and analyzing synergistic causation in complex systems.
method Partial Effective Information Decomposition (PEID) framework.
result Unified and computable characterization of synergistic causal relations.
Method estimates multivariate counterfactual distributions efficiently and accurately.
problem Estimating multivariate counterfactual distributions in causal models with correlation structures.
method Proposes a method leveraging a one-dimensional subspace to capture correlation structures and efficiently estimate multivariate counterfactual distributions.
result Demonstrates superior performance over existing methods on synthetic and real-world data.
We introduce a model for causal structure learning from multivariate functional data, even when graphs have cycles.
problem Discovering causal relationships from multivariate functional data with cycles.
method Functional linear structural equation model with a low-dimensional causal embedded space.
result The proposed model is causally identifiable under standard assumptions.
Neural optimal transport improves multivariate conformal prediction.
problem Multivariate quantile regression challenges and existing methods ignore joint distribution geometry.
method Combines neural optimal transport with amortized optimization for efficient training and faster inference.
result Constructs tighter and more informative predictive regions for multivariate conformal prediction.
New algorithms for interpreting complex multivariate functions.
problem Hard interpretation of multivariate functions due to many parameters.
method Filtered tensor decompositions of derivative information.
result Nonparametric estimates of smooth decoupled functions.
Proposes a new model for complex multivariate event data.
problem Modeling complex multivariate event data with spatio-temporal dynamics.
method Integrates spatial information into latent state evolution through learned temporal and spatial decay dynamics.
result Successfully recovers sensible temporal and spatial intensity structure in multivariate spatio-temporal point patterns.