A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Extracts the finest pattern of mutual independence from data.
Mutual independence is a key concept in statistics that characterizes the structural relationships between variables. Existing methods to investigate mutual independence rely on the definition of two competing models, one being nested into the other and used to generate a null distribution for a statistic of interest, …
We propose three measures of mutual dependence between multiple random vectors. All the measures are zero if and only if the random vectors are mutually independent. The first measure generalizes distance covariance from pairwise dependence to mutual dependence, while the other two measures are sums of squared distance…
IndiSeek learns disentangled representations by balancing independence and completeness.
We apply both distance-based (Jin and Matteson, 2017) and kernel-based (Pfister et al., 2016) mutual dependence measures to independent component analysis (ICA), and generalize dCovICA (Matteson and Tsay, 2017) to MDMICA, minimizing empirical dependence measures as an objective function in both deflation and parallel m…
The paper studies how quickly samples from Langevin dynamics become independent.
In this paper, we propose novel strategies for neutral vector variable decorrelation. Two fundamental invertible transformations, namely serial nonlinear transformation and parallel nonlinear transformation, are proposed to carry out the decorrelation. For a neutral vector variable, which is not multivariate Gaussian d…
This research designs a data-driven partition to test independence between continuous variables.
We propose a test of independence of two multivariate random vectors, given a sample from the underlying population. Our approach, which we call MINT, is based on the estimation of mutual information, whose decomposition into joint and marginal entropies facilitates the use of recently-developed efficient entropy estim…
This work improves independence tests for high-dimensional data.
Paper benchmarks mutual info estimators on diverse distributions.
Most existing works on disentangled representation learning are solely built upon an marginal independence assumption: all factors in disentangled representations should be statistically independent. This assumption is necessary but definitely not sufficient for the disentangled representations without additional induc…
The independence clustering problem is considered in the following formulation: given a set of random variables, it is required to find the finest partitioning of into clusters such that the clusters are mutually independent. Since mutual independence is the target, pairwise …
Several methods of estimating the mutual information of random variables have been developed in recent years. They can prove valuable for novel approaches to learning statistically independent features. In this paper, we use one of these methods, a mutual information neural estimation (MINE) network, to present a proof…
Self-Distilled Disentanglement improves counterfactual predictions by separating variables.
Cycles in causal learning cause feedback loops under intervention.
We propose a method for learning Markov network structures for continuous data without invoking any assumptions about the distribution of the variables. The method makes use of previous work on a non-parametric estimator for mutual information which is used to create a non-parametric test for multivariate conditional i…
New framework improves multivariate time series forecasting by minimizing redundant information.
Study reveals mutual information is crucial for understanding algorithm performance in stochastic convex optimization.
Optimal Bayesian feature filtering (OBF) is a supervised screening method designed for biomarker discovery. In this article, we prove two major theoretical properties of OBF. First, optimal Bayesian feature selection under a general family of Bayesian models reduces to filtering if and only if the underlying Bayesian m…
Estimates conditional mutual information using a minmax formulation.
Meta-learning bound uses conditional mutual information.
Half-AVAE enhances VAE for underdetermined ICA with adversarial training.
AutoTransfer improves subject transfer learning for biosignal datasets.
The paper explores tail diversification in financial markets using entropy and mutual information.
MSRL learns a representation maximizing mutual info with response variables.
New method shows links can be braided open book bindings.
The paper introduces submodular information measures for machine learning applications.
Paper proposes a method to extract style features from unlabeled data.
The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model inference, causal strength estimation and time-series problems. In several appl…
Paper proposes MIM-DRCFR to learn disentangled factors for better treatment effect estimation.
New framework IIA identifies innovations in general nonlinear vector autoregressive processes.
Feature selection methods are usually evaluated by wrapping specific classifiers and datasets in the evaluation process, resulting very often in unfair comparisons between methods. In this work, we develop a theoretical framework that allows obtaining the true feature ordering of two-dimensional sequential forward feat…
Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (…
Estimators of information theoretic measures such as entropy and mutual information are a basic workhorse for many downstream applications in modern data science. State of the art approaches have been either geometric (nearest neighbor (NN) based) or kernel based (with a globally chosen bandwidth). In this paper, we co…
Boosting improves ICA for better component recovery.
Most work in algorithmic fairness to date has focused on discrete outcomes, such as deciding whether to grant someone a loan or not. In these classification settings, group fairness criteria such as independence, separation and sufficiency can be measured directly by comparing rates of outcomes between subpopulations. …
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical physics methods, under the assumption that weight matrices are independent and ort…
In the field of machine learning, it is still a critical issue to identify and supervise the learned representation without manually intervening or intuition assistance to extract useful knowledge or serve for the downstream tasks. In this work, we focus on supervising the influential factors extracted by the variation…
MINDE estimates Mutual Information using neural diffusion models.
Multivariate pattern analyses approaches in neuroimaging are fundamentally concerned with investigating the quantity and type of information processed by various regions of the human brain; typically, estimates of classification accuracy are used to quantify information. While a extensive and powerful library of method…
We present a variational renormalization group (RG) approach using a deep generative model based on normalizing flows. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with reduced mutual information. Conversely, the neural net directly maps independent Gauss…
Proposes Infomax and Domain-Independent Representations for robust causal inference.
Maximizes image representation dependence for self-supervised learning.
We study the optimal investment problem for a continuous time incomplete market model such that the risk-free rate, the appreciation rates and the volatility of the stocks are all random; they are assumed to be independent from the driving Brownian motion, and they are supposed to be currently observable. It is shown t…
One of the most fundamental questions one can ask about a pair of random variables X and Y is the value of their mutual information. Unfortunately, this task is often stymied by the extremely large dimension of the variables. We might hope to replace each variable by a lower-dimensional representation that preserves th…
Improved statistical efficiency of Thompson Sampling for combinatorial semi-bandits.
Recent works investigated the generalization properties in deep neural networks (DNNs) by studying the Information Bottleneck in DNNs. However, the mea- surement of the mutual information (MI) is often inaccurate due to the density estimation. To address this issue, we propose to measure the dependency instead of MI be…