Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “dependence information”
Information-theoretic Bayesian regret bounds of Russo and Van Roy capture the dependence of regret on prior uncertainty. However, this dependence is through entropy, which can become arbitrarily large as the number of actions increases. We establish new bounds that depend instead on a notion of rate-distortion. Among o…
Information theory provides ideas for conceptualising information and measuring relationships between objects. It has found wide application in the sciences, but economics and finance have made surprisingly little use of it. We show that time series data can usefully be studied as information -- by noting the relations…
Improved Robust Price Bounds for Multi-Asset Derivatives under Market-Implied Dependence Informationq-fin.MF
Improved price bounds for multi-asset derivatives using market option data.
problem Creating robust price bounds for multi-asset derivatives under market-implied dependence.
method Extracting inter-asset dependence information from market option prices and applying modified martingale optimal transport.
result Improved price bounds for multi-asset derivatives, demonstrating relevance and tractability.
The paper proposes methods to extract and analyze individual variable information from complex dependencies.
problem Analyzing and understanding complex dependencies between multiple variables.
method Reversible normalization and iterative dependency reduction to extract individual information, and use it for direct mutual information and multi-feature Granger causality analysis.
result Decoupling of variables to analyze their individual information and direct mutual information transfers.
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter quantified by the mutual information. We define the information excess as a measure of d…
We investigated financial market data to determine which factors affect information flow between stocks. Two factors, the time dependency and the degree of efficiency, were considered in the analysis of Korean, the Japanese, the Taiwanese, the Canadian, and US market data. We found that the frequency of the significant…
We simplify information measure computation using learned features.
problem Computing information measures from raw data is computationally expensive.
method Developed a separable design for computing information measures from learned feature representations.
result A variety of information measures can be computed efficiently through learned feature representations.
MIC consistently estimates dependence in large datasets.
problem Estimating dependence between variable pairs in large datasets.
method Proving consistency of MIC as an estimator.
result MIC is a consistent estimator of population statistic MIC*.
New bounds show limitations of sample-wise information-theoretic generalization.
problem Limitations of sample-wise information-theoretic generalization bounds.
method Analysis of existing bounds and derivation of new bounds.
result No sample-wise information-theoretic bounds exist for expected squared generalization gap.
DOS improves language model generation by considering inter-token dependencies.
problem Lack of sequence-level information and inter-token dependencies in existing decoding strategies.
method Dependency-Oriented Sampler (DOS) that uses attention matrices to approximate inter-token dependencies.
result DOS consistently achieves superior performance on code generation and mathematical reasoning tasks.
In this work, we develop a novel regularizer to improve the learning of long-range dependency of sequence data. Applied on language modelling, our regularizer expresses the inductive bias that sequence variables should have high mutual information even though the model might not see abundant observations for complex lo…
Optimizes black-box functions with varying costs across multiple sources.
problem Optimizing black-box functions with varying costs across multiple sources.
method Uses Augmented Gaussian Process and Gaussian Process to model fidelity and location-dependent costs, respectively. Uses Confidence Bound acquisition function to select sources and locations.
result The approach significantly outperforms existing methods on Hyperparameters Optimization tasks.
Recent works investigated the generalization properties in deep neural networks (DNNs) by studying the Information Bottleneck in DNNs. However, the mea- surement of the mutual information (MI) is often inaccurate due to the density estimation. To address this issue, we propose to measure the dependency instead of MI be…
In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli (2019). Our main contributions are significantly improved mutual information bounds for Stochastic Gradient Langevin Dynamics via data-depe…
Investigates optimal portfolio strategies in markets with latent side information.
problem Investment problem in markets with latent dependence structure and side information.
method Dynamic and constant portfolio strategies, analyzing log-optimal portfolio as benchmark.
result Optimal dynamic strategy growth rate asymptotically converges to constant strategy in stationary markets.
Nonlinear dependencies on Brazilian equity network from mutual information minimum spanning treesphysics.soc-ph
Mutual information minimum spanning trees are used to explore nonlinear dependencies on Brazilian equity network in the periods from June/01/2015 to January/26/2016, in which Brazil was under the government of President Dilma Rousseff, and from January/27/2016 to September/08/2016 which includes the government transiti…
Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (…
In this paper we extend the theory of option pricing to take into account and explain the empirical evidence for asset prices such as non-Gaussian returns, long-range dependence, volatility clustering, non-Gaussian copula dependence, as well as theoretical issues such as asymmetric information and the presence of limit…
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
We study 'meta-dependence' in conditional independence tests across different empirical distributions.
problem Understanding the breakdown of conditional independence properties in finite data.
method Geometric intuition and information projections to measure meta-dependence between conditional independences.
result We provide a measure of meta-dependence that consolidates findings across synthetic and real-world data.
InfoAtlas speeds up MI estimation for real-time data analysis.
problem Efficiently measuring statistical dependency between high-dimensional datasets.
method Directly infers mutual information in a single forward pass using a pretrained model.
result Matches state-of-the-art accuracy with 100x speedup.
This paper improves dependency networks using information geometry.
problem Technical disadvantage in dependency networks' learned distribution.
method Interpret pseudo-Gibbs sampling as iterative m-projections onto manifolds.
result Dependency networks can learn faster and have similar performance to Bayesian networks.
We extend manifold capacity to nonlinear neural representations with contextual information.
problem Efficient processing of information through neural representations.
method Theoretical framework leveraging latent directions in input space related to contextual information.
result Derivation of an exact formula for context-dependent manifold capacity.
Limits of Transfer Learningstat.ML
New research on limits of transfer learning, proving key selection and dependence requirements.
problem Insufficient theoretical foundation for transfer learning.
method Proved novel results on transfer learning, emphasizing selection of information and dependence between domains.
result Upper bound on improvement possible with transfer learning, highlighting the need for careful selection.
Bounding the generalization error of learning algorithms has a long history, which yet falls short in explaining various generalization successes including those of deep learning. Two important difficulties are (i) exploiting the dependencies between the hypotheses, (ii) exploiting the dependence between the algorithm'…
Paper analyzes trade-offs between fairness, privacy, and accuracy using Chernoff Information.
problem The relationship between fairness and privacy in machine learning.
method Utilizes Chernoff Information to characterize trade-offs, proposes Chernoff Difference and Noisy Chernoff Difference, develops CINE for neural estimation.
result Shows three distinct behaviors of Noisy Chernoff Difference based on data distribution.
The paper argues that normalized mutual information is biased in clustering and community detection.
problem Bias in normalized mutual information for clustering and community detection.
method Introducing a modified version of mutual information to correct for information content and spurious dependence.
result The modified mutual information leads to different conclusions about which algorithms are best for community detection.
We consider a sequential learning problem with Gaussian payoffs and side information: after selecting an action , the learner receives information about the payoff of every action in the form of Gaussian observations whose mean is the same as the mean payoff, but the variance depends on the pair (and may…
A new deep learning framework captures multi-scale spatio-temporal dependencies.
problem Designing and analyzing deep learning models for complex spatio-temporal analytics.
method Developed an IDRNN model with three modules for integrating and learning multi-scale spatio-temporal data.
result The IDRNN model outperforms classical and state-of-the-art models in capturing meaningful multi-scale spatio-temporal dependencies.
Study improves sampling from non-log-concave distributions using Fisher information.
problem Sampling from non-log-concave distributions with high Fisher information guarantees.
method Proximal sampler with RGO implementation, leveraging log-concave sampling results.
result Improved complexity guarantee in relative Fisher information for non-log-concave sampling.
The paper analyzes cryptocurrency trading networks using pairwise and high-order dependencies.
problem Understanding information flows and dependencies in cryptocurrency markets.
method Defined a cryptocurrency trading network using weekly log returns, analyzed using Granger causality and O-information.
result High-order dependencies reveal that stable coins play a major role in high-order effects.
Measures dependence between two systems using Bayesian model comparison.
problem Quantifying dependence between two systems in a dataset.
method Bayesian model comparison of independence and dependence models.
result Dependence measure quantifies evidence for dependence in data.
The Mutual Information (MI) is an often used measure of dependency between two random variables utilized in information theory, statistics and machine learning. Recently several MI estimators have been proposed that can achieve parametric MSE convergence rate. However, most of the previously proposed estimators have th…
In many cases, feature selection is often more complicated than identifying a single subset of input variables that would together explain the output. There may be interactions that depend on contextual information, i.e., variables that reveal to be relevant only in some specific circumstances. In this setting, the con…
A new method extracts events and their arguments efficiently from text.
problem Efficiently extract event information from texts with long-range dependencies and associations.
method Graph Convolutional Networks with shortest dependency paths to capture syntactic relationships.
result Significant improvement over state-of-the-art methods.
Dropout technique is analyzed using information geometry.
problem Understanding the regularization performance of dropout in neural networks.
method Unified analysis from information geometry viewpoint.
result Dropout flattens the model manifold and its performance depends on curvature.
The paper analyzes convergence of Langevin dynamics with time-dependent metrics.
problem Analyzing convergence of Langevin dynamics with time-dependent metrics.
method Formulated a modified gradient flow of the Kullback-Leibler divergence, selected a time-dependent relative Fisher information functional, and developed a time-dependent Hessian matrix condition.
result Proved convergence conditions for various Langevin dynamics.
Paper tackles robust classification under class-dependent domain shift.
problem Class-dependent domain shift in machine learning.
method Defined a simple optimization problem with an information theoretic constraint and solved it using neural networks.
result Demonstrated that the proposed method can learn robust classifiers that generalize well to unseen domains.
The Information Sievestat.ML
We introduce a new framework for unsupervised learning of representations based on a novel hierarchical decomposition of information. Intuitively, data is passed through a series of progressively fine-grained sieves. Each layer of the sieve recovers a single latent factor that is maximally informative about multivariat…
New algorithms improve privacy in bandit problems with partial information.
problem Privacy constraints in multi-armed bandit problems with partial reward information.
method Proposed a generic framework for designing -global DP extensions of UCB and KL-UCB algorithms.
result AdaP-KLUCB algorithm achieves optimal regret bound under -global DP constraints.
Exact partial information decompositions for Gaussian systems based on dependency constraintscond-mat.stat-mech
The Partial Information Decomposition (PID) [arXiv:1004.2515] provides a theoretical framework to characterize and quantify the structure of multivariate information sharing. A new method (Idep) has recently been proposed for computing a two-predictor PID over discrete spaces. [arXiv:1709.06653] A lattice of maximum en…
New methods estimate point-wise dependency from neural MI models.
problem Estimating point-wise dependency between different events.
method Developed two methods: Probabilistic Classifier and Density-Ratio Fitting.
result Demonstrated effectiveness in MI estimation, self-supervised representation learning, and cross-modal retrieval.
This work improves independence tests for high-dimensional data.
problem Detecting subtle dependencies between high-dimensional random variables with complex distributions.
method Develops two approaches to learn powerful independence tests using variational mutual information and HSIC.
result Optimized HSIC tests generally outperform other approaches on detecting structured dependence.
The vast majority of optimization and online learning algorithms today require some prior information about the data (often in the form of bounds on gradients or on the optimal parameter value). When this information is not available, these algorithms require laborious manual tuning of various hyperparameters, motivati…
Information concentration of probability measures have important implications in learning theory. Recently, it is discovered that the information content of a log-concave distribution concentrates around their differential entropy, albeit with an unpleasant dependence on the ambient dimension. In this work, we prove th…
The identification of relevant features, i.e., the driving variables that determine a process or the properties of a system, is an essential part of the analysis of data sets with a large number of variables. A mathematical rigorous approach to quantifying the relevance of these features is mutual information. Mutual i…
The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.
problem Understanding the reliability and uncertainty of probabilistic predictions.
method Developed decomposition identities for proper losses, quantifying reliability, residual uncertainty, and information gain.
result A three-term identity for classification scores, revealing miscalibration, grouping term, and feature-level uncertainty.