This paper uses information theory to improve risk modeling in big data.
problem Insufficient application of information theory in actuarial science.
method Explores information theory to uncover performance limits of insurance big data systems.
result Guidance for risk modeling and actuarial pricing systems.
The paper explores how information theory aids in statistical learning models.
problem Characterizing fundamental performance limits in statistical learning models.
method Introduces divergence measures and evidence lower bound (ELBO) in model training.
result Provides a systematic derivation for generative diffusion models.
Tensor-based method simplifies causal skeleton discovery.
problem Discover causal relationships between variables.
method Express associations as tensors to reduce dimensionality.
result Causal skeleton can be determined using pair-wise tensors.
This work uses statistical mechanics to explain AI learning.
problem Understanding the statistical principles behind AI learning.
method Starting from sample concentration behaviors, the study applies statistical mechanics principles to AI and machine learning.
result Exponential families and statistical quantities are key in AI and machine learning.
Density destructors simplify complex PDFs to maximize entropy, linking to information theory.
problem Complex multivariate PDFs are hard to analyze.
method Invertible transforms that progressively remove structure from PDFs.
result Density destructors can improve estimates of information theoretic quantities.
Generalizes information theory for hierarchical partitions.
problem Understanding hierarchical decomposition of complex systems.
method Introducing a generalization of information theory for hierarchical partitions, revisiting Hierarchical Mutual Information (HMI), and proving its bounds and transformations.
result Derives hierarchical generalizations of information-theoretic quantities, including a non-metric variation of information.
Kendall transformation converts continuous data into categorical vectors for robust information theory.
problem Handling small number of observations and preserving ranking in continuous data.
method Kendall transformation converts ordered features into categorical vectors of pairwise order relations.
result Kendall transformation makes information theory methods applicable to continuous data robustly.
This paper reviews information theory in open-world machine learning.
problem Lack of a unified theoretical foundation for open-world machine learning.
method Synthesis of information theoretic approaches.
result Established a pathway toward provable and trustworthy open world intelligence.
This work connects conformal prediction to information theory for uncertainty estimation.
problem Uncertainty estimation in machine learning models.
method Information theory applied to conformal prediction.
result Proven ways to upper bound intrinsic uncertainty using information theory.
Study connects covariance cleaning theory to information theory for heavy-tailed distributions.
problem Optimizing covariance matrices for heavy-tailed distributions using information theory.
method Minimizing Frobenius norm and information loss between true and estimated covariance matrices.
result Asymptotic regime of large matrices minimizes information loss for Student's t distributions.
Statistical test rejects market efficiency using entropy from price returns.
problem Determining market efficiency using information theory.
method Symbolic representation of price returns, Shannon entropy, and statistical test.
result Rejects market efficiency hypothesis for various datasets.
The paper applies information theory to financial markets, improving risk management and asset allocation.
problem Improving risk management and asset allocation in financial markets.
method Information-theoretic measures (entropy, mutual information, etc.) applied to financial time series.
result Normalized mutual information (NMI) is a powerful measure of temporal dependence in financial markets.
Paper introduces a novel error measure for neural networks integrating statistical and information theory.
problem No single error measure is universally best for neural network training.
method Developed a novel error measure EExpAbs and integrated it into the Levenberg-Marquardt algorithm. result Self-adaptive, dynamic learning algorithm improves both model accuracy and training process.
This paper proposes a new subspace learning method, named Quantized Fisher Discriminant Analysis (QFDA), which makes use of both machine learning and information theory. There is a lack of literature for combination of machine learning and information theory and this paper tries to tackle this gap. QFDA finds a subspac…
Proposes a method to compute information theory measures via Gaussianization.
problem Challenges of computing information from multidimensional data.
method Indirect computation using a multivariate Gaussianization transform.
result Proposed methods outperform existing estimators, especially in high dimensions.
Investigates financial and economic systems using statistical mechanics and information theory.
problem Complexity, asymmetry, stochasticity, and non-linearity in financial and economic systems.
method Model-based and empirical analyses using statistical mechanics and information theory.
result Derives probability distribution functions for better understanding of financial and economic dynamics.
Examines predictability and complexity of economic time series using symbolic dynamics and entropy.
problem Understanding the predictability and complexity of economic time series.
method Symbolic dynamics and Information theory (entropy and uncertainty).
result Economic time series are complex and can be expressed in terms of information production.
A novel framework infers causal direction from symbolic sequences using pattern entropy.
problem Challenges in discovering causal direction from temporal symbolic data.
method Dictionary Based Pattern Entropy (DPE) framework integrating AIT and Shannon Information Theory. result Minimizing pattern level uncertainty yields a robust framework for causal discovery.
We adapt Shapley values to explain model uncertainty, connecting it to information theory.
problem Explaining uncertainty in model predictions.
method Adapted Shapley value framework to quantify feature contributions to predictive uncertainty.
result Deep connections between Shapley values and information theory quantities.
We consider the "partial information decomposition" (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for …
Novel method uses information theory to measure causal influences during transient neural events.
problem Characterizing network interactions during transient neural events.
method Structural Causal Models, Information Theory, Transfer Entropy, Dynamic Causal Strength, Relative Dynamic Causal Strength.
result Introduced a novel measure, relative Dynamic Causal Strength, with theoretical and empirical support.
Warped product affects divergences in information geometry.
problem Warped product's impact on divergences in information geometry.
method Study of warped product on information geometry.
result Warped product does not preserve canonical divergences.
A new framework for information theory considers computational constraints.
problem Understanding information in complex systems with computational limitations.
method Variational extension of Shannon's information theory with computational constraints.
result Predictive V-information can be created through computation and reliably estimated from data. We developed a strategic of optimal portfolio based on information theory and Tsallis statistics. The growth rate of a stock market is defined by using q-deformed functions and we find that the wealth after n days with the optimal portfolio is given by a q-exponential function. In this context, the asymptotic optim…
We propose a novel method for clustering data which is grounded in information-theoretic principles and requires no parametric assumptions. Previous attempts to use information theory to define clusters in an assumption-free way are based on maximizing mutual information between data and cluster labels. We demonstrate …
The paper establishes bounds for transductive learning using information theory.
problem Transductive learning generalization gap control.
method Information theory, PAC-Bayes, mutual information, conditional mutual information, different information measures.
result Established transductive information-theoretic and PAC-Bayesian bounds.
Mathematical study of excess growth rate connects info theory with finance.
problem Understanding the excess growth rate in portfolio theory.
method Axiomatic characterization theorems of excess growth rate in terms of relative entropy, Jensen's inequality gap, and logarithmic divergence.
result Established rich connections between information theory and finance.
New method selects diffusion scales for graph wavelets.
problem Choosing optimal diffusion scales for graph wavelets.
method Proposes an unsupervised method using information theory.
result Method selects diffusion scales for graph wavelets.
Bayesian optimization surveys information-theoretic acquisition functions.
problem Optimizing noisy, expensive, non-convex functions with unknown gradients.
method Bayesian optimization using Gaussian process surrogate models and information-theoretic acquisition functions.
result Information-theoretic acquisition functions outperform others in real scenarios.
The paper applies Gaussianization to analyze Earth data, simplifying complex multivariate distributions.
problem Challenges in accurately estimating information content in high-dimensional, heterogeneous Earth data.
method Multivariate Gaussianization for robust probability density estimation.
result Validates the method for estimating information-theoretic measures in Earth system data.
Model financial markets using information theory with a single parameter.
problem Capture the complexity of financial markets with a simple model.
method Derive an idealized model based on four information-theoretic assumptions, minimizing surprisal and divergence.
result The model uses squared radial Ornstein-Uhlenbeck processes for state variables and their sums.
The paper proves sampling methods using discrete-time processes and information theory.
problem Proving convergence guarantees for diffusion-based sampling methods.
method Directly works with discrete-time stochastic processes and uses information theory.
result Discrepancy between sampling and comparison processes is bounded using information theory.
Study OOD generalization in meta-reinforcement learning using information theory.
problem Understanding how meta-reinforcement learning handles distribution shifts.
method Information-theoretic analysis of Markov Decision Processes and gradient-based algorithms.
result Established fine-grained generalization bounds for meta-reinforcement learning.
Framework for understanding overfitting and underfitting using information theory.
problem Understanding and preventing overfitting and underfitting in machine learning.
method Information-theoretic framework measuring algorithm capacity and dataset information transfer.
result Upper-bounding algorithm capacity and establishing its relationship to machine learning quantities.
New approach to learning kernels from data using AIT principles.
problem Learning kernels from data in machine learning.
method Sparse Kernel Flows method based on AIT principles.
result Sparse Kernel Flows aligns with MDL principle and offers a robust theoretical foundation.
This paper analyzes the informational efficiency of oil market during the last three decades, and examines changes in informational efficiency with major geopolitical events, such as terrorist attacks, financial crisis and other important events. The series under study is the daily prices of West Texas Intermediate (WT…
We advocate the use of a notion of entropy that reflects the relative abundances of the symbols in an alphabet, as well as the similarities between them. This concept was originally introduced in theoretical ecology to study the diversity of ecosystems. Based on this notion of entropy, we introduce geometry-aware count…
InfoBridge uses diffusion bridges to estimate mutual information accurately.
problem Estimating mutual information between random variables.
method Formulated mutual information estimation as a domain transfer problem using diffusion bridge models.
result Demonstrated unbiased estimator for various data types.
The paper develops sum-of-squares relaxations for computing f-divergences.
problem Computing f-divergences from non-centered covariance matrices. method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.
BraidNet uses braid theory to optimize neural networks for image classification.
problem Image classification problems
method Procedural optimization of neural networks combining information theory and braid theory
result BraidNet outperforms other networks in learning speed and accuracy
The article introduces inferential moments for analyzing uncertain multivariable systems.
problem Handling inference tasks in uncertain multivariable systems.
method Bayesian inference and quantification of inferential moments.
result Quantification of inferential moments and their connection to mutual information.
The paper sets sample complexity bounds for identifying LTI systems from a finite set.
problem Identifying an LTI system from a finite set of possible systems using trajectory data.
method Maximum likelihood estimator and information theory tools.
result Upper and lower bounds for sample complexity are derived, independent of stability assumption.
The paper models financial markets using information theory to minimize information.
problem Understanding the dynamics of financial markets.
method Modeling financial market dynamics with independent stationary scalar diffusions, interpreting the market as a communication system, and minimizing information-theoretical joint information.
result Financial market dynamics are represented by squared radial Ornstein-Uhlenbeck processes with additivity and self-similarity properties.
New bounds on neural network convergence using information theory.
problem Quantifying convergence rates of neural networks to Gaussian distributions.
method Entropic inequalities and Gaussian approximations.
result Improved convergence rates in various distances for neural networks.
This paper analyzes Libor interest rates for seven different maturities and referred to operations in British Pounds, Euro, Swiss Francs and Japanese Yen, during the period years 2001 to 2015. The analysis is performed by means of two quantifiers derived from Information Theory: the permutation Shannon entropy and the …
Bayesian symbolic regression automates model discovery from data.
problem Learning closed-form mathematical models from data using heuristic methods.
method Probabilistic approach to symbolic regression, connecting to information theory and statistical physics.
result Probabilistic approach provides model plausibility and performance guarantees.
We analyze the relationships between game theory and quantum mechanics and the extensions to statistical physics and information theory. We use certain quantization relationships to assign quantum states to the strategies of a player. These quantum states are contained in a density operator which describes the new quan…
Value selection reduces model size while maintaining accuracy.
problem Space efficiency in model size reduction.
method Two probabilistic methods based on information theory's metric: PVS and P + VS.
result Value selection achieves balance between accuracy and model size reduction.