Shannon's theory sets limits on information transmission.
problem Limits of information transmission in systems.
method Mathematical theory of communication.
result Defines fundamental limits on information transmission.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “Shannon information theory”
This paper explores VAEs in Fisher-Shannon plane, revealing the relationship between Fisher information and Shannon entropy.
problem Understanding the relationship between Fisher information and Shannon entropy in VAEs.
method Investigation of VAEs in Fisher-Shannon plane, focusing on the trade-off between Fisher information and Shannon entropy.
result VAEs' representation learning and log-likelihood estimation are intrinsically related to Fisher information and Shannon entropy.
New findings show Shannon information measures fail to accurately assess multivariate dependencies.
problem Accurately measuring information flow in complex systems.
method Demonstrated that Shannon information measures fail to distinguish between dyadic and polyadic relationships.
result Shannon information measures are inadequate for discovering meaningful dependency structures in joint probability distributions.
Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithmscs.IT
New bound on machine learning model performance using Jensen-Shannon information.
problem Understanding the performance of machine learning models.
method Proposes a new information-theoretic bound on generalization error.
result Shows that the new bound can be tighter than mutual information-based bounds under certain conditions.
There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. The second, Shannon information…
A new framework for information theory considers computational constraints.
problem Understanding information in complex systems with computational limitations.
method Variational extension of Shannon's information theory with computational constraints.
result Predictive -information can be created through computation and reliably estimated from data.
Proposes a new framework for decomposing information in multivariate settings.
problem Decomposing information from multiple sources about a target variable.
method Formal analogy with set theory and Blackwell order to define PID.
result Framework can be generalized to various information theories.
Paper proposes adjustments for ARI and AMI measures.
problem Lack of clear guidelines for using ARI and AMI.
method Developed generalized IT measures and solved their statistical properties.
result Proposed guidelines for using ARI and AMI.
Paper compares Rényi min-entropy vs Shannon entropy for feature selection in machine learning.
problem Feature selection in machine learning to improve model performance.
method Proposes an algorithm based on conditional Rényi min-entropy for feature selection, comparing it to Shannon-based mutual information.
result Rényi-based algorithm tends to outperform Shannon-based in real datasets.
Study complexity in financial market using Shannon entropy.
problem Measuring complexity in financial market information traffic.
method Reconstructing financial dynamics from share prices, calculating Shannon entropy.
result Shannon entropy quantifies complexity in financial market information.
The background for the general mathematical link between utility and information theory investigated in this paper is a simple financial market model with two kinds of small traders: less informed traders and insiders, whose extra information is represented by an enlargement of the other agents' filtration. The expecte…
This paper uses information theory to improve risk modeling in big data.
problem Insufficient application of information theory in actuarial science.
method Explores information theory to uncover performance limits of insurance big data systems.
result Guidance for risk modeling and actuarial pricing systems.
New framework using Jensen-Shannon divergence improves domain adaptation theory.
problem Incoherence between empirical domain adversarial training and theoretical -divergence.
method Established new theoretical framework based on Jensen-Shannon divergence, derived bi-directional upper bounds.
result Framework exhibits flexibilities for various transfer learning problems.
A novel framework infers causal direction from symbolic sequences using pattern entropy.
problem Challenges in discovering causal direction from temporal symbolic data.
method Dictionary Based Pattern Entropy () framework integrating AIT and Shannon Information Theory.
result Minimizing pattern level uncertainty yields a robust framework for causal discovery.
New Bayes' theorem optimizes semantic channels for machine learning.
problem Class imbalance and semantic meaning evolution in natural language.
method Convert Shannon's channel to semantic channel using third kind of Bayes' theorem.
result CM algorithm explains natural language evolution and improves predictive models.
Inferring Volatility in the Heston Model and its Relatives -- an Information Theoretical Approachq-fin.ST
Study infers volatility from asset prices using information theory.
problem Estimating volatility from asset prices in stochastic models.
method Information theory, mutual information, Heston model, discrete time models.
result Large uncertainty in volatility estimates due to information theoretic reasons.
Statistical test rejects market efficiency using entropy from price returns.
problem Determining market efficiency using information theory.
method Symbolic representation of price returns, Shannon entropy, and statistical test.
result Rejects market efficiency hypothesis for various datasets.
CM algorithm matches Shannon's and semantic channels for multi-label classification.
problem Tackles label learning and selection for multi-label classification.
method Adheres to maximum semantic information criterion, uses Bayes' theorem, and trains truth functions.
result Shows improved performance and adaptability to changing source distributions.
A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clusteringcs.LG
This paper uses group theory to create data-free, feature-free clustering.
problem Creating data-free, feature-free hierarchical clustering.
method Symmetry-driven hierarchical clustering using group theory.
result A new clustering framework that is globally hierarchical and data-free.
Generalizes information theory to evolving belief.
problem Measuring change in belief over time.
method Derives a general theory of information from first principles.
result Recover all information measures and interprets entropy as expected gain.
Pragmatic Information Rates, Generalizations of the Kelly Criterion, and Financial Market Efficiencycs.IT
This paper is part of an ongoing investigation of "pragmatic information", defined in Weinberger (2002) as "the amount of information actually used in making a decision". Because a study of information rates led to the Noiseless and Noisy Coding Theorems, two of the most important results of Shannon's theory, we begin …
A new objective function using Jensen-Shannon divergence improves generative learning from multiple data types.
problem Learning from multiple data types efficiently and accurately.
method Proposes a novel objective function using Jensen-Shannon divergence to approximate multimodal posteriors directly.
result The mmJSD objective optimizes an ELBO and improves generative learning tasks.
Proposes SHADE, a new regularization scheme for deep learning.
problem Improving classification performance in deep learning.
method SHADE uses information theory to define a prior based on conditional entropy, decoupling representation learning from data fitting.
result Empirically validated improvements over standard regularization schemes.
A permutation Information Theory tour through different interest rate maturities: the Libor caseq-fin.ST
This paper analyzes Libor interest rates for seven different maturities and referred to operations in British Pounds, Euro, Swiss Francs and Japanese Yen, during the period years 2001 to 2015. The analysis is performed by means of two quantifiers derived from Information Theory: the permutation Shannon entropy and the …
Study shows how information loss and operation loss are related in feature representations.
problem Understanding the relationship between information loss and operation loss in feature representations.
method Analyzes the interplay between weak information loss and operation loss in continuous representations.
result Specific forms of vanishing information loss imply vanishing MPE loss in classification.
Proposes SHADE, a new regularization scheme for deep learning.
problem Improving classification performances in deep learning.
method SHADE uses information theory to define a prior based on conditional entropy, decoupling representation learning from data fitting.
result Empirically validated improvements over common regularization schemes.
New method improves understanding of machine learning model performance.
problem Understanding how well machine learning models generalize from training data to unseen data.
method Auxiliary Distribution Method to derive new generalization error bounds.
result Upper bounds on generalization errors are tighter and more applicable.
This paper generalizes BO uncertainty measures using decision-theoretic entropies.
problem Efficiently inferring optima of expensive black-box functions.
method Introduces a generalized entropy measure from statistical decision theory to optimize Bayesian optimization.
result Demonstrates strong empirical performance across various sequential decision-making tasks.
The FSRM uses a multifractional process to capture price multifractality, revealing serial information for forecasting.
problem Capturing multifractal price dynamics for better forecasting.
method Developed a fractional stochastic regularity model based on multifractional processes and information theory.
result The serial information of the regularity process can be theoretically determined, aiding in forecasting future price increments.
New method uses SVD entropy to price artworks.
problem Lack of fine measurements in traditional art pricing models.
method SVD entropy of painting images for content measurement.
result SVD entropy positively affects sales price at 1% significance level.
We discuss the behavior of two magnitudes, physical complexity and mutual information function of the outcome of a model of heterogeneous, inductive rational agents inspired in the El Farol Bar problem and the Minority Game. The first is a measure rooted in Kolmogorov-Chaitin theory and the second one a measure related…
MIM learns useful representations with high mutual information.
problem Learning useful representations for downstream tasks.
method Symmetric Jensen-Shannon divergence and mutual information regularizer in an encoder/decoder framework.
result MIM learns high mutual information representations without posterior collapse.
FSMJ selects features with JS-divergence for text categorization.
problem Feature selection for text categorization.
method Greedy feature selection based on Jensen-Shannon divergence for real-valued features.
result FSMJ outperforms state-of-the-art methods in text categorization experiments.
Entropy analysis via kernel methods for probabilistic inference.
problem Entropy analysis of probability distributions.
method Kernel methods and reproducing kernel Hilbert spaces for entropy estimation.
result New upper-bounds on log partition functions for probabilistic inference.
Estimates price impacts and finds asymmetric market structures.
problem Understanding asymmetric price impacts in financial markets.
method Quantifies price impacts using spectral statistics and Shannon entropy.
result Asymmetric and non-random price impacts across the market.
Framework measures learning task complexity, distinguishing from memorization.
problem Measuring and distinguishing learning from memorization in learning tasks.
method Introduces an asymmetric distance and a non-asymptotic framework to compute complexity.
result Framework can measure complexity in large-scale models and real-world datasets.
LogDet estimator improves entropy estimation in neural networks.
problem Inconsistent observations and diversified interpretation in neural networks.
method Proposes LogDet estimator for reliable entropy approximation.
result LogDet estimator overcomes distributional diversity issues.
The paper characterizes curvature-dimension conditions and related inequalities on Riemannian manifolds.
problem Curvature-dimension conditions and related inequalities on Riemannian manifolds.
method Information-theoretic approach to study curvature-dimension condition, rigidity theorems, and entropy differential inequalities.
result Equivalence of curvature-dimension condition and entropy differential inequalities on Riemannian manifolds.
An econophysics approach to analyse uncertainty in financial markets: an application to the Portuguese stock marketphysics.soc-ph
In recent years there has been a closer interrelationship between several scientific areas trying to obtain a more realistic and rich explanation of the natural and social phenomena. Among these it should be emphasized the increasing interrelationship between physics and financial theory. In this field the analysis of …
Revisits SWIFT method for option pricing using Shannon wavelets.
problem Improving option pricing under known characteristic functions.
method SWIFT method based on Shannon wavelets.
result Exposes drawbacks and discusses improvements.
New method improves structure discovery for continuous nonparametric multivariate models.
problem Discovering structure in continuous nonparametric multivariate models.
method Weighted ensemble divergence estimators with asymptotic central limit theorem.
result Achieves parametric convergence rates and facilitates statistical validation.
Entropy for uniform hypergraphs defined via tensor theory.
problem Entropy calculation for uniform hypergraphs.
method Probability distribution of generalized singular values from Laplacian tensors, Shannon entropy formula.
result Tensor entropy is a measure of regularity for uniform hypergraphs.
Synchronization and Control in Intrinsic and Designed Computation: An Information-Theoretic Analysis of Competing Models of Stochastic Computationcond-mat.stat-mech
We adapt tools from information theory to analyze how an observer comes to synchronize with the hidden states of a finitary, stationary stochastic process. We show that synchronization is determined by both the process's internal organization and by an observer's model of it. We analyze these components using the conve…
This review explores entropy applications in data analysis and machine learning.
problem Characterizing probability mass distributions in data analysis and machine learning.
method Review of various entropy types and their applications.
result Entropy's versatility in data analysis and machine learning.
The paper characterizes information complexity for testing binary distributions in the broadcast model.
problem Testing binary distributions with constant advantage in the broadcast model.
method Characterization of information complexity using mixed Hellinger--Jensen--Shannon inequality and optimisation over channels.
result Identifies three parameter regimes with optimal protocols based on different channel models.
Paper quantifies how past stock returns inform about volatility and future returns.
problem Inferring volatility and future returns from past returns in stochastic volatility models.
method Quantifies mutual information between past and future stock returns and volatility.
result Past stock returns provide significant information about future volatility and returns.
Study on geometric Jensen-Shannon divergence for Gaussian measures in Hilbert space.
problem Computing divergence between Gaussian measures in infinite-dimensional Hilbert space.
method Closed form expression and regularization for divergence calculation.
result Closed form expression and regularization for Geometric Jensen-Shannon divergence.
The paper develops sum-of-squares relaxations for computing -divergences.
problem Computing -divergences from non-centered covariance matrices.
method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.