Shannon's theory sets limits on information transmission.
problem Limits of information transmission in systems.
method Mathematical theory of communication.
result Defines fundamental limits on information transmission.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “Shannon's theory”
New framework using Jensen-Shannon divergence improves domain adaptation theory.
problem Incoherence between empirical domain adversarial training and theoretical -divergence.
method Established new theoretical framework based on Jensen-Shannon divergence, derived bi-directional upper bounds.
result Framework exhibits flexibilities for various transfer learning problems.
This paper explores VAEs in Fisher-Shannon plane, revealing the relationship between Fisher information and Shannon entropy.
problem Understanding the relationship between Fisher information and Shannon entropy in VAEs.
method Investigation of VAEs in Fisher-Shannon plane, focusing on the trade-off between Fisher information and Shannon entropy.
result VAEs' representation learning and log-likelihood estimation are intrinsically related to Fisher information and Shannon entropy.
Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithmscs.IT
New bound on machine learning model performance using Jensen-Shannon information.
problem Understanding the performance of machine learning models.
method Proposes a new information-theoretic bound on generalization error.
result Shows that the new bound can be tighter than mutual information-based bounds under certain conditions.
Study complexity in financial market using Shannon entropy.
problem Measuring complexity in financial market information traffic.
method Reconstructing financial dynamics from share prices, calculating Shannon entropy.
result Shannon entropy quantifies complexity in financial market information.
Paper compares Rényi min-entropy vs Shannon entropy for feature selection in machine learning.
problem Feature selection in machine learning to improve model performance.
method Proposes an algorithm based on conditional Rényi min-entropy for feature selection, comparing it to Shannon-based mutual information.
result Rényi-based algorithm tends to outperform Shannon-based in real datasets.
There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. The second, Shannon information…
Revisits SWIFT method for option pricing using Shannon wavelets.
problem Improving option pricing under known characteristic functions.
method SWIFT method based on Shannon wavelets.
result Exposes drawbacks and discusses improvements.
Entropy for uniform hypergraphs defined via tensor theory.
problem Entropy calculation for uniform hypergraphs.
method Probability distribution of generalized singular values from Laplacian tensors, Shannon entropy formula.
result Tensor entropy is a measure of regularity for uniform hypergraphs.
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this…
Study on geometric Jensen-Shannon divergence for Gaussian measures in Hilbert space.
problem Computing divergence between Gaussian measures in infinite-dimensional Hilbert space.
method Closed form expression and regularization for divergence calculation.
result Closed form expression and regularization for Geometric Jensen-Shannon divergence.
A new framework for information theory considers computational constraints.
problem Understanding information in complex systems with computational limitations.
method Variational extension of Shannon's information theory with computational constraints.
result Predictive -information can be created through computation and reliably estimated from data.
Proposes a new framework for decomposing information in multivariate settings.
problem Decomposing information from multiple sources about a target variable.
method Formal analogy with set theory and Blackwell order to define PID.
result Framework can be generalized to various information theories.
The note evaluates different methods for option pricing using Shannon Wavelets.
problem Efficient computation of Shannon Wavelet coefficients for option pricing.
method Evaluation of cosine expansion, direct algorithms, and Filon quadrature.
result Filon quadrature is more efficient for computing Shannon Wavelet coefficients.
The Shannon theorem is extended to locally compact groups.
problem Identifying the Poisson boundary of locally compact groups.
method Random walks and Shannon-McMillan-Breiman theorem.
result Generalized criteria for identifying Poisson boundaries.
Proposes a new divergence measure for probability distributions.
problem Challenges in estimating divergences from empirical samples.
method Embeds data into RKHS, computes Jensen-Shannon divergence between covariance operators.
result Establishes RJSD as a lower bound on Jensen-Shannon divergence, enabling variational estimation.
CM algorithm matches Shannon's and semantic channels for multi-label classification.
problem Tackles label learning and selection for multi-label classification.
method Adheres to maximum semantic information criterion, uses Bayes' theorem, and trains truth functions.
result Shows improved performance and adaptability to changing source distributions.
A new objective function using Jensen-Shannon divergence improves generative learning from multiple data types.
problem Learning from multiple data types efficiently and accurately.
method Proposes a novel objective function using Jensen-Shannon divergence to approximate multimodal posteriors directly.
result The mmJSD objective optimizes an ELBO and improves generative learning tasks.
New Bayes' theorem optimizes semantic channels for machine learning.
problem Class imbalance and semantic meaning evolution in natural language.
method Convert Shannon's channel to semantic channel using third kind of Bayes' theorem.
result CM algorithm explains natural language evolution and improves predictive models.
Proposes a new loss function for learning with noisy labels.
problem Improving model learnability with noisy labels.
method Uses generalized Jensen-Shannon divergence as a noise-robust loss function.
result Shows state-of-the-art results on noisy data.
The paper proves entropy power properties on Riemannian manifolds and Ricci flows.
problem Entropy power on Riemannian manifolds and Ricci flows.
method Proving concavity and convexity of Shannon entropy power for heat and conjugate heat equations on Riemannian manifolds and Ricci flows.
result Entropy power rigidity models on Einstein or quasi Einstein manifolds and shrinking Ricci solitons.
A novel framework infers causal direction from symbolic sequences using pattern entropy.
problem Challenges in discovering causal direction from temporal symbolic data.
method Dictionary Based Pattern Entropy () framework integrating AIT and Shannon Information Theory.
result Minimizing pattern level uncertainty yields a robust framework for causal discovery.
Adjusted for chance measures are widely used to compare partitions/clusterings of the same data set. In particular, the Adjusted Rand Index (ARI) based on pair-counting, and the Adjusted Mutual Information (AMI) based on Shannon information theory are very popular in the clustering community. Nonetheless it is an open …
In this report, we derive a non-negative series expansion for the Jensen-Shannon divergence (JSD) between two probability distributions. This series expansion is shown to be useful for numerical calculations of the JSD, when the probability distributions are nearly equal, and for which, consequently, small numerical er…
A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clusteringcs.LG
This paper uses group theory to create data-free, feature-free clustering.
problem Creating data-free, feature-free hierarchical clustering.
method Symmetry-driven hierarchical clustering using group theory.
result A new clustering framework that is globally hierarchical and data-free.
Paper tests for time-varying entropy in stock prices, finding periods of inefficiency.
problem Testing for time-varying entropy in stock price dynamics.
method Unbiased approximation of Shannon entropy variance, optimal rolling window selection, hypothesis testing.
result Existence of periods of market inefficiency for meme stocks.
Compressed Counting (CC) [22] was recently proposed for estimating the ath frequency moments of data streams, where 0 < a <= 2. CC can be used for estimating Shannon entropy, which can be approximated by certain functions of the ath frequency moments as a -> 1. Monitoring Shannon entropy for anomaly detection (e.g., DD…
The background for the general mathematical link between utility and information theory investigated in this paper is a simple financial market model with two kinds of small traders: less informed traders and insiders, whose extra information is represented by an enlargement of the other agents' filtration. The expecte…
We conduct an axiomatic study of the problem of estimating the strength of a known causal relationship between a pair of variables. We propose that an estimate of causal strength should be based on the conditional distribution of the effect given the cause (and not on the driving distribution of the cause), and study d…
Statistical Properties and Power Analysis of Divergence Measures for Credit Risk Model Monitoringq-fin.ST
Study compares statistical properties and power of divergence measures for credit risk monitoring.
problem Detecting distributional shifts in credit risk models.
method Derives statistical properties and chi-square benchmark values for Jensen-Shannon Divergence and Kullback-Leibler Divergence, demonstrating their applicability in credit risk monitoring.
result Jensen-Shannon Divergence and Kullback-Leibler Divergence follow chi-square distributions and reveal practical trade-offs in minimizing false positives vs. detecting changes.
Relationships that exist between the classical, Shannon-type, and geometric-based approaches to sampling are investigated. Some aspects of coding and communication through a Gaussian channel are considered. In particular, a constructive method to determine the quantizing dimension in Zador's theorem is provided. A geom…
This review explores entropy applications in data analysis and machine learning.
problem Characterizing probability mass distributions in data analysis and machine learning.
method Review of various entropy types and their applications.
result Entropy's versatility in data analysis and machine learning.
Researchers calculate Shannon entropy rates of hidden Markov processes efficiently.
problem No finite expression exists for Shannon entropy rates of hidden Markov processes.
method Developed an efficient method to calculate entropy rates and identify minimal predictive features.
result Entropy rates can be accurately calculated for hidden Markov processes.
This paper uses information theory to improve risk modeling in big data.
problem Insufficient application of information theory in actuarial science.
method Explores information theory to uncover performance limits of insurance big data systems.
result Guidance for risk modeling and actuarial pricing systems.
New method improves understanding of machine learning model performance.
problem Understanding how well machine learning models generalize from training data to unseen data.
method Auxiliary Distribution Method to derive new generalization error bounds.
result Upper bounds on generalization errors are tighter and more applicable.
This paper generalizes BO uncertainty measures using decision-theoretic entropies.
problem Efficiently inferring optima of expensive black-box functions.
method Introduces a generalized entropy measure from statistical decision theory to optimize Bayesian optimization.
result Demonstrates strong empirical performance across various sequential decision-making tasks.
The paper explores how GANs' learned distribution aligns with real data distribution.
problem Aligning GAN's learned distribution with real data distribution.
method Optimizing Jensen-Shannon divergence to force alignment, comparing gradients of different distances.
result Wasserstein may have desirable properties like reduced mode collapse.
We develop a complexity measure for large-scale economic systems based on Shannon's concept of entropy. By adopting Leontief's perspective of the production process as a circular flow, we formulate the process as a Markov chain. Then we derive a measure of economic complexity as the average number of bits required to e…
A new metric uses nonparametric comparison for fitting parametric distributions.
problem Measuring goodness-of-fit for nonlinear models using maximum likelihood estimation.
method Survival Jensen-Shannon divergence () and its empirical counterpart () for nonparametric comparison.
result The can be used as a measure of goodness-of-fit in maximum likelihood estimation.
The construction of efficient and effective decision trees remains a key topic in machine learning because of their simplicity and flexibility. A lot of heuristic algorithms have been proposed to construct near-optimal decision trees. ID3, C4.5 and CART are classical decision tree algorithms and the split criteria they…
Statistical test rejects market efficiency using entropy from price returns.
problem Determining market efficiency using information theory.
method Symbolic representation of price returns, Shannon entropy, and statistical test.
result Rejects market efficiency hypothesis for various datasets.
An econophysics approach to analyse uncertainty in financial markets: an application to the Portuguese stock marketphysics.soc-ph
In recent years there has been a closer interrelationship between several scientific areas trying to obtain a more realistic and rich explanation of the natural and social phenomena. Among these it should be emphasized the increasing interrelationship between physics and financial theory. In this field the analysis of …
Bayesian Monte-Carlo method assesses uncertainty in shear stress entropy models.
problem Uncertainty in evaluating shear stress entropy models remains an open question.
method Bayesian Monte-Carlo (BMC) uncertainty method to evaluate four entropy models.
result FOCB statistic index determines certainty of entropy models in shear stress estimation.
FINGER computes von Neumann graph entropy efficiently for online graph sequence analysis.
problem Efficiently compute von Neumann graph entropy for online graph sequence analysis.
method Fast Incremental von Neumann Graph Entropy (FINGER) framework.
result FINGER reduces VNGE computation complexity from cubic to linear.
Paper introduces ENZ to measure significant coefficients in sparse recovery, improving over classical methods.
problem Numerical noise creates long tails of negligible coefficients in sparse recovery.
method Entropy-based notion of effective sparsity (ENZ) to measure significant coefficients, proving stability under restricted isometry condition.
result ENZ decomposes into support cardinality and efficiency factor, providing a precise measure of sparsity.
Proposes SHADE, a new regularization scheme for deep learning.
problem Improving classification performance in deep learning.
method SHADE uses information theory to define a prior based on conditional entropy, decoupling representation learning from data fitting.
result Empirically validated improvements over standard regularization schemes.
A permutation Information Theory tour through different interest rate maturities: the Libor caseq-fin.ST
This paper analyzes Libor interest rates for seven different maturities and referred to operations in British Pounds, Euro, Swiss Francs and Japanese Yen, during the period years 2001 to 2015. The analysis is performed by means of two quantifiers derived from Information Theory: the permutation Shannon entropy and the …
Proposes SHADE, a new regularization scheme for deep learning.
problem Improving classification performances in deep learning.
method SHADE uses information theory to define a prior based on conditional entropy, decoupling representation learning from data fitting.
result Empirically validated improvements over common regularization schemes.