In this paper we consider an information theoretic approach for the accounting classification process. We propose a matrix formalism and an algorithm for calculations of information theoretic measures associated to accounting classification. The formalism may be useful for further generalizations and computer-based imp…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “information-theoretic measures”
Unified framework for comparing clusterings from information-theoretic and pair-counting perspectives.
problem Divergent evaluations of unsupervised models due to different clustering similarity measures.
method Developed an analytical framework that unifies pair-counting and information-theoretic clustering similarity measures.
result Unified framework clarifies when and why the two regimes diverge and provides a principled basis for selecting and interpreting clustering similarity measures.
The paper proposes a framework for information-theoretic predictive uncertainty measures.
problem The need for reliable estimation of predictive uncertainty in machine learning.
method Revisiting core concepts, categorizing predictive uncertainty measures based on model and approximation of true distribution.
result Identification of conditions under which certain predictive uncertainty measures excel.
Unified framework for information-theoretic bounds on learning algorithms.
problem Deriving generalization bounds for learning algorithms.
method Probabilistic decorrelation lemma, symmetrization, couplings, chaining, Young's inequality.
result New upper bounds on generalization error in expectation and high probability.
Study uses information-theoretic measures to analyze neural networks and neuron importance.
problem Understanding the importance of individual neurons in neural networks.
method Cumulative ablation of neurons, analyzing entropy, mutual information, and class selectivity.
result Class selectivity is not a good indicator for classification performance, while mutual information and selectivity are positively correlated with performance.
Information theoretical measures, such as entropy, mutual information, and various divergences, exhibit robust characteristics in image registration applications. However, the estimation of these quantities is computationally intensive in high dimensions. On the other hand, consistent estimation from pairwise distances…
Each compact manifold M of finite dimension k is differentiable and supports an intrinsic probability measure. There then exists a measurable transformation of M to the k-dimensional "surface" of the (k+1)-dimensional ball.
Model financial markets using information theory with a single parameter.
problem Capture the complexity of financial markets with a simple model.
method Derive an idealized model based on four information-theoretic assumptions, minimizing surprisal and divergence.
result The model uses squared radial Ornstein-Uhlenbeck processes for state variables and their sums.
New information-theoretic bounds improve machine learning generalization.
problem Improving machine learning generalization beyond traditional complexity-based methods.
method Introducing bounds using Wasserstein distance and structured methods to incorporate geometry and individual data dependence.
result Established connections between different bounds and introduced new tighter bounds for various loss functions.
Information-theoretic measures for non-linear causality detection: application to social media sentiment and cryptocurrency pricesphysics.data-an
Detects non-linear causality between social media sentiment and cryptocurrency prices.
problem Detecting causality between non-linear time series data.
method Use of transfer entropy to measure information transfer, validating against synthetic data, and applying significance tests.
result Significant non-linear causality detected, orders of magnitude greater than linear causality.
Study robust mean estimation under coordinate-level corruptions using Hamming distance.
problem Robust mean estimation under realistic coordinate-level corruptions.
method Introduce a novel Hamming distance-based measure and present information-theoretic analysis.
result Data cleaning-inspired approaches can match information theoretic bounds for robust mean estimation.
Framework for understanding overfitting and underfitting using information theory.
problem Understanding and preventing overfitting and underfitting in machine learning.
method Information-theoretic framework measuring algorithm capacity and dataset information transfer.
result Upper-bounding algorithm capacity and establishing its relationship to machine learning quantities.
Random forest improves nearest neighbor estimation of information-theoretic quantities.
problem Estimating information-theoretic quantities in high-dimensional and scale-different settings.
method Decision forest-based adaptive nearest neighbor estimators.
result Forest-based estimators effectively estimate posterior probabilities and mutual information.
We propose a novel method for clustering data which is grounded in information-theoretic principles and requires no parametric assumptions. Previous attempts to use information theory to define clusters in an assumption-free way are based on maximizing mutual information between data and cluster labels. We demonstrate …
Sharp asymptotics derived for phase retrieval and compressed sensing with random generative priors.
problem Phase retrieval and compressed sensing with random measurement matrices.
method Sharp asymptotics derived for optimal performance and polynomial algorithm for random generative priors.
result Compressed phase retrieval becomes tractable with random generative priors, unlike sparse priors.
Model selection in clustering requires (i) to specify a suitable clustering principle and (ii) to control the model order complexity by choosing an appropriate number of clusters depending on the noise level in the data. We advocate an information theoretic perspective where the uncertainty in the measurements quantize…
Softmax Is Not an Artificial Trick: An Information-Theoretic View of Softmax in Neural Networkscs.LG
Softmax emerges naturally in neural networks as a measure of conditional mutual information.
problem The artificial nature of softmax in neural networks.
method Information-theoretic perspective to derive log-softmax and evaluate conditional mutual information.
result Training deterministic neural networks through log-softmax maximises conditional mutual information.
An Analysis of the Value of Information when Exploring Stochastic, Discrete Multi-Armed Banditscs.AI
Optimal regret achieved in stochastic, discrete multi-armed bandits using information-theoretic exploration.
problem Optimal exploration vs. exploitation in stochastic, discrete multi-armed bandits.
method Proposes an information-theoretic strategy based on the value of information criterion, using simulated-annealing-like updates of a parameter.
result Achieves logarithmic optimal regret with respect to the number of episodes.
Financial Information Theoryq-fin.PM
The paper applies information theory to financial markets, improving risk management and asset allocation.
problem Improving risk management and asset allocation in financial markets.
method Information-theoretic measures (entropy, mutual information, etc.) applied to financial time series.
result Normalized mutual information (NMI) is a powerful measure of temporal dependence in financial markets.
Unified framework improves meta-learning generalization bounds.
problem Limited sharpness of existing meta-generalization bounds.
method Unified information-theoretic derivation for single-step bounds.
result Unified bounds exhibit tighter scaling and computational advantages.
Study on limits of recovering sparse variables from phaseless measurements.
problem Support recovery in phase retrieval model with noisy phaseless measurements.
method Information-theoretic analysis, considering discrete and Gaussian models, Gaussian measurement matrices.
result Sharp thresholds with near-matching constant factors for sparsity and signal-to-noise ratio in various scaling regimes.
An Information-Theoretic Route to Isoperimetric Inequalities via Heat Flow and Entropy Dissipationmath.DG
New method uses entropy dissipation to prove isoperimetric inequalities.
problem Proving isoperimetric inequalities in geometric settings.
method Information-theoretic approach based on entropy dissipation under heat flow.
result New proof of Euclidean isoperimetric inequality with sharp constant.
New framework controls generalization for heavy-tailed data in RLHF and SGLD.
problem Heavy-tailed data in modern learning pipelines.
method Tail-dependent information-theoretic framework for sub-Weibull data.
result Sharp generalization bounds for heavy-tailed data.
New algorithm recovers sparse binary vectors from generalized linear measurements efficiently.
problem Recovering sparse binary vectors from generalized linear measurements.
method Linear estimation algorithm and information theoretic lower bounds.
result Optimal sample complexity of for noisy one bit quantized linear measurements.
Bayesian approach scores influential training examples for model predictions.
problem Enhance interpretability and safety of machine learning models.
method Formulate TDA as a Bayesian information-theoretic problem, scoring subsets by information loss.
result Method aligns with classical influence scores while promoting diversity for subsets.
Improved bounds for SGLD via data-dependent estimates.
problem Improving generalization bounds for noisy iterative learning algorithms.
method Variational characterization of mutual information and data-dependent priors.
result Significantly improved mutual information bounds for SGLD.
Kernel embeddings separate distinct probability distributions, simplifying testing.
problem Testing equality of non-atomic probability distributions.
method Kernel covariance embeddings and Gaussian measures in reproducing kernel Hilbert spaces.
result Testing for singularity between Gaussian measures is equivalent to testing for equality of non-atomic probability distributions.
Paper reinterprets majorizing measure theorem in terms of coding theory.
problem Understanding boundedness of random processes.
method Information-theoretic perspective using variable-length codes.
result Boundedness of random processes linked to efficient coding.
New deep clustering network uses divergence measures for unlabeled data.
problem Discovering cluster structure in unlabeled data without supervision.
method Discriminative loss function incorporating geometric regularization.
result Competitive performance on synthetic and real datasets.
This paper establishes conditions for sparse signal recovery with sparse measurements.
problem Recovering the support of a sparse signal using noisy projections with sparse measurement matrices.
method Establishes sufficient conditions for successful sparse recovery using sparse measurement matrices.
result A phase transition threshold for sparse recovery in the sparse setting is discovered, revealing a trade-off between sampling complexity and measurement sparsity.
Proposes a new Helmholtz machine model outperforming previous models.
problem Improving performance of Helmholtz machines.
method Connecting bi-directional Helmholtz machines to information theory.
result Generalized Helmholtz machine outperforms previous models.
New measures for causal entropy and information gain studied.
problem Quantifying causal relationships in machine learning.
method Formal study of causal entropy and information gain.
result Established fundamental properties and relationships.
Social media has nonlinear causal effects on stock market prices, according to new research.
problem Understanding the nonlinear relationship between social media and stock market dynamics.
method Analyzed extensive social media data related to DJIA index components using information-theoretic measures.
result Social media has significant nonlinear causality on stocks' returns, with social media dominating the directional coupling.
The study learns causal graphs from time series data using entropy measures.
problem Learning causal graphs from time series data.
method Constraint-based framework, information-theoretic measures, generalized causation entropy, PC and FCI algorithms.
result The methods effectively construct causal graphs from time series data.
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
New causal measures improve feature selection in AI models.
problem Lack of causal interpretability in AI models.
method Introduces causal entropy and causal information gain to assess feature control.
result Demonstrates superiority of causal information gain in feature selection.
Improved Thompson Sampling for logistic bandits with information-theoretic analysis.
problem Optimizing binary reward probabilities in logistic bandit problems.
method Information-theoretic framework, focusing on the information ratio and minimax measure.
result Bound on Bayesian expected regret of for logistic bandits.
New framework for detecting complex interactions in multivariate data.
problem Insufficient pairwise measures fail to capture multivariate data complexities.
method Lattice theory and operator functions to derive higher-order information-theoretic measures.
result Streitberg Information fully characterizes all interactions among variables.
Study on sparse recovery with mixed-quality data, establishing sample-size conditions.
problem Sparse recovery with heterogeneous noise from high- and low-quality sources.
method Establishes linear trade-off for sufficient conditions, analyzes LASSO algorithm.
result Linear trade-off for sufficient conditions, robustness of LASSO to data heterogeneity.
Proposes PIC and POIC for measuring task difficulty in RL.
problem Lack of metrics to measure task difficulty in RL.
method Introduces policy information capacity (PIC) and policy-optimal information capacity (POIC) as metrics based on mutual information.
result Empirically shows PIC and POIC correlate with task solvability better than alternatives.
This paper investigates methods for quantifying similarity between audio signals, specifically for the task of of cover song detection. We consider an information-theoretic approach, where we compute pairwise measures of predictability between time series. We compare discrete-valued approaches operating on quantised au…
Quantifying Causal Coupling Strength: A Lag-specific Measure For Multivariate Time Series Related To Transfer Entropyphysics.data-an
While it is an important problem to identify the existence of causal associations between two components of a multivariate time series, a topic addressed in Runge et al. (2012), it is even more important to assess the strength of their association in a meaningful way. In the present article we focus on the problem of d…
New algorithm for signal reconstruction from multi-layered measurements.
problem Reconstructing signals from multi-layered non-linear measurements.
method Multi-Layer Approximate Message Passing (ML-AMP) algorithm and state evolution equations.
result Asymptotic free energy and minimal achievable error derived.
Density destructors simplify complex PDFs to maximize entropy, linking to information theory.
problem Complex multivariate PDFs are hard to analyze.
method Invertible transforms that progressively remove structure from PDFs.
result Density destructors can improve estimates of information theoretic quantities.
We study the localization of a cluster of activated vertices in a graph, from adaptively designed compressive measurements. We propose a hierarchical partitioning of the graph that groups the activated vertices into few partitions, so that a top-down sensing procedure can identify these partitions, and hence the activa…
Unified treatment of PAC-Bayesian and information-theoretic generalization bounds.
problem Generalization capabilities of machine learning algorithms.
method PAC-Bayesian and information-theoretic perspectives.
result Unified treatment and modular structure of proofs.
Meta-learning bound uses conditional mutual information.
problem Bounding generalization performance in meta-learning.
method Extends CMI framework to meta-learning with a meta-supersample.
result Explicit bound involving two CMI terms.
The paper proposes a new co-training method using information theory.
problem Predicting future sensations from past ones.
method Introduces a co-training objective based on mutual information.
result The method improves mutual information between past and future sensations.