The article presents a new entropy model for assessing stock market interest.
problem Assessing investor interest and market sentiment in exchange-traded securities.
method Intrinsic entropy model using actual trading data, without exogenous factors.
result Empirical evidence supports the model's ability to predict trading activity.
New method uses flow networks to optimize maximum entropy distributions.
problem Optimizing maximum entropy distributions directly is difficult.
method Normalizing flow networks to transform a simple distribution to a desired maximum entropy distribution.
result Demonstrated effectiveness and flexibility of maximum entropy flow networks.
Entropy-minimal measure calculated for a stochastic volatility model.
problem Calculating the entropy-minimal equivalent martingale measure in a stochastic volatility model.
method Revised related theory, calculated entropy-minimal measure.
result Entropy-minimal measure for the exponential Ornstein-Uhlenbeck model.
Machine learning for entropy calculation from binary signals.
problem Calculating entropy from binary configurations/signals.
method Transformed entropy calculation into supervised classification tasks using machine learning.
result Reproduced entropy and free energy of the 2D Ising model.
Optimizes topic number in topic models using Rényi and Tsallis entropies.
problem Determining the optimal number of topics in topic models for large textual collections.
method Uses Rényi and Tsallis entropies based on thermodynamics approach.
result Semantic stability plays a crucial role in determining the optimal number of topics.
ELBO converges to a sum of entropies for many generative models.
problem Understanding the convergence of variational lower bounds in unsupervised learning.
method Analyzing the ELBO for a broad class of generative models, showing it equals a sum of entropies.
result The ELBO is equal to a sum of entropies at stationary points for many generative models.
The study evaluates different probability models for uncertainty visualization using entropy calculations.
problem Choosing the right probability model affects memory use, run time, and accuracy in uncertainty visualization.
method Entropy calculation on ensemble data to compare various probability models (uniform, Gaussian, histogram, quantile).
result Models matching the ensemble data distribution have the lowest entropy, indicating better accuracy.
Entrocraft addresses RL performance saturation in LLMs by customizing entropy curves.
problem Performance saturation in RL algorithms for LLMs.
method Entrocraft uses rejection sampling to bias advantage distributions for customized entropy schedules.
result Entrocraft significantly improves generalization, output diversity, and long-term training in 4B models.
We investigate entropy as a financial risk measure. Entropy explains the equity premium of securities and portfolios in a simpler way and, at the same time, with higher explanatory power than the beta parameter of the capital asset pricing model. For asset pricing we define the continuous entropy as an alternative meas…
A new method infers causal direction from discrete variables using entropy of unobserved exogenous variables.
problem Identifying causal direction between two discrete random variables using observational data.
method Assumes simplicity of unobserved exogenous variable using Rényi entropy and proposes an efficient greedy algorithm.
result Proves that if exogenous variable has low entropy in true direction, it must have high entropy in wrong direction.
AER dynamically adjusts entropy regularization for better LLM reinforcement learning.
problem Policy entropy collapse in RLVR training limits exploration and reasoning performance.
method Adaptive Entropy Regularization (AER) with difficulty-aware coefficient allocation, initial-anchored target entropy, and dynamic global coefficient adjustment.
result AER consistently outperforms baselines on mathematical reasoning benchmarks, improving both accuracy and exploration.
The paper proves entropy power properties on Riemannian manifolds and Ricci flows.
problem Entropy power on Riemannian manifolds and Ricci flows.
method Proving concavity and convexity of Shannon entropy power for heat and conjugate heat equations on Riemannian manifolds and Ricci flows.
result Entropy power rigidity models on Einstein or quasi Einstein manifolds and shrinking Ricci solitons.
The intrinsic entropy model accurately estimates stock market volatility.
problem Accurately estimating historical volatility of stock market indices.
method Incorporates traded volumes alongside OHLC prices in daily data.
result Intrinsic entropy model delivers reliable estimates with lower coefficient of variation.
Paper proposes a policy-search algorithm to learn entropy-maximizing exploration policies in reward-free environments.
problem Reward-free learning in high-dimensional, continuous-control domains.
method Maximum Entropy POLicy optimization (MEPOL) algorithm that maximizes a non-parametric state entropy estimate.
result MEPOL learns a maximum-entropy exploration policy that facilitates learning various reward-based tasks.
REMEDI improves neural entropy estimation across various tasks.
problem Challenges in estimating information theoretic quantities in high-dimensional data.
method Combines minimization of cross-entropy with estimation of deviation from data density.
result Improves accuracy in entropy estimation on synthetic and natural data.
A new IRL model recovers reward and state structure from expert demonstrations.
problem Limitation of classical maximum entropy model in capturing state structure.
method Generalized maximum causal entropy for IRL models.
result Empirically outperforms classical models in recovering reward and state structure.
HCLM framework uses entropy regularization for open learning systems.
problem Real-world AI challenges and limitations of deep learning.
method Dynamical and information-theoretic framework with entropy regularization.
result Geometric entropy surrogates, especially log-determinant covariance entropy, induce stronger and more stable information forces.
Generative models' ELBOs converge to entropy sums, proving for various models.
problem Proving convergence of ELBOs to entropy sums for various generative models.
method Proofs for individual models under realistic conditions.
result ELBOs of various generative models converge to entropy sums at all stationary points.
Structured entropy improves classification performance on structured targets.
problem Cross-entropy loss fails to account for target variable structure.
method Proposes structured entropy, a generalization of entropy using random partitions.
result Structured cross-entropy loss yields better results on classification problems with known structure.
The financial market entropy is modeled using open quantum systems.
problem Understanding entropy in financial market dynamics.
method Using Open Quantum Systems to model entropy gain in financial markets.
result Interesting non-classical results generated by relaxing assumptions.
New measures for causal entropy and information gain studied.
problem Quantifying causal relationships in machine learning.
method Formal study of causal entropy and information gain.
result Established fundamental properties and relationships.
Unified framework for maximum entropy RL using Tsallis entropy.
problem Generalizing maximum entropy reinforcement learning with various entropies.
method Tsallis MDPs with Tsallis entropy maximization, controlling entropic index.
result Different entropic indices lead to different optimal policies and exploration tendencies.
ALIEN improves uncertainty estimation of language models by refining entropy-based methods.
problem Overconfidence in uncertainty estimation for language models, especially for difficult inputs.
method ALIEN refines entropy-based uncertainty by aligning it with prediction reliability, using a lightweight uncertainty head.
result ALIEN consistently outperforms strong baselines in detecting incorrect predictions and achieving the lowest calibration error.
GALILEO creates better mixture models for categorical data.
problem Generating high-quality mixture models for categorical data.
method Entropy-based density metric and annealing of components.
result GALILEO consistently finds high-quality clusters with the optimal number of components.
Entropy measures financial drawdowns, volatility, and volatility regimes.
problem Analyzing financial time series with entropy measures.
method Transfer entropy, state space models, entropy of realised volatility.
result Entropy reveals information flows and volatility regimes in financial markets.
The paper applies math and physics to language models, introducing entropy and geometric concepts.
problem Understanding and improving language models to approximate intelligent language.
method Formal definitions, functional analysis, topology, thermodynamics, and set theory.
result Entropy function reveals key obstacles for LLMs and offers insights into language models.
Entropy tracking reveals class commitment transitions in diffusion models.
problem Diffusion models lack reliable methods to detect semantic structure transitions.
method Tracking class-conditional entropy of latent variables.
result Entropy isolates noise regimes critical for semantic structure formation.
Bayesian Monte-Carlo method assesses uncertainty in shear stress entropy models.
problem Uncertainty in evaluating shear stress entropy models remains an open question.
method Bayesian Monte-Carlo (BMC) uncertainty method to evaluate four entropy models.
result FOCB statistic index determines certainty of entropy models in shear stress estimation.
The study calculates Weyl entropy in spacetime regions and shows its monotonic behavior.
problem Calculating and understanding Weyl entropy in spacetime regions.
method Introducing a candidate density for Weyl entropy in perfect fluid regions and analyzing its behavior in compact spacetime regions.
result Weyl entropy is shown to be monotonic in time and maximal in vacuum static metrics.
The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's razor" interpretation. Unfortunately, calculating the potentials in the maximum-entro…
We show vanishing results about the infimum of the topological entropy of the geodesic flow of homogeneous smooth four manifolds. We prove that any closed oriented geometric four manifold has zero minimal entropy if and only if it has zero simplicial volume. We also show that if a four manifold M admits a geometric dec…
New algorithms improve deep RL with entropy-based action selection and environment exploration.
problem Improving deep reinforcement learning algorithms for better sample efficiency and effectiveness.
method Proposes Tsallis entropy Actor-Critic (TAC), Renyi entropy Actor-Critic (RAC), and Ensemble Actor-Critic (EAC) algorithms.
result Empirically, TAC, RAC, and EAC outperform SAC and other algorithms in benchmark control tasks.
New approach to portfolio optimization shows entropy regularization is ineffective.
problem Entropy regularization in mean-variance portfolio optimization under drift uncertainty.
method Combining Bayesian filtering and stochastic policy optimization.
result Entropy regularization does not accelerate learning about unknown drift.
Bayesian Markowitz portfolio problem shows entropy regularization is ineffective.
problem Entropy regularization in Bayesian Markowitz portfolio optimization.
method Combines continuous-time Bayesian filtering with stochastic policy optimization.
result Entropy regularization does not accelerate learning of unknown drift.
Paper characterizes embeddability of function spaces into Lp-type RKBS via metric entropy.
problem Characterizing embeddability of function spaces into Lp-type RKBS. method Establishes a connection between metric entropy growth and embeddability.
result A bound on metric entropy growth allows embedding into Lp-type RKBS. Entropy corrections improve GBM's predictive accuracy for non-log-normal distributions.
problem Log-normal distribution limitations in GBM predictions.
method Entropy corrections to geometric Brownian motion (GBM).
result Improved predictive accuracy for non-log-normal distributions.
Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.
problem Tuning hierarchical topic models, especially determining the number of topics at each level, is challenging.
method The paper introduces a Renyi entropy-based metric for quality assessment and a practical tuning concept.
result The proposed method can estimate the number of topics for two hierarchical levels in hARTM model.
Fine-tuning improves information conveyance in language models by reorganizing uncertainty into more informative sequences.
problem Uncertainty reduction in large language models through fine-tuning is not fully understood, especially regarding output length.
method Proposed Canopy Entropy (CE⋆) to measure uncertainty in both output length and sequence, capturing total Shannon entropy. result Fine-tuned models exhibit stronger positive correlation between entropy rate and semantic diversity, indicating more informative and semantically meaningful generations.
CEB enhances model resilience through simple entropy bottleneck.
problem Improving model robustness against adversarial attacks.
method Conditional Entropy Bottleneck (CEB) combined with data augmentation.
result CEB significantly boosts adversarial robustness on various benchmarks.
New method uses SVD entropy to price artworks.
problem Lack of fine measurements in traditional art pricing models.
method SVD entropy of painting images for content measurement.
result SVD entropy positively affects sales price at 1% significance level.
Long memory and volatility clustering are two stylized facts frequently related to financial markets. Traditionally, these phenomena have been studied based on conditionally heteroscedastic models like ARCH, GARCH, IGARCH and FIGARCH, inter alia. One advantage of these models is their ability to capture nonlinear dynam…
The paper characterizes curvature-dimension conditions and related inequalities on Riemannian manifolds.
problem Curvature-dimension conditions and related inequalities on Riemannian manifolds.
method Information-theoretic approach to study curvature-dimension condition, rigidity theorems, and entropy differential inequalities.
result Equivalence of curvature-dimension condition and entropy differential inequalities on Riemannian manifolds.
Monotonicity and rigidity of W-entropy proved in singular spaces.
problem Entropy behavior in singular metric measure spaces.
method Space-time Wasserstein control to show monotonicity and rigidity.
result Entropy dissipation rate and rigidity models in singular spaces.
The paper proves properties of Renyi entropy power on Riemannian manifolds.
problem Properties of Renyi entropy power on Riemannian manifolds.
method Proof of concavity, rigidity models, Aronson-Benilan estimates, NIW formula, entropy isoperimetric inequality.
result Rigidity models and intrinsic relationships for Renyi entropy power.
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
problem Understanding the assumptions and dependencies in Bayesian hierarchical models.
method Demonstrates how canonical distributions and maximum entropy principles can be used to derive marginal priors in hierarchical models.
result Marginal priors in hierarchical models derived from maximum entropy principles have different constraints compared to the original priors.
Generative models use EOT cost for better image generation.
problem Developing models to learn implicit distributions for image generation.
method Two models: one-shot optimization with EOT cost and adversarial game with EOT cost.
result Improved image generation performance on MNSIT.
Enhances RL by controlling policy stochasticity through trajectory entropy constraints.
problem Non-stationary Q-value estimation and short-sighted entropy tuning in maximum entropy RL.
method Proposes TECRL framework with separate Q-functions for reward and entropy, enforcing a trajectory entropy constraint.
result DSAC-E algorithm achieves higher returns and better stability on OpenAI Gym benchmarks.
In this paper we study the consistency of an empirical minimum error entropy (MEE) algorithm in a regression setting. We introduce two types of consistency. The error entropy consistency, which requires the error entropy of the learned function to approximate the minimum error entropy, is shown to be always true if the…