A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
DAC enhances exploration in reinforcement learning with entropy regularization.
The article presents a new entropy model for assessing stock market interest.
We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent. The minimum entropy required for such a latent is known as common entropy in information theory. We extend this notion to Renyi common entropy by minimizing the Renyi entropy of the …
In this paper, we present a new class of Markov decision processes (MDPs), called Tsallis MDPs, with Tsallis entropy maximization, which generalizes existing maximum entropy reinforcement learning (RL). A Tsallis MDP provides a unified framework for the original RL problem and RL with various types of entropy, includin…
AR-DAE approximates entropy gradient for machine learning models.
Two hitherto disconnected threads of research, diverse exploration (DE) and maximum entropy RL have addressed a wide range of problems facing reinforcement learning algorithms via ostensibly distinct mechanisms. In this work, we identify a connection between these two approaches. First, a discriminator-based diversity …
Neural network MCMC sampler maximizes proposal entropy for efficient sampling.
The construction of efficient and effective decision trees remains a key topic in machine learning because of their simplicity and flexibility. A lot of heuristic algorithms have been proposed to construct near-optimal decision trees. ID3, C4.5 and CART are classical decision tree algorithms and the split criteria they…
Enhances RL by controlling policy stochasticity through trajectory entropy constraints.
Optimized fuzzy entropy framework improves feature selection and classification performance.
New measures for causal entropy and information gain studied.
In this paper, we develop the notion of entropy for uniform hypergraphs via tensor theory. We employ the probability distribution of the generalized singular values, calculated from the higher-order singular value decomposition of the Laplacian tensors, to fit into the Shannon entropy formula. We show that this tensor …
Paper develops MRCs for supervised classification using generalized maximum entropy.
Structured entropy improves classification performance on structured targets.
Proposes a new criterion for selecting Nash equilibria considering both utility and inequality.
Improved exploration methods for reinforcement learning with reduced sample complexity.
Entropy minimization has been widely used in unsupervised domain adaptation (UDA). However, existing works reveal that entropy minimization only may result into collapsed trivial solutions. In this paper, we propose to avoid trivial solutions by further introducing diversity maximization. In order to achieve the possib…
Causal discovery is a fundamental problem in statistics and has wide applications in different fields. Transfer Entropy (TE) is a important notion defined for measuring causality, which is essentially conditional Mutual Information (MI). Copula Entropy (CE) is a theory on measurement of statistical independence and is …
JES optimizes expensive functions by considering joint entropy over input and output spaces.
New method detects changes by maximizing cross-entropy, outperforming existing techniques.
Entropy regularization is used to get improved optimization performance in reinforcement learning tasks. A common form of regularization is to maximize policy entropy to avoid premature convergence and lead to more stochastic policies for exploration through action space. However, this does not ensure exploration in th…
A recent proposal by Ryu and Takayanagi for a holographic interpretation of entanglement entropy in conformal field theories dual to supergravity on anti-de Sitter (adS) is generalized to include entanglement entropy of black holes living on the boundary of adS. The generalized proposal is verified in boundary dimensio…
HCLM framework uses entropy regularization for open learning systems.
We translate the problem of calculating the entropy of a set of binary configurations/signals into a sequence of supervised classification tasks. Subsequently, one can use virtually any machine learning classification algorithm for computing entropy. This procedure can be used to compute entropy, and consequently the f…
In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of the proposed framework. First, the optimal solution of an MCTE problem is shown …
This paper studies an entropy-based multi-objective Bayesian optimization (MBO). The entropy search is successful approach to Bayesian optimization. However, for MBO, existing entropy-based methods ignore trade-off among objectives or introduce unreliable approximations. We propose a novel entropy-based MBO called Pare…
Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.
Paper proposes a policy-search algorithm to learn entropy-maximizing exploration policies in reward-free environments.
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic quantities as test statistics, that is, as quantities obtained from empirical data, p…
Proposes MGCE for improved classification performance.
This article proposes a method to quantify the structure of a bipartite graph using a network entropy per link. The network entropy of a bipartite graph with random links is calculated both numerically and theoretically. As an application of the proposed method to analyze collective behavior, the affairs in which parti…
Entropy Search (ES) and Predictive Entropy Search (PES) are popular and empirically successful Bayesian Optimization techniques. Both rely on a compelling information-theoretic motivation, and maximize the information gained about the of the unknown function; yet, both are plagued by the expensive computatio…
EDLP samples flat modes in discrete spaces using entropy.
Proposes integrating global and local entropy for more reliable LLMs.
AER dynamically adjusts entropy regularization for better LLM reinforcement learning.
The paper proposes a method to identify high-quality financial patterns using entropy.
Proposes a method to solve deep neural networks' local minimum problem.
LogDet estimator improves entropy estimation in neural networks.
Paper tests for time-varying entropy in stock prices, finding periods of inefficiency.
In this work, we study the credit assignment problem in reward augmented maximum likelihood (RAML) learning, and establish a theoretical equivalence between the token-level counterpart of RAML and the entropy regularized reinforcement learning. Inspired by the connection, we propose two sequence prediction algorithms, …
In this paper, we propose a general framework to learn a robust large-margin binary classifier when corrupt measurements, called anomalies, caused by sensor failure might be present in the training set. The goal is to minimize the generalization error of the classifier on non-corrupted measurements while controlling th…
This review explores entropy applications in data analysis and machine learning.
New method accelerates convergence for entropy-regularized reinforcement learning problems.
A neural network method estimates entropy production from system trajectories.
We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the unobserved exogenous variable: Inspired by Occam's razor, we assume that the exogenous variable is sim…
Entropy corrections improve GBM's predictive accuracy for non-log-normal distributions.
Proposes a new method to measure epistemic uncertainty in Bayesian neural networks.
New method uses SVD entropy to price artworks.