Paper characterizes embeddability of function spaces into Lp-type RKBS via metric entropy.
problem Characterizing embeddability of function spaces into Lp-type RKBS. method Establishes a connection between metric entropy growth and embeddability.
result A bound on metric entropy growth allows embedding into Lp-type RKBS. Entropy-regularized NPG converges linearly with linear function approximation.
problem Analyzing convergence of entropy-regularized NPG with function approximation.
method Established finite-time convergence analyses with entropy regularization and linear function approximation.
result Entropy-regularized NPG achieves linear convergence up to a function approximation error.
JES optimizes expensive functions by considering joint entropy over input and output spaces.
problem Optimizing expensive functions with limited evaluations.
method Joint Entropy Search (JES) considers joint entropy over input and output spaces.
result JES outperforms other information-theoretic methods in Bayesian optimization.
This paper introduces a new potential function using Tsallis entropy for neural network optimization.
problem The challenge of obtaining exponential convergence in neural network optimization.
method Utilizes a linearized potential function based on Csiszár type of Tsallis entropy.
result Derives an exponential convergence result in neural network optimization.
New entropy functionals for curved spaces help predict shape behavior.
problem Understanding entropy behavior in curved spaces.
method Introduced new entropy functionals for submanifolds of Cartan-Hadamard manifolds.
result Obtained sharp lower bounds on these entropies for certain closed hypersurfaces and observed a novel rigidity phenomenon.
Entropy measures geodesic flow complexity.
problem Measuring complexity of geodesic flows on manifolds.
method Introduced barcode entropy to measure exponential growth rate of not-too-short bars in Morse-theoretic barcodes.
result Barcode entropy bounds topological entropy and vice versa.
The article proves a new entropy formula for surfaces with boundaries.
problem Entropy formula for surfaces with boundaries.
method Established a monotonicity formula of Hamilton type entropy.
result Entropy functional and W-functional relation studied. We study a Boltzmann's type entropy functional (which appeared in existing literature) defined on Kähler metrics of a fixed Kähler class. The critical points of this functional are gradient Kähler-Ricci solitons, and the functional was known to be monotonically increasing along the Kähler-Ricci flow in the canonical cl…
The paper analyzes worst-case distortion risk metrics and weighted entropy under partial information.
problem Analyzing worst-case distortion risk metrics and weighted entropy with limited information.
method General distributions, partial information (mean and variance), various entropies and risk measures.
result Provides worst-case results for distortion risk metrics and weighted entropy.
VES-Gamma adapts EI using information-theoretic principles.
problem Optimizing black-box functions using Bayesian optimization.
method Variational Entropy Search (VES) and VES-Gamma algorithm.
result VES-Gamma improves EI by incorporating information-theoretic concepts.
New stability thresholds detect K-stability in Fano manifolds.
problem Detecting K-stability in Fano manifolds.
method Introducing new stability thresholds and studying geodesic rays in Kähler potentials.
result New entropy functional relates to radial entropy functional.
Study bounds self-shrinker entropy using Li-Yau volume and Colding-Minicozzi entropy.
problem Bounding entropy of self-shrinkers in arbitrary codimensions.
method Introduced stable conformal volume and virtual entropy to prove bounds.
result Entropy bounds are sharp and independent of codimension.
Study on non-archimedean μ-entropy for toric varieties, proving existence and uniqueness.
problem Exploring non-archimedean μ-entropy for toric varieties and its thermodynamical structure.
method Established a Rellich type compactness result for convex functions on simple polytope, proving existence and uniqueness of optimizer.
result Existence and uniqueness of optimizer for toric non-archimedean μ^λ-entropy for λ ≤ 0.
Paper develops MRCs for supervised classification using generalized maximum entropy.
problem Developing robust classifiers for decision problems.
method Generalized maximum entropy principle applied to minimax risk classifiers.
result Learning techniques for determining MRCs with performance guarantees.
Entropy of critical points generalizes Morse theory.
problem Extending Morse theory to group actions.
method Entropy of homologically detectable critical points.
result Entropy lower bound on critical points.
Enhances RL by controlling policy stochasticity through trajectory entropy constraints.
problem Non-stationary Q-value estimation and short-sighted entropy tuning in maximum entropy RL.
method Proposes TECRL framework with separate Q-functions for reward and entropy, enforcing a trajectory entropy constraint.
result DSAC-E algorithm achieves higher returns and better stability on OpenAI Gym benchmarks.
A q-Gaussian measure is a generalization of a Gaussian measure. This generalization is obtained by replacing the exponential function with the power function of exponent 1/(1−q) (q=1). The limit case q=1 recovers a Gaussian measure. For 1≤q<3, the set of all q-Gaussian densities over the real line …
This paper characterizes mu-cscK metrics using Perelman's W-entropy.
problem Characterizing mu-cscK metrics and understanding their properties.
method Using Perelman's W-entropy as a functional on the tangent bundle of Kähler metrics, the paper characterizes mu-cscK metrics as critical points of this functional.
result The W-entropy is monotonic along geodesics and provides a lower bound for mu-entropy.
Structured entropy improves classification performance on structured targets.
problem Cross-entropy loss fails to account for target variable structure.
method Proposes structured entropy, a generalization of entropy using random partitions.
result Structured cross-entropy loss yields better results on classification problems with known structure.
The maximum entropy principle can be used to assign utility values when only partial information is available about the decision maker's preferences. In order to obtain such utility values it is necessary to establish an analogy between probability and utility through the notion of a utility density function. According…
New method trains normalizing flows using entropy-regularized transport.
problem Training continuous normalizing flows efficiently.
method Formulates flows as gradients of scalar potentials, training only these potentials.
result Trains normalizing flows without explicit flow computation during training.
This paper generalizes BO uncertainty measures using decision-theoretic entropies.
problem Efficiently inferring optima of expensive black-box functions.
method Introduces a generalized entropy measure from statistical decision theory to optimize Bayesian optimization.
result Demonstrates strong empirical performance across various sequential decision-making tasks.
Extends finite entropy measures in Kähler geometry.
problem Analyzing finite entropy measures on compact Kähler manifolds.
method Defining finite p-entropy and demonstrating their inclusion in an energy class. result Stability result for the complex Monge-Ampère equation.
The notion of utility maximising entropy (u-entropy) of a probability density, which was introduced and studied by Slomczynski and Zastawniak (Ann. Prob 32 (2004) 2261-2285, arXiv:math.PR/0410115 v1), is extended in two directions. First, the relative u-entropy of two probability measures in arbitrary probability space…
Entropy regularization improves policy optimization in reinforcement learning.
problem Improving policy optimization in reinforcement learning.
method Entropy regularization is introduced to soften the greedy policy towards a more diverse softmax policy, leading to a continuously parameterized algorithm that interpolates between policy gradient and Q-learning.
result An intermediate algorithm can improve performance in reinforcement learning.
Entropy derived from Colding's volume on Ricci-flat manifolds.
problem Deriving Perelman's entropy from Colding's monotonic volume.
method Applying Colding's monotonic volume to Perelman's N-space for harmonic functions on Ricci-flat manifolds.
result Entropy is the limit of Colding's monotonic volume.
The paper studies Q-curvature and volume entropy on manifolds, proving polynomial growth polyharmonic function finiteness and rigidity.
problem Investigating Q-curvature and volume entropy on conformally flat manifolds.
method Introducing a new volume entropy, establishing identities, and proving rigidity results.
result Each polynomial growth polyharmonic function on such manifolds is of finite dimension, and the Cohn-Vossen inequality achieves equality under specific conditions.
Sharp bounds on neural network approximation rates and widths.
problem Estimating approximation rates, metric entropy, and n-widths of shallow neural networks.
method Introducing smoothly parameterized dictionaries and providing upper and lower bounds.
result Sharp bounds on approximation rates, metric entropy, and n-widths for neural networks with various activation functions.
The paper proves the monotonicity of a modified Perelman's W-entropy for mean curvature flow.
problem Proving the monotonicity of Perelman's W-entropy for mean curvature flow.
method Modified definition of K. Ecker's W-entropy and Hamilton's Harnack inequality.
result The modified W-entropy is monotonically decreasing in time.
This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create useful new ones easily. Fenchel-Young losses constructed from a generalized entropy, …
Unified framework connects EI and information-theoretic acquisition functions.
problem Distinguish between Expected Improvement and information-theoretic acquisition functions.
method Introduces Variational Entropy Search (VES) to unify EI and information-theoretic approaches.
result EI can be seen as a variational inference approximation of Max-value Entropy Search (MES).
Unified interpretation of softmax cross-entropy and negative sampling for knowledge graph embedding.
problem Lack of theoretical relationship between softmax cross-entropy and negative sampling loss functions in knowledge graph embedding.
method Used Bregman divergence to provide a unified interpretation of the two loss functions.
result Theoretical findings for fair comparison of softmax cross-entropy and negative sampling are derived.
Convolution Neural Networks (CNN) have recently achieved state-of-the art performance on handwritten Chinese character recognition (HCCR). However, most of CNN models employ the SoftMax activation function and minimize cross entropy loss, which may cause loss of inter-class information. To cope with this problem, we pr…
Theoretical analysis of cross-entropy loss functions and their robustness.
problem Guarantees for using cross-entropy as a surrogate loss function.
method Theoretical analysis of a broad family of loss functions, including cross-entropy.
result First H-consistency bounds for comp-sum losses and smooth adversarial comp-sum losses. Study on finite entropy and energy in Kähler geometry.
problem Finite entropy and energy measures in Kähler geometry.
method Refined Moser-Trudinger inequalities for quasi-plurisubharmonic functions.
result Quasi-plurisubharmonic potentials with finite entropy belong to the finite energy class En−1n. New algorithm estimates semi-continuous data density using entropy maximization.
problem Estimating density functions for semi-continuous data.
method Maximum entropy principle, requiring only constraint function samples.
result Estimate has significantly less bias compared to existing methods.
Entropy Search (ES) and Predictive Entropy Search (PES) are popular and empirically successful Bayesian Optimization techniques. Both rely on a compelling information-theoretic motivation, and maximize the information gained about the argmax of the unknown function; yet, both are plagued by the expensive computatio…
We propose the use of the functional determinant of geometric operators in constructing an entropy functional associated to geometric flows. Our approach is based on the direct computation of the partition function, with a well-defined set of microstates and macrostates in the canonical ensemble. The approach is motiva…
The Bregman divergence (Bregman distance, Bregman measure of distance) is a certain useful substitute for a distance, obtained from a well-chosen function (the "Bregman function"). Bregman functions and divergences have been extensively investigated during the last decades and have found applications in optimization, o…
The paper applies math and physics to language models, introducing entropy and geometric concepts.
problem Understanding and improving language models to approximate intelligent language.
method Formal definitions, functional analysis, topology, thermodynamics, and set theory.
result Entropy function reveals key obstacles for LLMs and offers insights into language models.
In this paper we introduce the log entropy functional and establish its monotonicity along the Ricci flow. One consequence of it is the monotonicity of the logarithmic Sobolev constant along the Ricci flow.
Ancient Ricci flows with bounded Nash entropy have uniform Sobolev inequalities.
problem Bounding Nash entropy in ancient Ricci flows.
method Uniformly bounded Nash entropy implies uniform bounds on the ν-functional, leading to uniform logarithmic and Sobolev inequalities.
result Uniform logarithmic and Sobolev inequalities on ancient Ricci flows with bounded Nash entropy.
Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptron with n input neurons, one output neuron, and no threshold bias term -- we prove that upon random initialisation of weights, the a priori p…
Quantizes Kähler-Ricci flow for Fano manifolds.
problem Optimal degeneration for Fano manifolds.
method Geometric quantization of Kähler-Ricci flow and entropy functional.
result Established convergence to original flow and entropy.
DAC enhances exploration in reinforcement learning with entropy regularization.
problem Improving exploration efficiency in reinforcement learning.
method Sample-aware entropy regularization using replay buffer action distributions.
result DAC significantly outperforms existing algorithms in reinforcement learning tasks.
Entropy study on synthetic spaces with curvature bounds.
problem Entropy functional on synthetic spaces with curvature bounds.
method Rigorous justification of entropy formula, monotonicity, and rigidity properties; heat kernel bounds.
result Bounds for heat equation solutions on synthetic spaces.
Entropy data replaces classical charts for smooth manifolds.
problem Establishing smooth structures on topological manifolds.
method Using entropy data to define admissible coordinate functions and reconstruct smooth atlases.
result Entropy-smooth structures are equivalent to classical smooth structures and stable under perturbations.
The entropy of a hypersurface is given by the supremum over all F-functionals with varying centers and scales, and is invariant under rigid motions and dilations. As a consequence of Huisken's monotonicity formula, entropy is non-increasing under mean curvature flow. We show here that a compact mean convex hypersurface…