Entropy asymmetry affects regularization in ERM, leading to biased solutions.
problem Analyzing the impact of relative entropy asymmetry in ERM regularization.
method Examined Type-I and Type-II ERM-RER, comparing their solutions and properties.
result Type-II ERM-RER regularization introduces a strong bias against training data.
New regularization method reduces support of empirical risk minimization solutions.
problem Regularization in empirical risk minimization with relative entropy.
method Introduces Type-II regularization, characterizes solutions, analyzes properties of relative entropy.
result Type-II regularization collapses solution support into reference measure's support.
Paper proves Jeffrey's update rule minimizes relative entropy.
problem Improving Bayesian learning algorithms.
method More concise proof of Jeffrey's update rule.
result Jeffrey's update rule reduces relative entropy.
Study finds optimal martingale coupling between two distributions with minimal entropy.
problem Finding the optimal martingale coupling between two distributions with minimal relative entropy.
method Solving a dual problem to find the log-density of the optimal coupling, which represents the marginal and martingale constraints.
result The log-density of the optimal coupling is given by a triplet of real functions representing the marginal and martingale constraints.
This paper optimizes trading strategies to minimize risk and maximize profit while accounting for market uncertainty.
problem Optimizing trading strategies to minimize risk and maximize profit while accounting for market uncertainty.
method Relative entropy-regularized robust optimal control problem, modeled as a stochastic differential game.
result Analytical expressions for optimal strategy and trajectory are derived under specific assumptions.
We give several Bishop-Gromov relative volume comparisons with integral Ricci curvature which improve the results in \cite{PW1}. Using one of these volume comparisons, we derive an estimate for the volume entropy in terms of integral Ricci curvature which substantially improves an earlier estimate in \cite{Au2} and giv…
The study proves compactness and existence of entropy minimizers for self-shrinking surfaces.
problem Understanding entropy in higher-codimension mean curvature flow.
method Measure-theoretical techniques and rigidity results for self-shrinkers.
result Existence of entropy minimizers and improved rigidity results.
REMEDI improves neural entropy estimation across various tasks.
problem Challenges in estimating information theoretic quantities in high-dimensional data.
method Combines minimization of cross-entropy with estimation of deviation from data density.
result Improves accuracy in entropy estimation on synthetic and natural data.
Mathematical analysis of SNE and t-SNE for dimension reduction.
problem Optimal mapping of high-dimensional data to low dimensions.
method Gradient flow of relative entropy to minimize the distance between points.
result The diameter of the evolving sets remains bounded for SNE but may blow up for t-SNE.
Study on relative entropy for hypersurfaces in hyperbolic space.
problem Understanding relative entropy for hypersurfaces in hyperbolic space.
method Relate relative entropy to renormalized area and apply monotonicity formula to mean curvature flows.
result Obtained a monotonicity formula for relative entropy in hyperbolic space.
Extends specific relative entropy to multidimensional continuous martingales.
problem Mutual singularity of martingale laws in continuous time.
method Extension of specific relative entropy from one to multiple dimensions, including closed-form expressions for simple examples.
result Establishes that the lower bound on specific relative entropy from Gantert carries over to higher dimensions and is tight.
A measure called relative cluster entropy distinguishes between correlated and uncorrelated sequences.
problem Distinguishing between sequences with different correlation degrees.
method Minimum relative entropy principle applied to cluster partitions of power-law correlated sequences.
result Optimal Hurst exponents are selected for market price series, indicating non-markovianity.
A pricing principle is introduced for non-attainable claims in incomplete markets.
problem Pricing non-attainable contingent claims in incomplete markets.
method Distorted Radon-Nikodym derivative and Tsallis relative entropy over a family of equivalent martingale measures.
result The pricing principle is closely related to backward stochastic differential equations and is arbitrage-free and time-consistent.
We define a relative entropy for two expanding solutions to mean curvature flow of hypersurfaces, asymptotic to the same cone at infinity. Adapting work of White and using recent results of Bernstein and Bernstein-Wang, we show that expanders with vanishing relative entropy are unique in a generic sense. This also impl…
New statistical test for change-point detection using relative entropy.
problem Offline change-point detection using divergence metrics.
method Study of empirical relative entropy distributions, derivation of approximations, introduction of new Berry-Esseen bounds.
result Theoretical and practical validation of relative entropy for change-point detection.
Inference for normal and Monte Carlo distributions using minimum relative entropy.
problem Inference from partial information on expectations and covariances.
method Minimum relative entropy sub-manifolds, analytical formulas, Monte Carlo simulations.
result Improved numerical implementation for inference from partial information.
Framework uses optimal transport for neural architecture search.
problem Optimizing neural architectures in deep learning.
method Semi-discrete optimization using optimal transport.
result Gradient flow and minimizing movement scheme converge to reaction-diffusion equations.
The notion of utility maximising entropy (u-entropy) of a probability density, which was introduced and studied by Slomczynski and Zastawniak (Ann. Prob 32 (2004) 2261-2285, arXiv:math.PR/0410115 v1), is extended in two directions. First, the relative u-entropy of two probability measures in arbitrary probability space…
Unique continuation result for expanding Ricci solitons.
problem Unique continuation of expanding Ricci solitons.
method Optimal relative integral convergence rate, relative entropy.
result Well-defined relative entropy for expanding solitons.
We introduce a class of generalized relative entropies (inspired by the Bregman divergence in information theory) on the Wasserstein space over a weighted Riemannian or Finsler manifold. We prove that the convexity of all the entropies in this class is equivalent to the combination of the nonnegative weighted Ricci cur…
Quantum ML predicts data with improved speed and accuracy.
problem Predicting data using maximum likelihood in a quantum setting.
method Quantum states embedding and minimization of quantum relative entropy.
result Unified framework for classical and quantum LLMs with performance guarantees.
We investigate the m-relative entropy, which stems from the Bregman divergence, on weighted Riemannian and Finsler manifolds. We prove that the displacement K-convexity of the m-relative entropy is equivalent to the combination of the nonnegativity of the weighted Ricci curvature and the K-convexity of the weig…
New clustering method adapts to data structure.
problem Natural adaptability of clustering order in EM algorithm.
method Minimizes mixing entropy to define relative entropic order.
result Empirical relative entropic order is consistent.
Accounting for the non-normality of asset returns remains challenging in robust portfolio optimization. In this article, we tackle this problem by assessing the risk of the portfolio through the "amount of randomness" conveyed by its returns. We achieve this by using an objective function that relies on the exponential…
Earlier studies have shown that stock market distributions can be well described by distributions derived from Tsallis entropy, which is a generalization of Shannon entropy to non-extensive systems. In this paper, Tsallis relative entropy (TRE), which is the generalization of Kullback-Leibler relative entropy (KLRE) to…
This work extends ME-RL using diffusion models to sample optimal policies.
problem Sampling from the optimal policy trajectory distribution in ME-RL.
method Introducing Diffusion-Augmented Markov Decision Processes (DA-MDPs) to minimize reverse KL divergence.
result DA-MDPs enable seamless integration into various ME-RL methods and outperform baselines.
Entropy analysis via kernel methods for probabilistic inference.
problem Entropy analysis of probability distributions.
method Kernel methods and reproducing kernel Hilbert spaces for entropy estimation.
result New upper-bounds on log partition functions for probabilistic inference.
Investing is a compression problem, maximizing growth by minimizing divergence.
problem Maximizing long-term wealth and minimizing risk of ruin in investing.
method Decomposes investing into three terms: money, entropy, and divergence. Uses Kelly Criterion and universal portfolio theory.
result Investing can be seen as a compression problem, with optimal strategies minimizing divergence.
Study shows how optimal transport behaves in higher dimensions.
problem Characterizing optimal transport in higher dimensions with Euclidean distance.
method Investigates the small regularization limit of entropic optimal transport.
result The limiting transport plan is supported on transport rays and uniquely minimizes a relative entropy functional.
A q-Gaussian measure is a generalization of a Gaussian measure. This generalization is obtained by replacing the exponential function with the power function of exponent 1/(1−q) (q=1). The limit case q=1 recovers a Gaussian measure. For 1≤q<3, the set of all q-Gaussian densities over the real line …
Study quantifies information flow in neural networks using relative entropy and RG analogy.
problem Quantifying information flow in deep neural networks.
method Explicit computation of relative entropy in Ising models and feedforward neural networks.
result Monotonic increase of relative entropy to an asymptotic value, confirming connection to c-theorem.
Two entropy measures quantify suboptimal portfolio performance.
problem Measuring suboptimality in investment portfolios.
method Relative entropy (KL divergence) calculations.
result Suboptimal portfolios appear better than Kelly portfolios under certain measures.
We derive generalization and excess risk bounds for neural nets using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural nets and by using the technique of chaining mutual information introduced in Asadi…
We study a notion of relative entropy motivated by self-expanders of mean curvature flow. In particular, we obtain the existence of this quantity for arbitrary hypersurfaces trapped between two disjoint self-expanders asymptotic to the same cone. This allows us to begin to develop the variational theory for the relativ…
The relative α-entropy is the Rényi analog of relative entropy and arises prominently in information-theoretic problems. Recent information geometric investigations on this quantity have enabled the generalization of the Cramér-Rao inequality, which provides a lower bound for the variance of an estimator of an escort…
We introduce a pathwise approach to analyze the relative performance of an equity portfolio with respect to a benchmark market portfolio. In this energy-entropy framework, the relative performance is decomposed into three components: a volatility term, a relative entropy term measuring the distance between the portfoli…
Synthesizes sensor likelihoods to enforce accuracy constraints in uncertain systems.
problem Designing sensing architectures for systems with uncertain or unavailable sensor models and accuracy requirements.
method Inverts the design flow, synthesizing measurement likelihoods that minimize Kullback-Leibler divergence from the prior while enforcing an accuracy bound.
result The method synthesizes a maximum-entropy posterior and induced likelihood, accommodating various discrepancy metrics.
This study analyzes theoretical guarantees for VI with fixed-variance Gaussian mixtures.
problem Theoretical analysis of variational inference with non-Gaussian distributions.
method Formulates variational inference as minimizing a mollified relative entropy, solving it through gradient descent on particle positions.
result Establishes descent lemma and approximation error bounds for optimization of variational inference.
A new method, REC, compresses images by encoding their latent representations efficiently.
problem Efficiently compressing single images with latent representations.
method Relative Entropy Coding (REC) that directly encodes latent representations with codelength close to relative entropy.
result REC is more efficient for single image compression compared to previous methods and is competitive for lossy compression.
Abstract: Necessary and sufficient conditions for gradient flows of relative entropy in Lindblad equations.
problem Conditions for gradient flows in finite-dimensional Lindblad equations.
method Analyzes conditions for a finite-dimensional Lindblad equation to have a gradient flow structure for the von Neumann relative entropy.
result A finite-dimensional Lindblad equation admits a gradient flow structure for the von Neumann relative entropy if and only if the BKM-detailed balance condition holds.
In this paper we focus on the uniqueness question for (expanding) solutions of the Harmonic map flow coming out of smooth 0-homogeneous maps with values into a closed Riemannian manifold. We introduce a relative entropy for two purposes. On the one hand, we prove the existence of two expanding solutions associated to a…
Extended characterization of RAAGs with zero minimal volume entropy.
problem Characterizing RAAGs with vanishing minimal volume entropy.
method Extended characterization from geometric dimension 2 to higher dimensions.
result Extended characterization of RAAGs with zero minimal volume entropy.
In the existing financial literature, entropy based ideas have been proposed in portfolio optimization, in model calibration for options pricing as well as in ascertaining a pricing measure in incomplete markets. The abstracted problem corresponds to finding a probability measure that minimizes the relative entropy (al…
Minimal volume entropy vanishes for mapping tori over 3-manifolds.
problem Volume entropy of mapping tori over 3-manifolds.
method A variation of amenable category and minimal volume entropy of a homology class.
result Minimal volume entropy vanishes.
Study shows rigidity for entropy minimizers in non-monotone cases.
problem Rigidity of entropy minimizers in non-monotone settings.
method Elementary proofs in non-monotone situations.
result Showed rigidity for minimizers of generalized Colding-Minicozzi entropies.
The null energy condition is characterized via convexity of entropy in Lorentzian manifolds.
problem Characterizing the null energy condition in Lorentzian manifolds.
method Characterization via convexity of the relative entropy along displacement interpolations on null hypersurfaces.
result The null energy condition is characterized in terms of convexity of the relative entropy.
New bound limits generalization gap for large models, independent of model complexity.
problem Understanding generalization gap in large-scale machine learning models.
method Established a model-independent upper bound for generalization gap using Rényi entropy.
result Generalization gap can be maintained with arbitrarily large models if data entropy is sufficient.
This paper uses normalizing flows to approximate transport maps between densities.
problem Approximating transport maps between given densities.
method Construct time-dependent controls using normalizing flows.
result Provides bounds on the number of switches for piecewise constant approximations.