New method synthesizes and analyzes probability measures using entropy-regularized optimal transport.
problem Synthesize and analyze probability measures with entropy-regularized optimal transport.
method Entropy-regularized Wasserstein-2 cost and Sinkhorn divergence for synthesis and analysis.
result Computed barycentric coefficients and their stability for classification of corrupted point cloud data.
New ACE cost function encourages diversity in neural networks.
problem Training multiple classifiers with controlled diversity.
method Mathematical derivation and gradient control.
result ACE yields better ensemble results than vanilla.
We investigate the use of entropy-regularized optimal transport (EOT) cost in developing generative models to learn implicit distributions. Two generative models are proposed. One uses EOT cost directly in an one-shot optimization problem and the other uses EOT cost iteratively in an adversarial game. The proposed gene…
Study on optimal information acquisition in Kyle model with entropy cost.
problem Optimal information acquisition in Kyle model with entropy cost.
method Continuous signals are optimal, and any signal with a logit posterior distribution yields the same ex-ante value.
result Posterior expected payoff becomes normally distributed as information acquisition cost increases.
Two entropy measures quantify suboptimal portfolio performance.
problem Measuring suboptimality in investment portfolios.
method Relative entropy (KL divergence) calculations.
result Suboptimal portfolios appear better than Kelly portfolios under certain measures.
New proof of energy functional monotonicity via geodesics in measure space.
problem Proving monotonicity of energy functional in generalized Ricci flow.
method Defining adapted cost functional, geodesics, and entropy functional.
result Monotonicity of cost along backwards heat flow and energy functional along generalized Ricci flow.
MESSY estimation recovers symbolic density functions from samples using maximum entropy.
problem Estimating probability density functions from limited samples.
method Maximum-Entropy approach with gradient flow and symbolic regression.
result Efficiently finds optimal symbolic expressions for unknown distributions.
Study on thermodynamic costs of simple linear regression.
problem Understanding thermodynamic costs in machine learning models.
method Approximated thermodynamic lower bounds for exact and stochastic linear regression.
result Derived scaling laws for optimal dataset size based on generalization error.
We approximate differential entropy for efficient Bayesian experimental design.
problem Efficiently estimating expected information gain in large-scale inference problems.
method Approximate differential entropy using Monte Carlo or quasi-Monte Carlo surrogates.
result Our approach achieves comparable or better convergence rates than state-of-the-art methods.
We introduce an algorithm to locate contours of functions that are expensive to evaluate. The problem of locating contours arises in many applications, including classification, constrained optimization, and performance analysis of mechanical and dynamical systems (reliability, probability of failure, stability, etc.).…
We derive weighted log-Sobolev inequalities from a class of super Poincaré inequalities. As an application, the Talagrand inequality with larger distances are obtained. In particular, on a complete connected Riemannian manifold, we prove that the $\log^\dd$ -Sobolev inequality with $\dd\in (1,2)$ implies the $L^{2/(2-\d…
This work optimizes RL algorithms using entropy regularisation for continuous-time LQ problems.
problem Designing RL algorithms to balance exploration and exploitation in noisy environments.
method Entropy regularisation in two formulations: exploratory control and proximal policy update.
result Regret of O ( N ) \mathcal{O}(\sqrt{N}) O ( N ) for both learning algorithms over N N N episodes. Researchers analyze inverse optimal transport, deriving theoretical and empirical insights.
problem Understanding the inverse problem of inferring cost matrices from optimal couplings.
method Formalized and analyzed using entropy-regularized optimal transport, with theoretical and empirical contributions.
result Characterization of the manifold of cross-ratio equivalent costs and derivation of an MCMC sampler.
The effect of proportional transaction costs on systematically generated portfolios is studied empirically. The performance of several portfolios (the index tracking portfolio, the equally-weighted portfolio, the entropy-weighted portfolio, and the diversity-weighted portfolio) in the presence of dividends and transact…
EntProp increases entropy of clean samples to generate out-of-distribution data for better DNN performance.
problem Improving deep neural networks' accuracy and robustness to out-of-distribution data.
method High entropy propagation using data augmentation and free adversarial training.
result EntProp achieves higher standard accuracy and robustness with lower training cost.
There are many industrial situations where rods are used to stir a fluid, or where rods repeatedly stretch a material such as bread dough or taffy. The goal in these applications is to stretch either material lines (in a fluid) or the material itself (for dough or taffy) as rapidly as possible. The growth rate of mater…
New bounds show empirical EOT adapts to simpler measure.
problem Statistical performance of empirical EOT estimators.
method Novel statistical bounds, empirical process theory, dual formulation.
result Empirical EOT and its unregularized version follow lower complexity adaptation.
New method detects changes by maximizing cross-entropy, outperforming existing techniques.
problem Detecting abrupt changes in data streams without labeled examples.
method Maximizes cross-entropy between segments to find change points, using dynamic programming.
result Outperforms three state-of-the-art approaches on challenging datasets.
Improved MESMOC+ optimizes constrained multi-objective problems efficiently.
problem Optimizing constrained multi-objective problems with expensive evaluations.
method Minimizes entropy of Pareto frontier to guide search, using linear cost and decoupled evaluation.
result Significantly faster than alternatives, with more accurate entropy estimation.
Inequalities linking entropy, Fisher info, Stein discrepancy, and Wasserstein distance on Riemannian manifolds.
problem Linking entropy, Fisher info, Stein discrepancy, and Wasserstein distance on Riemannian manifolds.
method Deriving inequalities linking these measures on Riemannian manifolds.
result Strengthening and extending existing inequalities to Riemannian manifolds.
Paper proposes a new uncertainty measure for active learning in neural networks.
problem Efficiently selecting informative data points in limited labeled data scenarios.
method BalEntAcq, a new uncertainty measure based on balanced entropy, approximated by Beta distributions.
result BalEntAcq outperforms existing uncertainty measures in active learning.
A new AI framework reduces costs and improves performance.
problem High costs and over-confidence in AI models.
method Entropy-optimizing reformulation of Boltzmann machines.
result Cheaper, more performant AI models with validated reliability.
MUMBO optimizes multiple tasks efficiently, even with low-cost related functions.
problem Efficiently optimizing multiple related functions with low-cost evaluations.
method Derives a novel multi-task version of entropy search.
result Robust performance with low computational overhead across various optimization challenges.
Study timelike Ricci curvature bounds via optimal transport with Orlicz-type costs.
problem Characterize timelike Ricci curvature bounds.
method Optimal transport with Orlicz-type costs, convexity of relative entropy.
result Characterize timelike Ricci curvature lower bounds via convexity of relative entropy.
A framework for efficient multi-objective optimization using entropy search.
problem Optimizing expensive black-box functions with multiple objectives.
method Output space entropy search (OSE) to minimize resource cost.
result Improves efficiency and accuracy in multi-objective optimization.
Proof of convergence for multi-objective optimization using inverse reinforcement learning.
problem Proving convergence in multi-objective optimization problems.
method Wasserstein inverse reinforcement learning with projective subgradient method and gradient descent.
result Convergence of inverse reinforcement learning for multi-objective optimization.
A new approach improves numerical tabular data imputation by addressing diffusion models' limitations.
problem Inaccurate and difficult training in numerical tabular data imputation.
method Kernelized Negative Entropy-regularized Wasserstein gradient flow Imputation (KnewImp) based on Wasserstein gradient flow (WGF) framework.
result KnewImp significantly outperforms existing methods in numerical tabular data imputation.
The cost of belief changes with precision and is a hyperbolic geometry.
problem The cost of belief changes with precision and is a hyperbolic geometry.
method The cost of belief changes with precision and is a hyperbolic geometry.
result The cost of belief changes with precision and is a hyperbolic geometry.
Paper develops a robust HVA measure for dynamic hedging under liquidity stress.
problem Valuation of dynamic hedging under liquidity stress.
method Defines robust HVA as worst-case expected loss over a relative-entropy neighborhood of loss distributions for no-trade bands.
result Wider no-trade bands lower rebalancing costs but increase hedge-error risk.
In recent years, deep reinforcement learning has been shown to be adept at solving sequential decision processes with high-dimensional state spaces such as in the Atari games. Many reinforcement learning problems, however, involve high-dimensional discrete action spaces as well as high-dimensional state spaces. This pa…
New curvature measure for optimal transport with specific cost function.
problem Optimal transport with specific cost function.
method Proposed generalized curvature measure.
result Non-negativity of the generalized curvature implies displacement convexity.
New method reduces regret for sparse adversarial SSP problems.
problem Sparse adversarial Stochastic Shortest Path problem.
method Proposed ℓ r \ell_r ℓ r -norm regularizers for adaptive sparsity. result Regret scales with log M \sqrt{\log M} log M instead of log S A \sqrt{\log SA} log S A . We present PESMO, a Bayesian method for identifying the Pareto set of multi-objective optimization problems, when the functions are expensive to evaluate. The central idea of PESMO is to choose evaluation points so as to maximally reduce the entropy of the posterior distribution over the Pareto set. Critically, the PES…
Algorithm learns Nash equilibria in stochastic games using entropy-regularized policies.
problem Learning Nash equilibria in zero-sum stochastic games is computationally expensive.
method Entropy-regularized soft policies for Q-function updates.
result Algorithm converges to Nash equilibrium under certain conditions.
CAGES optimizes expensive RL problems by efficiently learning gradients from multiple sources.
problem Optimizing expensive-to-evaluate functions in high-dimensional spaces.
method Cost-Aware Gradient Entropy Search (CAGES) for multi-fidelity Bayesian optimization.
result Significant performance improvements on synthetic and RL benchmark problems.
In a standard setting of Bayesian optimization (BO), the objective function evaluation is assumed to be highly expensive. Multi-fidelity Bayesian optimization (MFBO) accelerates BO by incorporating lower fidelity observations available with a lower sampling cost. In this paper, we focus on the information-based approac…
Recently deep reinforcement learning (DRL) has achieved outstanding success on solving many difficult and large-scale RL problems. However the high sample cost required for effective learning often makes DRL unaffordable in resource-limited applications. With the aim of improving sample efficiency and learning performa…
Comment on entropy learning for dynamic treatment regimes.
problem Evaluating dynamic treatment regimes using entropy loss.
method Optimization-based alternative to IPW estimate.
result Suggests optimization-based approach for evaluation.
Comparing with traditional learning criteria, such as mean square error (MSE), the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyis entropy estimator, called information potential (IP), is a popular MEE cost i…
Entropy-based GP adaptive design improves failure probability estimation.
problem Limited accuracy in failure probability estimation due to model evaluation costs.
method Entropy-based Gaussian process (GP) adaptive design combined with multifidelity importance sampling (MFIS).
result More accurate failure probability estimates and higher confidence.
RL solves discrete LQ control with Gaussian optimal policy.
problem Discrete-time linear-quadratic control problem.
method Entropy-based RL to find Gaussian optimal policy.
result RL algorithm solves mean-variance asset-liability management problem.
The study extends SPT to account for real-world transaction costs, improving portfolio performance.
problem Real-world transaction costs affect portfolio performance, especially during market stress.
method Developed a continuous-time model with stochastic transaction costs and derived lower bounds for cost-adjusted wealth.
result Functionally generated portfolios can still achieve relative arbitrage after accounting for transaction costs.
Efficient approximations reduce computation of matrix-based Renyi's entropy.
problem High computational complexity of matrix-based Renyi's entropy.
method Taylor, Chebyshev, and Lanczos approximations to reduce complexity.
result Reduced complexity to significantly less than O ( n 2 ) O(n^2) O ( n 2 ) with negligible accuracy loss. The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.
problem Optimal transport problem in Linear Quadratic optimal control systems.
method Well-posedness of the Monge problem, regularity of optimal transport map, displacement interpolation of measures.
result Derivation of general interpolation inequalities for entropy functionals.
EVODiff optimizes DM inference by reducing conditional entropy, improving image generation.
problem Slow and inaccurate inference in diffusion models.
method Entropy-aware variance optimization for efficient inference.
result Significant improvement in image generation quality and efficiency.
Study of discrete-time mean-variance model using reinforcement learning.
problem Discrete-time model with more general return distribution assumptions.
method Entropy-based exploration cost, reinforcement learning algorithm design.
result Optimal investment strategy with Gaussian density function.
Two log-linear approximations speed up optimal transport for deep learning applications.
problem Computing optimal transport in high dimensions is computationally expensive.
method Locality-sensitive hashing (LSH) and Nyström approximation with LSH-based sparse corrections.
result Log-linear time algorithms for entropy-regularized OT perform well in high-dimensional spaces.
There is a rapidly increasing interest in crowdsourcing for data labeling. By crowdsourcing, a large number of labels can be often quickly gathered at low cost. However, the labels provided by the crowdsourcing workers are usually not of high quality. In this paper, we propose a minimax conditional entropy principle to…