arXiv research

ELBO converges to a sum of entropies for many generative models.

Generative models' ELBOs converge to entropy sums, proving for various models.

Theoretical analysis of cross-entropy loss functions and their robustness.

ELBO of VAEs converges to a sum of three entropies.

New method detects changes by maximizing cross-entropy, outperforming existing techniques.

Algorithm learns Nash equilibria in stochastic games using entropy-regularized policies.

Entropy of critical points generalizes Morse theory.

New methods optimize sums of bivariate functions on finite domains.

We compute the Minimal Entropy of every closed, orientable $3$-manifold, showing that its cube equals the sum of the cubes of the minimal entropies of each hyperbolic component arising from the $JSJ$ decomposition of each prime summand. As a consequence we show that the cube of the Minimal Entropy is additive with resp…

DAC enhances exploration in reinforcement learning with entropy regularization.

Novel analysis of EFP for finite-sum problems in neural networks.

Theoretical analysis of entropy approximation for Gaussian mixtures.

New entropy-based objective for sparse coding improves learning.

We show that the minimal volume entropy of closed manifolds remains unaffected when nonessential manifolds are added in a connected sum. We combine this result with the stable cohomotopy invariant of Bauer-Furuta in order to present an infinite family of four-manifolds with the following properties: 1) They have positi…

New method for sequential probability assignment reduces regret using contextual Shtarkov sums.

Entropy-based decoding improves DLM sampling efficiency.

Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using …

We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations …

The study counts units and eigenvalue patterns in SL_n(Z) and Sp_{2n}(Z) in thin tubes.

New statistical test for change-point detection using relative entropy.

Study minimax regret in sequential probability assignment with and without side information.

The paper introduces a new intrinsic reward method for exploration in reinforcement learning.

Unified framework for estimating reward functions in competitive games.

GAN+VER improves GANs by regularizing entropy to reduce mode collapse.

Bayesian methods suffer from the problem of how to specify prior beliefs. One interesting idea is to consider worst-case priors. This requires solving a stochastic zero-sum game. In this paper, we extend well-known results from bandit theory in order to discover minimax-Bayes policies and discuss when they are practica…

New RL approach uses future state and action visitation measures for better exploration.

Introduces TSI, a variance-based measure for persistence barcodes.

Market dynamic is quantified in terms of the entropy $S(τ,n)$ of the clusters formed by the intersections between the series of the prices $p_t$ and the moving average $\widetilde{p}_{t,n}$. The entropy $S(τ,n)$ is defined according to Shannon as $\sum P(τ,n)\log P(τ,n),$ with $P(τ,n)$ the probability for the cluster t…

Here we present an application of two maxentropic procedures to determine the probability density distribution of compound sums of random variables, using only a finite number of empirically determined fractional moments. The two methods are the Standard method of Maximum Entropy (SME), and the method of Maximum Entrop…

Proposes a new method to measure epistemic uncertainty in Bayesian neural networks.

Proposes squentropy loss for improved classification accuracy and model calibration.

We present simple and computationally efficient nonparametric estimators of Rényi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over $\R^d$. The estimators are calculated as the sum of $p$-th powers of the Euclidean lengths of the edges of the `genera…

We show that if a closed manifold M admits an F-structure (possibly of rank 0) then its minimal entropy vanishes. In particular, this is the case if M admits a non-trivial circle action. As a corollary we obtain that the simplicial volume of a colsed manifold admitting an F-structure is zero. We also show that if M adm…

Improved MESMOC+ optimizes constrained multi-objective problems efficiently.

The dual Minkowski problem for even data asks what are the necessary and sufficient conditions on an even prescribed measure on the unit sphere for it to be the $q$-th dual curvature measure of an origin-symmetric convex body in $\mathbb{R}^n$. A full solution to this is given when $1 < q < n$. The necessary and suffic…

Study risk-sensitive market making with entropy regularization for better quote control.

Image recognition systems have demonstrated tremendous progress over the past few decades thanks, in part, to our ability of learning compact and robust representations of images. As we witness the wide spread adoption of these systems, it is imperative to consider the problem of unintended leakage of information from …

Study on convergence of Langevin dynamics for zero-sum games in probability distributions.

For any ReLU network there is a representation in which the sum of the absolute values of the weights into each node is exactly $1$, and the input layer variables are multiplied by a value $V$ coinciding with the total variation of the path weights. Implications are given for Gaussian complexity, Rademacher complexity,…

Assume (M,g,Ω) is a closed, oriented Riemannian surface equipped with an Anosov magnetic flow. We establish certain results on the surjectivity of the adjoint of the magnetic ray transform, and use these to prove the injectivity of the magnetic ray transform on sums of tensors of degree at most two. In the final sectio…

The paper develops sum-of-squares relaxations for computing $f$-divergences.

New analysis shows how cross-entropy training shapes attention in transformers.

We present results about financial market observables, specifically returns and traded volumes. They are obtained within the current nonextensive statistical mechanical framework based on the entropy $S_{q}=k\frac{1-\sum\limits_{i=1}^{W} p_{i} ^{q}}{1-q} (q\in \Re)$ ($S_{1} \equiv S_{BG}=-k\sum\limits_{i=1}^{W}p_{i} \l…

We analyze an $N+1$-player game and the corresponding mean field game with state space $\{0,1\}$. The transition rate of $j$-th player is the sum of his control $α^j$ plus a minimum jumping rate $η$. Instead of working under monotonicity conditions, here we consider an anti-monotone running cost. We show that the mean …

Maximizes coding rate difference for robust, discriminative features.

GSP improves global average pooling for deep metric learning by learning weights and selecting semantic entities.

Engle's ARCH algorithm is a generator of stochastic time series for financial returns (and similar quantities) characterized by a time-dependent variance. It involves a memory parameter $b$ ($b=0$ corresponds to {\it no memory}), and the noise is currently chosen to be Gaussian. We assume here a generalized noise, name…

We present PESMO, a Bayesian method for identifying the Pareto set of multi-objective optimization problems, when the functions are expensive to evaluate. The central idea of PESMO is to choose evaluation points so as to maximally reduce the entropy of the posterior distribution over the Pareto set. Critically, the PES…