New divergence measures improve KL approximation.
problem Improving KL divergence approximation without AC condition.
method Introduced -geodesical skew divergence.
result Properties of -geodesical skew divergence studied.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,738 papers · 148 categories
Trend · papers per month
48 results for “distribution divergence”
New KL-divergence for Gaussian distributions based on Wasserstein geometry.
problem Computing KL-divergence for Gaussian distributions efficiently.
method Introducing WKL-divergence based on Wasserstein geometry.
result WKL-divergence evaluates to squared distance between points for Dirac measures.
Proposes a new divergence measure for probability distributions.
problem Challenges in estimating divergences from empirical samples.
method Embeds data into RKHS, computes Jensen-Shannon divergence between covariance operators.
result Establishes RJSD as a lower bound on Jensen-Shannon divergence, enabling variational estimation.
Paper proposes f-DPG for aligning language models with preferences.
problem Aligning language models with user preferences.
method Uses f-divergence to approximate target distributions and minimizes a forward KL from it using DPG.
result Jensen-Shannon divergence often outperforms forward KL divergence, leading to significant improvements.
Paper studies regularized KKL divergence for distributions with disjoint supports.
problem Inability of original KKL divergence to handle distributions with disjoint supports.
method Proposes a regularized variant of KKL divergence, derives bounds, and provides closed-form expression.
result Regularized KKL divergence is well-defined for all distributions and has finite-sample bounds.
DAIS minimizes symmetrized KL divergence between initial and target distributions.
problem Optimizing over initial distributions in importance sampling.
method Differentiable annealed importance sampling (DAIS) minimizing symmetrized KL divergence.
result DAIS minimizes symmetrized KL divergence between initial and target distributions.
Measuring Model Riskq-fin.RM
We propose to interpret distribution model risk as sensitivity of expected loss to changes in the risk factor distribution, and to measure the distribution model risk of a portfolio by the maximum expected loss over a set of plausible distributions defined in terms of some divergence from an estimated distribution. The…
Develops a new divergence framework that combines -divergences and IPMs.
problem Comparing distributions that are not absolutely continuous.
method Introduces -divergences as a two-stage mass-redistribution/mass-transport process.
result Improves estimation, learning, and uncertainty quantification in GANs for heavy-tailed distributions.
A method to compute divergences between decomposable models, useful in supervised learning.
problem Computing exact divergences between high-dimensional distributions is intractable.
method Proposes an approach to compute exact alpha-beta divergences between marginal and conditional distributions of decomposable models.
result Tractable computation of marginal and conditional alpha-beta divergences.
Improved bounds for estimating discrete distributions in KL divergence.
problem Estimating discrete distributions in KL divergence with accuracy.
method Used Laplace estimator and established concentration bounds.
result Deviation from mean scales as for .
Statistical Properties and Power Analysis of Divergence Measures for Credit Risk Model Monitoringq-fin.ST
Study compares statistical properties and power of divergence measures for credit risk monitoring.
problem Detecting distributional shifts in credit risk models.
method Derives statistical properties and chi-square benchmark values for Jensen-Shannon Divergence and Kullback-Leibler Divergence, demonstrating their applicability in credit risk monitoring.
result Jensen-Shannon Divergence and Kullback-Leibler Divergence follow chi-square distributions and reveal practical trade-offs in minimizing false positives vs. detecting changes.
Unified deep metric learning approach using neural networks.
problem Learning embeddings of data and extending Euclidean distances.
method Deep Bregman divergences based on neural networks.
result Superior performance on benchmark datasets compared to existing methods.
Paper connects rejection learning to Bhattacharyya divergence.
problem Learning models to abstain from predictions.
method Developed a link between rejection and thresholding different statistical divergences, focusing on Bhattacharyya divergence.
result Rejector obtained by joint ideal distribution corresponds to thresholding of skewed Bhattacharyya divergence.
We describe the underlying probabilistic interpretation of alpha and beta divergences. We first show that beta divergences are inherently tied to Tweedie distributions, a particular type of exponential family, known as exponential dispersion models. Starting from the variance function of a Tweedie model, we outline how…
Paper shows robust generative learning with minimal assumptions on target distributions.
problem Learning generative models with minimal assumptions on target distributions.
method Lipschitz-regularized -divergences with minimal assumptions.
result Stable learning across various target distributions with minimal assumptions.
Construction of ambiguity set in robust optimization relies on the choice of divergences between probability distributions. In distribution learning, choosing appropriate probability distributions based on observed data is critical for approximating the true distribution. To improve the performance of machine learning …
Proves Sard conjecture for specific distributions, controlling divergence of vector fields.
problem Proving the Sard conjecture for certain types of distributions.
method Constructs a singular distribution capturing essential abnormal lifts, proving the conjecture for rank 3 distributions in dimension 4 and generic corank 1 distributions.
result Proves the Sard conjecture for generic co-rank one distributions.
Paper proposes SinkhornDRL for distributional RL using Sinkhorn divergence and regularized Wasserstein loss.
problem Improving distributional reinforcement learning by minimizing Bellman return distribution differences.
method Introduces SinkhornDRL, a distributional RL algorithm using Sinkhorn divergence and regularized Wasserstein loss.
result SinkhornDRL consistently outperforms or matches existing algorithms on Atari games, especially in multi-dimensional reward settings.
Fisher Auto-Encodersstat.ML
Fisher auto-encoders use Fisher divergence for more robust generative modeling.
problem Model uncertainty in generative models.
method Minimizing Fisher divergence between true and modeled joint distributions.
result Fisher auto-encoders can more accurately quantify model uncertainty.
Flow matching KL divergence bound derived for smooth distributions.
problem Estimating smooth distributions efficiently.
method Deterministic upper bound on KL divergence derived from flow-matching loss.
result Flow matching achieves nearly minimax-optimal efficiency under TV distance.
Introduces Cauchy-Schwarz divergence for domain adaptation.
problem Evaluating discrepancy between source and target domains in unsupervised domain adaptation.
method Introduces Cauchy-Schwarz divergence as a measure for evaluating discrepancy between marginal and conditional distributions.
result CS divergence offers a tighter generalization error bound than Kullback-Leibler divergence.
New divergences improve score-based methods for multi-modal distributions.
problem Blindness problem in score-based divergences for multi-modal distributions.
method Proposed a new family of divergences to mitigate blindness.
result Improved performance in density estimation compared to traditional approaches.
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize t…
Paper proposes f-EBM for training deep EBMs using various f-divergences.
problem Training deep EBMs with intractable partition functions.
method Introduces f-EBM framework and optimization algorithm for any f-divergence.
result f-EBM outperforms contrastive divergence and other f-divergences.
CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
The paper is devoted to differential geometry of singular distributions (i.e., of varying dimension) on a Riemannian manifold. Such distributions are defined as images of the tangent bundle under smooth endomorphisms. We prove the novel divergence theorem with the divergence type operator and deduce the Codazzi equatio…
VI struggles to fully quantify uncertainty when distributions don't factorize.
problem Uncertainty quantification in non-factorizable distributions.
method Analysis of variational inference trade-offs and divergence choices.
result Different divergences yield different measures of uncertainty in VI.
We extend CS divergence to conditional distributions and show its advantages in time series data and sequential decision making.
problem Quantifying the closeness between conditional distributions.
method Developed and estimated a conditional Cauchy-Schwarz divergence using kernel density estimation.
result Conditional CS divergence outperforms previous methods in time series clustering and sequential decision making.
Paper introduces symmetric divergence link models for probability distributions.
problem Symmetric divergence measures for probability distributions.
method Two general classes of link models: one for survival functions and another for cumulative probability distribution functions.
result Advantages of symmetric divergence measures over asymmetric measures for model averaging and feature assessment.
The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In particular, even for those estimators whose MSE convergence rates are known, the asympt…
We present a derivation of the Kullback Leibler (KL)-Divergence (also known as Relative Entropy) for the von Mises Fisher (VMF) Distribution in -dimensions.
New analysis of annealing paths in sampling and estimation.
problem Sampling from complex distributions and estimating normalization constants.
method Extending known results on Bregman divergence to quasi-arithmetic means under monotonic embedding.
result Analogous result for quasi-arithmetic means, highlighting the interplay between means, parametric families, and divergence functionals.
Paper explores how generative models can be made more creative.
problem Limitation of generative models in diverging from original data distribution.
method Proposes a novel training objective called Bounded Adversarial Divergence (BAD) to enable creative divergence.
result Preliminary results suggest BAD can enable creative divergence in generative models.
This work improves transferability by considering conditional distributions in feature representations.
problem Improving transferability across multiple domains by considering conditional distributions.
method Introducing von Neumann conditional divergence to quantify the functional dependence between features and desired response.
result Favorable performance in terms of smaller generalization error and less catastrophic forgetting.
Paper relaxes triangle inequality for KL divergence between Gaussian distributions.
problem KL divergence does not satisfy triangle inequality for Gaussian distributions.
method Investigates relaxed triangle inequality and finds supremum.
result Supremum of KL divergence is found and conditions for attaining it are determined.
Introduces a new divergence measure for optimal transport.
problem Optimal transport distances and information divergences.
method Infimal convolution formulation of proximal optimal transport divergence.
result Establishes connections to dynamic formulations and partial differential equations.
In this paper, we derive a useful lower bound for the Kullback-Leibler divergence (KL-divergence) based on the Hammersley-Chapman-Robbins bound (HCRB). The HCRB states that the variance of an estimator is bounded from below by the Chi-square divergence and the expectation value of the estimator. By using the relation b…
Paper analyzes kNN estimator for KL divergence, proving its optimality.
problem Estimating KL divergence from identical samples.
method kNN estimator based on nearest neighbor distances.
result kNN method is asymptotically rate optimal for KL divergence estimation.
We develop a method to combine Markov chain Monte Carlo (MCMC) and variational inference (VI), leveraging the advantages of both inference approaches. Specifically, we improve the variational distribution by running a few MCMC steps. To make inference tractable, we introduce the variational contrastive divergence (VCD)…
Estimates KL divergence with fairness considerations for sub-populations.
problem Fairly estimate KL divergence between distributions considering sub-populations.
method Proposes multi-group attribution for KL divergence estimation, derived from multi-calibration.
result Shows multi-group attribution provides better KL divergence estimates conditioned on sub-populations.
Parametric adversarial divergences, which are a generalization of the losses used to train generative adversarial networks (GANs), have often been described as being approximations of their nonparametric counterparts, such as the Jensen-Shannon divergence, which can be derived under the so-called optimal discriminator …
Proposes ATM method to improve domain adaptation.
problem Mitigating distribution divergence between source and target domains.
method Adversarial Tight Match (ATM) method using Maximum Density Divergence (MDD).
result New state-of-the-art performance on domain adaptation benchmarks.
RHMC accelerates sampling from log-concave distributions.
problem Sampling from log-concave probability distributions efficiently.
method RHMC uses simulated Hamiltonian dynamics with random integration times.
result RHMC converges exponentially fast in KL divergence for log-concave distributions.
A new approach to model rejection using density ratios.
problem Improving model performance through selective prediction.
method Optimization of a loss's risk with φ-divergence regularization to find an idealized data distribution.
result Model rejection can be made by comparing the density ratio of the idealized distribution to the actual data distribution.
Study of generalized Csiszár divergences and their application to Cramér-Rao bounds.
problem Deriving lower bounds for estimator variance using generalized divergences.
method Applied Eguchi's theory to derive Fisher information metric and dual affine connections.
result More widely applicable Cramér-Rao inequality for escort distributions.
The t-distributed Stochastic Neighbor Embedding (t-SNE) is a powerful and popular method for visualizing high-dimensional data. It minimizes the Kullback-Leibler (KL) divergence between the original and embedded data distributions. In this work, we propose extending this method to other f-divergences. We analytically a…
Optimized -posteriors reduce KL divergence from true posterior in parametric misspecification.
problem Reduction of KL divergence from true posterior in parametric model misspecification.
method Derivation of Bernstein-von Mises theorem and optimization of -posteriors.
result Optimized -posteriors minimize KL divergence from true posterior, especially in severe misspecification.
Unbiased methods for alpha-divergence minimization struggle in high dimensions.
problem The difficulty of unbiased alpha-divergence minimization in high dimensions.
method Signal-to-Noise Ratio (SNR) analysis of gradient estimators.
result The SNR of the gradient estimator worsens exponentially with dimensionality.