We review recent results about the maximal values of the Kullback-Leibler information divergence from statistical models defined by neural networks, including naive Bayes models, restricted Boltzmann machines, deep belief networks, and various classes of exponential families. We illustrate approaches to compute the max…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “divergence maximization”
Optimal payoff choice constrained by Bregman-Wasserstein divergence.
problem Maximizing utility under a deviation constraint from a benchmark.
method Solving the problem using Bregman-Wasserstein divergence with a convex function φ.
result Provided the optimal payoff choice in this setting.
We present new methods for batch anomaly detection in multivariate time series. Our methods are based on maximizing the Kullback-Leibler divergence between the data distribution within and outside an interval of the time series. An empirical analysis shows the benefits of our algorithms compared to methods that treat e…
Spread Divergencestat.ML
For distributions and with different supports or undefined densities, the divergence may not exist. We define a Spread Divergence on modified and and describe sufficient conditions for t…
Unified technique for sequential estimation of convex divergences.
problem Estimating convex divergences between distributions.
method Martingale methods and maximal inequalities for reverse submartingales.
result Valid time-uniform confidence sequences for arbitrary stopping times.
Generative adversarial networks (GANs) can be interpreted as an adversarial game between two players, a discriminator D and a generator G, in which D learns to classify real from fake data and G learns to generate realistic data by "fooling" D into thinking that fake data is actually real data. Currently, a dominating …
-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler divergence, chi-squared divergence, squared Hellinger distance, total variation distance e…
Geodesics in curved spaces spread evenly over time.
problem Equidistribution of geodesics in negatively curved spaces.
method Proving equidistribution of geodesic flow orbits towards measures of maximal entropy and Bowen-Margulis measure.
result Equidistribution of divergent geodesics in negative curvature as their complexity increases.
Automatic detection of anomalies in space- and time-varying measurements is an important tool in several fields, e.g., fraud detection, climate analysis, or healthcare monitoring. We present an algorithm for detecting anomalous regions in multivariate spatio-temporal time-series, which allows for spotting the interesti…
New -divergence measures improve robustness in noisy label learning.
problem Improving robustness in learning with noisy labels.
method Derived decoupling property of -divergence measures under label noise.
result Properly defined -divergence measures are robust with label noise.
A new VIS approach improves log-likelihood estimation in latent variable models.
problem Challenges in achieving high log-likelihood with VI for complex posterior distributions.
method Uses forward divergence to optimize proposal distribution for better log-likelihood estimation.
result Consistently outperforms state-of-the-art baselines in log-likelihood and parameter estimation.
The paper introduces a new divergence for portfolio management to outperform a benchmark.
problem Maximizing expected utility of outperformance over a benchmark with constraints.
method Uses -Bregman-Wasserstein divergence to penalize underperformance more than overperformance.
result Proves existence and uniqueness of optimal portfolio strategy and conditions for constraints binding.
Statistical Properties and Power Analysis of Divergence Measures for Credit Risk Model Monitoringq-fin.ST
Study compares statistical properties and power of divergence measures for credit risk monitoring.
problem Detecting distributional shifts in credit risk models.
method Derives statistical properties and chi-square benchmark values for Jensen-Shannon Divergence and Kullback-Leibler Divergence, demonstrating their applicability in credit risk monitoring.
result Jensen-Shannon Divergence and Kullback-Leibler Divergence follow chi-square distributions and reveal practical trade-offs in minimizing false positives vs. detecting changes.
Paper estimates optimal ROC curve arc length and AUC, improving classification performance.
problem Estimating optimal ROC curve arc length and AUC in imbalanced binary classification.
method Expresses arc length and AUC as variational objectives, estimating using positive and negative samples.
result Proposed classification procedure maximizes an approximate lower bound of maximal AUC.
Unified view of KL-divergence and IPMs via DRE, with new DRM metrics.
problem Unified understanding of KL-divergence and IPMs.
method Unified representation via maximum likelihood density-ratio estimation (DRE).
result Unified form of IPMs and novel DRM metrics.
EM optimizes tensor density estimation by relaxing -divergence to KL-divergence.
problem Analytical challenges in traditional -divergence optimization for tensor-based density estimation.
method EM algorithm: relaxes optimization to KL-divergence, then applies tensor many-body approximation.
result Flexible modeling of various low-rank structures and their mixtures.
Investing is a compression problem, maximizing growth by minimizing divergence.
problem Maximizing long-term wealth and minimizing risk of ruin in investing.
method Decomposes investing into three terms: money, entropy, and divergence. Uses Kelly Criterion and universal portfolio theory.
result Investing can be seen as a compression problem, with optimal strategies minimizing divergence.
Information Forestscs.LG
We describe Information Forests, an approach to classification that generalizes Random Forests by replacing the splitting criterion of non-leaf nodes from a discriminative one -- based on the entropy of the label distribution -- to a generative one -- based on maximizing the information divergence between the class-con…
Study improves density estimation for compact domains using -lifted KL divergence.
problem Estimating probability density functions on compact domains.
method Introduced -lifted Kullback--Leibler (KL) divergence for risk minimization.
result Proved bound on estimation error.
When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guaranteecs.LG
The paper proposes methods to optimize pAUC for deep learning using DRO.
problem Optimizing partial AUC for deep learning models.
method Proposes gradient-based methods using DRO formulations for pAUC maximization.
result Proves convergence of proposed algorithms for optimizing pAUC.
Causal KL improves on existing metrics for evaluating causal models.
problem Insufficient discrimination between causal models using edit-distance and KL divergence.
method Introducing Causal KL, an augmented KL divergence that considers causal relationships.
result Causal KL variants effectively distinguish between observationally equivalent models.
Study geodesic flows on hyperbolic manifolds without conjugate points, proving unique measure of maximal entropy.
problem Proving uniqueness of measure of maximal entropy for geodesic flows on specific manifolds.
method Analyzing geodesic flows on closed Riemannian manifolds without conjugate points, using properties of Gromov hyperbolic and residually finite groups.
result Proves geodesic flow has a unique measure of maximal entropy under appropriate assumptions.
Towards a Sharp Analysis of Offline Policy Learning for -Divergence-Regularized Contextual Banditscs.LG
Paper analyzes sample complexity for offline -divergence-regularized contextual bandits.
problem Lack of tight analyses for sample complexity in offline reinforcement learning.
method Novel pessimism-based analysis for reverse KL divergence, establishing sample complexity.
result Achieves sample complexity for reverse KL divergence, surpassing existing bounds.
A new approach to model rejection using density ratios.
problem Improving model performance through selective prediction.
method Optimization of a loss's risk with φ-divergence regularization to find an idealized data distribution.
result Model rejection can be made by comparing the density ratio of the idealized distribution to the actual data distribution.
New method to measure compositional generalization on realistic data.
problem Limited compositional generalization in machine learning.
method Maximizing compound divergence while ensuring small atom divergence.
result Machine learning architectures fail to generalize compositionally.
A typical goal of supervised dimension reduction is to find a low-dimensional subspace of the input space such that the projected input variables preserve maximal information about the output variables. The dependence maximization approach solves the supervised dimension reduction problem through maximizing a statistic…
CPR adds entropy maximization to improve continual learning methods.
problem Catastrophic forgetting in continual learning.
method Classifier-Projection Regularization (CPR) adds an entropy maximization term to existing regularization methods.
result CPR improves accuracy and plasticity in continual learning methods.
EM algorithm converges in KL divergence for exponential families via mirror descent.
problem Lack of understanding of EM's non-asymptotic convergence properties.
method Viewing EM as a mirror descent algorithm, showing convergence rates in KL divergence.
result KL divergence rates for EM in exponential families, invariant to parametrization.
Proposes ATM method to improve domain adaptation.
problem Mitigating distribution divergence between source and target domains.
method Adversarial Tight Match (ATM) method using Maximum Density Divergence (MDD).
result New state-of-the-art performance on domain adaptation benchmarks.
cGAN learns a distribution for causal inference without specifying P.
problem Enforcing strong ignorability in causal analyses of observational data.
method Generative adversarial network (GAN)-based model called the Counterfactual -GAN (cGAN).
result Minimizes Pearson divergence, maximizing coverage and minimizing variance of ATE estimates.
BaM improves BBVI by optimizing a score-based divergence, leading to faster convergence.
problem Slow convergence of black-box variational inference methods.
method Batch and match (BaM) approach based on a score-based divergence.
result BaM converges exponentially quickly to the target mean and covariance.
A theoretical framework for non-negative matrix factorization based on generalized dual Kullback-Leibler divergence, which includes members of the exponential family of models, is proposed. A family of algorithms is developed using this framework and its convergence proven using the Expectation-Maximization algorithm. …
Develops a Bayesian framework for portfolio choice with a new posterior distribution.
problem Estimation risk in parametric portfolio policies.
method Generalized Bayesian framework with Gibbs posterior, utility maximization, and KNEEDLE algorithm.
result Optimal scaling parameter controls the balance between prior and data.
Paper proposes a new ML framework to enhance creativity in music generation.
problem Current generative models struggle to produce music outside their training dataset.
method Develops a new ML objective to address creativity limitations.
result Proposed framework could improve generative models' creativity.
This article is devoted to the maximisation of HARA utilities of L{é}vy switching process on finite time interval via dual method. We give the description of all f-divergence minimal martingale measures in initially enlarged filtration, the expression of their Radon-Nikodym densities involving Hellinger and Kulback-Lei…
New approach categorizes objective functions for embodied agents.
problem Understanding how objectives relate to each other and discovering new objectives.
method Introducing Action Perception Divergence (APD) to categorize objective functions.
result Introduces a spectrum of objectives from narrow to general, explaining various unsupervised objectives.
Some variational principles associated with ODEs of maximal symmetry. Part 2: The general casemath.DG
The paper explores variational principles for equations of maximal symmetry, providing new insights and results.
problem Exploring variational principles for equations of maximal symmetry.
method Study of variational and divergence symmetries for linear and nonlinear equations of maximal symmetry, providing first integrals in explicit form.
result Significantly different results and more general variational symmetry algebra for linear and nonlinear equations compared to previous studies.
New algorithms improve robust estimation in contaminated Gaussian models.
problem Simultaneous estimation of location and variance matrix in contaminated Gaussian models.
method Tractable adversarial algorithms with spline discriminators for robust estimation.
result Achieve minimax optimal rates or near-optimal rates under Huber's contamination model.
This work explains why GANs are less used for NLP tasks.
problem Why adversarial approaches like GANs are not widely used for NLP tasks.
method Theoretical analysis and reductions showing that maximizing likelihood is equivalent to minimizing distinguishability for certain models.
result Maximizing likelihood is as effective as minimizing distinguishability for NLP tasks.
Optimizes information acquisition to reduce estimation risk and maximize utility.
problem Estimation risk in investor decision-making.
method Derives closed-form value functions using CARA and CRRA utility functions, employs variational methods to explore optimal acquisition.
result Acquiring information earlier is more valuable in reducing estimation risk and achieving higher utility.
New method optimizes policies without assuming known link functions between preferences and rewards.
problem Policy alignment with unknown and unrestricted link functions.
method Formulates an -divergence-constrained reward maximization problem, learning policies directly.
result Induces a semiparametric single-index binary choice model for policy alignment.
We consider model-based reinforcement learning in finite Markov De- cision Processes (MDPs), focussing on so-called optimistic strategies. In MDPs, optimism can be implemented by carrying out extended value it- erations under a constraint of consistency with the estimated model tran- sition probabilities. The UCRL2 alg…
The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…
Let be a proper geodesic Gromov hyperbolic metric space and let be a cocompact group of isometries of admitting a uniform lattice. Let be the Hausdorff dimension of the Gromov boundary . We define the critical exponent of any discrete invariant random subgroup of the locally compa…
New method improves neural spike train models by minimizing divergence directly, leading to better performance.
problem Poor performance and divergence issues in spike train models using maximum likelihood estimation.
method Directly minimize maximum mean discrepancy using spike train kernels and stochastic optimization.
result The proposed method generates well-behaved models with better control over feature trade-offs.
New probabilistic model for semi-nonnegative matrix factorization using Skellam distribution.
problem Automatic clustering of semi-nonnegative data.
method Skellam-SNMF model with EM and VBEM algorithms.
result New divergence and algorithms outperform classic SNMF.
New variational method solves submodular maximum coverage problem efficiently.
problem Submodular maximum coverage problem in various applications.
method Variational optimization using Nemhauser divergence, alternating E and M steps.
result Efficient solution to submodular maximum coverage problem.
Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defi…