The Bregman divergence (Bregman distance, Bregman measure of distance) is a certain useful substitute for a distance, obtained from a well-chosen function (the "Bregman function"). Bregman functions and divergences have been extensively investigated during the last decades and have found applications in optimization, o…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Bregman information”
Exponential families and mixture families are parametric probability models that can be geometrically studied as smooth statistical manifolds with respect to any statistical divergence like the Kullback-Leibler (KL) divergence or the Hellinger divergence. When equipping a statistical manifold with the KL divergence, th…
We investigate connections between information-theoretic and estimation-theoretic quantities in vector Poisson channel models. In particular, we generalize the gradient of mutual information with respect to key system parameters from the scalar to the vector Poisson channel model. We also propose, as another contributi…
It is well-known that there are a number of relations between theoretical finance theory and information theory. Some of these relations are exact and some are approximate. In this paper we will explore some of these relations and determine under which conditions the relations are exact. It turns out that portfolio the…
New analysis of annealing paths in sampling and estimation.
problem Sampling from complex distributions and estimating normalization constants.
method Extending known results on Bregman divergence to quasi-arithmetic means under monotonic embedding.
result Analogous result for quasi-arithmetic means, highlighting the interplay between means, parametric families, and divergence functionals.
Geometric analysis improves convergence of variational inference.
problem Challenges in analyzing convergence of variational inference due to non-convexity and non-smoothness.
method Exploits exponential family structure and Bregman divergences to geometrically analyze the optimization landscape.
result Establishes non-asymptotic convergence rates for gradient descent algorithms.
New geometry for optimal transport cost based on Bregman divergences.
problem Optimal transport cost calculation with Bregman divergences.
method Established properties, defined interpolations, constructed dualistic geometry.
result Derived generalized Pythagorean inequality and Bregman-Wasserstein barycenters.
This work proposes the Bregman-Tweedie classification model and analyzes the domain structure of the extended exponential function, an extension of the classic generalized exponential function with additional scaling parameter, and related high-level mathematical structures, such as the Bregman-Tweedie loss function an…
This work tackles community detection in networks with node attributes, achieving exact recovery.
problem Community detection in networks with correlated node attributes.
method Information-theoretic criterion and iterative clustering algorithm maximizing joint likelihood.
result Exact recovery of community labels under a general model for network and node attributes.
Bregman divergences play a central role in the design and analysis of a range of machine learning algorithms. This paper explores the use of Bregman divergences to establish reductions between such algorithms and their analyses. We present a new scaled isodistortion theorem involving Bregman divergences (scaled Bregman…
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
problem Sampling from distributions with nonsmooth convex composite potentials.
method Leveraging Bregman--Moreau envelopes and proximal operators in mirror descent.
result Efficiency in sampling from nonsmooth distributions, extending existing methods.
New algorithm speeds up learning of graphical models.
problem Learning graphical models with sparse structure efficiently.
method Vertex-greedy score-based algorithm for learning DAGs.
result Polynomial runtime for learning DAG models.
In this work, we extend some quantities introduced in "Optimization of conditional value-at-risk" of R.T Rockafellar and S. Uryasev to the case where the proximity between real numbers is measured by using a Bregman divergence. This leads to the definition of the Bregman superquantile. Axioms of a coherent measure of r…
Distances are fundamental primitives whose choice significantly impacts the performances of algorithms in machine learning and signal processing. However selecting the most appropriate distance for a given task is an endeavor. Instead of testing one by one the entries of an ever-expanding dictionary of {\em ad hoc} dis…
Paper introduces a new method for learning Bregman divergence from data.
problem Suboptimal performance of classic distance metrics in deep metric learning.
method Learning empirical Bregman divergence from data using deep learning.
result Empirical Bregman divergence method outperforms other methods on public datasets.
This manuscript develops the theory of agglomerative clustering with Bregman divergences. Geometric smoothing techniques are developed to deal with degenerate clusters. To allow for cluster models based on exponential families with overcomplete representations, Bregman divergences are developed for nondifferentiable co…
New findings show Bregman proximal algorithms can get stuck near non-stationary points.
problem Bregman proximal algorithms can get stuck near non-stationary points, misleadingly suggesting convergence.
method Analysis of Bregman proximal algorithms and their behavior near non-stationary points.
result Bregman proximal algorithms can get stuck near spurious stationary points, even in convex problems.
New algorithm solves maximal monotone inclusion problems.
problem Solving maximal monotone inclusion problems.
method Bregman Douglas-Rachford splitting method and variants.
result Convergence of algorithms under certain assumptions.
In this paper, we introduce new classes of divergences by extending the definitions of the Bregman divergence and the skew Jensen divergence. These new divergence classes (g-Bregman divergence and skew g-Jensen divergence) satisfy some properties similar to the Bregman or skew Jensen divergence. We show these g-diverge…
New method improves structure learning on sparse graphs.
problem Structure learning on sparse directed acyclic graphs (DAGs).
method Bregman proximal gradient method to address non-convex, high-curvature problem.
result Significantly improved convergence and efficiency.
Separable Bregman divergences induce Riemannian metric spaces that are isometric to the Euclidean space after monotone embeddings. We investigate fixed rate quantization and its codebook Voronoi diagrams, and report on experimental performances of partition-based, hierarchical, and soft clustering algorithms with respe…
We study the logarithmic -divergence which extrapolates the Bregman divergence and corresponds to solutions to novel optimal transport problems. We show that this logarithmic divergence is equivalent to a conformal transformation of the Bregman divergence, and, via an explicit affine immersion, is equivalent t…
We extend the recently introduced theory of Lovasz-Bregman (LB) divergences (Iyer & Bilmes 2012) in several ways. We show that they represent a distortion between a "score" and an "ordering", thus providing a new view of rank aggregation and order based clustering with interesting connections to web ranking. We show ho…
We extend the recently introduced theory of Lovasz-Bregman (LB) divergences (Iyer & Bilmes, 2012) in several ways. We show that they represent a distortion between a 'score' and an 'ordering', thus providing a new view of rank aggregation and order based clustering with interesting connections to web ranking. We show h…
Information geometry of perturbed gradient flow systems on hypergraphs: A perspective towards nonequilibrium physicscond-mat.stat-mech
The article reviews how gradient flow systems on hypergraphs connect to information geometry and nonequilibrium physics.
problem Understanding the geometry of perturbed gradient flow systems on hypergraphs.
method Formulating modern nonequilibrium principles within the framework of perturbed gradient flow systems on hypergraphs.
result New concepts like moduli spaces and thermodynamical area are introduced to understand speed limits.
Proposes BMME for optimizing nonsmooth nonconvex problems with block structure.
problem Optimizing nonsmooth nonconvex problems with block structure.
method Block Alternating Bregman Majorization Minimization with Extrapolation (BMME).
result Subsequential convergence to a first-order stationary point under mild assumptions, global convergence under stronger conditions.
We tackle the issue of classifier combinations when observations have multiple views. Our method jointly learns view-specific weighted majority vote classifiers (i.e. for each view) over a set of base voters, and a second weighted majority vote classifier over the set of these view-specific weighted majority vote class…
Generalizes bias-variance decomposition for Bregman divergences.
problem No specific problem stated; generalization of bias-variance for Bregman divergences.
method Provided a generalization of the bias-variance decomposition for Bregman divergences.
result A clear, standalone derivation of the bias-variance decomposition for Bregman divergences.
New insights into Markov chain geometry via positive transition measures.
problem Lack of statistical meaning in the space of transition probabilities.
method Constructing an extension of the space of transition probabilities using Amari's theory of positive measures.
result Introduction of a new dually flat structure for the space of positive transition measures.
Bregman perspective on CART provides a unified framework for impurity measures.
problem Unifying impurity measures in CART
method Bregman divergence approach
result Unified framework for impurity measures
The mirror descent algorithm (MDA) generalizes gradient descent by using a Bregman divergence to replace squared Euclidean distance. In this paper, we similarly generalize the alternating direction method of multipliers (ADMM) to Bregman ADMM (BADMM), which allows the choice of different Bregman divergences to exploit …
Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the f-divergences. In this paper, we show that the symmetric Bregman divergence can be …
Optimal payoff choice constrained by Bregman-Wasserstein divergence.
problem Maximizing utility under a deviation constraint from a benchmark.
method Solving the problem using Bregman-Wasserstein divergence with a convex function φ.
result Provided the optimal payoff choice in this setting.
Develops clustering methods based on likelihood and convergence proved.
problem Hard clustering based on likelihood.
method k-MLE, k-Bregman, k-VARs approaches.
result Convergence proved for clustering methods.
Unified deep metric learning approach using neural networks.
problem Learning embeddings of data and extending Euclidean distances.
method Deep Bregman divergences based on neural networks.
result Superior performance on benchmark datasets compared to existing methods.
Bregman divergences generalize measures such as the squared Euclidean distance and the KL divergence, and arise throughout many areas of machine learning. In this paper, we focus on the problem of approximating an arbitrary Bregman divergence from supervision, and we provide a well-principled approach to analyzing such…
The paper analyzes the bias-variance tradeoff for Bregman divergences.
problem Understanding the bias-variance tradeoff for Bregman divergences.
method Analyzes the bias-variance tradeoff through operations in dual space.
result Derives several results including a generalized law of total variance and ensembling operations.
New algorithm solves saddle point problems in Banach spaces.
problem Solving saddle point problems in real reflexive Banach spaces.
method Stochastic Bregman Primal-Dual Splitting Algorithm with relative smoothness and strong convexity assumptions.
result Almost sure convergence to saddle points under various conditions.
Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applicationscs.LG
A new test statistic measures discrepancy between conditional distributions.
problem Measuring the discrepancy between two conditional distributions.
method Proposes a Bregman matrix divergence-based statistic that avoids explicit distribution estimation.
result The new statistic inherits high-order statistics and demonstrates utility in multi-task learning, concept drift detection, and feature selection.
The family of f-divergences is ubiquitously applied to generative modeling in order to adapt the distribution of the model to that of the data. Well-definedness of f-divergences, however, requires the distributions of the data and model to overlap completely in every time step of training. As a result, as soon as the s…
This work broadens calibeating to various proper losses using Bregman divergence.
problem Calibration for a wide range of proper losses.
method Regret minimization and Bregman divergence approach.
result U-calibration results for a family of Tsallis losses with logarithmic regret and dimension independence.
This work generalizes calibeating for a broader range of proper losses using Bregman divergence.
problem Calibration for a wide range of proper losses beyond Brier and log loss.
method Regret minimization based on Bregman divergence for a family of proper losses.
result U-calibration results for a family of Tsallis losses with logarithmic regret and dimension independence.
A new method for distributed PCA using matrix β-mean.
problem Efficiently aggregating PCA results across multiple machines with reduced computational overhead.
method Proposes a novel DPCA method that incorporates eigenvalue information using the matrix β-mean.
result The matrix β-mean method improves robustness and stability of eigenvector ordering.
The paper improves support recovery in high-dimensional precision matrix estimation using meta learning.
problem Support recovery in high-dimensional precision matrix estimation with reduced sample complexity.
method Pooling samples from different tasks and using an improper -regularized log-determinant Bregman divergence to estimate a single precision matrix.
result The support of the improperly estimated single precision matrix is equal to the true support union with high probability.
The paper introduces a new divergence for portfolio management to outperform a benchmark.
problem Maximizing expected utility of outperformance over a benchmark with constraints.
method Uses -Bregman-Wasserstein divergence to penalize underperformance more than overperformance.
result Proves existence and uniqueness of optimal portfolio strategy and conditions for constraints binding.
Paper develops robust Bayesian models for linear regression under adversarial perturbations.
problem Ensuring reliable machine learning models under data perturbations.
method Formulates adversarial Bregman divergence loss, computes adversarial perturbation, introduces adversarially robust posteriors, derives generalization certificates.
result Derives first rigorous generalization certificates for adversarially robust Bayesian linear regression.
A new optimisation framework for neural networks without hyperparameters.
problem Lack of explicit architectural information in optimisation frameworks.
method Extends mirror descent to account for neural architecture, transforming a Bregman divergence.
result Automatic gradient descent: a first-order optimiser without hyperparameters.
Analysis of Generalized Bregman Surrogate Algorithms for Nonsmooth Nonconvex Statistical Learningmath.OC
Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.
problem Nonsmooth and nonconvex objectives in statistical learning.
method Bregman-surrogate algorithm framework, including local linear approximation, mirror descent, iterative thresholding, DC programming.
result Global convergence rates for nonconvex and nonsmooth objectives in high dimensions.