New method tackles nonconvex-nonconcave problems with local KL condition.
problem Nonconvex-nonconcave minimax problems under varying KL conditions.
method Inexact proximal gradient method for KL-structured subproblems.
result Complexity guarantees for approximate stationary points.
New method solves complex constrained optimization problems.
problem Constrained nonconvex-nonconcave minimax optimization problems.
method Inexact proximal gradient method using sequential convex programming.
result Established complexity guarantees for approximate stationary points.
Local mass perspective on Bayesian inference
problem Measuring distributional discrepancy in Bayesian inference
method Introducing Mass Index and Regularised Extended KL
result Proving inequalities for comparing local small-ball masses
Kurdyka-Lojasiewicz (KL) exponent plays an important role in estimating the convergence rate of many contemporary first-order methods. In particular, a KL exponent of 21 for a suitable potential function is related to local linear convergence. Nevertheless, KL exponent is in general extremely hard to estimate. I…
EGMU optimizes portfolios using KL divergence, ensuring positive solutions.
problem Constructing multi-factor target-exposure portfolios efficiently and accurately.
method Convex optimization framework minimizing KL divergence, with explicit solvers.
result Established feasibility and uniqueness of strictly positive solutions under convex-hull conditions.
Decentralized Bayesian learning reduces KL-divergence exponentially.
problem Efficiently learning posterior distributions in a decentralized setting.
method Decentralized Langevin dynamics in a non-convex setting.
result The algorithm converges to the target posterior distribution with exponential decrease in KL-divergence and polynomial decrease in error contributions.
Improved KL bounds and Wasserstein guarantees for diffusion flow matching under minimal conditions.
problem Theoretical convergence properties of Brownian motion based diffusion flow matching.
method Refined analysis under Kullback-Leibler and 2-Wasserstein distances.
result State-of-the-art scaling in KL convergence bounds under minimal conditions.
We quantify forgetting in post-training models, distinguishing mass and drift.
problem Understanding and preventing forgetting in post-training generative models.
method Developed theoretical results under a two-mode mixture abstraction, formalizing mass and drift forgetting.
result Forgetting can be precisely quantified based on divergence direction, geometric overlap, and training regime.
Private KL distribution estimation improved with instance-optimality.
problem Minimizing KL divergence between true and estimated distributions.
method Construct minimax optimal private estimators, then focus on instance-optimality.
result Achieved instance-optimality up to constant factors for KL estimation.
EM algorithm converges in KL divergence for exponential families via mirror descent.
problem Lack of understanding of EM's non-asymptotic convergence properties.
method Viewing EM as a mirror descent algorithm, showing convergence rates in KL divergence.
result KL divergence rates for EM in exponential families, invariant to parametrization.
Unified analysis of KL divergence using shifted composition for sampling.
problem Sampling from target distributions with KL divergence guarantees.
method Shifted composition rule applied to KL divergence, combining local error analysis and Girsanov's theorem.
result Unified KL guarantees for strongly log-concave, weakly log-concave, and log-Sobolev distributions.
New algorithm minimizes inclusive KL for VI, improving accuracy.
problem Improving variational inference accuracy with KL(p||q).
method Markovian score climbing (MSC) using stochastic gradients.
result MSC converges to local optimum of inclusive KL without bias.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
New bounds for sequential tests under power-one error levels.
problem Determining stopping times for sequential tests with power-one error levels.
method Proved two lower bounds for stopping times under specific conditions.
result Upper and lower bounds for sequential tests are shown to be tight.
Efficiently addresses federated learning challenges with reduced communication and sample complexity.
problem Heterogeneity in data volumes and distributions at different clients compromises model generalization ability.
method Introduces algorithms for communication-efficient Federated Group Distributionally Robust Optimization (FGDRO).
result Communication complexity reduced to O(1/ε4) for FGDRO-CVaR and O(1/ε3) for FGDRO-KL. In this paper, we study the Kurdyka-Łojasiewicz (KL) exponent, an important quantity for analyzing the convergence rate of first-order methods. Specifically, we develop various calculus rules to deduce the KL exponent of new (possibly nonconvex and nonsmooth) functions formed from functions with known KL exponents. In …
Estimates KL divergence with fairness considerations for sub-populations.
problem Fairly estimate KL divergence between distributions considering sub-populations.
method Proposes multi-group attribution for KL divergence estimation, derived from multi-calibration.
result Shows multi-group attribution provides better KL divergence estimates conditioned on sub-populations.
SRFE clarifies KL divergences without unifying learning frameworks.
problem Inductive biases of KL divergences and their limitations.
method Introducing SRFE, a log-moment-based functional of the likelihood ratio.
result SRFE recovers KL divergences as limits and reveals a mean-variance tradeoff.
New method proves dimension-free convergence for ULD in KL divergence.
problem Polynomial scaling of existing convergence guarantees in high dimensions.
method Refined KL local error framework, focusing on tr(H) instead of d.
result First dimension-free KL divergence bounds for discretized ULD.
New schemes improve error estimates for sampling from non-log-concave distributions.
problem Improving sampling from non-log-concave distributions with super-linear drift growth.
method Developed tamed Euler and randomized Euler schemes with error estimates.
result Near-optimal error bounds for sampling and optimization problems.
Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.
problem Solving nonconvex and nonsmooth DC composite optimization problems.
method Inexact linearized proximal algorithm (iLPA) for DC composite optimization problems.
result The iLPA achieves local R-linear convergence rate under the Kurdyka-Łöjasiewicz property.
Improved KL convergence bounds for score diffusion models without restrictive assumptions.
problem Lack of comprehensive quantitative results for diffusion models, especially in non-regular scores and estimators.
method Score diffusion models with fixed step size from Ornstein-Uhlenbeck and kinetic semigroups, providing explicit and sharp KL convergence bounds.
result Explicit and sharp convergence bounds in KL applicable to any data distribution with finite Fisher information.
Paper relaxes triangle inequality for KL divergence between Gaussian distributions.
problem KL divergence does not satisfy triangle inequality for Gaussian distributions.
method Investigates relaxed triangle inequality and finds supremum.
result Supremum of KL divergence is found and conditions for attaining it are determined.
CPR adds entropy maximization to improve continual learning methods.
problem Catastrophic forgetting in continual learning.
method Classifier-Projection Regularization (CPR) adds an entropy maximization term to existing regularization methods.
result CPR improves accuracy and plasticity in continual learning methods.
Robust VB framework handles contamination using min-max median aggregation.
problem Handling contamination and outliers in datasets.
method Partition data into subsets, formulate robust optimization problem, use min-max median KL divergence.
result Min-max median formulation improves robustness and statistical rates.
New method learns disentangled signals without prior or model constraints.
problem Learning disentangled signals from data without prior or model constraints.
method Minimizes conditional KL divergence using a sequential algorithm to learn de-mixing flow models.
result Method learns self-sufficient signals that can reconstruct missing values.
FORE evaluates occupancy ratios without requiring Bellman completeness.
problem Offline reinforcement learning occupancy ratio estimation.
method Fitted occupancy-ratio evaluation (FORE) using adjoint Bellman recursion.
result FORE achieves convergence in KL without Bellman completeness.
New method samples from non-log-concave distributions with weak dissipativity.
problem Sampling from distributions that are not log-concave and weakly dissipative.
method Taming scheme tailored to growth and decay properties of the target distribution.
result Explicit non-asymptotic guarantees for KL, TV, and Wasserstein distances.
A1GM method improves efficiency in reconstructing missing data using KL divergence.
problem Efficiently reconstructing missing data in matrices.
method Fast non-gradient-based rank-1 NMF using KL divergence.
result A1GM outperforms gradient methods in efficiency with competitive reconstruction errors.
New algorithm for private non-convex optimization with optimal rates.
problem Private optimization of non-convex functions under KL condition.
method Variance-reduced gradient descent and proximal point method.
result Achieves nearly optimal rates for excess empirical risk.
Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. However, existing understandings of the convergence rate of CR are conditioned on special types of geometrical properties of the objective function. In…
New method guarantees global convergence in variational inference.
problem Limited convergence to local optima in variational inference.
method Minimizes inclusive KL divergence using neural networks and neural tangent kernel.
result Gradient descent dynamics converge to a unique solution in function space.
New samplers minimize KL divergence for constrained and non-Euclidean geometries.
problem Efficient sampling from constrained and non-Euclidean distributions.
method Stein Variational Mirror Descent and Mirrored Stein Variational Gradient Descent.
result New samplers converge more rapidly and accurately than prior methods.
New bounds show diffusion models converge nearly linearly in data dimension.
problem Improving convergence bounds for diffusion models.
method Refined discretization of reverse SDE using stochastic localization.
result Linear convergence in data dimension with logarithmic factors.
Proposes DR algorithms for distributionally robust off-policy evaluation and learning.
problem Sensitive to environment distribution shifts in offline observational data.
method Doubly robust and distributionally robust approaches for OPE/L.
result Achieves semiparametric efficiency and fast regret rate.
New findings show many popular bandit algorithms are unstable, contradicting minimax optimality.
problem Challenges in statistical inference from bandit algorithms due to adaptive, non-i.i.d. nature.
method Analysis of stability properties of optimism-based bandit algorithms.
result Widely used minimax-optimal UCB-style algorithms are unstable.
Paper proposes f-EBM for training deep EBMs using various f-divergences.
problem Training deep EBMs with intractable partition functions.
method Introduces f-EBM framework and optimization algorithm for any f-divergence.
result f-EBM outperforms contrastive divergence and other f-divergences.
Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important. We study a simple communication-efficient learning framework that first calculates the local maximum likelihood estimates (MLE) based on the data subsets, and then combines the local MLEs t…
Study shows how neural networks generalize with minimal training data.
problem Understanding how neural networks generalize with limited data.
method Mean-field analysis of KL-regularized empirical risk minimization.
result Generalization error rate is O(1/n) for large n. This paper provides guarantees for DFM models using KL divergence.
problem Ensuring generative models match target distributions efficiently.
method Using KL divergence and Brownian motion bridge for generative models.
result Non-asymptotic guarantees for DFM models under specific conditions.
Paper develops a method to compare generative models using KL divergence.
problem Lack of principled uncertainty quantification for generative models.
method Employ Kullback-Leibler divergence to measure generative model distance.
result Effective coverage rates and higher power compared to kernel-based methods.
A Bayesian factor graph reduced to normal form consists in the interconnection of diverter units (or equal constraint units) and Single-Input/Single-Output (SISO) blocks. In this framework localized adaptation rules are explicitly derived from a constrained maximum likelihood (ML) formulation and from a minimum KL-dive…
DualVDT improves time-series forecasting with a novel dual reparametrized structure.
problem Time-series forecasting with improved performance and analytical rigor.
method Dual reparametrized variational mechanisms on VAE, latent score based generative model, reverse time stochastic differential equation, variational ancestral sampling, KL divergence reduction.
result Advanced performance in time-series forecasting with reduced KL divergence.
Paper analyzes inclusive KL inference using Wasserstein gradient flows.
problem Analyzing inclusive KL inference with mathematical tools.
method Gradient flows derived from PDE analysis.
result Unified view of existing sampling algorithms as inclusive-KL inference.
Study compares chi-squared divergence and KL-divergence posteriors for PAC-Bayesian bounds.
problem Investigates optimal posteriors for PAC-Bayesian bounds using chi-squared divergence.
method Analyzes bounds for three distance functions, derives FP equations for computation.
result Chi-squared divergence based posteriors have weaker bounds and worse test errors.
TSC uses HMC and adaptive transport maps to optimize forward KL for variational inference.
problem Variational inference underestimates uncertainty when minimizing reverse KL.
method TSC uses Hamiltonian Monte Carlo and adaptive transport maps to optimize KL(p||q).
result TSC achieves competitive performance in training variational autoencoders on large-scale data.
We study the asymptotic consistency properties of α-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the α-Rényi divergence from the true posterior. Unique to o…
We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …