This paper introduces f-DPO, a generalized approach to Direct Preference Optimization using diverse divergence constraints.
problem Aligning large language models with human preferences while mitigating safety risks.
method Incorporates diverse divergence constraints to simplify the relationship between reward and optimal policy, eliminating the need for estimating the normalizing constant.
result Optimizes LLMs to align with human preferences more efficiently and under a broader set of divergence constraints.
Paper explores how text generation quality and diversity metrics relate to distribution fitting.
problem Unclear relation between text generation quality and diversity metrics and distribution fitting.
method Theoretical approach to prove a linear combination of quality and diversity metrics can be a divergence metric.
result CR/NRR proposed as a better substitute for BLEU/Self-BLEU metrics.
New theory explains GAN's high quality but low diversity.
problem Lack of theoretical justification for non-saturating GAN training.
method Showed non-saturating GAN training approximately minimizes a specific f-divergence.
result Non-saturating GAN training minimizes a particular f-divergence.
The paper introduces MU for NMF with β-divergences and disjoint constraints.
problem Nonnegative matrix factorization with constraints.
method Design multiplicative updates for NMF based on β-divergences with disjoint constraints. result Multiplicative updates satisfy constraints and decrease the objective function.
We present a framework to understand GAN training as alternating density ratio estimation and approximate divergence minimization. This provides an interpretation for the mismatched GAN generator and discriminator objectives often used in practice, and explains the problem of poor sample diversity. We also derive a fam…
The paper introduces a new divergence for portfolio management to outperform a benchmark.
problem Maximizing expected utility of outperformance over a benchmark with constraints.
method Uses α-Bregman-Wasserstein divergence to penalize underperformance more than overperformance. result Proves existence and uniqueness of optimal portfolio strategy and conditions for constraints binding.
Optimal payoff choice constrained by Bregman-Wasserstein divergence.
problem Maximizing utility under a deviation constraint from a benchmark.
method Solving the problem using Bregman-Wasserstein divergence with a convex function φ.
result Provided the optimal payoff choice in this setting.
Paper proposes f-DPG for aligning language models with preferences.
problem Aligning language models with user preferences.
method Uses f-divergence to approximate target distributions and minimizes a forward KL from it using DPG.
result Jensen-Shannon divergence often outperforms forward KL divergence, leading to significant improvements.
Quantum channels' contraction under privacy constraints studied.
problem Understanding the privacy constraints on quantum channel contractions.
method Established upper bounds on contraction coefficients for specific divergences under QLDP constraints.
result Upper bounds and full characterization of contraction coefficients for specific quantum distances.
Unified framework for unlearning in diffusion models using KL divergence and likelihood constraints.
problem Removing undesirable data or concepts while preserving utility of pretrained models.
method Constrained optimization framework based on reverse and forward KL divergences, and likelihood constraints.
result Our KL-constrained approach achieves superior retention-unlearning tradeoffs compared to weight-based baselines.
New optimal transport divergences derived from scoring functions.
problem Developing new divergences for optimal transport.
method Using scoring functions as cost functions in optimal transport.
result Comonotonic coupling is optimal for many new divergences.
While Generative Adversarial Networks (GANs) have empirically produced impressive results on learning complex real-world distributions, recent works have shown that they suffer from lack of diversity or mode collapse. The theoretical work of Arora et al. suggests a dilemma about GANs' statistical properties: powerful d…
Master-slave architecture tackles combinatorial multi-armed bandits with diversity constraints.
problem Solving top-K combinatorial multi-armed bandits with non-linear feedback and diversity constraints. method Master-slave architecture with six slave models, teacher learning, and policy co-training.
result Significantly outperforms existing algorithms in synthetic and real datasets.
Proposes practical kernel tests for f-divergences with theoretical guarantees.
problem Two-sample testing and machine unlearning evaluation.
method Regularized f-divergence kernel tests, adaptive to hyperparameters. result Different f-divergences highlight localized differences. In standard reinforcement learning (RL), a learning agent seeks to optimize the overall reward. However, many key aspects of a desired behavior are more naturally expressed as constraints. For instance, the designer may want to limit the use of unsafe actions, increase the diversity of trajectories to enable exploratio…
We know SGAN may have a risk of gradient vanishing. A significant improvement is WGAN, with the help of 1-Lipschitz constraint on discriminator to prevent from gradient vanishing. Is there any GAN having no gradient vanishing and no 1-Lipschitz constraint on discriminator? We do find one, called GAN-QP. To construct a …
New RL algorithm learns good actions from offline data, reducing uncertainty and divergence.
problem Limited applicability of current RL algorithms in real-world settings due to high costs of exploration.
method Proposes an algorithm for batch RL using a fixed offline dataset, with penalties for policy and value constraints.
result Compared favorably to state-of-the-art methods on 32 continuous-action benchmarks.
Two hitherto disconnected threads of research, diverse exploration (DE) and maximum entropy RL have addressed a wide range of problems facing reinforcement learning algorithms via ostensibly distinct mechanisms. In this work, we identify a connection between these two approaches. First, a discriminator-based diversity …
Generative models often misrepresent class frequencies; this paper calibrates them.
problem Miscalibration of class frequencies in generative models.
method Formulated as constrained optimization, using surrogate objectives to approximate constraints.
result Significant reduction in calibration error across various models and applications.
Develops deep NMF models using β-divergences for feature extraction.
problem Inadequate evaluation metrics for deep NMF on diverse datasets.
method Introduces new deep NMF models using Kullback-Leibler divergence.
result Improves feature extraction quality across different types of data.
New tools quantify deep generative models' performance.
problem Measuring the quality-diversity trade-off in deep generative models.
method Established non-asymptotic bounds on sample complexity and introduced frontier integrals.
result Smoothed estimators improve convergence rates of divergence frontiers.
CPRA efficiently finds diverse solutions in CO problems using UL and parallelization.
problem Finding optimal solutions often requires diverse outcomes in real-world applications.
method CPRA, an UL-based framework, discovers shared representations to generate diverse solutions.
result CPRA outperforms existing UL-based solvers in generating diverse solutions.
f-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler divergence, chi-squared divergence, squared Hellinger distance, total variation distance e…
Study introduces a variational approach for efficient KL divergence estimation in Dirichlet mixture models.
problem Efficient estimation of KL divergence in Dirichlet mixture models.
method Variational approach for a closed-form solution.
result Superior efficiency and accuracy compared to Monte Carlo methods.
Pantypes improve prototypical models by capturing diverse input distributions.
problem Prototypical models lack sufficient data representation in low density regions.
method Introducing pantypes, a sparse set of diverse objects to represent the full diversity of input distribution.
result Pantypes empower prototypical models to foster high diversity, interpretability, and fairness.
Method estimates posterior model for boundary value problems with uncertain constraints.
problem Estimating posterior probability model for stochastic boundary value problems with uncertain constraints.
method Probabilistic learning inference using Kullback-Leibler divergence and MCMC.
result Method successfully estimates posterior probability measure with constraints.
New method improves text classification without labeled target data.
problem Improving text classification under domain shift without labeled target data.
method Diversity-based generalization using multi-head attention with diversity constraints.
result Method matches state-of-the-art performance without labeled target data.
Method prevents model divergence in rapidly changing ad markets.
problem Model divergence due to rapid ad turnover and discontinuity.
method Dual ascent optimization with latent vector constraints.
result Significant reduction in diverging instances and improved user experience/revenue.
Learning word representations has garnered greater attention in the recent past due to its diverse text applications. Word embeddings encapsulate the syntactic and semantic regularities of sentences. Modelling word embedding as multi-sense gaussian mixture distributions, will additionally capture uncertainty and polyse…
A study on α-GANs proving convergence and estimation guarantees.
problem Analyzing the convergence and estimation guarantees of α-GANs. method Proved a correspondence between α-GANs and f-divergences, and provided estimation bounds. result Estimation bounds indicate diverse GAN behavior as a function of α. EGAB algorithms improve online portfolio selection.
problem Online portfolio selection problem.
method Generalized exponentiated gradient (EG) updates with Alpha-Beta divergence regularization.
result EGAB algorithms enhance portfolio performance, especially with transaction costs.
In the existing financial literature, entropy based ideas have been proposed in portfolio optimization, in model calibration for options pricing as well as in ascertaining a pricing measure in incomplete markets. The abstracted problem corresponds to finding a probability measure that minimizes the relative entropy (al…
We consider the problem of metric learning subject to a set of constraints on relative-distance comparisons between the data items. Such constraints are meant to reflect side-information that is not expressed directly in the feature vectors of the data items. The relative-distance constraints used in this work are part…
New method improves neural architecture search by optimizing for both performance and diversity.
problem Traditional multi-objective NAS fails to address practical constraints and niches.
method Formulated as quality diversity optimization, introduces multifidelity optimizers.
result Quality diversity NAS outperforms multi-objective NAS in quality and efficiency.
The paper improves QD policy ensembles using distribution ratio estimators.
problem Training diverse and high-quality reinforcement learning agents.
method Using Stein variational gradient descent and distribution ratio estimators.
result The method generates diverse and high-quality reinforcement learning agents.
Sampling methods that choose a subset of the data proportional to its diversity in the feature space are popular for data summarization. However, recent studies have noted the occurrence of bias (under- or over-representation of a certain gender or race) in such data summarization methods. In this paper we initiate a s…
LDP is equivalent to contraction of E_γ-divergence, impacting privacy and utility.
problem Analyzing trade-offs between privacy and utility in estimation problems.
method Equivalence of LDP constraints to contraction coefficients of E_γ-divergence, using f-divergences and estimation-theoretic tools.
result LDP guarantees can be expressed in terms of contraction coefficients of arbitrary f-divergences.
Study scaling of optimal solutions for reliability constraints in resource provisioning.
problem Achieving high reliability in resource provisioning under stringent requirements.
method Chance-constrained optimization, distributionally robust optimization, f-divergence balls, line search.
result Correct scaling properties of optimal decisions are preserved by using appropriate f-divergence balls, leading to conservative yet near-optimal solutions.
KL-constrained API shows optimization issues and improved with regularization.
problem Optimization issues in KL-constrained API algorithms.
method Comparison of KL divergence as a constraint vs. regularizer, empirical evaluation.
result KL-constrained API is not guaranteed to converge and incurs linear regret.
Study statistical guarantees for DRO with OT and OT-regularized divergences.
problem Enhancing adversarial robustness in machine learning models.
method Derive concentration inequalities for supervised learning via DRO-based adversarial training.
result First to cover soft-constraint costs and reweighting mechanisms in adversarial training.
CAKD framework optimizes knowledge transfer by focusing on influential components of distillation.
problem Balancing and optimizing knowledge transfer in distillation models.
method Decouple KL divergence into BCD, SCD, and WCD; prioritize influential components.
result CAKD framework consistently outperforms baseline across diverse models and datasets.
This work optimizes model performance while ensuring fairness through AUC constraints.
problem Ensuring fairness in machine learning models, especially for protected populations.
method Formulates fairness-aware machine learning model training as AUC optimization subject to fairness constraints, solves using stochastic first-order methods.
result Demonstrates effectiveness of the approach on real-world data under different fairness metrics.
Active learning selects high-quality examples for text-to-SQL systems.
problem Efficiently annotate large language models for text-to-SQL systems.
method Formalizes example selection as a constrained experimental design problem over semantic query embeddings, proposing a stratified greedy algorithm that maximizes heteroscedastic mutual information.
result Proposed method significantly reduces labeling effort while maintaining high text-to-SQL retrieval accuracy.
New algorithm speeds up NMF with β-divergence.
problem Efficiently factorize nonnegative matrices with β-divergence. method Joint majorization-minimization with multiplicative updates.
result Significant reduction in computation time for NMF.
Mixreg improves RL generalization by mixing diverse training environments.
problem Overfitting in RL agents trained in limited environments.
method Trains on a mixture of diverse observations and imposes linearity constraints.
result Mixreg outperforms baselines on unseen testing environments.
Paper resolves open problems on sample complexity in binary hypothesis testing.
problem Open problems in distributed simple binary hypothesis testing under information constraints.
method One-shot lower bound on Bayes error, streamlined sample complexity formula, reverse data-processing inequality.
result Optimally tight sample complexity bounds for communication-constrained simple binary hypothesis testing.
DASH improves ensemble generalizability by encouraging diverse, flat loss landscapes.
problem Improving generalization and robustness of deep ensembles.
method DASH promotes diversity and flatness in deep ensembles by encouraging base learners to move towards low-loss regions of minimal sharpness.
result DASH improves ensemble generalizability, as demonstrated by extensive empirical evidence.
This paper provides a non-robust interpretation of the distributionally robust optimization (DRO) problem by relating the distributional uncertainties to the chance probabilities. Our analysis allows a decision-maker to interpret the size of the ambiguity set, which is often lack of business meaning, through the chance…