arXiv research

The paper improves semi-supervised learning using $f$-divergences and $α$-Rényi divergences.

ERM with $f$-divergence regularization yields unique solution.

New algorithm optimizes unimodal bandits using empirical divergence.

Paper presents ERM with $f$-divergence regularization and its properties.

The paper evaluates biased methods for alpha-divergence minimization.

Neural networks estimate statistical divergences with performance guarantees.

New framework using Jensen-Shannon divergence improves domain adaptation theory.

Unified technique for sequential estimation of convex divergences.

Replacing MSE with f-divergence in diffusion models improves robustness under data contamination.

Paper introduces a new method for learning Bregman divergence from data.

The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In particular, even for those estimators whose MSE convergence rates are known, the asympt…

The empirical NTK diverges from the NTK in classification problems during overtraining.

Proposes a new divergence measure for probability distributions.

Dual optimization connects ERM-fDR to normalization function.

New divergences improve score-based methods for multi-modal distributions.

Paper proposes SinkhornDRL for distributional RL using Sinkhorn divergence and regularized Wasserstein loss.

New theory explains GAN's high quality but low diversity.

New PAC-Bayes bounds derived using Legendre transform and f-divergences.

This paper provides performance guarantees for neural estimation of statistical distances.

Paper shows robust generative learning with minimal assumptions on target distributions.

LinMED is a new linear bandit algorithm with near-optimal regret bound.

Regularizes $f$-divergences with MMD to analyze Wasserstein flows.

LLM safety alignment explained as divergence estimation.

The Dirichlet mechanism protects privacy while minimizing KL divergence.

Paper introduces f-divergence variational inference for broader application.

The t-distributed Stochastic Neighbor Embedding (t-SNE) is a powerful and popular method for visualizing high-dimensional data. It minimizes the Kullback-Leibler (KL) divergence between the original and embedded data distributions. In this work, we propose extending this method to other f-divergences. We analytically a…

We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on t…

DAIS minimizes symmetrized KL divergence between initial and target distributions.

Stabilizes policy optimization with off-policy data using divergence augmentation.

We extend a variational framework to estimate calibration errors for $L_p$ divergences.

Bregman divergences generalize measures such as the squared Euclidean distance and the KL divergence, and arise throughout many areas of machine learning. In this paper, we focus on the problem of approximating an arbitrary Bregman divergence from supervision, and we provide a well-principled approach to analyzing such…

Rank-statistic method approximates $f$-divergences without density-ratio estimation.

New method for bidirectional generative modeling using adversarial gradient estimation.

In high-dimensional data, many sparse regression methods have been proposed. However, they may not be robust against outliers. Recently, the use of density power weight has been studied for robust parameter estimation and the corresponding divergences have been discussed. One of such divergences is the $γ$-divergence a…

Jeffreys Flow improves robustness of Boltzmann generators for rare event sampling.

Proposes a method to prevent overfitting in deep DRE models.

New divergences improve estimation and GAN training performance.

New method improves imitation learning from expert observations.

We present new methods for batch anomaly detection in multivariate time series. Our methods are based on maximizing the Kullback-Leibler divergence between the data distribution within and outside an interval of the time series. An empirical analysis shows the benefits of our algorithms compared to methods that treat e…

Contrastive Divergence (CD) and Persistent Contrastive Divergence (PCD) are popular methods for training the weights of Restricted Boltzmann Machines. However, both methods use an approximate method for sampling from the model distribution. As a side effect, these approximations yield significantly different biases and…

A density ratio is defined by the ratio of two probability densities. We study the inference problem of density ratios and apply a semi-parametric density-ratio estimator to the two-sample homogeneity test. In the proposed test procedure, the f-divergence between two probability densities is estimated using a density-r…

This work develops a unified framework for RLHF with general $f$-divergence regularization.

This paper introduces $f$-DPO, a generalized approach to Direct Preference Optimization using diverse divergence constraints.

New loss functions based on f-divergences improve language model performance.

We address the problem of imitation learning with multi-modal demonstrations. Instead of attempting to learn all modes, we argue that in many tasks it is sufficient to imitate any one of them. We show that the state-of-the-art methods such as GAIL and behavior cloning, due to their choice of loss function, often incorr…

The coefficient of determination, known as $R^2$, is commonly used as a goodness-of-fit criterion for fitting linear models. $R^2$ is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case basis to deal with specific models such as the logistic model. Assume we are fittin…

Paper studies regularized KKL divergence for distributions with disjoint supports.

This work investigates the properties of Gaussian-smoothed sliced divergences for comparing distributions.