arXiv research

New $\alpha$-divergence loss function improves neural density ratio estimation.

EGFs use ergodicity to simplify generative flows for easier training and imitation learning.

We consider the problem of training probabilistic conditional random fields (CRFs) in the context of a task where performance is measured using a specific loss function. While maximum likelihood is the most common approach to training CRFs, it ignores the inherent structure of the task's loss function. We describe alte…

New inequality for ternary variables improves on existing measures.

The Dirichlet mechanism protects privacy while minimizing KL divergence.

Quantum models face barren plateaus, but specific losses can be trainable.

Flow matching KL divergence bound derived for smooth distributions.

SLERP interpolation optimizes dynamic weight rebalancing in AMMs.

New research shows posterior collapse in VAEs isn't just about KL-divergence.

This work improves online regression and contextual bandits using neural networks.

This paper is concerned with the factorization form of the rank regularized loss minimization problem. To cater for the scenario in which only a coarse estimation is available for the rank of the true matrix, an $\ell_{2,0}$-norm regularized term is added to the factored loss function to reduce the rank adaptively; and…

New loss functions based on f-divergences improve language model performance.

New bounds close the score matching gap for diffusion models.

VCL adds uncertainty to contrastive learning models.

New algorithm for private non-convex optimization with optimal rates.

Efficiently addresses federated learning challenges with reduced communication and sample complexity.

DRO-NPE improves neural posterior estimation by reducing overconfidence and overfitting.

The problem of estimating an unknown discrete distribution from its samples is a fundamental tenet of statistical learning. Over the past decade, it attracted significant research effort and has been solved for a variety of divergence measures. Surprisingly, an equally important problem, estimating an unknown Markov ch…

This paper is concerned with the squared F(robenius)-norm regularized factorization form for noisy low-rank matrix recovery problems. Under a suitable assumption on the restricted condition number of the Hessian for the loss function, we derive an error bound to the true matrix for the non-strict critical points with r…

Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important. We study a simple communication-efficient learning framework that first calculates the local maximum likelihood estimates (MLE) based on the data subsets, and then combines the local MLEs t…

Personalized sleep staging achieved with single-night data using KL-divergence regularization.

New PAC-Bayes method updates priors without losing confidence information.

Generalized dual discriminator GANs improve upon traditional GANs by using two discriminators and a flexible loss function.

Paper improves PAC-Bayes bounds using a better-than-KL divergence.

Study shows how neural networks generalize with minimal training data.

Our study analyzes how neural network initialization affects privacy and utility in overparameterized models.

Beta diffusion generates bounded data using multiplicative transitions.

Mutual information bounds generalization error in variational classifiers.

In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…

New PAC-Bayes bounds use Wasserstein distances to improve generalization.

Unified algorithm for tensor decomposition supports multiple loss functions and models.

The paper improves risk certificate tightness for neural networks using PAC-Bayes bounds.

Mixability of a loss is known to characterise when constant regret bounds are achievable in games of prediction with expert advice through the use of Vovk's aggregating algorithm. We provide a new interpretation of mixability via convex analysis that highlights the role of the Kullback-Leibler divergence in its definit…

Two synthetic likelihood methods learn EBM of likelihood from simulator data for SBI.

A new method uses MCMC-assisted normalizing flows for efficient Bayesian sampling.

Study optimizes tree-based models for better alignment of predicted scores and actual probabilities.

New reweighted losses improve diffusion model training and image quality.

Optimizes deep neural network initialization variance for better performance.

The paper proves learning-curve monotonicity for maximum likelihood estimators in various parametric settings.

Paper analyzes inclusive KL inference using Wasserstein gradient flows.

TSC uses HMC and adaptive transport maps to optimize forward KL for variational inference.

A classic setting of the stochastic K-armed bandit problem is considered in this note. In this problem it has been known that KL-UCB policy achieves the asymptotically optimal regret bound and KL-UCB+ policy empirically performs better than the KL-UCB policy although the regret bound for the original form of the KL-UCB…

Study quantifies model risk in dynamic portfolio selection using KL divergence.

The t-distributed Stochastic Neighbor Embedding (t-SNE) is a powerful and popular method for visualizing high-dimensional data. It minimizes the Kullback-Leibler (KL) divergence between the original and embedded data distributions. In this work, we propose extending this method to other f-divergences. We analytically a…

Improved fast rates for decision making with forward-KL regularization in contextual bandits.

Causal KL improves on existing metrics for evaluating causal models.

New algorithm minimizes inclusive KL for VI, improving accuracy.

Sharp analysis improves RLHF sample complexity with KL-regularization.