arXiv research

Paper compares UCB policy to new adaptive RL methods.

Efficiently computes indices for UCB and DMED algorithms in reinforcement learning.

New algorithm optimizes unimodal bandits using empirical divergence.

A new method for averaging model predictions using minimum divergence.

LinMED is a new linear bandit algorithm with near-optimal regret bound.

The study tightens bounds on binomial probabilities and minimums using KL-divergence.

Information divergence functions play a critical role in statistics and information theory. In this paper we show that a non-parametric f-divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the sa…

In this work, a novel solution to the speaker identification problem is proposed through minimization of statistical divergences between the probability distribution (g). of feature vectors from the test utterance and the probability distributions of the feature vector corresponding to the speaker classes. This approac…

We study the $K$-armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. We introduce a tight asymptotic regret lower bound that is based on the information divergence. An algorithm that is inspired by the Deterministic…

A new robust PCA estimator combining M-estimators and minimum divergence estimators.

The paper introduces a new method for estimating optimal policies in dynamic treatment regimes using information geometry.

We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergen…

DM framework improves robustness and efficiency in latent-mixture models.

New strategies avoid forced exploration for unimodal bandits.

Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize t…

Unified framework improves neural network robustness against label noise and adversarial attacks.

We present transductive Boltzmann machines (TBMs), which firstly achieve transductive learning of the Gibbs distribution. While exact learning of the Gibbs distribution is impossible by the family of existing Boltzmann machines due to combinatorial explosion of the sample space, TBMs overcome the problem by adaptively …

Proposes NRS to find flat minima in deep neural networks.

Study finds the minimum number of finite Gaussian mixtures for best approximation.

Inflating the minimum norm interpolator improves linear regression generalization error.

The paper improves semi-supervised learning using $f$-divergences and $α$-Rényi divergences.

Private KL distribution estimation improved with instance-optimality.

New algorithms improve robust estimation in contaminated Gaussian models.

Proposes a general method to derive regret bounds for multi-armed bandit algorithms.

ERM with $f$-divergence regularization yields unique solution.

Paper proposes efficient communication scheme for statistical learning.

Paper presents ERM with $f$-divergence regularization and its properties.

The paper evaluates biased methods for alpha-divergence minimization.

We study the K-armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. The hardness of recommending Copeland winners, the arms that beat the greatest number of other arms, is characterized by deriving an asymptotic regr…

We study strictly proper scoring rules in the Reproducing Kernel Hilbert Space. We propose a general Kernel Scoring rule and associated Kernel Divergence. We consider conditions under which the Kernel Score is strictly proper. We then demonstrate that the Kernel Score includes the Maximum Mean Discrepancy as a special …

We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks. We exploit the primal-dual view of the MLE with a kinetics augmented model to obtain an estimate associated with an adversarial dua…

Neural networks estimate statistical divergences with performance guarantees.

Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function and its derivatives. Here we propose a new parameter estimation technique that does not require computing an intractable normalization factor or sampling from the equilibrium distribution of the model. T…

Derives metrics for DeFi vaults, addressing credit risk.

When maximum likelihood estimation is infeasible, one often turns to score matching, contrastive divergence, or minimum probability flow to obtain tractable parameter estimates. We provide a unifying perspective of these techniques as minimum Stein discrepancy estimators, and use this lens to design new diffusion kerne…

In this report, we present an unsupervised machine learning method for determining groups of molecular systems according to similarity in their dynamics or structures using Ward's minimum variance objective function. We first apply the minimum variance clustering to a set of simulated tripeptides using the information …

New framework using Jensen-Shannon divergence improves domain adaptation theory.

Unified technique for sequential estimation of convex divergences.

SRFE clarifies KL divergences without unifying learning frameworks.

Regularized nonlinear acceleration (RNA) estimates the minimum of a function by post-processing iterates from an algorithm such as the gradient method. It can be seen as a regularized version of Anderson acceleration, a classical acceleration scheme from numerical analysis. The new scheme provably improves the rate of …

Replacing MSE with f-divergence in diffusion models improves robustness under data contamination.

Paper introduces a new method for learning Bregman divergence from data.

The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In particular, even for those estimators whose MSE convergence rates are known, the asympt…

Machine learning factors outperform traditional portfolio optimization methods.

The empirical NTK diverges from the NTK in classification problems during overtraining.

Generative adversarial network (GAN) is a minimax game between a generator mimicking the true model and a discriminator distinguishing the samples produced by the generator from the real training samples. Given an unconstrained discriminator able to approximate any function, this game reduces to finding the generative …

Proposes a new divergence measure for probability distributions.

Optimal posterior distributions improve SVM classifiers and parameter selection.