arXiv research

Unified representation of density-power-based divergences simplifies estimation to M-estimation.

New method minimizes robust density power-based divergences for general parametric densities.

Study explores relationship between Hölder and FDPD divergences.

A new robust PCA estimator combining M-estimators and minimum divergence estimators.

E$^2$M optimizes tensor density estimation by relaxing $α$-divergence to KL-divergence.

New Stein operator improves robustness in model inference.

For distributions $\mathbb{P}$ and $\mathbb{Q}$ with different supports or undefined densities, the divergence $\textrm{D}(\mathbb{P}||\mathbb{Q})$ may not exist. We define a Spread Divergence $\tilde{\textrm{D}}(\mathbb{P}||\mathbb{Q})$ on modified $\mathbb{P}$ and $\mathbb{Q}$ and describe sufficient conditions for t…

We extend CS divergence to conditional distributions and show its advantages in time series data and sequential decision making.

The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…

Improves Bridge estimators using f-GAN to minimize RMSE.

Improved VAE for heavy-tailed data using Student's t-distributions.

New method uses SoS densities and α-divergences for efficient sequential transport maps.

Variational Autoencoder (VAE), a simple and effective deep generative model, has led to a number of impressive empirical successes and spawned many advanced variants and theoretical investigations. However, recent studies demonstrate that, when equipped with expressive generative distributions (aka. decoders), VAE suff…

A density ratio is defined by the ratio of two probability densities. We study the inference problem of density ratios and apply a semi-parametric density-ratio estimator to the two-sample homogeneity test. In the proposed test procedure, the f-divergence between two probability densities is estimated using a density-r…

New $\alpha$-divergence loss function improves neural density ratio estimation.

Unified view of KL-divergence and IPMs via DRE, with new DRM metrics.

Study improves density estimation for compact domains using $h$-lifted KL divergence.

Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity…

New robust learning framework for regression NNs using β-divergences.

We propose a direct estimation method for Rényi and f-divergence measures based on a new graph theoretical interpretation. Suppose that we are given two sample sets $X$ and $Y$, respectively with $N$ and $M$ samples, where $η:=M/N$ is a constant value. Considering the $k$-nearest neighbor ($k$-NN) graph of $Y$ in the j…

In high-dimensional data, many sparse regression methods have been proposed. However, they may not be robust against outliers. Recently, the use of density power weight has been studied for robust parameter estimation and the corresponding divergences have been discussed. One of such divergences is the $γ$-divergence a…

Rank-statistic method approximates $f$-divergences without density-ratio estimation.

Adapts RKHS methods to estimate density ratios with optimal error.

A new metric evaluates generative models by comparing real and generated samples.

Reflective Hamiltonian Monte Carlo struggles with high-dimensional sampling.

New divergences improve score-based methods for multi-modal distributions.

Unified framework for robust, stable, and efficient density ratio estimation.

Paper analyzes robustness of MDPDE under INH setups.

Bi-Lipschitz flows approximate a wide range of distributions.

Improved GANs estimate convergence rate for density estimation.

Locally private mechanisms' output divergence bounds derived.

A new approach to model rejection using density ratios.

Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.

New analysis of annealing paths in sampling and estimation.

Truncated densities are probability density functions defined on truncated domains. They share the same parametric form with their non-truncated counterparts up to a normalizing constant. Since the computation of their normalizing constants is usually infeasible, Maximum Likelihood Estimation cannot be easily applied t…

Moser Flow generates models for complex geometries on manifolds without ODE solvers.

Method uses normalizing flows to efficiently sample from complex target densities.

Generative adversarial networks (GANs) are successful deep generative models. GANs are based on a two-player minimax game. However, the objective function derived in the original motivation is changed to obtain stronger gradients when learning the generator. We propose a novel algorithm that repeats the density ratio e…

Proposes ATM method to improve domain adaptation.

Estimation of density derivatives is a versatile tool in statistical data analysis. A naive approach is to first estimate the density and then compute its derivative. However, such a two-step approach does not work well because a good density estimator does not necessarily mean a good density-derivative estimator. In t…

Machine learning classification limits estimated using Kullback-Leibler divergence and Cohen's Kappa.

This paper improves active learning by using robust divergences for committee disagreement.

Proposes a new method for kernel density estimation using stagewise minimization and a simple dictionary.

Proximal Diffusion Models improve generative model efficiency.

TRE improves density-ratio estimation for highly dissimilar densities.

Study compares statistical properties and power of divergence measures for credit risk monitoring.

Improved Bayesian inference using power priors with historical data.

New method trains neural samplers to sample from multi-modal distributions efficiently.