Adapts RKHS methods to estimate density ratios with optimal error.
problem Estimating density ratios from limited data.
method Minimizes regularized Bregman divergence in RKHS, with Lepskii type parameter choice.
result Adaptive minimax optimal error rate for quadratic loss.
Optimizes kernel density ratios for better predictions and information measures.
problem Improving accuracy of kernel density estimates for density ratios.
method Derives an optimal weight function using calculus of variations.
result Reduces bias in kernel density estimates, leading to improved prediction posteriors and information-theoretic measures.
Paper tackles unbounded density ratio estimation for covariate shift adaptation.
problem Understudied challenge in statistical learning: unbounded density ratios.
method Three-step estimation method: relative density ratio, truncation, and transformation.
result Established rigorous convergence guarantees for density ratio and regression estimators.
Stable and consistent model alignment for language models without assuming human preference models.
problem Lack of statistical consistency in existing alignment methods.
method Relative density ratio optimization between preferred and mixture of preferred and non-preferred data distributions.
result Our approach achieves statistical consistency and stability, providing tighter convergence guarantees.
BO method improved by density-ratio estimation for better efficiency and scalability.
problem Limitations in Bayesian optimization due to analytical tractability of predictive models.
method Reformulated Bayesian optimization by casting expected improvement as a binary classification problem.
result Improved efficiency and scalability of Bayesian optimization.
A density ratio is defined by the ratio of two probability densities. We study the inference problem of density ratios and apply a semi-parametric density-ratio estimator to the two-sample homogeneity test. In the proposed test procedure, the f-divergence between two probability densities is estimated using a density-r…
In this work, we propose new objective functions to train deep neural network based density ratio estimators and apply it to a change point detection problem. Existing methods use linear combinations of kernels to approximate the density ratio function by solving a convex constrained minimization problem. Approximating…
Bayesian optimization improves efficiency with semi-supervised learning.
problem Efficiently find global optima of expensive functions.
method Density ratio estimation combined with semi-supervised learning.
result Improved accuracy in identifying global optima with unlabeled data.
Paper develops estimators for unbounded density ratios with applications in error control.
problem Estimating density ratios with unbounded domains and ranges.
method Least squares and logistic regression loss functions for density ratio estimation.
result Established upper bounds on estimation errors with optimal rates for unbounded density ratios.
Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.
problem Statistical inconsistency in aligning LLMs with human preferences.
method Direct Density Ratio Optimization (DDRO) estimates density ratio directly.
result DDRO is statistically consistent, converging to true human preferences as data grows.
We develop a new loss function for estimating quasiprobabilistic density ratios.
problem Discontinuous or non-surjective relationships between optimal classifiers and target densities.
method Introduce a convex loss function compatible with both probabilistic and quasiprobabilistic densities.
result Achieve state-of-the-art results in estimating di-Higgs production in particle physics.
The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…
New algorithm uses density ratios for efficient online reinforcement learning.
problem Challenges in collecting exploratory data for online reinforcement learning.
method Density ratio modeling for online exploration, combining truncation and optimism.
result Sample-efficient online exploration achieved with GLOW and HyGLOW.
Unified framework for robust, stable, and efficient density ratio estimation.
problem Density-chasm and support-chasm problems in density ratio estimation.
method Dequantified diffusion-Schrödinger bridge (D3RE) framework with DDBI and DSBI.
result Offers uniform approximation and bounded time scores in theory and empirical performance.
We propose a \textbf{uni}fied \textbf{f}ramework for \textbf{i}mplicit \textbf{ge}nerative \textbf{m}odeling (UnifiGem) with theoretical guarantees by integrating approaches from optimal transport, numerical ODE, density-ratio (density-difference) estimation and deep neural networks. First, the problem of implicit gene…
Various problems in Engineering and Statistics require the computation of the likelihood ratio function of two probability densities. In classical approaches the two densities are assumed known or to belong to some known parametric family. In a data-driven version we replace this requirement with the availability of da…
New α-divergence loss function improves neural density ratio estimation.
problem Optimization challenges in existing DRE methods, especially overfitting and high sample requirements.
method Derived α-divergence loss function (α-Div) for neural density ratio estimation. result The α-divergence loss function (α-Div) offers stable and effective optimization for DRE. Estimates density ratio for two-sample comparison using tree models.
problem Comparing two distributions given i.i.d. observations.
method Additive tree models with balancing loss for density ratio estimation.
result Bayesian inference provides uncertainty quantification for density ratio.
The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), and feature selection (mutual information). Recently, several methods of directly estimating the density ratio have been deve…
Develops a new density ratio estimator for causal inference.
problem Estimation of density ratio functions in statistics.
method Super learning approach with a novel loss function.
result Empirical validation of the density ratio super learner's performance.
Meta-learning improves relative density-ratio estimation from limited data.
problem Estimating relative density-ratios from few instances.
method Meta-learning using neural networks to extract and embed dataset information for relative DRE.
result Meta-learning enables efficient and effective adaptation to few instances for relative DRE.
TRE improves density-ratio estimation for highly dissimilar densities.
problem Density-ratio estimation fails for significantly different densities.
method Telescoping density-ratio estimation (TRE) framework.
result TRE yields substantial improvements over existing methods for mutual information estimation.
New method resolves density ratio estimation saturation issues.
problem Error saturation in density ratio estimation methods.
method Iterated regularization to improve kernel methods.
result Achieves fast error rates on regular learning problems.
Filtering out unrealistic images from trained generative adversarial networks (GANs) has attracted considerable attention recently. Two density ratio based subsampling methods---Discriminator Rejection Sampling (DRS) and Metropolis-Hastings GAN (MH-GAN)---were recently proposed, and their effectiveness in improving GAN…
Study shows robust method for estimating density ratios even with heavy contamination.
problem Estimating density ratios in the presence of heavy contamination.
method Weighted density ratio estimation (DRE) with doubly strong robustness.
result Weighted DRE achieves sparse consistency under heavy contamination.
Unified framework for OOD detection using class ratio estimation.
problem Density-based OOD detection is unreliable for OOD images.
method Unified framework that builds energy-based models and employs differing base distributions, directly estimating the density ratio through class ratio estimation.
result Competitive results on OOD image problems compared to recent work.
Featurization improves density ratio estimation for complex data.
problem Difficulty in estimating density ratios for high-dimensional, different distributions.
method Invertible generative model to map distributions into a common feature space.
result Improved accuracy in density ratio estimation through feature space.
A new approach to model rejection using density ratios.
problem Improving model performance through selective prediction.
method Optimization of a loss's risk with φ-divergence regularization to find an idealized data distribution.
result Model rejection can be made by comparing the density ratio of the idealized distribution to the actual data distribution.
MBORE optimizes multi-objective problems using density-ratio estimation.
problem Optimizing complex, multi-objective functions with expensive evaluations.
method Extends BORE to multi-objective Bayesian optimisation, using density-ratio estimation.
result MBORE outperforms BO on high-dimensional and real-world problems.
Adaptive multi-stage density ratio estimation improves learning of latent space EBM.
problem Learning energy-based models in latent space is computationally expensive and challenging.
method Adaptive multi-stage density ratio estimation using NCE to bridge the gap between prior and posterior densities.
result The method enables more expressive prior models and sharpens the latent space EBM.
New method corrects bias in density ratio estimation for missing data.
problem Missing data bias in density ratio estimation.
method Adapted KLIEP method (M-KLIEP) for MNAR data.
result M-KLIEP restores consistency and minimax optimality.
Spectral algorithms improve under covariate shift with novel weighted techniques.
problem Improving spectral algorithms' performance under covariate shift.
method Analysis of spectral algorithms in non-parametric regression over RKHS, proposing a weighted spectral algorithm with clipped weights.
result Normalized weighted spectral algorithm achieves optimal capacity-independent convergence rates, and clipped weights can approach optimal capacity-dependent rates.
New method estimates density ratio for well-separated distributions using multi-class logistic regression.
problem Challenges in estimating density ratio for well-separated distributions.
method Uses multi-class logistic regression with auxiliary densities to estimate log(p/q).
result Demonstrates superior performance on density ratio estimation, mutual information, and representation learning tasks.
New binary loss functions improve density ratio estimation accuracy.
problem Improving accuracy of density ratio estimators using binary classifiers.
method Characterized loss functions based on prescribed error measures in Bregman divergences.
result Novel loss functions prioritize accurate estimation of large density ratio values.
Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity…
A new method for estimating density ratios using geodesics on statistical manifolds.
problem Stability of density ratio estimation when distributions are distant.
method Iterative sampling along generalized geodesics on the Riemannian manifold.
result The proposed method outperforms existing incremental mixture methods.
Proposes a robust method for predicting missing outcomes in covariate shift adaptation.
problem Predicting missing outcomes in test data with covariate shift.
method Doubly robust estimator for covariate shift adaptation via importance weighting, incorporating an additional estimator for the regression function.
result Shows robustness against density-ratio estimation errors, maintaining consistency if either estimator is consistent.
Unified framework for estimating density ratios in causal inference.
problem Estimating density ratios for causal inference is challenging due to instability and curse of dimensionality.
method Bregman-Riesz regression unifies three methods: Bregman divergences, probabilistic classification, and Riesz loss.
result Unified framework improves density ratio estimation in causal inference.
The paper proposes using density ratio estimation to evaluate synthetic data quality.
problem Improving the quality and utility of synthetic data for analysis.
method Density ratio estimation to measure synthetic data quality.
result Density ratio estimation yields more accurate global utility estimates than existing methods.
Riesz regression connects to density ratio estimation for causal inference.
problem Estimating average treatment effects in causal inference.
method Riesz regression as a signed density ratio and least-squares importance fitting.
result Riesz regression and DRE are equivalent, allowing transfer of DRE results.
Modes and ridges of the probability density function behind observed data are useful geometric features. Mode-seeking clustering assigns cluster labels by associating data samples with the nearest modes, and estimation of density ridges enables us to find lower-dimensional structures hidden in data. A key technical cha…
Generative adversarial networks (GANs) are successful deep generative models. GANs are based on a two-player minimax game. However, the objective function derived in the original motivation is changed to obtain stronger gradients when learning the generator. We propose a novel algorithm that repeats the density ratio e…
Study proves NN matching is equivalent to Riesz regression for debiased machine learning.
problem Addressing bias in machine learning models.
method Interprets NN matching as Riesz regression and derives it from LSIF.
result NN matching is shown to be equivalent to Riesz regression.
DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.
problem Inferring causal direction between continuous and discrete variables from observational data.
method Density Ratio-based Causal Discovery (DRCD) method.
result DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.
New method combines strengths of two PCL approaches without density ratio estimation.
problem Estimating causal functions in Proxy Causal Learning with unobserved confounders and proxies.
method Kernel-based doubly robust estimators combining treatment and outcome bridges, density ratio-free.
result Outperforms existing methods on PCL benchmarks, including a prior doubly robust method.
Unified framework for estimating density ratios across multiple distributions.
problem Binary density ratio estimation for multiple distributions.
method Unified framework based on Bregman divergence minimization.
result Generalization of binary DRE methods to multiple distributions.
Study post-hoc Learning to Defer using density-ratio losses.
problem Optimizing decision-making between models and experts.
method Density-ratio losses for post-hoc L2D scorers, derived from class-probability estimation.
result The approach recovers known results and introduces new connections to expert comparison and anomaly detection.
Paper proposes a novel approach to density ratio estimation using projection pursuit.
problem Density ratio estimation challenges in high dimensions and model misspecification.
method The approach uses projection pursuit to approximate density ratios, addressing high dimensionality and model flexibility issues.
result The proposed estimator is consistent and converges at a certain rate, outperforming existing methods in experiments.