Paper tackles unbounded density ratio estimation for covariate shift adaptation.
problem Understudied challenge in statistical learning: unbounded density ratios.
method Three-step estimation method: relative density ratio, truncation, and transformation.
result Established rigorous convergence guarantees for density ratio and regression estimators.
Paper develops estimators for unbounded density ratios with applications in error control.
problem Estimating density ratios with unbounded domains and ranges.
method Least squares and logistic regression loss functions for density ratio estimation.
result Established upper bounds on estimation errors with optimal rates for unbounded density ratios.
Develops a new density ratio estimator for causal inference.
problem Estimation of density ratio functions in statistics.
method Super learning approach with a novel loss function.
result Empirical validation of the density ratio super learner's performance.
Meta-learning improves relative density-ratio estimation from limited data.
problem Estimating relative density-ratios from few instances.
method Meta-learning using neural networks to extract and embed dataset information for relative DRE.
result Meta-learning enables efficient and effective adaptation to few instances for relative DRE.
Study shows robust method for estimating density ratios even with heavy contamination.
problem Estimating density ratios in the presence of heavy contamination.
method Weighted density ratio estimation (DRE) with doubly strong robustness.
result Weighted DRE achieves sparse consistency under heavy contamination.
In this work, we propose new objective functions to train deep neural network based density ratio estimators and apply it to a change point detection problem. Existing methods use linear combinations of kernels to approximate the density ratio function by solving a convex constrained minimization problem. Approximating…
New method resolves density ratio estimation saturation issues.
problem Error saturation in density ratio estimation methods.
method Iterated regularization to improve kernel methods.
result Achieves fast error rates on regular learning problems.
A new method for estimating density ratios using geodesics on statistical manifolds.
problem Stability of density ratio estimation when distributions are distant.
method Iterative sampling along generalized geodesics on the Riemannian manifold.
result The proposed method outperforms existing incremental mixture methods.
Adapts RKHS methods to estimate density ratios with optimal error.
problem Estimating density ratios from limited data.
method Minimizes regularized Bregman divergence in RKHS, with Lepskii type parameter choice.
result Adaptive minimax optimal error rate for quadratic loss.
Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity…
The paper proposes using density ratio estimation to evaluate synthetic data quality.
problem Improving the quality and utility of synthetic data for analysis.
method Density ratio estimation to measure synthetic data quality.
result Density ratio estimation yields more accurate global utility estimates than existing methods.
We develop a new loss function for estimating quasiprobabilistic density ratios.
problem Discontinuous or non-surjective relationships between optimal classifiers and target densities.
method Introduce a convex loss function compatible with both probabilistic and quasiprobabilistic densities.
result Achieve state-of-the-art results in estimating di-Higgs production in particle physics.
A density ratio is defined by the ratio of two probability densities. We study the inference problem of density ratios and apply a semi-parametric density-ratio estimator to the two-sample homogeneity test. In the proposed test procedure, the f-divergence between two probability densities is estimated using a density-r…
Stable and consistent model alignment for language models without assuming human preference models.
problem Lack of statistical consistency in existing alignment methods.
method Relative density ratio optimization between preferred and mixture of preferred and non-preferred data distributions.
result Our approach achieves statistical consistency and stability, providing tighter convergence guarantees.
New binary loss functions improve density ratio estimation accuracy.
problem Improving accuracy of density ratio estimators using binary classifiers.
method Characterized loss functions based on prescribed error measures in Bregman divergences.
result Novel loss functions prioritize accurate estimation of large density ratio values.
Featurization improves density ratio estimation for complex data.
problem Difficulty in estimating density ratios for high-dimensional, different distributions.
method Invertible generative model to map distributions into a common feature space.
result Improved accuracy in density ratio estimation through feature space.
New method estimates density ratio for well-separated distributions using multi-class logistic regression.
problem Challenges in estimating density ratio for well-separated distributions.
method Uses multi-class logistic regression with auxiliary densities to estimate log(p/q).
result Demonstrates superior performance on density ratio estimation, mutual information, and representation learning tasks.
Proposes a robust method for predicting missing outcomes in covariate shift adaptation.
problem Predicting missing outcomes in test data with covariate shift.
method Doubly robust estimator for covariate shift adaptation via importance weighting, incorporating an additional estimator for the regression function.
result Shows robustness against density-ratio estimation errors, maintaining consistency if either estimator is consistent.
Riesz regression connects to density ratio estimation for causal inference.
problem Estimating average treatment effects in causal inference.
method Riesz regression as a signed density ratio and least-squares importance fitting.
result Riesz regression and DRE are equivalent, allowing transfer of DRE results.
Generative adversarial networks (GANs) are successful deep generative models. GANs are based on a two-player minimax game. However, the objective function derived in the original motivation is changed to obtain stronger gradients when learning the generator. We propose a novel algorithm that repeats the density ratio e…
Study proves NN matching is equivalent to Riesz regression for debiased machine learning.
problem Addressing bias in machine learning models.
method Interprets NN matching as Riesz regression and derives it from LSIF.
result NN matching is shown to be equivalent to Riesz regression.
Adaptive multi-stage density ratio estimation improves learning of latent space EBM.
problem Learning energy-based models in latent space is computationally expensive and challenging.
method Adaptive multi-stage density ratio estimation using NCE to bridge the gap between prior and posterior densities.
result The method enables more expressive prior models and sharpens the latent space EBM.
New algorithm uses density ratios for efficient online reinforcement learning.
problem Challenges in collecting exploratory data for online reinforcement learning.
method Density ratio modeling for online exploration, combining truncation and optimism.
result Sample-efficient online exploration achieved with GLOW and HyGLOW.
Optimizes kernel density ratios for better predictions and information measures.
problem Improving accuracy of kernel density estimates for density ratios.
method Derives an optimal weight function using calculus of variations.
result Reduces bias in kernel density estimates, leading to improved prediction posteriors and information-theoretic measures.
TRE improves density-ratio estimation for highly dissimilar densities.
problem Density-ratio estimation fails for significantly different densities.
method Telescoping density-ratio estimation (TRE) framework.
result TRE yields substantial improvements over existing methods for mutual information estimation.
BO method improved by density-ratio estimation for better efficiency and scalability.
problem Limitations in Bayesian optimization due to analytical tractability of predictive models.
method Reformulated Bayesian optimization by casting expected improvement as a binary classification problem.
result Improved efficiency and scalability of Bayesian optimization.
DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.
problem Inferring causal direction between continuous and discrete variables from observational data.
method Density Ratio-based Causal Discovery (DRCD) method.
result DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.
New method combines strengths of two PCL approaches without density ratio estimation.
problem Estimating causal functions in Proxy Causal Learning with unobserved confounders and proxies.
method Kernel-based doubly robust estimators combining treatment and outcome bridges, density ratio-free.
result Outperforms existing methods on PCL benchmarks, including a prior doubly robust method.
Study post-hoc Learning to Defer using density-ratio losses.
problem Optimizing decision-making between models and experts.
method Density-ratio losses for post-hoc L2D scorers, derived from class-probability estimation.
result The approach recovers known results and introduces new connections to expert comparison and anomaly detection.
Unified framework for estimating density ratios in causal inference.
problem Estimating density ratios for causal inference is challenging due to instability and curse of dimensionality.
method Bregman-Riesz regression unifies three methods: Bregman divergences, probabilistic classification, and Riesz loss.
result Unified framework improves density ratio estimation in causal inference.
Paper proposes a novel approach to density ratio estimation using projection pursuit.
problem Density ratio estimation challenges in high dimensions and model misspecification.
method The approach uses projection pursuit to approximate density ratios, addressing high dimensionality and model flexibility issues.
result The proposed estimator is consistent and converges at a certain rate, outperforming existing methods in experiments.
Unified framework for OOD detection using class ratio estimation.
problem Density-based OOD detection is unreliable for OOD images.
method Unified framework that builds energy-based models and employs differing base distributions, directly estimating the density ratio through class ratio estimation.
result Competitive results on OOD image problems compared to recent work.
Unified framework for robust, stable, and efficient density ratio estimation.
problem Density-chasm and support-chasm problems in density ratio estimation.
method Dequantified diffusion-Schrödinger bridge (D3RE) framework with DDBI and DSBI.
result Offers uniform approximation and bounded time scores in theory and empirical performance.
Machine learning models, especially based on deep architectures are used in everyday applications ranging from self driving cars to medical diagnostics. It has been shown that such models are dangerously susceptible to adversarial samples, indistinguishable from real samples to human eye, adversarial samples lead to in…
Unified framework for estimating density ratios across multiple distributions.
problem Binary density ratio estimation for multiple distributions.
method Unified framework based on Bregman divergence minimization.
result Generalization of binary DRE methods to multiple distributions.
Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.
problem Statistical inconsistency in aligning LLMs with human preferences.
method Direct Density Ratio Optimization (DDRO) estimates density ratio directly.
result DDRO is statistically consistent, converging to true human preferences as data grows.
Bayesian optimization improves efficiency with semi-supervised learning.
problem Efficiently find global optima of expensive functions.
method Density ratio estimation combined with semi-supervised learning.
result Improved accuracy in identifying global optima with unlabeled data.
Estimates density ratio for two-sample comparison using tree models.
problem Comparing two distributions given i.i.d. observations.
method Additive tree models with balancing loss for density ratio estimation.
result Bayesian inference provides uncertainty quantification for density ratio.
Density ratio estimation is a vital tool in both machine learning and statistical community. However, due to the unbounded nature of density ratio, the estimation procedure can be vulnerable to corrupted data points, which often pushes the estimated ratio toward infinity. In this paper, we present a robust estimator wh…
A new method improves density ratio estimation with fewer function evaluations.
problem Stable and accurate estimation of density ratios with high variance issues.
method Diffusion Secant Alignment for Score-Based Density Ratio Estimation (ISA-DRE)
result ISA-DRE achieves comparable or superior results with fewer function evaluations.
This paper tackles continuous covariate shift by adaptively training predictors.
problem Continuous covariate shift where input distributions change over time.
method Online density ratio estimation method to adaptively train predictors.
result Excess risk guarantee for the predictor through dynamic regret bound.
A new method for estimating density ratios in high dimensions.
problem Difficulty in accurately comparing probability distributions in high-dimensional settings.
method Divide-and-conquer approach via an infinite continuum of bridge distributions and time score matching.
result The proposed method effectively estimates density ratios and performs well on complex datasets.
Paper connects contrastive learning to MI maximization and establishes robust methods for nonlinear ICA and subspace estimation.
problem Understanding and improving unsupervised representation learning and density ratio estimation.
method The paper connects contrastive learning to MI maximization, establishes new recovery conditions for nonlinear ICA, and proposes a practical outlier-robust method for nonlinear subspace estimation.
result The proposed methods can be seen as maximizing MI, performing nonlinear ICA, or estimating nonlinear subspaces, and are robust to outliers.
A new approach to model rejection using density ratios.
problem Improving model performance through selective prediction.
method Optimization of a loss's risk with φ-divergence regularization to find an idealized data distribution.
result Model rejection can be made by comparing the density ratio of the idealized distribution to the actual data distribution.
Unified view of KL-divergence and IPMs via DRE, with new DRM metrics.
problem Unified understanding of KL-divergence and IPMs.
method Unified representation via maximum likelihood density-ratio estimation (DRE).
result Unified form of IPMs and novel DRM metrics.
The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…
New method corrects bias in density ratio estimation for missing data.
problem Missing data bias in density ratio estimation.
method Adapted KLIEP method (M-KLIEP) for MNAR data.
result M-KLIEP restores consistency and minimax optimality.
CDRE estimates density ratios in streaming data without historical samples.
problem Online learning with shifting data distributions.
method Iterative estimation of density ratios between initial and current distributions.
result CDRE outperforms standard DRE in estimating divergences between distributions.