Unified framework for OOD detection using class ratio estimation.
problem Density-based OOD detection is unreliable for OOD images.
method Unified framework that builds energy-based models and employs differing base distributions, directly estimating the density ratio through class ratio estimation.
result Competitive results on OOD image problems compared to recent work.
New method estimates density ratio for well-separated distributions using multi-class logistic regression.
problem Challenges in estimating density ratio for well-separated distributions.
method Uses multi-class logistic regression with auxiliary densities to estimate log(p/q).
result Demonstrates superior performance on density ratio estimation, mutual information, and representation learning tasks.
Unified framework for estimating density ratios across multiple distributions.
problem Binary density ratio estimation for multiple distributions.
method Unified framework based on Bregman divergence minimization.
result Generalization of binary DRE methods to multiple distributions.
A new method for estimating density ratios using geodesics on statistical manifolds.
problem Stability of density ratio estimation when distributions are distant.
method Iterative sampling along generalized geodesics on the Riemannian manifold.
result The proposed method outperforms existing incremental mixture methods.
The paper improves QD policy ensembles using distribution ratio estimators.
problem Training diverse and high-quality reinforcement learning agents.
method Using Stein variational gradient descent and distribution ratio estimators.
result The method generates diverse and high-quality reinforcement learning agents.
Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity…
A new method for estimating density ratios in high dimensions.
problem Difficulty in accurately comparing probability distributions in high-dimensional settings.
method Divide-and-conquer approach via an infinite continuum of bridge distributions and time score matching.
result The proposed method effectively estimates density ratios and performs well on complex datasets.
New method estimates velocity fields for minimizing f-divergences without overfitting.
problem Minimizing statistical discrepancies between target and particle distributions.
method Directly estimate velocity fields using interpolation techniques, proving consistency under mild conditions.
result Consistent estimators of velocity fields improve accuracy in applications like domain adaptation and missing data imputation.
Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right tail. This routinely occurs when there are aspects of the target distribution that are not well captur…
Sharpe ratio is widely used in asset management to compare and benchmark funds and asset managers. It computes the ratio of the excess return over the strategy standard deviation. However, the elements to compute the Sharpe ratio, namely, the expected returns and the volatilities are unknown numbers and need to be esti…
Density Ratio Estimation has attracted attention from the machine learning community due to its ability to compare the underlying distributions of two datasets. However, in some applications, we want to compare distributions of random variables that are \emph{inferred} from observations. In this paper, we study the pro…
Proposes a robust method for predicting missing outcomes in covariate shift adaptation.
problem Predicting missing outcomes in test data with covariate shift.
method Doubly robust estimator for covariate shift adaptation via importance weighting, incorporating an additional estimator for the regression function.
result Shows robustness against density-ratio estimation errors, maintaining consistency if either estimator is consistent.
Paper detects changes in graph-based data streams using likelihood-ratios.
problem Detecting changes in synchronized graph-based data streams.
method Kernel-based likelihood-ratio estimation over graph nodes.
result Effective detection and localization of change-points.
RATIO improves neural network robustness and explainability.
problem Neural networks' lack of robustness to adversarial changes and uncertainty on out-distribution samples.
method RATIO: Adversarial Training on In- and Out-distribution.
result RATIO leads to robust models with reliable confidence estimates on out-distribution samples.
A new method improves density ratio estimation efficiency and accuracy.
problem Density ratio estimation trade-off between quality and efficiency.
method One-step Score-based Density Ratio Estimation (OS-DRE) combining analytic and solver-free approach.
result OS-DRE offers a favorable balance between estimation quality and inference efficiency.
This paper tackles continuous covariate shift by adaptively training predictors.
problem Continuous covariate shift where input distributions change over time.
method Online density ratio estimation method to adaptively train predictors.
result Excess risk guarantee for the predictor through dynamic regret bound.
CDRE estimates density ratios in streaming data without historical samples.
problem Online learning with shifting data distributions.
method Iterative estimation of density ratios between initial and current distributions.
result CDRE outperforms standard DRE in estimating divergences between distributions.
Study post-hoc Learning to Defer using density-ratio losses.
problem Optimizing decision-making between models and experts.
method Density-ratio losses for post-hoc L2D scorers, derived from class-probability estimation.
result The approach recovers known results and introduces new connections to expert comparison and anomaly detection.
Paper tackles unbounded density ratio estimation for covariate shift adaptation.
problem Understudied challenge in statistical learning: unbounded density ratios.
method Three-step estimation method: relative density ratio, truncation, and transformation.
result Established rigorous convergence guarantees for density ratio and regression estimators.
Featurization improves density ratio estimation for complex data.
problem Difficulty in estimating density ratios for high-dimensional, different distributions.
method Invertible generative model to map distributions into a common feature space.
result Improved accuracy in density ratio estimation through feature space.
When performing imitation learning from expert demonstrations, distribution matching is a popular approach, in which one alternates between estimating distribution ratios and then using these ratios as rewards in a standard reinforcement learning (RL) algorithm. Traditionally, estimation of the distribution ratio requi…
Paper develops estimators for unbounded density ratios with applications in error control.
problem Estimating density ratios with unbounded domains and ranges.
method Least squares and logistic regression loss functions for density ratio estimation.
result Established upper bounds on estimation errors with optimal rates for unbounded density ratios.
We develop a new loss function for estimating quasiprobabilistic density ratios.
problem Discontinuous or non-surjective relationships between optimal classifiers and target densities.
method Introduce a convex loss function compatible with both probabilistic and quasiprobabilistic densities.
result Achieve state-of-the-art results in estimating di-Higgs production in particle physics.
A new metric evaluates generative models by comparing real and generated samples.
problem Evaluating the quality of generative models.
method Relative Density Ratio (RDR) function, optimization on variational form of φ-divergence.
result The RDR function provides a clear, interpretable, and numerically stable evaluation metric.
Paper tackles moment estimation under covariate shift with a two-stage algorithm.
problem Estimating moments under covariate shift when source and target distributions differ.
method Proposes a two-stage algorithm: first, an optimal estimator for the source distribution; second, likelihood ratio reweighting for calibration.
result Achieves minimax optimal bound for moment estimation.
We propose a \textbf{uni}fied \textbf{f}ramework for \textbf{i}mplicit \textbf{ge}nerative \textbf{m}odeling (UnifiGem) with theoretical guarantees by integrating approaches from optimal transport, numerical ODE, density-ratio (density-difference) estimation and deep neural networks. First, the problem of implicit gene…
Unified framework for robust, stable, and efficient density ratio estimation.
problem Density-chasm and support-chasm problems in density ratio estimation.
method Dequantified diffusion-Schrödinger bridge (D3RE) framework with DDBI and DSBI.
result Offers uniform approximation and bounded time scores in theory and empirical performance.
New equivalences found between subsampling and ridge regularization methods.
problem Establishing precise structural and risk equivalences between subsampling and ridge regularization.
method Proved structural and risk equivalences between subsample ridge estimators and different ridge regularization levels and subsample aspect ratios.
result Optimally tuned ridge regression exhibits a monotonic prediction risk in the data aspect ratio.
A new method optimizes a generalized Kullback-Leibler divergence for better simulation-based inference.
problem Optimizing likelihood functions when they are only known implicitly.
method Optimizes a generalized Kullback-Leibler divergence that accounts for normalization constants in unnormalized distributions.
result Unified approach that combines Neural Posterior Estimation and Neural Ratio Estimation.
Unified framework for estimating density ratios in causal inference.
problem Estimating density ratios for causal inference is challenging due to instability and curse of dimensionality.
method Bregman-Riesz regression unifies three methods: Bregman divergences, probabilistic classification, and Riesz loss.
result Unified framework improves density ratio estimation in causal inference.
In real-world classification problems, the class balance in the training dataset does not necessarily reflect that of the test dataset, which can cause significant estimation bias. If the class ratio of the test dataset is known, instance re-weighting or resampling allows systematical bias correction. However, learning…
Estimates density ratio for two-sample comparison using tree models.
problem Comparing two distributions given i.i.d. observations.
method Additive tree models with balancing loss for density ratio estimation.
result Bayesian inference provides uncertainty quantification for density ratio.
The Sharpe ratio, which is defined as the ratio of the excess expected return of an investment to its standard deviation, has been widely cited in the financial literature by researchers and practitioners. However, very little attention has been paid to the statistical properties of the estimation of the ratio. Lo (200…
FORE evaluates occupancy ratios without requiring Bellman completeness.
problem Offline reinforcement learning occupancy ratio estimation.
method Fitted occupancy-ratio evaluation (FORE) using adjoint Bellman recursion.
result FORE achieves convergence in KL without Bellman completeness.
New MC simulation methods use classifiers to estimate pdf ratios without explicit pdfs.
problem Estimating ratios of probability density functions (pdfs) without explicit pdfs.
method Proposes classifier-based pdf-free versions of MC simulation algorithms.
result Enables pdf-free simulation algorithms using surrogate functions computed by classifiers.
Rank-statistic method approximates f-divergences without density-ratio estimation.
problem Approximating f-divergences without explicit density-ratio estimation. method Mapping distribution rank histograms to discrete f-divergence and averaging over random projections. result The rank-statistic estimator is a lower bound of the true f-divergence and converges under mild conditions. REGS samples from unnormalized distributions using gradient flow and neural networks.
problem Sampling from unnormalized distributions with high accuracy and efficiency.
method REGS is a particle method that iteratively transforms samples from a reference distribution to match an unnormalized target distribution using Wasserstein gradient flow and neural networks.
result REGS outperforms state-of-the-art methods in sampling from challenging multimodal distributions and real datasets.
The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…
The paper analyzes the Rashomon ratio for infinite classifier families and shows its importance for choosing good classifiers.
problem Analyzing the Rashomon ratio for infinite classifier families.
method Quantifying the Rashomon ratio in two examples and providing guarantees for estimating it.
result A large Rashomon ratio guarantees choosing a classifier with good empirical accuracy will not significantly increase empirical loss.
SIXO improves inference by learning smoothing distributions from all observations.
problem Inference limitations due to ignoring future observations in filtering distributions.
method Density ratio estimation to warp filtering distributions into smoothing distributions, then use SMC with learned targets.
result Proves tighter log marginal lower bounds and more accurate inferences and estimates.
Bayesian optimization improves efficiency with semi-supervised learning.
problem Efficiently find global optima of expensive functions.
method Density ratio estimation combined with semi-supervised learning.
result Improved accuracy in identifying global optima with unlabeled data.
Proposes a new method for uncertainty estimation in neural networks.
problem Uncertainty quantification in neural networks for high-risk applications.
method Intuitive framework based on signal-to-noise ratio and variance-gated measure.
result Demonstrates a collapse in diversity of committee machines.
The signal-noise ratio of a portfolio of p assets, its expected return divided by its risk, is couched as an estimation problem on the sphere. When the portfolio is built using noisy data, the expected value of the signal-noise ratio is bounded from above via a Cramer-Rao bound, for the case of Gaussian returns. The bo…
A new method improves text generation quality and diversity.
problem Exposure bias in Maximum Likelihood Estimation for text generation.
method ψ-MLE, a new training scheme based on density ratio estimation.
result ψ-MLE outperforms Maximum Likelihood Estimation and other models in text generation quality and diversity.
Omega ratio, defined as the probability-weighted ratio of gains over losses at a given level of expected return, has been advocated as a better performance indicator compared to Sharpe and Sortino ratio as it depends on the full return distribution and hence encapsulates all information about risk and return. We comput…
A new method improves density ratio estimation with fewer function evaluations.
problem Stable and accurate estimation of density ratios with high variance issues.
method Diffusion Secant Alignment for Score-Based Density Ratio Estimation (ISA-DRE)
result ISA-DRE achieves comparable or superior results with fewer function evaluations.
DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.
problem Inferring causal direction between continuous and discrete variables from observational data.
method Density Ratio-based Causal Discovery (DRCD) method.
result DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.
The quantification problem consists of determining the prevalence of a given label in a target population. However, one often has access to the labels in a sample from the training population but not in the target population. A common assumption in this situation is that of prior probability shift, that is, once the la…