Neural networks approximate likelihood ratios for complex models.
problem Difficulty in computing likelihood ratios for modern models.
method Applying the likelihood ratio trick with neural network classifiers.
result Different neural network setups can approximate likelihood ratios with varying performance.
Novel neural likelihood ratio estimation for negative data in particle physics.
problem Estimating likelihood ratios with negative probability densities and weights.
method Introducing a novel loss function and a new model architecture based on signed mixture models.
result Demonstrated improved estimation on a real-world example from particle physics.
Paper detects changes in graph-based data streams using likelihood-ratios.
problem Detecting changes in synchronized graph-based data streams.
method Kernel-based likelihood-ratio estimation over graph nodes.
result Effective detection and localization of change-points.
New machine learning methods for inference from simulated data.
problem Modeling score and likelihood ratio functions from sampled data.
method InferoStatic Networks (ISN), Kernel Score Estimation (KSE), Kernel Likelihood Ratio Estimation (KLRE).
result Improved inference methods for complex models.
In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio. We show that the gap can be upper bounded by some form of dispersion measure of the likelihood ratio, which suggests the bias of variational inference can be reduced by making the distribution of the like…
Optimal selective classification using likelihood ratios improves model reliability.
problem Enhancing predictive model reliability by allowing uncertain predictions.
method Neyman--Pearson lemma applied to likelihood ratios for optimal selection.
result Neyman--Pearson-informed methods outperform existing baselines under covariate shifts.
Optimally tackles covariate shift in RKHS-based nonparametric regression.
problem Covariate shift in nonparametric regression over RKHS.
method Two families of covariate shift problems defined using likelihood ratios. Minimax rate-optimal estimators for KRR and reweighted KRR.
result KRR is minimax rate-optimal and strictly sub-optimal compared to naive estimator under covariate shift.
Study detects signals in spiked Wigner models using log likelihood ratio.
problem Detecting signals in rank-one spiked Wigner models with non-Gaussian noise.
method Proved asymptotic normality of log likelihood ratio and computed error thresholds.
result Optimal signal-to-noise ratio threshold for reliable detection.
The paper proposes a method to construct confidence sets using likelihood ratios for sequential decision-making.
problem Constructing valid uncertainty estimates for unknown quantities in sequential decision-making.
method The method uses likelihood ratios to create any-time valid confidence sequences without specialized treatment for each application.
result The proposed confidence sets maintain the prescribed coverage in a model-agnostic manner and their size depends on the choice of estimator sequence.
Extends likelihood ratio exponential families to analyze various optimization methods.
problem Analyzing optimization methods like rate-distortion and information bottleneck.
method Linking geometric mixture paths to exponential families and using hypothesis testing.
result Provides a common mathematical framework for understanding these methods.
DeepLR constructs confidence intervals for neural networks with asymmetric expansions.
problem Uncertainty estimation for neural network predictions.
method Likelihood-ratio-based approach for constructing asymmetric confidence intervals.
result DeepLR offers asymmetric intervals expanding in regions with limited data.
Direct neural ratio estimator for likelihood-free inference.
problem Efficient likelihood estimation for complex models.
method Amortized likelihood ratio estimation using neural networks.
result DNRE often outperforms previous ratio estimators.
In this work, a deep learning-based method for log-likelihood ratio (LLR) lossy compression and quantization is proposed, with emphasis on a single-input single-output uncorrelated fading communication setting. A deep autoencoder network is trained to compress, quantize and reconstruct the bit log-likelihood ratios cor…
Bayesian optimization improves by focusing on outputs with the likelihood ratio method.
problem Improving Bayesian optimization by accurately estimating output importance.
method Importance-sampling theory and likelihood ratio for guiding search towards low objective function values.
result Likelihood-weighted acquisition functions outperform unweighted ones in various applications.
New method for robust distribution alignment using log-likelihood ratio and normalizing flows.
problem Distribution alignment challenges in deep learning.
method Log-likelihood ratio statistic and normalizing flows.
result Minimizing the proposed objective yields robust domain alignment.
Graph-based LRE estimates likelihood-ratios collaboratively for nodes.
problem Comparing unknown pdfs at graph nodes with graph structure.
method Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF).
result Collaborative estimation improves performance compared to independent methods.
The paper proposes an efficient nested simulation design using likelihood ratio method.
problem Designing nested simulations with fixed outer scenarios and minimizing simulation effort.
method Proposes a bi-level optimization problem to decide inner replications and pooling strategies.
result Optimized design achieves $\cO(Γ^{-1})$ mean squared error of estimators.
In many fields of science, generalized likelihood ratio tests are established tools for statistical inference. At the same time, it has become increasingly common that a simulator (or generative model) is used to describe complex processes that tie parameters θ of an underlying theory and measurement apparatus to hig…
Sequential hypothesis testing is a desirable decision making strategy in any time sensitive scenario. Compared with fixed sample-size testing, sequential testing is capable of achieving identical probability of error requirements using less samples in average. For a binary detection problem, it is well known that for k…
FF algorithm uses goodness as a likelihood-ratio test for scalar normalization.
problem Training each layer locally with scalar goodness.
method FF algorithm uses a likelihood-ratio test with squared goodness as the sufficient statistic.
result The FF algorithm generalizes to anisotropic and heavy-tailed populations.
FF algorithm uses goodness as a measure of input quality, derived from likelihood-ratio tests.
problem Training each layer locally with a goodness measure.
method FF algorithm uses a likelihood-ratio test to define goodness, which is the sum of squared activations normalized between layers.
result The goodness measure is a sufficient statistic for a likelihood-ratio test, explaining the FF algorithm's performance.
A novel kernel-based test detects equality versus singularity of two probability measures.
problem Detecting equality versus singularity of two probability distributions.
method Combines kernel mean and kernel covariance embeddings to construct a likelihood ratio test statistic.
result The test statistic satisfies a '0/\infty' law, vanishing under the null and diverging under the alternative.
The paper exposes common misconceptions about OOD detection and proposes a new framework.
problem Density-based OOD detection fails in deep learning settings.
method Proposes the OOD proxy framework to unify likelihood-ratio-based methods.
result Likelihood ratio is a principled method for OOD detection.
Proposes a conservative LR estimator for infrequent data near a frequency threshold.
problem Overestimation of likelihood ratios for infrequent data near a frequency threshold.
method Conservative likelihood ratio estimator for frequencies slightly above a threshold.
result Improves prediction accuracy in named entity context prediction.
A new method optimizes diffusion models with recursive likelihood ratios.
problem Efficiently aligning pre-trained diffusion models for specific applications.
method Recursive Likelihood Ratio (RLR) optimizer for Half-Order (HO) fine-tuning.
result The RLR method achieves unbiased and lower-variance gradients, improving model performance.
New method for valid prediction sets in high-dimensional covariate shifts.
problem Valid prediction sets in high-dimensional covariate shifts.
method Likelihood-ratio regularized quantile regression (LR-QR) algorithm.
result LR-QR constructs valid prediction sets with desired coverage in target domain.
In deep neural network, the cross-entropy loss function is commonly used for classification. Minimizing cross-entropy is equivalent to maximizing likelihood under assumptions of uniform feature and class distributions. It belongs to generative training criteria which does not directly discriminate correct class from co…
Various problems in Engineering and Statistics require the computation of the likelihood ratio function of two probability densities. In classical approaches the two densities are assumed known or to belong to some known parametric family. In a data-driven version we replace this requirement with the availability of da…
We propose a general method for constructing hypothesis tests and confidence sets that have finite sample guarantees without regularity conditions. We refer to such procedures as "universal." The method is very simple and is based on a modified version of the usual likelihood ratio statistic, that we call "the split li…
Unified view of LR and RP gradients explained via divergence theorem.
problem Explaining the nature and relationship of LR and RP gradients.
method First principles approach using divergence theorem.
result Characterization of all possible estimators combining LR and RP.
A new algorithm detects changes in data with constant cost per iteration.
problem Detecting changes in data with low computational cost.
method Adapting pruning and maximisation techniques from Gaussian data to exponential family models.
result The algorithm can detect changes in a wide range of models with a constant per-iteration cost.
A method detects changes in heterogeneous data streams over graph nodes.
problem Detecting changes in data streams from nodes of a graph.
method Online non-parametric method using likelihood-ratio estimation.
result The method accurately identifies change-points in real-world applications.
A pretrained LLM and its finetuned version can detect OOD data effectively.
problem Detecting out-of-distribution data in language models.
method Using the likelihood ratio between a pretrained and finetuned LLM.
result The likelihood ratio is an effective criterion for OOD detection.
Paper introduces an online method for estimating the difference between two probability distributions.
problem Estimating the difference between two probability density functions using available data.
method Non-parametric online likelihood-ratio estimation using Pearson-divergence functional minimization.
result The proposed method provides efficient online updates and theoretical guarantees for performance.
Study benchmarks TSC algorithms in distinguishing diffusions using the likelihood ratio test.
problem Benchmarking optimality of TSC algorithms in distinguishing diffusion processes.
method Proposes to benchmark TSC algorithms using the likelihood ratio test (LRT).
result LRT benchmarks are computationally efficient and can be applied to various time series types.
We introduce a Bayesian solution for the problem in forensic speaker recognition, where there may be very little background material for estimating score calibration parameters. We work within the Bayesian paradigm of evidence reporting and develop a principled probabilistic treatment of the problem, which results in a…
These notes survey and explore an emerging method, which we call the low-degree method, for predicting and understanding statistical-versus-computational tradeoffs in high-dimensional inference problems. In short, the method posits that a certain quantity -- the second moment of the low-degree likelihood ratio -- gives…
Paper tackles moment estimation under covariate shift with a two-stage algorithm.
problem Estimating moments under covariate shift when source and target distributions differ.
method Proposes a two-stage algorithm: first, an optimal estimator for the source distribution; second, likelihood ratio reweighting for calibration.
result Achieves minimax optimal bound for moment estimation.
This paper presents a sequential method to identify the topological ordering of causal DAGs using likelihood ratio scores.
problem Identifying the causal relationships in a data mining scenario with ambiguity of causal directions.
method A general sequential sorting procedure that orders variables one at a time, starting at root nodes, followed by children of the root nodes, and so on until completion. Simple likelihood ratio scores are used to decide the next node to append to the current partial ordering.
result The population version of the procedure provably identifies a true ordering of the underlying DAG under mild assumptions.
A test for neural networks identifies genetic associations.
problem Testing complex associations in neural networks.
method Sieve quasi-likelihood ratio test for neural networks with one hidden layer.
result The test statistic has an asymptotic chi-squared distribution.
In this work, we propose a generalized likelihood ratio method capable of training the artificial neural networks with some biological brain-like mechanisms,.e.g., (a) learning by the loss value, (b) learning via neurons with discontinuous activation and loss functions. The traditional back propagation method cannot tr…
This article studies local and global inference for smoothing spline estimation in a unified asymptotic framework. We first introduce a new technical tool called functional Bahadur representation, which significantly generalizes the traditional Bahadur representation in parametric models, that is, Bahadur [Ann. Inst. S…
We are interested in the design of generative networks. The training of these mathematical structures is mostly performed with the help of adversarial (min-max) optimization problems. We propose a simple methodology for constructing such problems assuring, at the same time, consistency of the corresponding solution. We…
Markov regime switching models have been used in numerous empirical studies in economics and finance. However, the asymptotic distribution of the likelihood ratio test statistic for testing the number of regimes in Markov regime switching models has been an unresolved problem. This paper derives the asymptotic distribu…
Financial econometrics has become an increasingly popular research field. In this paper we review a few parametric and nonparametric models and methods used in this area. After introducing several widely used continuous-time and discrete-time models, we study in detail dependence structures of discrete samples, includi…
Reparameterization (RP) and likelihood ratio (LR) gradient estimators are used throughout machine and reinforcement learning; however, they are usually explained as simple mathematical tricks without providing any insight into their nature. We use a first principles approach to explain LR and RP, and show a connection …
This paper develops embeddings that preserve likelihood-based statistical inference.
problem Modern machine learning embeddings destroy the geometric structure required for likelihood-based inference.
method Developed a rigorous theory of likelihood-preserving embeddings and introduced the Likelihood-Ratio Distortion metric.
result Controlling the distortion Δn is necessary and sufficient for preserving inference. Unified framework for DRO using OT with constraints.
problem Handling ambiguity in likelihood ratios and outcomes.
method Unified framework leveraging optimal transport with conditional moment constraints.
result Unified approach enables adversarial perturbation of likelihood ratios and outcomes.