arXiv research

Paper proposes energy objective for training normalizing flows without determinants.

We consider the problem of training probabilistic conditional random fields (CRFs) in the context of a task where performance is measured using a specific loss function. While maximum likelihood is the most common approach to training CRFs, it ignores the inherent structure of the task's loss function. We describe alte…

Maximum likelihood training improves the performance of score-based diffusion models.

This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.

A new method normalizes EBM training by introducing a learnable parameter.

This paper improves normalizing flows by combining MLE and sliced-Wasserstein distance for better data fidelity.

Gaussian surrogates improve Poisson imaging performance at low doses.

Paper proposes efficient training for normalizing flows in Boltzmann generators.

Simplified explanation of ML for mixtures and OT.

Geodesic descent optimizes likelihood in dually flat spaces.

This work investigates training infinite mixtures with maximum likelihood for improved uncertainty quantification.

Innovative game theory approach optimizes survival analysis metrics.

Optimizes MMD learning for generative models with theoretical guarantees.

Connectionist temporal classification (CTC) is widely used for maximum likelihood learning in end-to-end speech recognition models. However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e.g., word error rate (WER). This results in a mismatch…

With an eye towards human-centered automation, we contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax selection in models of human decision-making, we study the maximum likelihood parameter estimation problem for sof…

Score matching fails to train VAEs robustly, revealing autoencoding loss insights.

We simplify inference for TPP models with latent structures.

A new method optimizes neural sequence models for better task performance.

Modern applications and progress in deep learning research have created renewed interest for generative models of text and of images. However, even today it is unclear what objective functions one should use to train and evaluate these models. In this paper we present two contributions. Firstly, we present a critique o…

We present an asymptotic analysis of Viterbi Training (VT) and contrast it with a more conventional Maximum Likelihood (ML) approach to parameter estimation in Hidden Markov Models. While ML estimator works by (locally) maximizing the likelihood of the observed data, VT seeks to maximize the probability of the most lik…

Label shift refers to the phenomenon where the prior class probability p(y) changes between the training and test distributions, while the conditional probability p(x|y) stays fixed. Label shift arises in settings like medical diagnosis, where a classifier trained to predict disease given symptoms must be adapted to sc…

DFM simplifies CNF training without interpolants.

When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter's estimator of the marginal likelihood, the …

New algorithm optimizes Bayesian network learning from Gaussian data.

There are many models, often called unnormalized models, whose normalizing constants are not calculated in closed form. Maximum likelihood estimation is not directly applicable to unnormalized models. Score matching, contrastive divergence method, pseudo-likelihood, Monte Carlo maximum likelihood, and noise contrastive…

This paper proposes a new RV prediction model using neural distributional transformation and co-training.

Maximum likelihood estimation of energy-based models is a challenging problem due to the intractability of the log-likelihood gradient. In this work, we propose learning both the energy function and an amortized approximate sampling mechanism using a neural generator network, which provides an efficient approximation o…

Estimates log-likelihood of interacting particle systems using virtual particles.

Maximum likelihood estimation fails to be well-posed in Gaussian process regression.

Proposes a model to generate 3D-aware images from 2D images.

New method for robust distribution alignment using log-likelihood ratio and normalizing flows.

Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explici…

We propose a new neural sequence model training method in which the objective function is defined by $α$-divergence. We demonstrate that the objective function generalizes the maximum-likelihood (ML)-based and reinforcement learning (RL)-based objective functions as special cases (i.e., ML corresponds to $α\to 0$ and R…

Spectral clustering for directed graphs using likelihood estimation.

DLGNet improves dialogue response generation by leveraging transformer architecture.

X-CAL improves survival model calibration without sacrificing predictive power.

New method trains energy-based models faster and more stably.

Paper improves likelihood estimation for discrete distributions.

Corrects pseudo log-likelihood method issues in various applications.

Many machine learning tasks can be formulated in terms of predicting structured outputs. In frameworks such as the structured support vector machine (SVM-Struct) and the structured perceptron, discriminative functions are learned by iteratively applying efficient maximum a posteriori (MAP) decoding. However, maximum li…

This work tackles the challenge of aligning generative models without explicit reward signals.

We prove that stochastic gradient descent efficiently converges to the global optimizer of the maximum likelihood objective of an unknown linear time-invariant dynamical system from a sequence of noisy observations generated by the system. Even though the objective function is non-convex, we provide polynomial running …

We propose a novel method for maximum likelihood-based parameter inference in nonlinear and/or non-Gaussian state space models. The method is an iterative procedure with three steps. At each iteration a particle filter is used to estimate the value of the log-likelihood function at the current parameter iterate. Using …

In recent work on both generative and discriminative score to log-likelihood-ratio calibration, it was shown that linear transforms give good accuracy only for a limited range of operating points. Moreover, these methods required tailoring of the calibration training objective functions in order to target the desired r…

Graphical lasso may fail to fit models when data points are insufficient.

We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…

A new method speeds up quantum state estimation.

ConvNP improves SP prediction with translation equivariance and coherent samples.