A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Maximum likelihood training improves the performance of score-based diffusion models.
In this article we use rate-distortion theory, a branch of information theory devoted to the problem of lossy compression, to shed light on an important problem in latent variable modeling of data: is there room to improve the model? One way to address this question is to find an upper bound on the probability (equival…
In deep neural network, the cross-entropy loss function is commonly used for classification. Minimizing cross-entropy is equivalent to maximizing likelihood under assumptions of uniform feature and class distributions. It belongs to generative training criteria which does not directly discriminate correct class from co…
Noise-Contrastive Estimation improves efficiency for estimating log-likelihood of complex point processes.
Enhances VAEs for sharper image synthesis.
We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanatio…
Separable losses are inconsistent for structured prediction models.
Improved estimation for imbalanced data using log odds correction and optimal sampling.
We consider the finite sample properties of the regularized high-dimensional Cox regression via lasso. Existing literature focuses on linear models or generalized linear models with Lipschitz loss functions, where the empirical risk functions are the summations of independent and identically distributed (iid) losses. T…
Quantum machine learning uses quantum cross entropy to minimize loss, but measurement loss affects this process.
Invertible DenseNets improve model efficiency and performance.
By exploiting the property that the RBM log-likelihood function is the difference of convex functions, we formulate a stochastic variant of the difference of convex functions (DC) programming to minimize the negative log-likelihood. Interestingly, the traditional contrastive divergence algorithm is a special case of th…
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
DNLL loss improves deep LDA accuracy and consistency.
We consider log-supermodular models on binary variables, which are probabilistic models with negative log-densities which are submodular. These models provide probabilistic interpretations of common combinatorial optimization tasks such as image segmentation. In this paper, we focus primarily on parameter estimation in…
ELU algorithm improves on EM for over-specified Gaussian mixtures.
We show that the log-likelihood of several probabilistic graphical models is Lipschitz continuous with respect to the lp-norm of the parameters. We discuss several implications of Lipschitz parametrization. We present an upper bound of the Kullback-Leibler divergence that allows understanding methods that penalize the …
Paper improves likelihood estimation for discrete distributions.
PACMAN provides bounds for classification tasks considering accuracy vs. negative log-loss mismatch.
Proposes a deep neural network for event intensity estimation.
Corrects pseudo log-likelihood method issues in various applications.
Test log-likelihood comparisons can be misleading.
GBHT uses gradient boosting for density estimation with theoretical guarantees.
In this paper we revisit the weighted likelihood bootstrap, a method that generates samples from an approximate Bayesian posterior of a parametric model. We show that the same method can be derived, without approximation, under a Bayesian nonparametric model with the parameter of interest defined as minimising an expec…
Active-set algorithm improves Cox regression for shape-restricted covariates.
A new VIS approach improves log-likelihood estimation in latent variable models.
wd1 improves reasoning in dLLMs by optimizing policies without policy ratios.
We empirically investigate the (negative) expected accuracy as an alternative loss function to cross entropy (negative log likelihood) for classification tasks. Coupled with softmax activation, it has small derivatives over most of its domain, and is therefore hard to optimize. A modified, leaky version is evaluated on…
Study reveals structure of local minima in GMMs, identifying key cluster centers.
New method optimizes clustering with better log-likelihood landscape.
New method calculates DMN log-likelihood faster.
Log-concavity proven for multinomial likelihoods under specific constraints.
Novel neural likelihood ratio estimation for negative data in particle physics.
This work investigates power laws in deep neural network ensembles and predicts their performance.
Gaussian Process Factor Analysis (GPFA) has been broadly applied to the problem of identifying smooth, low-dimensional temporal structure underlying large-scale neural recordings. However, spike trains are non-Gaussian, which motivates combining GPFA with discrete observation models for binned spike count data. The dra…
The fate of scientific hypotheses often relies on the ability of a computational model to explain the data, quantified in modern statistical approaches by the likelihood function. The log-likelihood is the key element for parameter estimation and model evaluation. However, the log-likelihood of complex models in fields…
PSO optimizes model parameters in nonstandard distributions.
The Restricted Boltzmann Machines (RBM) can be used either as classifiers or as generative models. The quality of the generative RBM is measured through the average log-likelihood on test data. Due to the high computational complexity of evaluating the partition function, exact calculation of test log-likelihood is ver…
The paper analyzes LASSO penalization for high-dimensional Beta regression models.
Markov random fields (MRFs) are difficult to evaluate as generative models because computing the test log-probabilities requires the intractable partition function. Annealed importance sampling (AIS) is widely used to estimate MRF partition functions, and often yields quite accurate results. However, AIS is prone to ov…
A new method normalizes EBM training by introducing a learnable parameter.
We propose a supervised anomaly detection method based on neural density estimators, where the negative log likelihood is used for the anomaly score. Density estimators have been widely used for unsupervised anomaly detection. By the recent advance of deep learning, the density estimation performance has been greatly i…
Survival regression method improves log-likelihood scores.
Learning RBMs using standard algorithms such as CD(k) involves gradient descent on the negative log-likelihood. One of the terms in the gradient, which involves expectation w.r.t. the model distribution, is intractable and is obtained through an MCMC estimate. In this work we show that the Hessian of the log-likelihood…
Improved Gaussian process regression with tighter log marginal likelihood bounds.
In this short note we provide an unbiased multilevel Monte Carlo estimator of the log marginal likelihood and discuss its application to variational Bayes.
We consider Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained. This occurs for example when complex simulator-based statistical models are fitted to data, and synthetic likelihood (SL) method is used to form the noisy log-likelihood estimates using computationally costly …
Likelihood-based generative models are the backbones of lossless compression due to the guaranteed existence of codes with lengths close to negative log likelihood. However, there is no guaranteed existence of computationally efficient codes that achieve these lengths, and coding algorithms must be hand-tailored to spe…