New method detects anomalies using GAN with minimum likelihood regularization.
problem Detecting abnormal events in various domains.
method Generative Adversarial Networks with minimum likelihood regularization.
result Significant improvement over other methods on Cifar10 and UCI datasets.
Improved MMD estimator for likelihood-free inference.
problem Computational challenges in estimating MMD for likelihood-free inference.
method Optimally-weighted MMD estimator with improved sample complexity.
result Significantly improved sample complexity for accurate MMD estimation.
Unified perspective on score matching and new estimators designed.
problem Infeasibility of maximum likelihood estimation in complex models.
method Minimum Stein discrepancy estimators, diffusion kernel Stein discrepancy (DKSD), diffusion score matching (DSM).
result Consistency, asymptotic normality, and robustness of DKSD and DSM estimators.
This paper proposes a new method for inferring the latent dimension of linear dynamical systems.
problem Manual specification of latent dimension is impractical and leads to model limitations.
method The paper introduces a minimum description length criterion to infer latent dimension.
result The proposed method effectively infers latent dimension and improves model performance.
In this work, a novel solution to the speaker identification problem is proposed through minimization of statistical divergences between the probability distribution (g). of feature vectors from the test utterance and the probability distributions of the feature vector corresponding to the speaker classes. This approac…
New metrics improve learning of Gaussian networks.
problem Lack of suitable metrics for continuous Bayesian network learning.
method Introduce three-part minimum description length and renormalized normalized maximum likelihood metrics.
result Proposed metrics outperform BIC/AIC metrics in accuracy and convergence.
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
Paper proposes MWDE for estimating finite location-scale mixtures.
problem Estimating finite location-scale mixtures using MLE is problematic.
method Investigates minimum Wasserstein distance estimators (MWDE).
result MWDE is consistent and provides a numerical solution.
New method improves MMD estimation without convexity assumptions.
problem Lack of theoretical guarantees for MMD estimation algorithms.
method Preconditioned gradient descent (PGD) scheme for MMD optimization.
result PGD scheme converges globally under specific conditions.
Improved OOD detection across various shifts using multi-encoder fusion of RDMs.
problem Out-of-distribution detection across multiple types of distribution shifts.
method Statistical identification of encoder sensitivity, EncMin2L fusion, and Tippett minimum combination.
result Achieves AUROC ≥ 0.94 across four shift types, outperforming state-of-the-art detectors.
A new topic modeling method that minimizes topic simplex volume.
problem Topic modeling efficiency and accuracy.
method Reformulates LDA as minimizing topic simplex volume, uses convex relaxation and ADMM.
result Relaxed problem has same global minimum as original under assumptions.
The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-li…
This research simplifies PCA model selection using MDL principle.
problem Choosing the right number of principal components in PCA.
method Reduces NML problems to lower-dimension problems and bounds PCA NML.
result Bound the NML of PCA by terms of the NML of linear regression.
Graphical lasso may fail to fit models when data points are insufficient.
problem When does graphical lasso fail to select and fit a graphical model?
method Computational experiments with graphical lasso.
result Graphical lasso may fail when the number of data points is less than the maximum likelihood threshold.
Adversarial dynamics embedding improves MLE of exponential family models.
problem Maximum likelihood estimation of exponential family models with neural network parametrization.
method Adversarial dynamics embedding to estimate the dual sampler and primal model simultaneously.
result Adversarial dynamics embedding leads to more effective learning and improved estimators compared to existing methods.
Develops efficient method for nonconvex problems using Regula Falsi.
problem Nonconvex inverse problems with likelihood constraints.
method Regula Falsi root-finding techniques applied to level-set formulations.
result Proves extension of level-set methods to nonconvex problems.
The paper explores how over-parameterized linear regression models generalize without violating learning theory principles.
problem Understanding how over-parameterized linear regression models generalize without violating learning theory principles.
method The paper uses the predictive normalized maximum likelihood (pNML) learner to investigate the minimum norm solution of over-parameterized linear regression models.
result The model generalizes well when the test sample lies in a subspace spanned by eigenvectors associated with large eigenvalues of the training data.
The NL score optimizes speaker recognition tasks.
problem Improving speaker recognition accuracy.
method Established the theory of optimal scores based on normalized likelihood.
result NL score is equivalent to PLDA likelihood ratio under certain conditions.
We consider an on-line system identification setting, in which new data become available at given time steps. In order to meet real-time estimation requirements, we propose a tailored Bayesian system identification procedure, in which the hyper-parameters are still updated through Marginal Likelihood maximization, but …
Proposes Deep LTMLE for estimating dynamic treatment effects in longitudinal studies.
problem Estimating counterfactual mean outcomes under dynamic treatment policies in longitudinal settings.
method Uses a transformer architecture with temporal-difference learning for initial estimation, followed by TMLE correction and statistical inference.
result Demonstrates superior performance in complex, long-term scenarios compared to existing methods.
The MoN loss fails to accurately represent ground truth probability density functions in probabilistic trajectory prediction.
problem Improving the diversity of probabilistic trajectory predictions in autonomous driving and robot planning.
method Proof and validation of the MoN loss's inaccuracy and proposed solutions to correct it.
result The MoN loss approximates the square root of the ground truth probability density function, not the function itself.
Spectral clustering for directed graphs using likelihood estimation.
problem Clustering directed graphs with edge directions.
method Maximum likelihood estimation on stochastic block models.
result Significant performance gains over existing methods.
Sequence-to-sequence models, such as attention-based models in automatic speech recognition (ASR), are typically trained to optimize the cross-entropy criterion which corresponds to improving the log-likelihood of the data. However, system performance is usually measured in terms of word error rate (WER), not log-likel…
IBS efficiently estimates log-likelihood without bias.
problem Estimating log-likelihood for complex models.
method Inverse Binomial Sampling (IBS) for efficient, unbiased estimation.
result IBS produces lower error in estimated parameters and maximum log-likelihood values.
We present a multi-task learning approach to jointly estimate the means of multiple independent data sets. The proposed multi-task averaging (MTA) algorithm results in a convex combination of the single-task maximum likelihood estimates. We derive the optimal minimum risk estimator and the minimax estimator, and show t…
Conservation laws improve diffusion model training by optimizing likelihood.
problem Training diffusion models with denoising objectives.
method Developed conservation laws based on GEXIT functions for memoryless noise processes.
result Unified characterization of diffusion model likelihood, reducing training to learning marginal posteriors.
We propose a minimum distance estimation method for robust regression in sparse high-dimensional settings. The traditional likelihood-based estimators lack resilience against outliers, a critical issue when dealing with high-dimensional noisy data. Our method, Minimum Distance Lasso (MD-Lasso), combines minimum distanc…
Maximum Likelihood Estimators (MLE) has many good properties. For example, the asymptotic variance of MLE solution attains equality of the asymptotic Cram{é}r-Rao lower bound (efficiency bound), which is the minimum possible variance for an unbiased estimator. However, obtaining such MLE solution requires calculating t…
Researchers develop a method for statistical inference in models with intractable likelihoods.
problem Statistical inference for models with intractable likelihoods.
method Minimum distance estimators using maximum mean discrepancy (MMD) in reproducing kernel Hilbert space.
result The estimators are consistent, asymptotically normal, and robust to model misspecification.
A comprehensive guide to MDL principle, a statistical inference theory.
problem Statistical inference and pattern recognition.
method MDL principle, model selection, hypothesis testing, averaging.
result Unified perspective on various statistical methods.
New method identifies common cause in causal insufficiency, revealing complex phase transitions.
problem Identifying common cause in causal insufficiency with observed joint probability.
method Generalized maximum likelihood method, closely related to maximum entropy principle.
result Identifies consistent common cause that aligns with the common cause principle.
New method finds minimum in noisy data, useful for model selection.
problem Finding the index of the minimum value in noisy observations.
method Developed an asymptotically normal test statistic integrating cross-validation and differential privacy.
result Achieves a favorable bias-variance trade-off in practical scenarios.
New method prevents posterior collapse in generative models.
problem Posterior collapse weakens generative model capacity or requires complex objectives.
method Proposes δ-VAEs that constrain the posterior variational family to a minimum distance from the prior. result Achieves state-of-the-art log-likelihood on CIFAR-10 and ImageNet 32x32.
Paper certifies intersection of minimum-volume confidence sets for multinomial outcomes.
problem Certifying intersection of minimum-volume confidence sets for multinomial outcomes.
method Exploits likelihood ordering to induce halfspace constraints, enabling adaptive geometric partitioning and computable bounds on p-values.
result Efficient and provably sound algorithm for certifying intersection, disjointness, or indeterminate result.
ACNML method improves uncertainty estimation for deep networks.
problem Uncertainty estimation and calibration for deep neural networks under distribution shift.
method Approximate Bayesian inference to approximate CNML distribution.
result ACNML compares favorably to prior techniques for uncertainty estimation.
Paper proves method for calculating NML code length works for continuous models.
problem Uncertainty in calculating NML code length for continuous models.
method Introduced a novel decomposition approach based on the coarea formula to prove correctness for continuous cases.
result Method accurately calculates NML code length for continuous models.
We tackle the problem of penalty selection of regularization on the basis of the minimum description length (MDL) principle. In particular, we consider that the design space of the penalty function is high-dimensional. In this situation, the luckiness-normalized-maximum-likelihood(LNML)-minimization approach is favorab…
Prob-PIT improves speech separation by considering output-label permutations as random variables.
problem Overconfident output-label assignment in PIT leads to unreliable speech separation.
method Prob-PIT treats output-label permutations as a discrete latent random variable with a uniform prior distribution and maximizes the log-likelihood function.
result Prob-PIT significantly outperforms PIT in terms of Signal to Distortion Ratio and Signal to Interference Ratio.
Improved EM algorithm for faster convergence of mixture models.
problem Slow or invalid convergence of EM algorithm for mixture models.
method CM-EM algorithm with a step to optimize mixture ratios and maximize G.
result Global convergence proof for CM-EM algorithm using variational methods.
We introduce a deep, generative autoencoder capable of learning hierarchies of distributed representations from data. Successive deep stochastic hidden layers are equipped with autoregressive connections, which enable the model to be sampled from quickly and exactly via ancestral sampling. We derive an efficient approx…
Wariness affects poverty traps and equilibrium diversity in economic models.
problem The impact of wariness on poverty traps and equilibrium diversity in economic models.
method An overlapping generations model to explore the effects of wariness on poverty traps and equilibrium diversity.
result Wariness can amplify or mitigate the likelihood of poverty traps and can lead to multiple intertemporal equilibria.
New algorithm optimizes Bayesian network learning from Gaussian data.
problem Learning Bayesian networks from Gaussian observational data.
method Proposes a coordinate descent algorithm for ℓ0-penalized maximum likelihood estimation. result The algorithm converges to a coordinate-wise minimum and achieves optimal objective value as sample size increases.
Study proposes a machine learning method to predict stock price crashes based on investor sentiment.
problem Predicting stock price crashes due to investor sentiment.
method Minimum covariance determinant methodology and cross-sectional regression analysis.
result The proposed method effectively captures stock price crash risk and is robust across different firm sizes.
Adaptor 'E' extends gradient-based optimizers to explore loss landscapes, improving generalization.
problem Finding lower and better-generalizing minima in deep learning.
method Proposes an adaptor 'E' to extend gradient-based optimizers, encouraging exploration along landscape valleys.
result Adapted optimizers increase test accuracy by an average of 2.5% in large-batch training tasks.
Paper introduces Ddim, a new measure of model complexity, for MDL-based learning and change detection.
problem Characterizing the complexity of probabilistic models for efficient learning and change detection.
method Introduces descriptive dimension (Ddim) as a measure of model complexity. Derives convergence rates and error probabilities for MDL-based learning and change detection.
result Ddim characterizes the performance of MDL-based learning and change detection.
OMASGAN generates anomalous samples on distribution boundary to improve anomaly detection.
problem Missed anomalies and low AD performance due to OoD samples in generative models.
method OMASGAN generates anomalous samples on the estimated distribution boundary using a GAN approach, refining AD models.
result OMASGAN improves AD performance by at least 0.24 and 0.07 points on average on MNIST and CIFAR-10 datasets.
Unified framework improves neural network robustness against label noise and adversarial attacks.
problem High sensitivity of neural networks to data contamination, including label noises and adversarial perturbations.
method Unified minimum-divergence estimation problem, rSDNet framework.
result Improves robustness to label corruption and adversarial attacks while maintaining competitive accuracy on clean data.
We extend the theory of matrix completion to the case where we make Poisson observations for a subset of entries of a low-rank matrix. We consider the (now) usual matrix recovery formulation through maximum likelihood with proper constraints on the matrix M, and establish theoretical upper and lower bounds on the rec…