Develops a nonparametric method to estimate isotropic covariance functions efficiently.
problem Estimating isotropic covariance functions without assuming a specific parametric form.
method Uses Bernstein polynomials and sieve maximum likelihood estimation.
result Consistent estimator with improved performance compared to parametric and nonparametric alternatives.
Nonparametric neural-network estimation of current-status data
problem Estimation of conditional cumulative distribution function with current-status data
method Neural-network sieve maximum likelihood estimator
result Explicit convergence rate for Hölder smoothness
Nonparametric density deconvolution and denoising using simulation-based inference
problem Learning latent signals and their distributions in the presence of measurement noise
method Convolutional maximum mean discrepancy (convMMD) loss and likelihood-free framework
result Learn a latent generative model matching observed data distribution
A test for neural networks identifies genetic associations.
problem Testing complex associations in neural networks.
method Sieve quasi-likelihood ratio test for neural networks with one hidden layer.
result The test statistic has an asymptotic chi-squared distribution.
Develops asymptotic theory for adversarial estimators.
problem Estimating unknown functions in machine learning and econometrics.
method Derives convergence rates and normality of A-estimators under various conditions.
result Normality of neural-net M-estimators, overcoming previous technical issues.
Paper develops methods for inference on time series data using neural networks and sieves.
problem Inference on time series data with nonparametric conditional moment restrictions.
method GN-QLR based inference using general nonlinear sieves and multilayer neural networks.
result Optimally weighted GN-QLR statistic is asymptotically Chi-square distributed.
Study nonparametric density estimation via measure transport, achieving optimal rates.
problem Nonparametric density estimation with optimal rates.
method Measure transport, penalized maximum likelihood, and sieved wavelet estimators.
result Achieve minimax optimal convergence rates over Hölder classes of densities.
We study likelihood-based methods for distribution regression with deep generative models.
problem Distribution regression with high-dimensional responses concentrated on a low-dimensional manifold.
method Likelihood-based approach using conditional deep generative models.
result Convergence rates for estimating conditional distributions in Hellinger and Wasserstein metrics.
New method constructs synthetic treatment groups without mean exchangeability assumption.
problem Violations of mean exchangeability assumption in randomized controlled trials.
method Weighted mixture of treatment groups from source populations, minimizing conditional maximum mean discrepancy.
result Asymptotic normality of synthetic treatment group estimator established.
CORES2 removes noisy labels by sieving out corrupted examples.
problem Instance-dependent label noise degrades DNN performance.
method CORES2 (COnfidence REgularized Sample Sieve) progressively sieves out corrupted examples.
result CORES2 provides theoretical guarantees for filtering out corrupted examples.
Maximum likelihood estimation fails to be well-posed in Gaussian process regression.
problem Establishing well-posedness of maximum likelihood estimation in Gaussian process regression.
method Analyzing the conditions under which maximum likelihood estimation is not Lipschitz in the data with respect to the Hellinger distance.
result Maximum likelihood estimation is not well-posed in the noiseless data setting for any Gaussian process with a stationary covariance function whose lengthscale parameter is estimated using maximum likelihood.
We develop a novel procedure for constructing confidence bands for components of a sparse additive model. Our procedure is based on a new kernel-sieve hybrid estimator that combines two most popular nonparametric estimation methods in the literature, the kernel regression and the spline method, and is of interest in it…
Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explici…
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
Corrects pseudo log-likelihood method issues in various applications.
problem Log-likelihood function unbounded issues in pseudo log-likelihood methods.
method Provided a counterexample and corrected algorithms in previous literature.
result Ensured well-definedness of maximum pseudo log-likelihood estimation.
AR-Sieve Bootstrap improves Random Forest time series prediction accuracy.
problem Inaccurate time series prediction due to inadequate resampling methods.
method Combines Random Forest with AR-Sieve Bootstrap for better resampling.
result AR-Sieve Bootstrap leads to more accurate predictions compared to other methods.
We introduce a new framework for unsupervised learning of representations based on a novel hierarchical decomposition of information. Intuitively, data is passed through a series of progressively fine-grained sieves. Each layer of the sieve recovers a single latent factor that is maximally informative about multivariat…
Graphical lasso may fail to fit models when data points are insufficient.
problem When does graphical lasso fail to select and fit a graphical model?
method Computational experiments with graphical lasso.
result Graphical lasso may fail when the number of data points is less than the maximum likelihood threshold.
Develops deep generative models for stratified learning.
problem Challenges in learning distributions on stratified spaces with varying dimensions and singularities.
method Two generative frameworks: sieve maximum likelihood and diffusion-based.
result Establishes convergence rates and consistency for estimating intrinsic dimensions and number of strata.
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…
A new method speeds up quantum state estimation.
problem Exponential growth in sample size and dimension for quantum state tomography.
method Stochastic mirror descent with Burg entropy.
result Optimization error vanishes at a O ( ( 1 / t ) d log t ) O (\sqrt{ ( 1 / t ) d \log t }) O ( ( 1/ t ) d log t ) rate. Invertibility conditions for observation-driven time series models often fail to be guaranteed in empirical applications. As a result, the asymptotic theory of maximum likelihood and quasi-maximum likelihood estimators may be compromised. We derive considerably weaker conditions that can be used in practice to ensure t…
Proposes a new approach to approximate maximum likelihood for complex models.
problem Intractable likelihood functions in complex parametric models.
method Simulation-based constrained approximation to the structural model.
result Estimators nearly as efficient as maximum likelihood, feasible in many cases.
Paper optimizes estimation of quadratic functionals in nonparametric IV models.
problem Optimal estimation of a nonlinear functional in ill-posed inverse regression.
method Adaptive, minimax estimation using leave-one-out, sieve NPIV estimator with data-driven sieve dimension selection.
result Adaptive estimator achieves minimax optimal rate in various ill-posed cases.
This paper improves normalizing flows by combining MLE and sliced-Wasserstein distance for better data fidelity.
problem Normalizing flows struggle with generating realistic data and detecting out-of-distribution data.
method Proposes a hybrid objective function combining MLE and sliced-Wasserstein distance.
result Shows better generative abilities and lower likelihood of out-of-distribution data.
Machine learning should incorporate maximum likelihood for better estimation.
problem Lack of rigorous foundational theory in machine learning.
method Integrate maximum likelihood estimation into machine learning models.
result Foundationally rigorous machine learning models have greater practical impact.
Proposes a guaranteed regularization method for maximum likelihood estimation using gauge symmetry in Kullback-Leibler divergence.
problem Overfitting in maximum likelihood estimation.
method Introduces a regularization approach based on gauge symmetry in Kullback-Leibler divergence.
result The method provides a theoretically guaranteed optimal model without frequent hyperparameter tuning.
We propose a robust estimator to improve maximum likelihood in probabilistic models.
problem Overfitting and sensitivity to noise in maximum likelihood estimation.
method Distributionally robust maximum likelihood estimator that minimizes worst-case expected log-loss.
result The robust estimator is statistically consistent and performs well in regression and classification tasks.
Geometric approach solves maximum likelihood for Cauchy-like distributions.
problem Estimating center and scatter robustly from heavy-tailed data.
method Geodesic convexity and symmetry spaces of noncompact type.
result Efficient numerical solution for robust estimates of location and spread.
Proposes a deep learning framework for interval-censored survival data.
problem Lack of deep learning methods for interval-censored survival data.
method Partially linear transformation models with DNN approximations for nonlinear effects.
result DNN estimator achieves minimax-optimal convergence and superior performance.
New particle algorithms optimize latent variable models.
problem Optimizing latent variable models for maximum likelihood estimation.
method Identify gradient flows associated with free energy functional and discretize them to create particle-based algorithms.
result Novel particle algorithms scale to high-dimensional settings and perform well in experiments.
This work investigates training infinite mixtures with maximum likelihood for improved uncertainty quantification.
problem Improving uncertainty quantification in neural networks.
method Investigates training infinite mixtures with maximum likelihood instead of variational inference.
result The proposed method leads to stochastic networks with increased predictive variance, improved robustness, and higher entropy on out-of-distribution data.
New method estimates latent gene expression factors without overlap with known confounders.
problem Estimating latent variance components in gene expression data with known confounders.
method Restricted maximum-likelihood method maximizing likelihood on orthogonal subspace.
result Method reduces runtime and attains greater likelihood values than gradient-based optimizers.
Bayesian networks with latent variables are characterized and their likelihoods compared.
problem Characterizing and comparing likelihoods of Bayesian networks with latent variables.
method Characterized likelihood function and empirical Bayesian network. Proved dominance of global maximum likelihood from empirical model.
result The global maximum likelihood of the original Bayesian network is attained if and only if parameters are consistent with empirical model.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
Paper quantifies label shift robustly.
problem Quantifying label shift in datasets.
method Robust estimators of label distribution.
result Maximum Likelihood Estimator is a robust estimator.
Paper presents a method for estimating Hawkes process parameters.
problem Estimating parameters of Hawkes processes with self-excitation or inhibition.
method Maximum likelihood estimation for Hawkes processes with self-excitation or inhibition.
result The proposed estimator provides more accurate estimations in the inhibition context.
Geodesic descent optimizes likelihood in dually flat spaces.
problem Maximum likelihood estimation in exponential families.
method m-geodesic and e-geodesic updates on dually flat spaces.
result Geodesic updates can reach maximum likelihood estimator in one step.
This work explores maximum likelihood optimization of neural networks through hypernetworks. A hypernetwork initializes the weights of another network, which in turn can be employed for typical functional tasks such as regression and classification. We optimize hypernetworks to directly maximize the conditional likelih…
New approach resolves ambiguity in PPCA model's maximum likelihood estimation.
problem Ambiguity in maximum likelihood estimation of PPCA model due to rotational symmetry.
method Using quotient topological spaces, the approach resolves ambiguity and shows consistency of the maximum likelihood solution.
result Maximum likelihood solution is consistent in an appropriate quotient Euclidean space.
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
A boosting method improves nonparametric density estimation without smoothing assumptions.
problem Overfitting in nonparametric data fitting.
method Introduces a boosting algorithm for univariate nonparametric maximum likelihood estimation.
result Demonstrates the effectiveness of the boosting approach through simulations and real data experiments.
A new method improves text generation quality and diversity.
problem Exposure bias in Maximum Likelihood Estimation for text generation.
method ψ-MLE, a new training scheme based on density ratio estimation.
result ψ-MLE outperforms Maximum Likelihood Estimation and other models in text generation quality and diversity.
Investigates numerical issues in GP interpolation parameter estimation.
problem Numerical issues in maximum likelihood parameter estimation for Gaussian process interpolation.
method Investigates and proposes strategies to improve open-source software implementations.
result Improves reliability and reproducibility of studies relying on GP implementations.
We improve maximum likelihood for location estimation in finite samples.
problem Estimating a parameter from samples with unknown or varying distribution.
method Use smoothed Fisher information for finite sample size and varying distributions.
result Recover optimal estimation theory for finite n n n and arbitrary f f f . Connectionist temporal classification (CTC) is widely used for maximum likelihood learning in end-to-end speech recognition models. However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e.g., word error rate (WER). This results in a mismatch…
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving …
Label shift refers to the phenomenon where the prior class probability p(y) changes between the training and test distributions, while the conditional probability p(x|y) stays fixed. Label shift arises in settings like medical diagnosis, where a classifier trained to predict disease given symptoms must be adapted to sc…