Regularized MLE improves MoE models for high-dimensional data.
problem Modeling with high-dimensional predictors and feature selection.
method Gaussian gating network, ℓ1-regularized MLE, EM-Lasso algorithm, BIC-like criterion. result Regularized MLE outperforms standard MLE in clustering and regression tasks.
The paper develops methods for high-dimensional inference in Markov random fields.
problem Statistical inference for high-dimensional Markov random fields.
method Markov Chain Monte Carlo Maximum Likelihood Estimation (MCMC-MLE) with Elastic-net regularization.
result The proposed methods achieve ℓ1-consistency and false discovery rate control. The paper strengthens the classical result of MLE convergence to a Gaussian distribution.
problem The classical result of MLE convergence to a Gaussian distribution.
method Sub-Gaussian concentration and entropic normality of the normalized MLE.
result Entropic central limit theorem for a smoothed version of the estimator.
We consider the class of optimization problems arising from computationally intensive L1-regularized M-estimators, where the function or gradient values are very expensive to compute. A particular instance of interest is the L1-regularized MLE for learning Conditional Random Fields (CRFs), which are a popular class of …
The L1-regularized Gaussian maximum likelihood estimator (MLE) has been shown to have strong statistical guarantees in recovering a sparse inverse covariance matrix, or alternatively the underlying graph structure of a Gaussian Markov Random Field, from very limited samples. We propose a novel algorithm for solving the…
Maximum Likelihood Estimators (MLE) has many good properties. For example, the asymptotic variance of MLE solution attains equality of the asymptotic Cram{é}r-Rao lower bound (efficiency bound), which is the minimum possible variance for an unbiased estimator. However, obtaining such MLE solution requires calculating t…
The paper improves ranking by integrating covariates and sparse intrinsic scores.
problem Ranking items with incomplete preference scores explained by covariates.
method Extends BTL model with covariate information and sparse intrinsic scores, using penalized MLE.
result Developed debiased estimator for penalized MLE with distributional properties.
Paper proposes an alternative to MLE for GLMs with non-canonical link functions.
problem Challenges in MLE for GLMs with non-canonical link functions.
method Variational Inequality (VI) estimation framework.
result Established finite-sample error bounds and asymptotic normality for VI estimator.
Paper proposes MWDE for estimating finite location-scale mixtures.
problem Estimating finite location-scale mixtures using MLE is problematic.
method Investigates minimum Wasserstein distance estimators (MWDE).
result MWDE is consistent and provides a numerical solution.
MLE and CVE are equivalent under exponential families, leading to faster and more stable EM algorithms.
problem Finding maximum likelihood estimators (MLE) efficiently and stably.
method Proved equivalence between MLE and CVE under exponential families, leading to an EM algorithm.
result EM algorithm achieves the same asymptotic variance as MLE and is faster and more stable.
Estimates network structure from correlated node outputs of wide-sense stationary processes.
problem Learning edge connectivity from node outputs of latent inputs.
method Wide-sense stationary stochastic processes, Laplacian matrix estimation, ℓ1-regularized Whittle's MLE.
result The MLE recovers the sparsity pattern of the Laplacian matrix with high probability.
A new method connects GLM and MLE for neuroimaging analysis.
problem Limited mathematical elegance and interpretation of MLE for neuroimaging.
method Derives a refined statistical test using SVR-iGLM and RFT.
result MLE and GLM parameter estimations are significantly related to functional tasks.
We describe k-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expec…
Optimizes ranking of top-k players from partial comparison data.
problem Identifying the top-k players from incomplete pairwise comparisons.
method Maximum Likelihood Estimator (MLE) and Spectral Method.
result MLE achieves optimal partial and exact recovery, while Spectral Method is sub-optimal.
The paper analyzes RLHF with human feedback and provides convergence results for MLE and pessimistic MLE.
problem Improving RLHF with human feedback from pairwise or K-wise comparisons. method Theoretical framework for RLHF with convergence analysis of MLE and pessimistic MLE.
result MLE fails but pessimistic MLE provides improved policies under certain coverage assumptions.
This paper improves topic model estimation for sparse distributions and applies it to Wasserstein distances.
problem Estimating sparse topic distributions in topic models with high-dimensional data.
method MLE for topic weights when A is known, plug-in estimator for unknown A. result MLE can be exactly sparse and contain true zero pattern of topic weights.
This paper rigorously establishes that the existence of the maximum likelihood estimate (MLE) in high-dimensional logistic regression models with Gaussian covariates undergoes a sharp `phase transition'. We introduce an explicit boundary curve hMLE, parameterized by two scalars measuring the overall magnitu…
Improved MLE for Hawkes Processes stabilizes unstable optimization.
problem Unstable Maximum Likelihood Estimation (MLE) for Hawkes Processes.
method Simple stabilization procedure to improve MLE without restrictive assumptions.
result Stabilized MLE outperforms traditional methods over various sequence lengths.
Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.
problem Estimation of normal mean in multivariate settings with correlated observations.
method Approximate risk minimization over a functional class of shrinkage-thresholding rules.
result Unified estimator NOMAD for shrinkage, thresholding, and regularization.
Develops a functional mix-of-experts model for multiclass classification.
problem Multiclass classification with univariate functional predictors.
method Functional mix-of-experts model with regularization and sparsity constraints.
result Regularized maximum likelihood estimation yields interpretable coefficient functions.
The paper derives upper bounds on the MLE error for BTL model under general graphs.
problem Estimating the MLE of BTL model parameters with ℓ∞-loss under general graphs. method Novel upper bounds on ℓ∞ estimation error dependent on algebraic connectivity and graph topology. result Upper bounds on ℓ∞ error are sharp and match minimax lower bounds under certain graph topologies. This paper is concerned with the problem of top-K ranking from pairwise comparisons. Given a collection of n items and a few pairwise comparisons across them, one wishes to identify the set of K items that receive the highest ranks. To tackle this problem, we adopt the logistic parametric model --- the Bradley-Te…
CMLE reduces spurious correlations in deep models.
problem Spurious correlations in deep learning models.
method Counterfactual Maximum Likelihood Estimation (CMLE) on interventional distribution.
result CMLE outperforms regular MLE in out-of-domain generalization and spurious correlation reduction.
New anomaly estimator reduces bias in MLE for normally distributed data.
problem Bias in Maximum Likelihood Estimation of structured anomalies.
method Derive a new anomaly estimator using a mixture model.
result New estimator is asymptotically unbiased regardless of anomaly family size.
New approach uses inverse reinforcement learning to improve language model training.
problem Training large language models using imitation learning methods.
method Developed a new method of inverse reinforcement learning to optimize sequences directly.
result IRL-based fine-tuning leads to better performance and diversity in language generation.
Paper establishes MLE consistency for market microstructure models.
problem Estimating parameters in partially observed diffusion models.
method Tractable sufficient condition for MLE consistency based on stationary distribution.
result Maximum likelihood estimators are consistent for market microstructure parameters.
In this paper we consider the task of estimating the non-zero pattern of the sparse inverse covariance matrix of a zero-mean Gaussian random vector from a set of iid samples. Note that this is also equivalent to recovering the underlying graph structure of a sparse Gaussian Markov Random Field (GMRF). We present two no…
The Chirikov standard map and the 2D Froeschlé map are investigated. A few thousand values of the Hurst exponent (HE) and the maximal Lyapunov exponent (mLE) are plotted in a mixed space of the nonlinear parameter versus the initial condition. Both characteristic exponents reveal remarkably similar structures in this s…
Image-to-image networks speed up SAR model parameter estimation.
problem Computational infeasibility of MLE for large, non-stationary spatial fields.
method Used image-to-image networks to estimate SAR model parameters.
result Image-to-image networks enable faster and more accurate parameter estimation.
Simple Deep LDA models achieve accuracy competitive with softmax baselines.
problem Training Deep LDA models by maximum likelihood estimation leads to overlapping or collapsed class clusters.
method Proposed a constrained Deep LDA formulation with geometric constraints to fix class means and covariance.
result MLE becomes stable under geometric constraints, yielding well-separated class clusters.
We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations…
New estimators improve Rasch model item parameter estimation for sparse data.
problem Estimating item parameters in sparse Rasch model data.
method Random pairing maximum likelihood estimator (RP-MLE) and its bootstrapped variant (MRP-MLE).
result RP-MLE and MRP-MLE are minimax optimal and provide precise item parameter estimates.
Operational risk models commonly employ maximum likelihood estimation (MLE) to fit loss data to heavy-tailed distributions. Yet several desirable properties of MLE (e.g. asymptotic normality) are generally valid only for large sample-sizes, a situation rarely encountered in operational risk. In this paper, we study how…
This paper explores the preference-based top-K rank aggregation problem. Suppose that a collection of items is repeatedly compared in pairs, and one wishes to recover a consistent ordering that emphasizes the top-K ranked items, based on partially revealed preferences. We focus on the Bradley-Terry-Luce (BTL) model…
Advocates for MLE in regression and forecasting for better inductive biases and post-hoc optimization.
problem Designing effective loss functions for regression and forecasting.
method Maximum Likelihood Estimation (MLE) approach for regression and forecasting.
result MLE approach outperforms direct empirical risk minimization under certain conditions and for various datasets.
We study the estimation of β for the nonlinear model $y = f(X\sp{\top}β) + ε$ when f is a nonlinear transformation that is known, β has sparse nonzero coordinates, and the number of observations can be much smaller than that of parameters (n≪p). We show that in order to bound the L2 error of the L0 reg…
This letter proposes a low-computational Bayesian algorithm for noisy sparse recovery in the context of one bit compressed sensing with sensing matrix perturbation. The proposed algorithm which is called BHT-MLE comprises a sparse support detector and an amplitude estimator. The support detector utilizes Bayesian hypot…
Proposes MLEs for MMJDM with EM-algorithm.
problem Estimating stock prices with varying drift and volatility.
method EM-algorithm for MLEs of MMJDM.
result Validated with simulated data and fitted to Amazon and Netflix stock prices.
Develops new Markov processes with switching rates and past dependence.
problem Modeling processes with dynamic switching rates and path dependence.
method Introduces a new class of Markov jump processes with regime switching and path dependence. Derives distributional properties and maximum likelihood estimates.
result Maximum likelihood estimates of the process parameters are derived in closed form and have asymptotic normality.
DMLE improves active learning by correcting MLE for sample dependencies.
problem Dependencies among samples in active learning affect model parameter estimation.
method Dependency-aware Maximum Likelihood Estimation (DMLE).
result DMLE achieves superior performance across multiple benchmark datasets.
MLE works best for covariate shift without modifications.
problem OOD generalization under covariate shift.
method Maximum Likelihood Estimation (MLE) without modifications.
result MLE achieves minimax optimality for covariate shift under well-specified setting.
Paper explores Elliptical Wishart distributions in signal processing and machine learning.
problem Estimating parameters of Elliptical Wishart distributions.
method Proposes fixed point and Riemannian optimization algorithms for maximum likelihood estimation.
result Characterizes existence, uniqueness, and convergence of the MLE.
We consider a stable Cox--Ingersoll--Ross process driven by a standard Wiener process and a spectrally positive strictly stable Lévy process, and we study asymptotic properties of the maximum likelihood estimator (MLE) for its growth rate based on continuous time observations. We distinguish three cases: subcritical, c…
Improved convergence rates for MLE in mixture models using penalized log-likelihood.
problem Convergence rates for MLE in finite mixture models.
method Penalizing log-likelihood to discourage vanishing mixing weights, using Wasserstein distance and new loss functions.
result Improved convergence rates for some mixture components, faster than traditional methods.
Unified detector calibration and simulation using MLE from generative models.
problem Combining detector calibration and simulation using traditional methods.
method Maximum likelihood estimation from conditional generative models.
result Prior-independent and non-Gaussian resolutions possible.
Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important. We study a simple communication-efficient learning framework that first calculates the local maximum likelihood estimates (MLE) based on the data subsets, and then combines the local MLEs t…
A new ranking model with dynamic covariates improves statistical analysis.
problem Statistical ranking with varying covariates across comparisons.
method Introduced a Plackett--Luce framework for covariate-assisted ranking, providing conditions for model identifiability and MLE existence, and developing an alternating maximization algorithm.
result Uniform consistency of the Maximum Likelihood Estimation (MLE) under suitable assumptions on graph design and covariates.
New study shows MLE can avoid model collapse with gradual synthetic data addition.
problem Model collapse in generative models trained on synthetic data.
method Theoretical study of maximum likelihood estimation (MLE) under iterative training with accumulating synthetic data.
result Non-asymptotic bounds show MLE can avoid model collapse even as real data fraction vanishes.