Paper proposes a new method to predict partial rankings from crowdsourced data.
problem Ambiguity in pairwise comparisons leads to incomplete rankings, requiring a better method.
method Margin-based Maximum Likelihood Estimate (MLE) framework for probabilistic partial order learning.
result The proposed MLE method improves accuracy over traditional algorithms.
Consider a classification problem where we have both labeled and unlabeled data available. We show that for linear classifiers defined by convex margin-based surrogate losses that are decreasing, it is impossible to construct any semi-supervised approach that is able to guarantee an improvement over the supervised clas…
New margin-based learning guarantees improve generalization bounds.
problem Improving generalization bounds for machine learning models.
method Relative deviation margin bounds using empirical margin loss and Rademacher complexity.
result Distribution-dependent generalization bounds for unbounded loss functions.
New method prevents class collapse in metric learning with margin-based losses.
problem Class collapse in metric learning due to diverse intra-class samples.
method Proposed a sampling method to select nearest same-class samples as positive elements in tuple.
result Demonstrated clear benefits on various fine-grained image retrieval datasets.
IMMIGRATE selects features with interaction terms using margin-based weights.
problem Unclear differentiation of feature interactions from marginal effects.
method Includes and trains weights for interaction terms, applies large margin principle, considers robustness and local/global information.
result Achieves state-of-the-art results on several tasks.
MLE and CVE are equivalent under exponential families, leading to faster and more stable EM algorithms.
problem Finding maximum likelihood estimators (MLE) efficiently and stably.
method Proved equivalence between MLE and CVE under exponential families, leading to an EM algorithm.
result EM algorithm achieves the same asymptotic variance as MLE and is faster and more stable.
Improves GATs by adding margin-based constraints to prevent over-fitting and over-smoothing.
problem Over-fitting and over-smoothing in GATs.
method Margin-based constraints on attention weights and graph structure.
result Significant improvements over previous GATs on various datasets.
We describe k-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expec…
Regularized MLE improves MoE models for high-dimensional data.
problem Modeling with high-dimensional predictors and feature selection.
method Gaussian gating network, ℓ1-regularized MLE, EM-Lasso algorithm, BIC-like criterion. result Regularized MLE outperforms standard MLE in clustering and regression tasks.
The paper analyzes RLHF with human feedback and provides convergence results for MLE and pessimistic MLE.
problem Improving RLHF with human feedback from pairwise or K-wise comparisons. method Theoretical framework for RLHF with convergence analysis of MLE and pessimistic MLE.
result MLE fails but pessimistic MLE provides improved policies under certain coverage assumptions.
The paper shows a phase transition for the existence of MLE in high-dimensional logistic regression.
problem The existence of the maximum likelihood estimate in high-dimensional logistic regression models.
method Established a phase transition boundary curve hextMLE parameterized by scalars measuring the magnitude of regression coefficients. result The existence of the MLE in high-dimensional logistic regression models undergoes a sharp phase transition.
New margin-based regularization and selective sampling improve deep neural network performance.
problem Improving deep neural network performance on various classification tasks.
method Multi-margin regularization (MMR) and minimal margin score (MMS) for selective sampling.
result Improved results on multiple classification tasks across domains.
New bound on neural network generalization error using geometric complexity.
problem Understanding the generalization capabilities of deep neural networks.
method Derive a new upper bound on generalization error using margin-normalized geometric complexity.
result Empirical validation of the bound for ResNet-18 on CIFAR-10 and CIFAR-100 datasets.
Improved MLE for Hawkes Processes stabilizes unstable optimization.
problem Unstable Maximum Likelihood Estimation (MLE) for Hawkes Processes.
method Simple stabilization procedure to improve MLE without restrictive assumptions.
result Stabilized MLE outperforms traditional methods over various sequence lengths.
New algorithms and bounds for contextual bandits using surrogate losses.
problem Efficiently solving contextual bandit problems with margin-based regret bounds.
method Use of surrogate losses (ramp and hinge) to derive new regret bounds and algorithms.
result Derives new margin-based regret bounds and efficient algorithms for contextual bandits.
New anomaly estimator reduces bias in MLE for normally distributed data.
problem Bias in Maximum Likelihood Estimation of structured anomalies.
method Derive a new anomaly estimator using a mixture model.
result New estimator is asymptotically unbiased regardless of anomaly family size.
Analyzes large-margin classifiers under high-dimensional data.
problem Selecting the best classifier among various margin-based methods.
method Investigates asymptotic performance of large-margin classifiers under two component mixture models.
result Analytical results closely match with Monte Carlo simulations.
Paper establishes MLE consistency for market microstructure models.
problem Estimating parameters in partially observed diffusion models.
method Tractable sufficient condition for MLE consistency based on stationary distribution.
result Maximum likelihood estimators are consistent for market microstructure parameters.
The Chirikov standard map and the 2D Froeschlé map are investigated. A few thousand values of the Hurst exponent (HE) and the maximal Lyapunov exponent (mLE) are plotted in a mixed space of the nonlinear parameter versus the initial condition. Both characteristic exponents reveal remarkably similar structures in this s…
We present a simple noise-robust margin-based active learning algorithm to find homogeneous (passing the origin) linear separators and analyze its error convergence when labels are corrupted by noise. We show that when the imposed noise satisfies the Tsybakov low noise condition (Mammen, Tsybakov, and others 1999; Tsyb…
Develops active learning method for linear optimization with margin-based criterion.
problem Optimizing decisions in linear optimization problems with limited labeled data.
method Smart Predict-then-Optimize (SPO) loss and margin-based active learning algorithm.
result Algorithm achieves significantly fewer labels than naive supervised learning, especially for minimizing SPO loss.
The paper improves SVM margin-based generalization bounds.
problem Improving generalization bounds for SVMs.
method Revisiting and improving classic generalization bounds in terms of margins, complementing with a nearly matching lower bound.
result Almost settles the generalization performance of SVMs in terms of margins.
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Study generalization of voting classifiers using margin-based bounds.
problem Understanding the generalization of ensemble classifiers like voting.
method Proved margin-based generalization bounds using PAC-Bayes theory and Dirichlet posteriors.
result Provided state-of-the-art guarantees on classification tasks.
New estimators improve Rasch model item parameter estimation for sparse data.
problem Estimating item parameters in sparse Rasch model data.
method Random pairing maximum likelihood estimator (RP-MLE) and its bootstrapped variant (MRP-MLE).
result RP-MLE and MRP-MLE are minimax optimal and provide precise item parameter estimates.
We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations…
Paper studies statistical properties of DP data synthesis algorithms based on Bayesian networks.
problem Ensuring differential privacy in synthetic data generation for high-dimensional data.
method Introduces random noise to low-dimensional marginals of a probabilistic graphical model (BN) to achieve differential privacy.
result Establishes a rigorous accuracy guarantee for BN-based DP synthetic data generators using total variation (TV) distance.
Study on MLE growth rate for stable CIR process, proving consistency and normality.
problem Estimating the growth rate of a stable CIR process from continuous observations.
method Maximum likelihood estimation for a specific type of process.
result Strong consistency and asymptotic normality in subcritical and supercritical cases, asymptotic mixed normality in supercritical, open in critical case.
Operational risk models commonly employ maximum likelihood estimation (MLE) to fit loss data to heavy-tailed distributions. Yet several desirable properties of MLE (e.g. asymptotic normality) are generally valid only for large sample-sizes, a situation rarely encountered in operational risk. In this paper, we study how…
This paper explores the preference-based top-K rank aggregation problem. Suppose that a collection of items is repeatedly compared in pairs, and one wishes to recover a consistent ordering that emphasizes the top-K ranked items, based on partially revealed preferences. We focus on the Bradley-Terry-Luce (BTL) model…
New margin bound improves generalization for voting classifiers.
problem Improving generalization bounds for voting classifiers.
method Established a new margin-based generalization bound.
result Derives an optimal weak-to-strong learner with matching theoretical lower bound.
Advocates for MLE in regression and forecasting for better inductive biases and post-hoc optimization.
problem Designing effective loss functions for regression and forecasting.
method Maximum Likelihood Estimation (MLE) approach for regression and forecasting.
result MLE approach outperforms direct empirical risk minimization under certain conditions and for various datasets.
This letter proposes a low-computational Bayesian algorithm for noisy sparse recovery in the context of one bit compressed sensing with sensing matrix perturbation. The proposed algorithm which is called BHT-MLE comprises a sparse support detector and an amplitude estimator. The support detector utilizes Bayesian hypot…
Optimized deferral improves accuracy in imbalanced settings.
problem Imbalance in expert predictions leads to suboptimal performance in two-stage learning to defer.
method Developed novel cost-sensitive learning algorithms and margin-based loss functions tailored for expert imbalance.
result MILD algorithm shows clear improvements over baselines in image classification and LLM routing tasks.
Improved fairness in pairwise comparisons using MLE with a simple modification.
problem Fairness in pairwise comparisons using MLE is suboptimal.
method Proposed a simple modification to MLE to improve fairness (bias) without sacrificing accuracy.
result Improved rate in bias while maintaining minimax-optimality in mean squared error.
Develops new Markov processes with switching rates and past dependence.
problem Modeling processes with dynamic switching rates and path dependence.
method Introduces a new class of Markov jump processes with regime switching and path dependence. Derives distributional properties and maximum likelihood estimates.
result Maximum likelihood estimates of the process parameters are derived in closed form and have asymptotic normality.
Proposes MLEs for MMJDM with EM-algorithm.
problem Estimating stock prices with varying drift and volatility.
method EM-algorithm for MLEs of MMJDM.
result Validated with simulated data and fitted to Amazon and Netflix stock prices.
DMLE improves active learning by correcting MLE for sample dependencies.
problem Dependencies among samples in active learning affect model parameter estimation.
method Dependency-aware Maximum Likelihood Estimation (DMLE).
result DMLE achieves superior performance across multiple benchmark datasets.
Adversarial dynamics embedding improves MLE of exponential family models.
problem Maximum likelihood estimation of exponential family models with neural network parametrization.
method Adversarial dynamics embedding to estimate the dual sampler and primal model simultaneously.
result Adversarial dynamics embedding leads to more effective learning and improved estimators compared to existing methods.
The paper proposes a tree model for interval-valued regression.
problem Learning a real-valued function from interval-valued data.
method Minimizing a margin-based discriminative objective function using a tree structure and dynamic programming.
result The proposed algorithm achieves state-of-the-art speed and accuracy.
MLE works best for covariate shift without modifications.
problem OOD generalization under covariate shift.
method Maximum Likelihood Estimation (MLE) without modifications.
result MLE achieves minimax optimality for covariate shift under well-specified setting.
Paper explores Elliptical Wishart distributions in signal processing and machine learning.
problem Estimating parameters of Elliptical Wishart distributions.
method Proposes fixed point and Riemannian optimization algorithms for maximum likelihood estimation.
result Characterizes existence, uniqueness, and convergence of the MLE.
Improved convergence rates for MLE in mixture models using penalized log-likelihood.
problem Convergence rates for MLE in finite mixture models.
method Penalizing log-likelihood to discourage vanishing mixing weights, using Wasserstein distance and new loss functions.
result Improved convergence rates for some mixture components, faster than traditional methods.
Unified detector calibration and simulation using MLE from generative models.
problem Combining detector calibration and simulation using traditional methods.
method Maximum likelihood estimation from conditional generative models.
result Prior-independent and non-Gaussian resolutions possible.
Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important. We study a simple communication-efficient learning framework that first calculates the local maximum likelihood estimates (MLE) based on the data subsets, and then combines the local MLEs t…
New study shows MLE can avoid model collapse with gradual synthetic data addition.
problem Model collapse in generative models trained on synthetic data.
method Theoretical study of maximum likelihood estimation (MLE) under iterative training with accumulating synthetic data.
result Non-asymptotic bounds show MLE can avoid model collapse even as real data fraction vanishes.
A new ranking model with dynamic covariates improves statistical analysis.
problem Statistical ranking with varying covariates across comparisons.
method Introduced a Plackett--Luce framework for covariate-assisted ranking, providing conditions for model identifiability and MLE existence, and developing an alternating maximization algorithm.
result Uniform consistency of the Maximum Likelihood Estimation (MLE) under suitable assumptions on graph design and covariates.
A fast method for estimating radar amplitude density parameters.
problem Accurate estimation of amplitude density function parameters in radar applications.
method Projecting amplitude data onto horizontal and vertical axes, then using MLE for α-stale distribution parameters. result The average of computed MLEs based on two projections is a fast and accurate estimator for amplitude distribution parameters.