New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Deep learning improves Bayes factor computation for likelihood-free models.
problem Computing Bayes factors for likelihood-free models is challenging.
method Proposes a deep learning estimator of Bayes factors using simulated data.
result Establishes consistency of the Deep Bayes Factor estimator.
Neural Empirical Bayes estimates source distributions from noisy simulations.
problem Estimating source distributions from noisy, simulated data.
method Uses neural density estimators to estimate a prior or source distribution over uncorrupted samples, then performs posterior inference.
result Recovering ground truth source distributions up to symmetries.
Empirical Bayes rates via variational approximations and prior decomposition.
problem Nonparametric and high-dimensional inference convergence rates.
method Variational perspective and prior decomposition.
result Empirical Bayes posterior rates derived from variational Bayes.
Naive Bayes estimator is widely used in text classification problems. However, it doesn't perform well with small-size training dataset. We propose a new method based on Naive Bayes estimator to solve this problem. A correlation factor is introduced to incorporate the correlation among different classes. Experimental r…
Researchers estimate optimal PAC-Bayes bounds using Hamiltonian Monte Carlo.
problem Estimating tight PAC-Bayes bounds with restricted posterior families.
method Sampling from optimal Gibbs posterior using Hamiltonian Monte Carlo, estimating KL divergence, and proposing high-probability bounds.
result Significant tightness gaps in PAC-Bayes bounds, up to 5-6% in some cases.
A new method estimates Bayes error for deep networks, suggesting they may have reached the limit.
problem Evaluating the performance of deep learning models and detecting overfitting.
method A simple and direct Bayes error estimator based on uncertainty of class assignments.
result Deep networks may have reached the Bayes error limit for benchmark datasets.
Graph neural networks extend neural Bayes estimators to irregular spatial data.
problem Estimating parameters from irregular spatial data with computational efficiency.
method Employing graph neural networks to approximate Bayes estimators for irregular spatial data.
result Extending neural Bayes estimation to irregular spatial data with computational benefits.
New method for high-dimensional linear regression using empirical Bayes.
problem Estimating prior in high-dimensional linear regression.
method Variational empirical Bayes approach with NPMLE and mean field approximation.
result Established asymptotic consistency and computational efficiency of the method.
New method reduces computational cost for estimating PAC-Bayes bounds.
problem High computational cost in estimating PAC-Bayes bounds.
method General alternative method that makes computational savings.
result Reduces computational cost on the order of the dataset size.
We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in d-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accurac…
A new sequential method estimates Poisson means in streaming data, achieving optimality and efficiency.
problem Estimating Poisson means in a streaming, or online, framework.
method A quasi-Bayesian approach based on Newton's algorithm for a sequential estimate.
result Established frequentist guarantees including consistency and asymptotic optimality.
We address the problem of learning to benchmark the best achievable classifier performance. In this problem the objective is to establish statistically consistent estimates of the Bayes misclassification error rate without having to learn a Bayes-optimal classifier. Our learning to benchmark framework improves on previ…
Neural Bayes methods simplify fitting complex bivariate extremal models.
problem Inference on complex multivariate extremal dependence models with computationally expensive likelihood functions.
method Use neural networks to approximate Bayes estimators and classifiers for model selection.
result Proposed neural Bayes methods enable routine implementation of complex extreme-value dependence models.
The paper analyzes high-dimensional linear regression using parametric empirical Bayes methods.
problem Estimation of i.i.d. priors in high-dimensional Bayesian linear regression with random design.
method Parametric empirical Bayes estimation, variational lower bound maximization, phase transition analysis.
result The vEB estimator is information theoretically optimal up to p=o(n2/3) but sub-optimal in higher dimensions. Unified approach improves accuracy in private estimation.
problem Protecting sensitive information in statistical estimation.
method Unified Bayes Private Point (UBaPP) approach to Bayes point estimation under differential privacy constraints.
result Achieves better accuracy-privacy trade-off than traditional methods.
EB-PCA reduces noise in high-dimensional PCA by estimating a joint prior distribution.
problem High-dimensional PCA noise in samples comparable to or larger than data.
method Empirical Bayes PCA using Kiefer-Wolfowitz MLE, random matrix theory, and AMP algorithm.
result EB-PCA achieves Bayes-optimal accuracy in spiked models and significantly improves over PCA in simulations and real data.
Efficient neural Bayes estimators for censored peaks-over-threshold models improve inference speed and accuracy.
problem Computational burden in inference with spatial extremal dependence models due to intractable or censored likelihoods.
method Developed neural Bayes estimators using data augmentation techniques to encode censoring information.
result Significant gains in computational and statistical efficiency compared to traditional methods.
In data science and machine learning, hierarchical parametric models, such as mixture models, are often used. They contain two kinds of variables: observable variables, which represent the parts of the data that can be directly measured, and latent variables, which represent the underlying processes that generate the d…
Paper estimates FPR of Bayes classifier using soft labels.
problem Determining optimal classifier performance.
method Uses soft labels and denoising technique.
result Consistent and unbiased FPR estimator developed.
We propose an empirical Bayes estimator based on Dirichlet process mixture model for estimating the sparse normalized mean difference, which could be directly applied to the high dimensional linear classification. In theory, we build a bridge to connect the estimation error of the mean difference and the misclassificat…
Paper extends Bayes Theorem for interval probability estimates.
problem Real-world input probabilities are often interval estimates, not precise.
method Developed IT2 version of Bayes Theorem and a novel algorithm for encoding intervals.
result Conservative method avoids invalid output results from inconsistent input.
Improved model-based estimation through tempered Bayes filter.
problem Improving predictive accuracy in partially-observable stochastic systems.
method Developed tempered Bayes filter combining likelihood and full posterior tempering.
result Tempered Bayes filter achieves improved predictive performance over the Bayes filter baseline.
We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…
Unified empirical and variational Bayes for unnormalized densities.
problem Approximating unnormalized densities using latent variable models.
method Formulate a latent variable model for Y=X+N(0,σ2Id), use ELBO as parametrization of Y's energy function, and estimate X with empirical Bayes least-squares. result UVB has higher capacity to approximate energy functions than MLPs in DEEN.
New bound improves on weighted majority vote risk estimation.
problem Improving risk estimation for weighted majority vote.
method Novel Chebyshev-Cantelli inequality and PAC-Bayes-Bennett inequality.
result New bounds improve on existing methods.
In this study, we consider an empirical Bayes method for Boltzmann machines and propose an algorithm for it. The empirical Bayes method allows estimation of the values of the hyperparameters of the Boltzmann machine by maximizing a specific likelihood function referred to as the empirical Bayes likelihood function in t…
PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our …
Deep neural networks are optimal for dependent data using PAC-Bayes bounds.
problem Optimizing deep neural networks for dependent data.
method PAC-Bayes oracle inequalities and Bernstein inequality.
result Upper and lower bounds match, proving minimax optimality.
New PAC-Bayes meta-learning method improves few-shot learning accuracy and calibration.
problem Few-shot learning with limited data.
method PAC-Bayes framework extended to meta-learning, estimating task-specific posteriors.
result State-of-the-art calibration and classification results on benchmarks.
This paper compares classical parametric methods with recently developed Bayesian methods for system identification. A Full Bayes solution is considered together with one of the standard approximations based on the Empirical Bayes paradigm. Results regarding point estimators for the impulse response as well as for conf…
We rebias estimates to improve interval calibration and prediction accuracy.
problem Constructing accurate intervals for noisy and biased estimates.
method Empirical Bayes rebiasing strategy that learns bias distribution from data.
result Substantial precision gains in prediction-powered inference.
Enhances normal mean estimation with side info using NIT approach.
problem Compound estimation of normal means with side information.
method Empirical Bayes, nonparametric integrative Tweedie (NIT) approach.
result NIT approach improves estimation risk and convergence rate with increasing auxiliary data.
New method predicts binary matrix entries using empirical Bayes and low-rank structure.
problem Predicting unobserved entries in binary matrices.
method Empirical Bayes method motivated by Efron--Morris estimator, exploiting low-rank structure.
result Superior performance in predictive accuracy, calibration, and efficiency compared to existing methods.
NBE method speeds up Lévy process parameter estimation.
problem Challenging parameter estimation for Lévy processes with unavailable or costly likelihoods.
method Neural Bayes estimation (NBE) framework using permutation-invariant neural networks.
result NBE provides accurate and consistent estimators with reduced runtime.
Transformers solve Poisson means estimation via empirical Bayes.
problem Estimating Poisson means under empirical Bayes setting.
method Pre-trained transformer learns to adapt to unknown prior and do in-context learning.
result Transformers achieve vanishing regret with large models and outperform classical algorithms.
New estimator reduces risk in slate bandits by leveraging Bayes risk criterion.
problem Evaluating slate policies using logged data when policies factorize over slots.
method Developed a new estimator using a control variate approach, showing risk improvement over existing methods.
result The new estimator has lower risk than the pseudoinverse estimator in slate bandit problems.
Smart Bayes integrates generative and discriminative features for improved classification.
problem Improving classification performance by combining generative and discriminative modeling.
method Integrates generative likelihood-ratio features into a logistic-regression-style classifier.
result Often outperforms logistic regression and Naive Bayes in simulations and real data.
Proposes a non-convex optimization method for a parsimonious weighted naive Bayes classifier.
problem Improving naïve Bayes classifier performance with a large number of input variables.
method Sparse regularization of model log-likelihood for direct estimation of variable weights.
result Optimization-based weighted naïve Bayes classifiers achieve equivalent performance to averaging-based classifiers.
Paper tightens PAC-Bayes bounds using coin-betting for better estimates.
problem Estimating mean of random elements with possibly S-dependent parameters.
method Refined PAC-Bayes proof strategy based on coin-betting framework.
result Derives tighter concentration inequalities for all sample sizes.
New method uses SURE to denoise signals, outperforming NPMLE.
problem Learning to optimally denoise signals corrupted by Gaussian noise.
method Hyvärinen's score matching (SM) is shown equivalent to SURE minimization.
result SURE achieves nearly parametric rates of convergence in empirical Bayes settings.
This work bounds classification error in machine learning for low Bayes error conditions.
problem Understanding the error mismatch between Bayes error and model-based classification error.
method Applying classification error bounds to study the relationship with Kullback-Leibler divergence and proposing a linear approximation for low Bayes error conditions.
result A linear approximation of the classification error bound for low Bayes error conditions is proposed.
Paper extends transfer learning for decision rules, improving treatment rule estimation.
problem Estimating optimal individualized treatment rules under changing conditions.
method Bayes decision rules and low-dimensional empirical risk minimization.
result Consistent estimators and risk bounds established under mild conditions.
Recently, a number of mostly ℓ1-norm regularized least squares type deterministic algorithms have been proposed to address the problem of \emph{sparse} adaptive signal estimation and system identification. From a Bayesian perspective, this task is equivalent to maximum a posteriori probability estimation under a …
Quantum method speeds up VB estimation in machine learning.
problem Prohibitively expensive natural gradient in high dimensions.
method Regression-based natural gradient estimation with quantum matrix inversion.
result Quantum method enables efficient VB estimation.
Adversarial meta-learning computes Gamma-minimax estimators for vague prior knowledge.
problem Estimating parameters with vague prior knowledge.
method Adversarial meta-learning algorithms for Gamma-minimax estimators.
result Convergence guarantees and neural network class for selection.
New insights into empirical Bayes and compound decision problems with improved regret bounds.
problem Estimating means of normally or Poisson distributed vectors under squared loss.
method Combines Bayesian and frequentist approaches using data-driven estimators.
result Optimal regret bounds for Poisson and normal mean models, resolving conjectures.
Generative models learn smoother densities to sample from unknown distributions.
problem Sampling from unknown distributions in high-dimensional spaces.
method Formalizes sampling problem, introduces multimeasurement noise model, derives Bayes estimator, and uses underdamped Langevin MCMC.
result Formulation leads to efficient sampling methods and theoretical connections with denoising autoencoders.