Adversarial meta-learning computes Gamma-minimax estimators for vague prior knowledge.
problem Estimating parameters with vague prior knowledge.
method Adversarial meta-learning algorithms for Gamma-minimax estimators.
result Convergence guarantees and neural network class for selection.
Two EM algorithms estimate prior distributions in mixture of linear regressions.
problem Estimating prior distributions in mixture of linear regressions.
method Two EM algorithms: one for continuous priors, one for discrete priors.
result Both algorithms accurately estimate prior distributions and the number of clusters.
We consider a problem of learning a binary classifier only from positive data and unlabeled data (PU learning) and estimating the class-prior in unlabeled data under the case-control scenario. Most of the recent methods of PU learning require an estimate of the class-prior probability in unlabeled data, and it is estim…
Algorithm estimates graph structure with prior information and Langevin diffusion.
problem Support estimation of partially known Gaussian graphical models.
method Proposes an algorithm using annealed Langevin diffusion and graph neural networks to estimate the posterior distribution of the graph.
result Demonstrates the benefits of the approach through numerical experiments.
Framework expands particle filtering to estimate states beyond prior boundaries.
problem Limitations of traditional particle filtering in estimating states outside prior support.
method Diffusion-Enhanced Particle Filtering Framework with adaptive diffusion, entropy-driven regularisation, and kernel-based perturbations.
result Framework significantly improves state estimation accuracy and success rates for out-of-boundary targets.
Proposes new priors for neural networks to improve generalization and uncertainty.
problem Improving generalization and uncertainty estimation in neural networks.
method Exploits scalable and structured posteriors as priors with generalization guarantees.
result Improves generalization and uncertainty estimation with non-vacuous bounds.
Algorithm learns shared demand structure across dynamic pricing experiments.
problem Learning shared demand parameters across multiple dynamic pricing experiments.
method Meta dynamic pricing algorithm that learns prior online while solving Thompson sampling experiments.
result Algorithm achieves sublinear meta regret in experiment-rich environments.
Adaptive multi-stage density ratio estimation improves learning of latent space EBM.
problem Learning energy-based models in latent space is computationally expensive and challenging.
method Adaptive multi-stage density ratio estimation using NCE to bridge the gap between prior and posterior densities.
result The method enables more expressive prior models and sharpens the latent space EBM.
New method needed for class prior estimation when covariates are reduced.
problem Class prior estimation fails under covariate shift when covariates are reduced.
method Propose a probing algorithm for class prior estimation.
result Provable transformations preserving covariate shift are necessary for class prior estimation.
Posterior sampling estimator achieves near-optimal recovery guarantees for signals from any prior distribution.
problem Characterizing measurement complexity for signals from any prior distribution, including the entire space.
method Characterization of measurement complexity using posterior sampling estimator for Gaussian measurements and any prior distribution.
result Posterior sampling estimator achieves near-optimal recovery guarantees for signals from any prior distribution, robust to model mismatch.
Paper presents a robust transfer learning method for active level set estimation.
problem Efficiently identifying regions of a black-box function with limited function evaluations.
method Incorporates prior knowledge from a related function while locally adapting it.
result The method achieves better convergence of level sets compared to standard transfer learning.
Estimates class prior for unlabeled data using kernel embedding.
problem Estimating class prior in PU learning scenario where only positive and full population samples are available.
method Direct estimator based on distribution matching and kernel embedding in Reproducing Kernel Hilbert Space.
result Asymptotic consistency and explicit deviation bound for the estimator.
Regression Prior Networks improve ensemble performance on regression tasks.
problem Improving ensemble performance on regression tasks.
method Extending Prior Networks and Ensemble Distribution Distillation (EnD2) to regression tasks using the Normal-Wishart distribution. result Regression Prior Networks yield performance competitive with ensemble approaches on regression tasks.
The paper explores how prior functions and bootstrapping improve ensemble uncertainty estimation.
problem Improving uncertainty estimation in machine learning models.
method Investigates the benefits of prior functions and bootstrapping in ensemble models.
result Prior functions and bootstrapping enhance ensemble agents' uncertainty estimation across different inputs.
New priors can update posteriors without re-estimating likelihoods.
problem Degradation of classification approaches when class priors change.
method Recompute posteriors using recovered likelihoods from original posteriors and new priors.
result Dynamic update of original posteriors is possible without re-estimating likelihoods.
This paper simplifies finding least favorable priors by reducing dimensionality.
problem Finding least favorable priors is challenging due to infinite-dimensional optimization.
method Develops a dimensionality reduction method using Bregman divergences.
result Allows use of gradient ascent algorithms for finding least favorable priors.
Neural-g models mixtures of densities with flexibility and accuracy.
problem Accurately estimating prior densities in g-models. method Neural network with softmax output for valid probability densities.
result Neural-g captures various prior shapes including flat, heavy-tailed, and discontinuous.
Obtaining reliable uncertainty estimates of neural network predictions is a long standing challenge. Bayesian neural networks have been proposed as a solution, but it remains open how to specify their prior. In particular, the common practice of an independent normal prior in weight space imposes relatively weak constr…
This paper analyzes and guarantees convergence of prior-guided ZO algorithms.
problem Understanding convergence properties of prior-guided zeroth-order optimization algorithms.
method Analysis of convergence under a greedy descent framework with various gradient estimators, and development of ARS algorithm.
result Convergence guarantee for prior-guided random gradient-free (PRGF) algorithms and accelerated random search (ARS) algorithm.
New method improves causal structure discovery with Prior-Fitted Networks.
problem Errors in likelihood estimation limit proper causal structure discovery.
method Amortized causal discovery with Prior-Fitted Networks.
result Significant gains in structure recovery compared to baselines.
GS-B3SE improves label shift estimation by smoothing priors on a graph.
problem Label shift adaptation when source and target distributions share conditional but not marginal probabilities.
method Graph-Smoothed Bayesian Black-Box Shift Estimator (GS-B3SE) places Laplacian-Gaussian priors on log-priors and confusion-matrix columns tied by a label-similarity graph. result GS-B3SE produces a tractable posterior with HMC or Newton-CG schemes, proving identifiability, contraction, and robustness. MAXENT method outperforms ML in sparse data with specific prior correlations.
problem Evaluating MAXENT method's validity limits and comparing it with ML.
method Bayesian decision theory, Dirichlet density, KL distance, regularized maximum likelihood.
result MAXENT can outperform ML in sparse data with specific prior correlations.
Novel prior for orthogonal functions improves functional component estimation.
problem Improving orthogonality in functional principal component analysis.
method Sequential adaptive priors for orthogonal functions using hierarchical conditionally normal distributions.
result Proposed prior leads to nearly orthogonal posterior estimates.
The MEM method uses data-driven priors for linear inverse problems, proving convergence and estimating differences.
problem Linear inverse problems with approximate priors.
method Maximum Entropy on the Mean (MEM) method with data-driven priors.
result Empirical mean convergence and estimates for prior differences based on epigraphical distance.
Study characterizes training and test risks for MAP regression with Gaussian priors.
problem Understanding high-dimensional behavior of regularized linear regression with informative priors.
method Maximum a posteriori (MAP) regression with Gaussian priors, using random matrix theory.
result Closed-form risk formulas reveal the bias-variance-prior tradeoff and explain double descent.
We consider the problem of estimating the class prior in an unlabeled dataset. Under the assumption that an additional labeled dataset is available, the class prior can be estimated by fitting a mixture of class-wise data distributions to the unlabeled data distribution. However, in practice, such an additional labeled…
We propose a generalized double Pareto prior for Bayesian shrinkage estimation and inferences in linear models. The prior can be obtained via a scale mixture of Laplace or normal distributions, forming a bridge between the Laplace and Normal-Jeffreys' priors. While it has a spike at zero like the Laplace density, it al…
The article introduces a new estimator for regression that combines bridge regression with prior information.
problem Estimating parameters and selecting variables in linear models with prior information.
method Restricted Bridge Estimator (RBRIDGE) using local quadratic approximation.
result The RBRIDGE estimator provides a closed-form solution and outperforms other estimators in simulations and real data analysis.
The paper improves Bayesian precision matrix estimation for high-dimensional sparse data.
problem Estimating sparse precision matrices in high-dimensional settings.
method Tempered posterior with fully specified horseshoe prior.
result Concentration results and theoretical oracle inequality for posterior.
R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.
problem Bayesian neural networks struggle with choosing appropriate priors, leading to over-shrinkage or poor predictive performance.
method Proposes R2D2-Net with an R^2-induced Dirichlet Decomposition prior and variational Gibbs inference algorithm.
result R2D2-Net effectively shrinks irrelevant coefficients while preventing key features from over-shrinkage.
Estimation of response functions is an important task in dynamic medical imaging. This task arises for example in dynamic renal scintigraphy, where impulse response or retention functions are estimated, or in functional magnetic resonance imaging where hemodynamic response functions are required. These functions can no…
This work shows that Gaussian is the only prior for optimal linear estimation in L1 loss.
problem Optimal linear estimation of a random variable from noisy observations under L1 fidelity criterion. method Analyzes the conditions under which the conditional median is a linear estimator and identifies the Gaussian distribution as the only prior that induces linearity.
result Gaussian is the only prior distribution that induces linearity in the conditional median for L1 loss. The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…
We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver precise estimates in test datasets under prior probability and more general dataset…
Proposes I-prior extension for additive interaction models.
problem Challenges in estimating and selecting models with interactions.
method Extends I-prior methodology to multiple covariates, introducing a parsimonious model specification.
result Improves estimation and model selection for additive interaction models.
This work considers an estimation task in compressive sensing, where the goal is to estimate an unknown signal from compressive measurements that are corrupted by additive pre-measurement noise (interference, or clutter) as well as post-measurement noise, in the specific setting where some (perhaps limited) prior knowl…
Existing methods for sparse channel estimation typically provide an estimate computed as the solution maximizing an objective function defined as the sum of the log-likelihood function and a penalization term proportional to the l1-norm of the parameter of interest. However, other penalization terms have proven to have…
Unrolled networks learn optimal Bayesian inference for unknown priors.
problem Optimizing Bayesian inference when the prior is unknown.
method Unrolling neural networks to simulate iterations of inference algorithms.
result Unrolled networks approximate convergence to optimal denoisers for product priors.
The hyperparameters in Gaussian process regression (GPR) model with a specified kernel are often estimated from the data via the maximum marginal likelihood. Due to the non-convexity of marginal likelihood with respect to the hyperparameters, the optimization may not converge to the global maxima. A common approach to …
Study MAP estimation for PnP priors with SGD, proving convergence and demonstrating practical applications.
problem Theoretical analysis and practical implementation of PnP priors for Bayesian imaging problems.
method Maximum-a-posteriori estimation with Plug & Play priors and stochastic gradient descent.
result Convergence proof for MAP computation by PnP-SGD under realistic assumptions on the denoiser.
The quantification problem consists of determining the prevalence of a given label in a target population. However, one often has access to the labels in a sample from the training population but not in the target population. A common assumption in this situation is that of prior probability shift, that is, once the la…
Bayesian optimization usually assumes that a Bayesian prior is given. However, the strong theoretical guarantees in Bayesian optimization are often regrettably compromised in practice because of unknown parameters in the prior. In this paper, we adopt a variant of empirical Bayes and show that, by estimating the Gaussi…
Proposes a tail-adaptive shrinkage method for robust sparse estimation.
problem Robust Bayesian methods for high-dimensional regression under diverse sparse regimes.
method Global-local-tail (GLT) Gaussian mixture distribution with tail-adaptive shrinkage.
result GLT posterior contracts at minimax optimal rate for sparse normal mean models.
Bayesian framework estimates label shift for improved classifier performance.
problem Label shift in supervised learning leading to degraded classifier performance.
method Bayesian framework with dynamic Dirichlet priors and online EM algorithms.
result Significant improvements in classifier accuracy over state-of-the-art methods.
Proposes EM for sparse horseshoe estimation.
problem Sparse estimation of sparse parameter vectors using the horseshoe prior.
method Expectation-Maximisation (EM) procedure for MAP estimates.
result Our approach performs comparable or superior to state-of-the-art methods.
The estimation of probabilities of default (PDs) for low default portfolios by means of upper confidence bounds is a well established procedure in many financial institutions. However, there are often discussions within the institutions or between institutions and supervisors about which confidence level to use for the…
Improved VAEs by training a contrastive prior to match posterior.
problem Prior hole problem in VAEs, leading to poor image generation.
method Introduced a contrastive energy-based prior and trained it using noise contrastive estimation.
result Significant improvement in VAE generative performance on various datasets.
In many signal detection and classification problems, we have knowledge of the distribution under each hypothesis, but not the prior probabilities. This paper is aimed at providing theory to quantify the performance of detection via estimating prior probabilities from either labeled or unlabeled training data. The erro…