Bayesian neural networks' performance varies with prior choice, affecting their ability to identify unknowns.
problem The impact of prior choice on Bayesian neural networks' ability to identify unknowns.
method Evaluation of different prior distributions on classification tasks using BNNs and NNs with Monte Carlo dropout.
result Prior choice significantly impacts BNNs' ability to identify unknowns, affecting true and false positive rates.
In this paper, we study the effects of different prior and likelihood choices for Bayesian matrix factorisation, focusing on small datasets. These choices can greatly influence the predictive performance of the methods. We identify four groups of approaches: Gaussian-likelihood with real-valued priors, nonnegative prio…
This paper examines how the choice of prior distribution affects likelihoods of out-of-distribution inputs in deep generative models.
problem Mismatch between prior and data distributions causes deep generative models to assign higher likelihoods to out-of-distribution inputs.
method Proposes using a mixture distribution as a prior to make likelihoods of out-of-distribution inputs more sensitive.
result A mixture prior lowers the out-of-distribution likelihood with respect to real image data sets.
The study assesses sensitivity to prior choices in Bayesian nonparametric models.
problem Difficulty in specifying priors for Bayesian nonparametric models.
method Utilizes variational Bayesian methods to assess sensitivity to concentration parameter and stick-breaking distribution.
result Demonstrates how to evaluate sensitivity to prior choices in Dirichlet process mixtures and related models.
Randomly initialized sentence encoders perform well on tasks, suggesting learning is key.
problem The role of sentence encoder architectures in language tasks.
method Random initialization and fixed architecture approach to evaluate sentence encoders.
result Priors do not leverage additional information, learning is necessary.
Review of priors in Bayesian deep learning models.
problem The importance of prior choices in Bayesian deep learning models.
method Overview of different priors and methods of learning priors from data.
result Motivate practitioners to think carefully about prior specification.
Study compares priors for ABNs to improve model accuracy.
problem Inadequate priors lead to model selection issues in ABNs.
method Simulation study with three priors: Gaussian, Student's t, and strongly informative Gaussian.
result Informative Student's t-prior performs best, mitigating Lindley's paradox.
Paper develops Bayesian inference for discrete-choice mnp models with Gaussian priors.
problem Estimating parameters of discrete-choice multinomial probit models with Gaussian priors.
method Adapts Fasano and Durante's results to a specific mnp model with zero mean and independent Gaussian priors, simplifying posterior distribution parameters and providing a new variational algorithm.
result Simplified expressions for posterior distribution parameters and a novel variational algorithm.
A new Weyl prior is proposed for Bayesian statistics, offering a more canonical choice for parameter α.
problem Choosing a prior distribution for Bayesian inference.
method Proposed a new Weyl prior based on the Weyl structure on a statistical manifold.
result The Weyl prior is a special case of the α-parallel prior with α = -n, where n is the dimension of the statistical manifold.
New optimal prior avoids bias in complex models with limited data.
problem Bias in inference from limited data using Jeffreys prior.
method Developed a principled choice of measure that avoids bias, dependent on data quantity.
result Optimal prior leads to unbiased inference in complex models.
This study explores how choosing noninformative priors affects Thompson Sampling in multiparameter bandit models.
problem The optimality of Thompson Sampling (TS) in multiparameter bandit models depends on the choice of priors, especially when models are complex.
method The study extends regret analysis to uniform distributions and proposes a modified TS policy, TS-T, to achieve asymptotic optimality.
result Changing noninformative priors can significantly affect the expected regret in multiparameter bandit models.
The paper discusses the impact of prior densities on Bayesian model selection.
problem The sensitivity of marginal likelihood to prior choice in Bayesian model selection.
method Analyzes the role of prior densities in model selection, discusses improper priors, and proposes solutions.
result Marginal likelihood can be sensitive to prior choice, but improper priors can still be used with caution.
Develops a Bayesian framework for portfolio choice with a new posterior distribution.
problem Estimation risk in parametric portfolio policies.
method Generalized Bayesian framework with Gibbs posterior, utility maximization, and KNEEDLE algorithm.
result Optimal scaling parameter λ controls the balance between prior and data. New method automates asymmetric choice for better skill transfer in reinforcement learning.
problem Improving sample efficiency and transferability of reinforcement learning agents.
method Attentive Priors for Expressive and Transferable Skills (APES) using hierarchical KL-regularization.
result APES automates asymmetric choice, leading to better skill transfer across sequential tasks.
The empirically successful Thompson Sampling algorithm for stochastic bandits has drawn much interest in understanding its theoretical properties. One important benefit of the algorithm is that it allows domain knowledge to be conveniently encoded as a prior distribution to balance exploration and exploitation more eff…
Paper proposes a method to break symmetries in Bayesian matrix factorization.
problem Symmetries in posterior distribution reduce MCMC sampling efficiency.
method Modification to Gaussian prior mean and covariance to break symmetries.
result Breaking symmetries leads to lower autocorrelation and reconstruction errors.
This work tackles the challenge of Bayesian deep learning by proposing a new framework for matching Gaussian process priors with neural network parameters.
problem The challenge of specifying priors over neural network parameters, which affects the induced functional prior and is uncontrolled.
method The approach involves defining functional priors using Gaussian processes and matching these priors with the functional prior of neural networks through the minimization of Wasserstein distance.
result The proposed framework offers systematic performance improvements over alternative priors and approximate Bayesian deep learning approaches.
We present a mixed multinomial logit (MNL) model, which leverages the truncated stick-breaking process representation of the Dirichlet process as a flexible nonparametric mixing distribution. The proposed model is a Dirichlet process mixture model and accommodates discrete representations of heterogeneity, like a laten…
GPs' decisions can vary significantly with different kernels, even if kernels are qualitatively similar.
problem Robustness of GP decisions to kernel choice.
method Solved a constrained optimization problem over a finite-dimensional space to identify changes in GP decisions.
result Decisions made with a GP can be non-robust to kernel choice, even with qualitatively similar kernels.
Bayesian method corrects for model selection multiplicity in regression.
problem Model selection multiplicity in regression analysis.
method Developed a Bayesian prior distribution based on Holm procedure analogy.
result Adequate multiplicity correction requires sparsity not provided by recommended priors.
The paper refines disentanglement in VAEs by defining it as latent overlap and prior structure.
problem Improving the disentanglement of latent variables in Variational Autoencoders (VAEs).
method Develops a new perspective on disentanglement as latent overlap and prior structure, and introduces a training objective to control both factors.
result The β-VAE controls latent overlap and maintains prior structure, leading to better disentanglement. We study the problem of finding probability densities that match given European call option prices. To allow prior information about such a density to be taken into account, we generalise the algorithm presented in Neri and Schneider (2011) to find the maximum entropy density of an asset price to the relative entropy c…
Proposes a new technique for VAEs using manifold-valued variables.
problem Effect of prior distribution choice on VAE learning capacity.
method Embedding-reparameterization procedure (ER) for manifold-valued latent variables.
result ER technique outperforms conventional VAE on a toy benchmark.
Nonparametric Bayesian models are often based on the assumption that the objects being modeled are exchangeable. While appropriate in some applications (e.g., bag-of-words models for documents), exchangeability is sometimes assumed simply for computational reasons; non-exchangeable models might be a better choice for a…
While Bayesian methods are praised for their ability to incorporate useful prior knowledge, in practice, convenient priors that allow for computationally cheap or tractable inference are commonly used. In this paper, we investigate the following question: for a given model, is it possible to compute an inference result…
This paper proposes learning priors for adversarial autoencoders to improve model expressiveness.
problem The choice of priors in deep latent factor models can significantly affect model expressiveness, especially for models with limited capacity.
method The authors introduce code generators to transform simple priors into ones that better characterize the data distribution for adversarial autoencoders.
result The proposed model generates better image quality and learns better disentangled representations than standard AAEs in supervised and unsupervised settings.
We develop a Bayesian nonparametric extension of the popular Plackett-Luce choice model that can handle an infinite number of choice items. Our framework is based on the theory of random atomic measures, with the prior specified by a gamma process. We derive a posterior characterization and a simple and effective Gibbs…
Bayesian optimisation framework for multi-objective decision-making from choice data.
problem Optimizing multi-objective functions via choice judgements.
method Gaussian process prior and novel likelihood model for choice data.
result Proposes a novel Bayesian framework for learning latent functions from choice data.
A new prior for VAEs improves model capacity by allowing a more flexible latent space.
problem Standard Gaussian priors in VAEs limit model capacity and performance.
method Proposed a Riemannian Brownian motion prior over a Riemannian structure of the latent space.
result The new prior significantly increases model capacity with only one additional scalar parameter.
Paper tackles cold-start domain adaptation with language descriptions.
problem Cold-start domain adaptation failure with scarce target data.
method Leverages textual descriptions to learn preferences from LLM, integrates into EM algorithm.
result Framework guides source selection for weak target signals, improves as data accumulates.
Study nonconcave portfolio choice with smooth ambiguity and Bayesian learning.
problem Nonconcave portfolio choice under smooth ambiguity and Bayesian learning.
method Developed a general framework for dynamic, non-concave asset allocation.
result Dynamic consistency achieved through a robust representation.
Designs a DNN with alternative-specific utility functions for improved choice analysis.
problem Challenges in reconciling domain-specific knowledge with generic DNN.
method Integrates prior behavioral knowledge into DNN architecture with alternative-specific utility functions.
result 2-3% higher prediction accuracy than fully connected DNN over hyperparameter space.
Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem. The AMP framework provides modularity in the choice of signal prior; here we propose a hierarchical form of the Gauss-Bernouilli prior which utilizes a Restricted Boltzmann Machin…
Study shows prior Lipschitz continuity can improve adversarial robustness of Bayesian Neural Networks.
problem Improving adversarial robustness of Bayesian Neural Networks.
method Analysis of i.i.d., zero-mean Gaussian priors and posteriors approximated via mean-field variational inference.
result Adversarial robustness is sensitive to the prior variance.
A VB method for high-dimensional regression with student-t priors achieves nearly optimal performance and computational efficiency.
problem High-dimensional linear model inferences with heavy-tailed shrinkage priors.
method Variational Bayesian (VB) procedure for high-dimensional linear models with student-t priors.
result The VB method achieves nearly optimal contraction rate and computational efficiency, outperforming MCMC methods.
We consider optimal consumption and portfolio choice in the presence of Knightian uncertainty in continuous-time. We embed the problem into the new framework of stochastic calculus for such settings, dealing in particular with the issue of non-equivalent multiple priors. We solve the problem completely by identifying t…
In this article we provide a formulation of empirical bayes described by Atchade (2011) to tune the hyperparameters of priors used in bayesian set up of collaborative filter. We implement the same in MovieLens small dataset. We see that it can be used to get a good initial choice for the parameters. It can also be used…
Proposes efficient sensitivity analysis for complex Bayesian models.
problem Inefficiency of sensitivity analyses in complex Bayesian models.
method SA-ABI: weight sharing and neural network rapid inference.
result Efficiently integrates sensitivity analyses into Bayesian inference.
Estimates optimal transport maps with known cost functions.
problem Ensuring optimal transport maps correspond to real-world usefulness.
method Differentiable neural ground costs with known Monge map forms.
result General approach for incorporating prior information.
We use the language of uninformative Bayesian prior choice to study the selection of appropriately simple effective models. We advocate for the prior which maximizes the mutual information between parameters and predictions, learning as much as possible from limited data. When many parameters are poorly constrained by …
High-dimensional shrinkage risk depends on the default prior for the common scale.
problem Choosing the default prior for the common scale in high-dimensional shrinkage.
method Using radial-power benchmark to compare variance-flat and standard deviation-flat priors.
result The standard deviation-flat prior has a one-unit asymptotic risk advantage near the origin.
Deep audio prior uses neural networks to solve audio problems without data.
problem Challenging audio problems like source separation, editing, and synthesis.
method Randomly-initialized neural network with carefully designed audio prior.
result Superior audio results on Universal-150 benchmark dataset.
Improved Bayesian inference using power priors with historical data.
problem Improving Bayesian inference with historical data.
method Generalized power priors that adapt to the α parameter of Amari's α-divergence. result Improved performance through appropriate choices of the α parameter. Optimizes Bayesian priors for matrix factorization without posterior inference.
problem Selecting optimal priors for Bayesian models in machine learning.
method Prior predictive distribution and virtual statistics matching user-provided or observed data statistics.
result Analytically determines hyperparameters for Poisson factorization models.
Generative model captures how individuals process travel information under uncertainty.
problem Travel decisions are sensitive to uncertainty and information processing constraints.
method Data-driven generative learning model based on rational inattention theory.
result Generative model demonstrates strong correlation with rational inattention theory.
Bayesian Optimization with a Prior for the Optimum (BOPrO) improves efficiency and accuracy.
problem Bayesian Optimization's standard priors are not intuitive for domain experts.
method BOPrO injects expert knowledge into the optimization process using priors about the optimum.
result BOPrO is 6.67x faster than state-of-the-art methods and achieves new state-of-the-art performance.
This research explores how different discrete diffusion kernels affect graph generation quality.
problem The impact of different discrete diffusion kernels on graph generation quality.
method Developed a family of discrete diffusion kernels that converge to different Bernoulli priors.
result The quality of generated graphs is sensitive to the prior used, challenging previous intuitions.
Improves robustness of information bottleneck framework with sparsity-inducing prior.
problem Fixed-dimensional priors restrict flexibility and restrict robustness.
method Sparsity-inducing spike-slab categorical prior that learns dimension distribution per data point.
result Improves accuracy and robustness compared to traditional priors and other methods.