A new Weyl prior is proposed for Bayesian statistics, offering a more canonical choice for parameter α.
problem Choosing a prior distribution for Bayesian inference.
method Proposed a new Weyl prior based on the Weyl structure on a statistical manifold.
result The Weyl prior is a special case of the α-parallel prior with α = -n, where n is the dimension of the statistical manifold.
Develops methods for constructing likelihoods and priors for Bayesian networks.
problem Learning parameters and structure of Bayesian networks from limited data.
method Introduces assumptions for constructing likelihoods and priors from small assessments.
result Allows construction of likelihoods and priors for a wide range of network structures.
New optimal prior avoids bias in complex models with limited data.
problem Bias in inference from limited data using Jeffreys prior.
method Developed a principled choice of measure that avoids bias, dependent on data quantity.
result Optimal prior leads to unbiased inference in complex models.
Bayesian models that mix multiple Dirichlet prior parameters, called Multi-Dirichlet priors (MD) in this paper, are gaining popularity. Inferring mixing weights and parameters of mixed prior distributions seems tricky, as sums over Dirichlet parameters complicate the joint distribution of model parameters. This paper s…
This work tackles the challenge of Bayesian deep learning by proposing a new framework for matching Gaussian process priors with neural network parameters.
problem The challenge of specifying priors over neural network parameters, which affects the induced functional prior and is uncontrolled.
method The approach involves defining functional priors using Gaussian processes and matching these priors with the functional prior of neural networks through the minimization of Wasserstein distance.
result The proposed framework offers systematic performance improvements over alternative priors and approximate Bayesian deep learning approaches.
Regularization methods, specifically those which directly alter weights like L1 and L2, are an integral part of many learning algorithms. Both the regularizers mentioned above are formulated by assuming certain priors in the parameter space and these assumptions, in some cases, induce sparsity in the parameter sp…
New method samples Jeffreys prior for objective Bayesian inference.
problem Sampling from Jeffreys prior is challenging.
method Metropolis-Adjusted Langevin Algorithm
result Samples can be directly used in Bayesian methods.
We propose a novel approach for nonlinear regression using a two-layer neural network (NN) model structure with sparsity-favoring hierarchical priors on the network weights. We present an expectation propagation (EP) approach for approximate integration over the posterior distribution of the weights, the hierarchical s…
We use the language of uninformative Bayesian prior choice to study the selection of appropriately simple effective models. We advocate for the prior which maximizes the mutual information between parameters and predictions, learning as much as possible from limited data. When many parameters are poorly constrained by …
Study shows priors are crucial for accurate causal learning from unlabeled data.
problem Improving causal learning from unlabeled data.
method Investigated causal learning using Bayesian methods and analyzed the impact of priors.
result Factorized priors lead to factorized posteriors, aligning with independent causal mechanisms.
The paper deals with learning probability distributions of observed data by artificial neural networks. We suggest a so-called gradient conjugate prior (GCP) update appropriate for neural networks, which is a modification of the classical Bayesian update for conjugate priors. We establish a connection between the gradi…
Framework for games with uncertain parameters, ensuring no player can improve by changing strategy.
problem Non-cooperative games with globally uncertain parameters and no common prior.
method Mixed strategies and subjective priors, Extended Equilibrium defined by fixed-point argument.
result Existence of Extended Equilibrium under certain conditions.
We consider the scenario where the parameters of a probabilistic model are expected to vary over time. We construct a novel prior distribution that promotes sparsity and adapts the strength of correlation between parameters at successive timesteps, based on the data. We derive approximate variational inference procedur…
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
problem Understanding the assumptions and dependencies in Bayesian hierarchical models.
method Demonstrates how canonical distributions and maximum entropy principles can be used to derive marginal priors in hierarchical models.
result Marginal priors in hierarchical models derived from maximum entropy principles have different constraints compared to the original priors.
Additive Bayesian networks are types of graphical models that extend the usual Bayesian generalized linear model to multiple dependent variables through the factorisation of the joint probability distribution of the underlying variables. When fitting an ABN model, the choice of the prior of the parameters is of crucial…
Develops methods for constructing parameter priors in DAG models.
problem Constructing parameter priors for model choice among DAG models.
method Introduces assumptions and methods for parameter priors construction and marginal likelihood computation.
result The only parameter prior for complete Gaussian DAG models that satisfies assumptions is the normal-Wishart distribution.
Thompson sampling used for linear bandits with normal-gamma priors.
problem Optimizing decisions in uncertain environments with linear dependencies and unknown parameters.
method Bayesian Thompson sampling with multivariate normal-gamma priors.
result Derivation of a Bayesian regret bound for the approach.
Proposes a new prior for complex models to improve prediction accuracy.
problem Difficulty in specifying priors for complex models like neural networks.
method Predictive complexity priors defined by comparing model predictions to a reference model, transferred to parameters via change of variables.
result Improves model predictions by reducing unintuitive effects of traditional priors.
New method reduces over-pessimism in Bayesian control under parameter uncertainty.
problem Over-pessimism in Bayesian control due to misspecified priors.
method Distributionally robust Bayesian control (DRBC) with strong duality and optimization.
result Validated algorithm on synthetic and real data, reducing over-pessimism.
New method improves Robbins-Monro algorithm convergence with prior information.
problem Improving convergence speed of Robbins-Monro algorithm.
method Integrates prior information into Robbins-Monro iteration without regression model.
result Prior-information Robbins-Monro sequence converges faster than standard.
The study optimizes Gaussian process approximations for finite-rank models.
problem Posterior behavior of finite-rank approximations differs from parent GP priors.
method Locally supported basis expansions with dependent Gaussian coefficients.
result Finite-rank expansions inherit the same posterior contraction rate as parent GP priors.
New method uses KL-divergence to create non-informative priors for multivariate Gaussian.
problem Handling hyperparameters for non-informative limits in multivariate Gaussian conjugate priors.
method Using scaled KL-divergence between multivariate Gaussians to construct Wishart and normal-Wishart conjugate priors.
result Forming non-informative priors without violating Wishart shape parameter restrictions.
So-called sparse estimators arise in the context of model fitting, when one a priori assumes that only a few (unknown) model parameters deviate from zero. Sparsity constraints can be useful when the estimation problem is under-determined, i.e. when number of model parameters is much higher than the number of data point…
Proposes I-prior extension for additive interaction models.
problem Challenges in estimating and selecting models with interactions.
method Extends I-prior methodology to multiple covariates, introducing a parsimonious model specification.
result Improves estimation and model selection for additive interaction models.
Proposes method for eliciting non-parametric joint priors using normalizing flows.
problem Learning complex non-parametric joint priors for model parameters.
method Expert elicitation combined with normalizing flows for generative modeling.
result Framework supports elicitation of both parametric and non-parametric priors.
A new meta-learning method using shared variational inference.
problem Meta-learning with uncertainty over model parameters.
method Shared amortized variational inference network for conditional prior and posterior.
result Prevents collapse of conditional prior to Dirac delta function.
Paper proposes a new method for Bayesian linear regression using spike-and-slab priors.
problem Identifying predictors with similar relationships in linear regression models.
method Hierarchical Bayesian models with spike-and-slab priors and a Gibbs sampler.
result The proposed method outperforms previous methods in simulations and real data analysis.
In this paper, we derive a Bayesian model order selection rule by using the exponentially embedded family method, termed Bayesian EEF. Unlike many other Bayesian model selection methods, the Bayesian EEF can use vague proper priors and improper noninformative priors to be objective in the elicitation of parameter prior…
In this paper we present decomposable priors, a family of priors over structure and parameters of tree belief nets for which Bayesian learning with complete observations is tractable, in the sense that the posterior is also decomposable and can be completely determined analytically in polynomial time. This follows from…
PriorGuide adapts diffusion models to new priors at test time.
problem Limited applicability of prior distributions in diffusion-based inference.
method PriorGuide uses a guidance approximation to adapt diffusion models to new priors at test time.
result Enhances the versatility of pre-trained inference models by allowing flexible adaptation to new priors.
Bayesian priors and penalties are equivalent in variational inference.
problem Understanding the relationship between Bayesian priors and penalties in variational inference.
method Characterizing the regularizers that can arise in variational inference and providing a systematic way to compute the prior corresponding to a given penalty.
result Equivalence between Bayesian priors and penalties in variational inference.
New method uses quotient predictor space for better PAC-Bayes bounds, reducing KL divergence and improving model performance.
problem Overparameterized models with continuous symmetries can lead to biased predictions.
method Perform PAC-Bayesian analysis on quotient predictor space, constructing a canonical prior that reflects model's implicit bias.
result The new prior reduces KL divergence and improves model performance in experiments.
A method for converting NIW parameters for better estimation.
problem Estimating parameters of multivariate normal distribution.
method Convergent procedure for converting mean parameters to natural parameters in NIW family.
result Maximum likelihood estimation of natural parameters from observed statistics.
Optimality of TS with noninformative priors proven for Pareto model.
problem Optimality of Thompson Sampling with noninformative priors for Pareto bandits.
method Proved optimality of TS with certain probability matching priors, showed suboptimality with others, and found effectiveness of truncation procedures.
result TS with certain probability matching priors achieves optimal regret bound for Pareto model.
Bayesian neural networks with functional priors improve surrogate modeling in mechanics.
problem Challenges in integrating prior knowledge and quantifying uncertainties in high-dimensional NN parameter spaces.
method Anchored ensembling to integrate a priori information and learn low-rank correlations between NN parameters.
result Effective transfer of knowledge between function-space and parameter-space priors improves surrogate model accuracy and uncertainty estimation.
We show that the only parameter prior for complete Gaussian DAG models that satisfies global parameter independence, complete model equivalence, and some weak regularity assumptions, is the normal-Wishart distribution. Our analysis is based on the following new characterization of the Wishart distribution: let W be an …
Proposes a new method to learn meta-priors from data.
problem Improving learning systems with domain knowledge and controlling parameter learning rates.
method Hierarchical Empirical Bayes approach to decouple learning rates of features.
result Meta-prior learning improves performance and convergence time in various applications.
GRASP simplifies Bayesian regression with grouped predictors using an adaptive NBP prior.
problem Regression with grouped predictors and adaptive shrinkage.
method Normal Beta Prime (NBP) prior with tunable hyperparameters for flexible sparsity control.
result Empirical validation of robust and versatile GRASP across various sparsity and signal-to-noise ratios.
Bayesian framework uses AI-generated data to improve parameter estimation.
problem Parameter estimation in models with unknown or unspecified likelihood.
method Exponentially tilted empirical likelihood with Dirichlet process posterior.
result AI-generated data can provide useful regularization for parameter estimation.
MARS meta-learns function scores for improved predictive accuracy and uncertainty.
problem Difficulty in specifying expressive priors for Bayesian meta-learning.
method Meta-learning the score function of data-generating process marginals in the function space.
result State-of-the-art predictive accuracy and improved uncertainty estimates.
New Bayesian method for joint sparse parameter inference.
problem Inference of jointly sparse parameter vectors from multiple measurements.
method Hierarchical Bayesian learning with joint sparsity-promoting priors.
result New algorithms consistently outperform existing methods in numerical experiments.
Bayesian neural networks use ridgelet prior for uncertainty quantification.
problem Combining strong predictive performance with uncertainty quantification in Bayesian neural networks.
method Proposes a ridgelet prior that approximates a Gaussian process covariance function in the output space of the network.
result Establishes universality property allowing Bayesian neural networks to approximate any Gaussian process.
Bayesian neural networks' performance varies with prior choice, affecting their ability to identify unknowns.
problem The impact of prior choice on Bayesian neural networks' ability to identify unknowns.
method Evaluation of different prior distributions on classification tasks using BNNs and NNs with Monte Carlo dropout.
result Prior choice significantly impacts BNNs' ability to identify unknowns, affecting true and false positive rates.
Modeling buildings' heat dynamics is a complex process which depends on various factors including weather, building thermal capacity, insulation preservation, and residents' behavior. Gray-box models offer a causal inference of those dynamics expressed in few parameters specific to built environments. These parameters …
Develops a simulation-based method to translate expert knowledge into prior distributions for Bayesian models.
problem Effective incorporation of expert knowledge into prior distributions for diverse model structures.
method Simulation-based stochastic gradient descent to learn hyperparameters of parametric priors from expert knowledge.
result Method is adaptable to various elicitation techniques and independent of model structure.
Paper proposes an efficient algorithm for nonnegative binary matrix factorization.
problem Decomposing binary data using matrix factorization.
method Majorization-minimization algorithm with Beta prior for improved performance.
result Proposed algorithm offers excellent trade-off between performance, complexity, and interpretability.
New theory for BNNs with Gaussian priors achieves optimal posterior concentration rates.
problem Lack of theoretical results for BNNs with Gaussian priors.
method New approximation theory for non-sparse DNNs with bounded parameters.
result BNNs with non-sparse general priors can achieve near-minimax optimal posterior concentration rates.
SAHMM-VAE separates sources adaptively using hidden Markov priors.
problem Unsupervised blind source separation.
method Source-wise adaptive Hidden Markov prior variational autoencoder.
result Different latent dimensions align with different source-specific temporal organizations.