Optimality of TS with noninformative priors proven for Pareto model.
problem Optimality of Thompson Sampling with noninformative priors for Pareto bandits.
method Proved optimality of TS with certain probability matching priors, showed suboptimality with others, and found effectiveness of truncation procedures.
result TS with certain probability matching priors achieves optimal regret bound for Pareto model.
Study proposes learning optimal priors from data for better Bayesian inference.
problem Challenges the use of noninformative uniform priors in Bayesian inference.
method Machine learning approach to learn optimal priors from data using a target function.
result Study models consistently outperformed baseline models in Wikipedia category classification.
GOAT improves attention mechanisms by learning better priors.
problem Standard attention mechanisms use a naive uniform prior, limiting flexibility and generalization.
method GOAT introduces a trainable, continuous prior that replaces the uniform assumption, maintaining compatibility with optimized kernels.
result GOAT avoids representational trade-offs and learns an extrapolatable prior that combines positional flexibility with length generalization.
Bayesian Optimization with a Prior for the Optimum (BOPrO) improves efficiency and accuracy.
problem Bayesian Optimization's standard priors are not intuitive for domain experts.
method BOPrO injects expert knowledge into the optimization process using priors about the optimum.
result BOPrO is 6.67x faster than state-of-the-art methods and achieves new state-of-the-art performance.
This study explores how choosing noninformative priors affects Thompson Sampling in multiparameter bandit models.
problem The optimality of Thompson Sampling (TS) in multiparameter bandit models depends on the choice of priors, especially when models are complex.
method The study extends regret analysis to uniform distributions and proposes a modified TS policy, TS-T, to achieve asymptotic optimality.
result Changing noninformative priors can significantly affect the expected regret in multiparameter bandit models.
Paper formulates mutual information optimal control for discrete-time systems.
problem Optimal control of discrete-time linear systems with mutual information.
method Formulates MIOCP as an extension of MEOCP, derives optimal policy and prior, proposes alternating minimization algorithm.
result Proposes an alternating minimization algorithm for MIOCP.
HyperBO+ pre-trains a universal prior for Bayesian optimization across different domains.
problem Bayesian optimization requires domain-specific priors, limiting its applicability.
method Two-step pre-training method for hierarchical Gaussian processes.
result HyperBO+ achieves lower regrets on unseen search spaces.
This paper solves aggregation of Pareto optimal models by using Bayesian priors and weighted averaging.
problem How to rationally aggregate Pareto optimal models while preserving Pareto efficiency.
method Four logical steps: 1) Bayesian models, 2) Prior as preference ranking, 3) Consistent aggregation, 4) Weighted average of priors.
result All rational/consistent aggregation rules follow a generalized hierarchical Bayesian model.
New optimal prior avoids bias in complex models with limited data.
problem Bias in inference from limited data using Jeffreys prior.
method Developed a principled choice of measure that avoids bias, dependent on data quantity.
result Optimal prior leads to unbiased inference in complex models.
Unrolled networks learn optimal Bayesian inference for unknown priors.
problem Optimizing Bayesian inference when the prior is unknown.
method Unrolling neural networks to simulate iterations of inference algorithms.
result Unrolled networks approximate convergence to optimal denoisers for product priors.
Bounded rationality investigates utility-optimizing decision-makers with limited information-processing power. In particular, information theoretic bounded rationality models formalize resource constraints abstractly in terms of relative Shannon information, namely the Kullback-Leibler Divergence between the agents' pr…
The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the stan…
A VB method for high-dimensional regression with student-t priors achieves nearly optimal performance and computational efficiency.
problem High-dimensional linear model inferences with heavy-tailed shrinkage priors.
method Variational Bayesian (VB) procedure for high-dimensional linear models with student-t priors.
result The VB method achieves nearly optimal contraction rate and computational efficiency, outperforming MCMC methods.
Adaptive optimal transport priors improve few-shot learning robustness.
problem Limited supervision and distribution shifts in few-shot learning.
method Prototype-Guided Distributionally Robust Optimization (PG-DRO) framework.
result PG-DRO achieves stronger robust generalization in few-shot scenarios.
Bayesian optimization usually assumes that a Bayesian prior is given. However, the strong theoretical guarantees in Bayesian optimization are often regrettably compromised in practice because of unknown parameters in the prior. In this paper, we adopt a variant of empirical Bayes and show that, by estimating the Gaussi…
C-VAE improves VAE by resolving prior issues and generating better samples.
problem Low-quality samples from VAE due to prior issues.
method Formulates VAE as OT, allows flexible priors, and uses OT formulations.
result C-VAE generates higher quality samples and latent representations.
This paper analyzes and guarantees convergence of prior-guided ZO algorithms.
problem Understanding convergence properties of prior-guided zeroth-order optimization algorithms.
method Analysis of convergence under a greedy descent framework with various gradient estimators, and development of ARS algorithm.
result Convergence guarantee for prior-guided random gradient-free (PRGF) algorithms and accelerated random search (ARS) algorithm.
ColaBO accelerates optimization with user beliefs.
problem Optimizing expensive functions with limited prior knowledge.
method General Bayesian framework for incorporating user beliefs.
result Significant optimization acceleration with accurate prior information.
Meta-learning priors improves safe Bayesian optimization.
problem Optimizing robot controllers under safety constraints.
method Meta-learning priors from offline data using F-PACOH.
result Meta-learned priors accelerate safe BO convergence.
Study shows generative priors improve rank-one matrix recovery with optimal sample complexity.
problem Recovering a rank-one signal matrix from noisy data with additional prior information.
method Analysis of a nonlinear least squares objective with a favorable global optimization landscape.
result Established optimal sample complexity for generative priors in rank-one matrix recovery.
Posterior sampling estimator achieves near-optimal recovery guarantees for signals from any prior distribution.
problem Characterizing measurement complexity for signals from any prior distribution, including the entire space.
method Characterization of measurement complexity using posterior sampling estimator for Gaussian measurements and any prior distribution.
result Posterior sampling estimator achieves near-optimal recovery guarantees for signals from any prior distribution, robust to model mismatch.
Prior design is one of the most important problems in both statistics and machine learning. The cross validation (CV) and the widely applicable information criterion (WAIC) are predictive measures of the Bayesian estimation, however, it has been difficult to apply them to find the optimal prior because their mathematic…
Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques fo…
The behavior of many Bayesian models used in machine learning critically depends on the choice of prior distributions, controlled by some hyperparameters that are typically selected by Bayesian optimization or cross-validation. This requires repeated, costly, posterior inference. We provide an alternative for selecting…
New theory for BNNs with Gaussian priors achieves optimal posterior concentration rates.
problem Lack of theoretical results for BNNs with Gaussian priors.
method New approximation theory for non-sparse DNNs with bounded parameters.
result BNNs with non-sparse general priors can achieve near-minimax optimal posterior concentration rates.
Study on GEPs with generative priors, showing optimal statistical rates and proposing an iterative algorithm.
problem Generalized eigenvalue problems with generative priors.
method Assumption of Lipschitz continuous generative model, Projected Rayleigh Flow Method (PRFM).
result PRFM converges linearly to an estimated vector achieving the optimal statistical rate.
PFNs4BO uses neural processes for flexible Bayesian Optimization.
problem Efficient surrogate modeling for Bayesian Optimization.
method In-context learning of PFNs to approximate posterior predictive distribution.
result PFNs outperform traditional GP, BNN in BO tasks.
We consider the stochastic multi-armed bandit problem with a prior distribution on the reward distributions. We are interested in studying prior-free and prior-dependent regret bounds, very much in the same spirit as the usual distribution-free and distribution-dependent bounds for the non-Bayesian stochastic bandit. B…
P-BO reduces black-box adversarial attacks by 10x with Bayesian optimization and function prior.
problem Efficiently generating adversarial examples against black-box models.
method Prior-guided Bayesian Optimization (P-BO) with a function prior initialized from a surrogate model.
result Significantly reduces the number of queries needed for adversarial attacks.
We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available. We introduce a framework that conceptually unifies much of the existing work on black-box attacks, and we demonstrate that the current state-of-the-art methods are optimal in a natura…
TSFlow uses Gaussian processes to match priors for better time series forecasting.
problem Difficulties in aligning generative models' priors with time series data.
method Conditional flow matching (CFM) with Gaussian processes, optimal transport, and data-dependent priors.
result TSFlow produces high-quality unconditional samples and competitive forecasting results.
One of the most interesting features of Bayesian optimization for direct policy search is that it can leverage priors (e.g., from simulation or from previous tasks) to accelerate learning on a robot. In this paper, we are interested in situations for which several priors exist but we do not know in advance which one fi…
In this letter, we address sparse signal recovery using spike and slab priors. In particular, we focus on a Bayesian framework where sparsity is enforced on reconstruction coefficients via probabilistic priors. The optimization resulting from spike and slab prior maximization is known to be a hard non-convex problem, a…
Bayesian optimization improves with nonstationary covariance functions.
problem Stationary covariance functions fail to capture prior information in high dimensions.
method Proposes nonstationary covariance functions to encode prior information and adaptively promote local exploration.
result Nonstationary covariance functions increase sample efficiency in high dimensions.
The study optimizes Gaussian process approximations for finite-rank models.
problem Posterior behavior of finite-rank approximations differs from parent GP priors.
method Locally supported basis expansions with dependent Gaussian coefficients.
result Finite-rank expansions inherit the same posterior contraction rate as parent GP priors.
SAHMM-VAE separates sources adaptively using hidden Markov priors.
problem Unsupervised blind source separation.
method Source-wise adaptive Hidden Markov prior variational autoencoder.
result Different latent dimensions align with different source-specific temporal organizations.
Berry et al. (1997) initiated the development of the infinite arms bandit problem. They derived a regret lower bound of all allocation strategies for Bernoulli rewards with uniform priors, and proposed strategies based on success runs. Bonald and Proutière (2013) proposed a two-target algorithm that achieves the regret…
BioBO optimizes gene perturbation design using Bayesian optimization with biological priors.
problem Efficient design of genomic perturbation experiments in drug discovery.
method Integrates Bayesian optimization with multimodal gene embeddings and enrichment analysis.
result Improves labeling efficiency by 25-40% and identifies top-performing perturbations more effectively.
Recent research has demonstrated that feature attribution methods for deep networks can themselves be incorporated into training; these attribution priors optimize for a model whose attributions have certain desirable properties -- most frequently, that particular features are important or unimportant. These attributio…
This work shows that Gaussian is the only prior for optimal linear estimation in L1 loss.
problem Optimal linear estimation of a random variable from noisy observations under L1 fidelity criterion. method Analyzes the conditions under which the conditional median is a linear estimator and identifies the Gaussian distribution as the only prior that induces linearity.
result Gaussian is the only prior distribution that induces linearity in the conditional median for L1 loss. We propose an online convex optimization algorithm (RescaledExp) that achieves optimal regret in the unconstrained setting without prior knowledge of any bounds on the loss functions. We prove a lower bound showing an exponential separation between the regret of existing algorithms that require a known bound on the los…
DPPS uses DP priors for Bayesian non-parametric multi-arm bandits.
problem Optimizing multi-arm bandit environments with prior beliefs.
method Bayesian non-parametric algorithm based on Dirichlet Process priors.
result DPPS provides principled incorporation of prior beliefs and is optimal in Bayesian regret setup.
Optimizes expensive experiments by incorporating expert knowledge.
problem Expensive experiments require minimizing the number of trials.
method Bayesian optimization with posterior sampling of expert knowledge.
result Demonstrates significant efficiency gains in experiments and hyperparameter tuning.
The vast majority of optimization and online learning algorithms today require some prior information about the data (often in the form of bounds on gradients or on the optimal parameter value). When this information is not available, these algorithms require laborious manual tuning of various hyperparameters, motivati…
New method improves GP uncertainty quantification for misspecified priors.
problem Uncertainty quantification for GPs under incorrect priors.
method Constructs a confidence sequence using martingale techniques.
result Empirically outperforms standard GP methods in robustness and utility for Bayesian Optimization.
Bayesian framework for sphere regression using Gaussian fields.
problem Nonparametric regression on the sphere with Gaussian priors.
method Isotropic Gaussian field priors, harmonic structure, exact posterior distributions, optimal spectral truncation, posterior contraction rates.
result Sharp posterior contraction rates for Gaussian priors with polynomially decaying angular power spectra.
This paper simplifies finding least favorable priors by reducing dimensionality.
problem Finding least favorable priors is challenging due to infinite-dimensional optimization.
method Develops a dimensionality reduction method using Bregman divergences.
result Allows use of gradient ascent algorithms for finding least favorable priors.
Spike-and-slab priors are popular Bayesian solutions for high-dimensional linear regression problems. Previous theoretical studies on spike-and-slab methods focus on specific prior formulations and use prior-dependent conditions and analyses, and thus can not be generalized directly. In this paper, we propose a class o…