Paper introduces a method to generate physically feasible dynamics with physical priors.
problem Challenges in generating physically feasible dynamics under physical priors.
method Seamlessly incorporates physical priors into diffusion-based generative models.
result Efficient generation of physically realistic dynamics across various physical phenomena.
GOAT improves attention mechanisms by learning better priors.
problem Standard attention mechanisms use a naive uniform prior, limiting flexibility and generalization.
method GOAT introduces a trainable, continuous prior that replaces the uniform assumption, maintaining compatibility with optimized kernels.
result GOAT avoids representational trade-offs and learns an extrapolatable prior that combines positional flexibility with length generalization.
Proposes new priors for neural networks to improve generalization and uncertainty.
problem Improving generalization and uncertainty estimation in neural networks.
method Exploits scalable and structured posteriors as priors with generalization guarantees.
result Improves generalization and uncertainty estimation with non-vacuous bounds.
Spike-and-slab priors are popular Bayesian solutions for high-dimensional linear regression problems. Previous theoretical studies on spike-and-slab methods focus on specific prior formulations and use prior-dependent conditions and analyses, and thus can not be generalized directly. In this paper, we propose a class o…
Improved VAEs by training a contrastive prior to match posterior.
problem Prior hole problem in VAEs, leading to poor image generation.
method Introduced a contrastive energy-based prior and trained it using noise contrastive estimation.
result Significant improvement in VAE generative performance on various datasets.
Introduces foundation priors for using model-generated data in empirical research.
problem Using model-generated data as real observations in empirical research.
method Introduces foundation priors as an exponential-tilted, generalized Bayesian update of the user's primitive prior.
result Synthetic data reflects both model patterns and user's priors, enabling principled use in empirical work.
Improves AI-prior reliability for Bayesian inference.
problem Error propagation from predictive models into posterior inference.
method Rectified AI-informed prior elicitation framework.
result Significant reduction in bias and improvement in predictive performance.
Generative models improve inverse problems by providing tailored priors.
problem Analyzing the error in inverse problems solved with generative priors.
method Quantitative error bounds for minimum Wasserstein-2 generative models.
result The error in the posterior due to the generative prior is bounded by the prior's error in Wasserstein-1 distance.
Generative models produce realistic objects in many domains, including text, image, video, and audio synthesis. Most popular models---Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs)---usually employ a standard Gaussian distribution as a prior. Previous works show that the richer family of pri…
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
TSFlow uses Gaussian processes to match priors for better time series forecasting.
problem Difficulties in aligning generative models' priors with time series data.
method Conditional flow matching (CFM) with Gaussian processes, optimal transport, and data-dependent priors.
result TSFlow produces high-quality unconditional samples and competitive forecasting results.
The paper extends and applies a new shrinkage prior in Bayesian factor analysis.
problem Estimating the number of factors in sparse Bayesian factor analysis.
method Introduces and extends a generalized cumulative shrinkage process (CUSP) prior.
result Exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index increases.
C-VAE improves VAE by resolving prior issues and generating better samples.
problem Low-quality samples from VAE due to prior issues.
method Formulates VAE as OT, allows flexible priors, and uses OT formulations.
result C-VAE generates higher quality samples and latent representations.
Most deep latent factor models choose simple priors for simplicity, tractability or not knowing what prior to use. Recent studies show that the choice of the prior may have a profound effect on the expressiveness of the model,especially when its generative network has limited capacity. In this paper, we propose to lear…
We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix. We derive sharp asymptotics for the information-theoretically optimal performance and for the best known polynomial algorithm for an ensemble of generative priors consisting of fully connected deep neural …
Bayesian weight priors improve neural network learning of identity relations.
problem Neural networks struggle to learn abstract and systematic relations, especially identity relations.
method Extended RBP approach using Bayesian weight priors as a regularization term.
result Bayesian weight priors lead to perfect generalization for identity relations and do not hinder standard neural network learning.
A new Weyl prior is proposed for Bayesian statistics, offering a more canonical choice for parameter α.
problem Choosing a prior distribution for Bayesian inference.
method Proposed a new Weyl prior based on the Weyl structure on a statistical manifold.
result The Weyl prior is a special case of the α-parallel prior with α = -n, where n is the dimension of the statistical manifold.
Recent work has shown that deep generative models assign higher likelihood to out-of-distribution inputs than to training data. We show that a factor underlying this phenomenon is a mismatch between the nature of the prior distribution and that of the data distribution, a problem found in widely used deep generative mo…
Study shows generative priors improve rank-one matrix recovery with optimal sample complexity.
problem Recovering a rank-one signal matrix from noisy data with additional prior information.
method Analysis of a nonlinear least squares objective with a favorable global optimization landscape.
result Established optimal sample complexity for generative priors in rank-one matrix recovery.
SIGMA prior enables federated learning for non-factorizable models.
problem Current FL methods assume conditional independence, limiting applicability to non-factorizable models.
method SIGMA prior approximates deep generative model to induce conditional independence structure.
result SIGMA prior expands FL applicability to fields requiring modeling dependencies.
Bayesian method uses deep learning prior for CT reconstruction.
problem Imaging inverse problems in CT reconstruction.
method SA-Roundtrip prior with HMC-pCN sampler.
result Outperforms state-of-the-art methods in CT reconstruction.
A new method learns hierarchical EBM models with diffusion schemes.
problem Challenges in learning EBM models with multi-modal distributions.
method Proposes a diffusion probabilistic scheme to learn EBM models in hierarchical latent spaces.
result Demonstrates superior performance on various tasks with diffusion-learned EBM.
Bayesian inference is known to provide a general framework for incorporating prior knowledge or specific properties into machine learning models via carefully choosing a prior distribution. In this work, we propose a new type of prior distributions for convolutional neural networks, deep weight prior (DWP), that exploi…
While Bayesian methods are praised for their ability to incorporate useful prior knowledge, in practice, convenient priors that allow for computationally cheap or tractable inference are commonly used. In this paper, we investigate the following question: for a given model, is it possible to compute an inference result…
A new model trains prior and encoder/decoder networks simultaneously for efficient generation.
problem Complex autoregressive prior in VQ-VAE models leads to slow generation.
method Builds a diffusion bridge between continuous and non-informative prior distributions.
result Model is competitive and efficient in optimization and sampling.
Adversarial meta-learning computes Gamma-minimax estimators for vague prior knowledge.
problem Estimating parameters with vague prior knowledge.
method Adversarial meta-learning algorithms for Gamma-minimax estimators.
result Convergence guarantees and neural network class for selection.
New findings suggest latent regularization is unnecessary for high-quality image generation.
problem Improving image generation quality without latent regularization.
method Investigated the effect of latent regularization on image generation using learned priors.
result In the case of a sufficiently expressive prior, latent regularization is not necessary and may harm image quality.
Deep convolutional networks have become a popular tool for image generation and restoration. Generally, their excellent performance is imputed to their ability to learn realistic image priors from a large number of example images. In this paper, we show that, on the contrary, the structure of a generator network is suf…
In recent years, a rich variety of shrinkage priors have been proposed that have great promise in addressing massive regression problems. In general, these new priors can be expressed as scale mixtures of normals, but have more complex forms and better properties than traditional Cauchy and double exponential priors. W…
SAHMM-VAE separates sources adaptively using hidden Markov priors.
problem Unsupervised blind source separation.
method Source-wise adaptive Hidden Markov prior variational autoencoder.
result Different latent dimensions align with different source-specific temporal organizations.
Variational dropout (VD) is a generalization of Gaussian dropout, which aims at inferring the posterior of network weights based on a log-uniform prior on them to learn these weights as well as dropout rate simultaneously. The log-uniform prior not only interprets the regularization capacity of Gaussian dropout in netw…
Proposes using equivariant generative models for compressed sensing with unknown orientations.
problem Recovering signals with unknown orientations from underdetermined systems of linear measurements.
method Equivariant variational autoencoder as a generative prior for compressed sensing.
result Signals with unknown orientations can be recovered using iterative gradient descent on the latent space of equivariant models.
New method improves generative modeling on convex domains using regularized mirror maps and Student-t priors.
problem Challenges in generative modeling on convex domains with heavy-tailed targets.
method Mirror Flow Matching with regularized mirror maps and Student-t priors.
result Empirically outperforms baselines and achieves competitive sample quality.
Improved Bayesian inference using power priors with historical data.
problem Improving Bayesian inference with historical data.
method Generalized power priors that adapt to the α parameter of Amari's α-divergence. result Improved performance through appropriate choices of the α parameter. Bounded rationality investigates utility-optimizing decision-makers with limited information-processing power. In particular, information theoretic bounded rationality models formalize resource constraints abstractly in terms of relative Shannon information, namely the Kullback-Leibler Divergence between the agents' pr…
This research explores how different discrete diffusion kernels affect graph generation quality.
problem The impact of different discrete diffusion kernels on graph generation quality.
method Developed a family of discrete diffusion kernels that converge to different Bernoulli priors.
result The quality of generated graphs is sensitive to the prior used, challenging previous intuitions.
Bayesian algorithms perform well even with misspecified priors, especially in meta-learning.
problem Performance degradation of Bayesian algorithms with misspecified priors.
method Thompson sampling and meta-learning analysis with misspecified priors.
result Thompson sampling's performance degrades gracefully with misspecification, with a bound of ildeO(H2ε). Bayesian metalearning improves performance in linear bandits with misspecified priors.
problem Improper priors lead to suboptimal performance in sequential decision-making.
method Proves performance bounds for metalearning priors in stochastic linear bandits and develops a metalearning algorithm.
result Metalearning can improve performance by learning the prior from multiple tasks.
Residual Prior Diffusion integrates coarse latent priors with diffusion models for better generative tasks.
problem Diffusion models struggle with representing both large-scale and fine-scale details in data distributions.
method Two-stage framework: first a coarse prior model captures large-scale structure, then a diffusion model represents the residual.
result RPD accurately captures fine-scale details while preserving large-scale structure, outperforming standard diffusion models.
Deep neural networks as image priors have been recently introduced for problems such as denoising, super-resolution and inpainting with promising performance gains over hand-crafted image priors such as sparsity and low-rank. Unlike learned generative priors they do not require any training over large datasets. However…
X-VAE uses data-adaptive Gaussian priors to improve latent space modeling.
problem Limitations of standard Gaussian priors in complex datasets.
method Data-adaptive Gaussian prior derived from pretrained autoencoder latent codes.
result Improved latent space modeling and generation quality.
The empirically successful Thompson Sampling algorithm for stochastic bandits has drawn much interest in understanding its theoretical properties. One important benefit of the algorithm is that it allows domain knowledge to be conveniently encoded as a prior distribution to balance exploration and exploitation more eff…
Unrolled networks learn optimal Bayesian inference for unknown priors.
problem Optimizing Bayesian inference when the prior is unknown.
method Unrolling neural networks to simulate iterations of inference algorithms.
result Unrolled networks approximate convergence to optimal denoisers for product priors.
Algorithm estimates graph structure with prior information and Langevin diffusion.
problem Support estimation of partially known Gaussian graphical models.
method Proposes an algorithm using annealed Langevin diffusion and graph neural networks to estimate the posterior distribution of the graph.
result Demonstrates the benefits of the approach through numerical experiments.
Posterior sampling estimator achieves near-optimal recovery guarantees for signals from any prior distribution.
problem Characterizing measurement complexity for signals from any prior distribution, including the entire space.
method Characterization of measurement complexity using posterior sampling estimator for Gaussian measurements and any prior distribution.
result Posterior sampling estimator achieves near-optimal recovery guarantees for signals from any prior distribution, robust to model mismatch.
HyperBO+ pre-trains a universal prior for Bayesian optimization across different domains.
problem Bayesian optimization requires domain-specific priors, limiting its applicability.
method Two-step pre-training method for hierarchical Gaussian processes.
result HyperBO+ achieves lower regrets on unseen search spaces.
PriorGrad improves speech synthesis models by using data-dependent adaptive priors.
problem Inefficiency in denoising diffusion models due to mismatch between prior and data distributions.
method Proposes PriorGrad, an adaptive prior derived from data statistics based on conditional information.
result PriorGrad achieves faster convergence and superior performance in speech synthesis models.
Proposes a new model for testing causal structural priors and synthesizing data.
problem Testing and synthesizing causal structural priors using nonparametric knowledge and neural networks.
method Causal Structural Hypothesis Testing (C-SHT) and Causal Structural Variational Hypothesis Testing (C-SVHT) using deep neural networks.
result Demonstrates out-of-distribution generalization error as a proxy for causal structural prior hypothesis testing.