Hi-fi priors enhance BNNs by learning flexible activations.
problem Challenging to impose function-space priors on BNNs.
method Optimization techniques to learn flexible activations.
result BNNs with flexible activations can achieve desired priors.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “flexible priors”
PFNs4BO uses neural processes for flexible Bayesian Optimization.
problem Efficient surrogate modeling for Bayesian Optimization.
method In-context learning of PFNs to approximate posterior predictive distribution.
result PFNs outperform traditional GP, BNN in BO tasks.
Neural network based models for collaborative filtering have started to gain attention recently. One branch of research is based on using deep generative models to model user preferences where variational autoencoders were shown to produce state-of-the-art results. However, there are some potentially problematic charac…
GOAT improves attention mechanisms by learning better priors.
problem Standard attention mechanisms use a naive uniform prior, limiting flexibility and generalization.
method GOAT introduces a trainable, continuous prior that replaces the uniform assumption, maintaining compatibility with optimized kernels.
result GOAT avoids representational trade-offs and learns an extrapolatable prior that combines positional flexibility with length generalization.
AR-Flow VAE improves blind source separation with flexible autoregressive priors.
problem Unsupervised blind source separation of latent signals from mixtures.
method AR-Flow VAE uses autoregressive flows to model latent sources, enhancing flexibility and capturing complex dependencies.
result AR-Flow VAE effectively separates latent sources, demonstrating improved performance over conventional methods.
From Partial Exchangeability to Predictive Probability: A Bayesian Perspective on Classificationstat.ME
Bayesian model improves classification performance with flexible uncertainty modeling.
problem Improving classification performance with flexible uncertainty modeling.
method Combines Gaussian process and Dirichlet process priors for latent function and link function, respectively.
result Outperforms standard logistic regression on simulated data.
C-VAE improves VAE by resolving prior issues and generating better samples.
problem Low-quality samples from VAE due to prior issues.
method Formulates VAE as OT, allows flexible priors, and uses OT formulations.
result C-VAE generates higher quality samples and latent representations.
DeepRV accelerates spatiotemporal inference using neural priors.
problem Intractable scaling of Gaussian Processes for large datasets.
method Neural-network surrogate replacing GP prior sampling with complexity.
result DeepRV achieves highest fidelity to exact GPs while significantly speeding up inference.
Empirical Gaussian Processes learn flexible priors from data.
problem Limited effectiveness of standard Gaussian process kernels.
method Estimate mean and covariance functions empirically from data.
result Empirical GPs converge to closest GP to real data generating process.
We consider the problem of training generative models with deep neural networks as generators, i.e. to map latent codes to data points. Whereas the dominant paradigm combines simple priors over codes with complex deterministic models, we argue that it might be advantageous to use more flexible code distributions. We de…
New method tunes prior IP to data for flexible predictive distributions.
problem Challenges in approximate inference for large models with high parameter dependencies.
method Inducing-point representation of prior IP to approximate posterior process.
result Scalable method that tunes prior IP to data and provides accurate non-Gaussian predictive distributions.
Labels distilled from images improve model training efficiency and flexibility.
problem Creating synthetic labels for a small set of real images to train models effectively.
method Introduce a more robust and flexible meta-learning algorithm for distillation and an effective first-order strategy based on convex optimization layers.
result Label distillation leads to improved results and greater flexibility in neural architectures.
We propose a novel approach for nonlinear regression using a two-layer neural network (NN) model structure with sparsity-favoring hierarchical priors on the network weights. We present an expectation propagation (EP) approach for approximate integration over the posterior distribution of the weights, the hierarchical s…
A new method for deep Wishart processes improves kernel-based models.
problem Inference in deep Wishart processes is challenging due to the need for flexible distributions over positive semi-definite matrices.
method Developed a novel approach to flexible distributions over positive semi-definite matrices using the Bartlett decomposition of the Wishart probability density. Used this to create an approximate posterior for the DWP.
result Improved performance of inference in the DWP compared to DGP with equivalent prior.
Introduces PELP for graph-enhanced word embeddings.
problem Combining graph side-information into static word embeddings.
method Probabilistic embeddings using Laplacian priors.
result Unified and flexible approach to various embedding methods.
BI-EqNO: Generalized Approximate Bayesian Inference with an Equivariant Neural Operator Frameworkstat.ML
BI-EqNO improves Bayesian inference with flexible neural operators.
problem Inaccurate estimation of marginal likelihoods in approximate Bayesian methods.
method Equivariant neural operator framework for generalized approximate Bayesian inference.
result BI-EqNO enhances both deterministic and stochastic approaches to Bayesian inference.
Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.
problem Sparse logistic regression challenges in machine learning.
method Empirical Bayes approach with mean-field variational inference, tuning-free and scalable.
result Superior predictive performance in sparse logistic regression compared to existing methods.
Improves robustness of information bottleneck framework with sparsity-inducing prior.
problem Fixed-dimensional priors restrict flexibility and restrict robustness.
method Sparsity-inducing spike-slab categorical prior that learns dimension distribution per data point.
result Improves accuracy and robustness compared to traditional priors and other methods.
Neural-g models mixtures of densities with flexibility and accuracy.
problem Accurately estimating prior densities in -models.
method Neural network with softmax output for valid probability densities.
result Neural-g captures various prior shapes including flat, heavy-tailed, and discontinuous.
An implementation of a nonparametric Bayesian approach to solving binary classification problems on graphs is described. A hierarchical Bayesian approach with a randomly scaled Gaussian prior is considered. The prior uses the graph Laplacian to take into account the underlying geometry of the graph. A method based on a…
Flexible empirical Bayes for large-scale multiple linear regression.
problem Large-scale multiple linear regression with flexible priors and efficient computation.
method Adaptive shrinkage priors combined with variational approximations for hyperparameter estimation.
result The posterior mean from the empirical Bayes method solves a penalized regression problem.
In this paper we propose a class of prior distributions on decomposable graphs, allowing for improved modeling flexibility. While existing methods solely penalize the number of edges, the proposed work empowers practitioners to control clustering, level of separation, and other features of the graph. Emphasis is placed…
Framework adapts to new tasks based on prior knowledge.
problem Models struggle to adapt to novel tasks without direct experience.
method Learned task representations and meta-mappings to transform them.
result Meta-mapping achieves 80-90% performance on novel tasks.
We propose Learned Accept/Reject Sampling (LARS), a method for constructing richer priors using rejection sampling with a learned acceptance function. This work is motivated by recent analyses of the VAE objective, which pointed out that commonly used simple priors can lead to underfitting. As the distribution induced …
We present a non-parametric Bayesian latent variable model capable of learning dependency structures across dimensions in a multivariate setting. Our approach is based on flexible Gaussian process priors for the generative mappings and interchangeable Dirichlet process priors to learn the structure. The introduction of…
Improves joint distribution learning for high-dimensional datasets with complex correlations.
problem Conditional independence assumption limitations in VAE decoders for high-dimensional datasets.
method Cramer-Wold distance regularization and two-step learning method for flexible prior modeling.
result Effective joint distributional learning for high-dimensional datasets with multiple categorical variables.
New PAC-Bayesian framework for flexible meta-learning.
problem Limitations in transferring knowledge between tasks in meta-learning.
method PAC-Bayesian theory applied to learning the learning algorithm.
result Flexibility in meta-learning mechanisms and improved prediction quality.
We consider a nonlinear state-space model with the state transition and observation functions expressed as basis function expansions. The coefficients in the basis function expansions are learned from data. Using a connection to Gaussian processes we also develop priors on the coefficients, for tuning the model flexibi…
Inference is an integral part of probabilistic topic models, but is often non-trivial to derive an efficient algorithm for a specific model. It is even much more challenging when we want to find a fast inference algorithm which always yields sparse latent representations of documents. In this article, we introduce a si…
FlexAE addresses bias-variance trade-off in RAEs by learning latent priors.
problem Improving generation quality of deterministic AE models.
method Introducing flexibly learnable latent priors in WAEs to optimize the latent distribution.
result FlexAE achieves state-of-the-art performance in AE-based generative models.
Novel prior for orthogonal functions improves functional component estimation.
problem Improving orthogonality in functional principal component analysis.
method Sequential adaptive priors for orthogonal functions using hierarchical conditionally normal distributions.
result Proposed prior leads to nearly orthogonal posterior estimates.
Exemplar-based clustering methods have been shown to produce state-of-the-art results on a number of synthetic and real-world clustering problems. They are appealing because they offer computational benefits over latent-mean models and can handle arbitrary pairwise similarity measures between data points. However, when…
In variational autoencoders, the prior on the latent codes is often treated as an afterthought, but the prior shapes the kind of latent representation that the model learns. If the goal is to learn a representation that is interpretable and useful, then the prior should reflect the ways in which the high-level fact…
Student- processes have recently been proposed as an appealing alternative non-parameteric function prior. They feature enhanced flexibility and predictive variance. In this work the use of Student- processes are explored for multi-objective Bayesian optimization. In particular, an analytical expression for the h…
The paper provides theoretical guarantees for transformation-based models in variational inference.
problem Theoretical justification for transformation-based models in variational inference.
method Theoretical analysis of non-linear latent variable models and Gaussian process priors.
result Theoretical guarantees for implicit variational inference, achieving optimal risk bounds and approximating the true posterior.
Bayesian deep learning uses function-space priors to improve model uncertainty and robustness.
problem Bayesian deep learning struggles with model-specific weight-space priors that are hard to interpret and specify.
method Apply a Dirichlet prior in predictive space and perform approximate function-space variational inference.
result The approach improves uncertainty quantification, scalability, and adversarial robustness in large-scale image classification.
Flexible model for complex relationships using Bayesian nonparametrics.
problem Complex relationships between variables not well captured by simple models.
method Hierarchical generation of nonlinear features, Bayesian inference, variable selection.
result Find interpretable models with a small set of important features.
The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult,…
Unified framework simplifies DPO algorithms for LLM alignment.
problem Vast number of DPO variants complicates model alignment.
method Mutual information inspired unifying framework with flexible priors.
result Many DPO variants can be derived from the new framework.
GRASP simplifies Bayesian regression with grouped predictors using an adaptive NBP prior.
problem Regression with grouped predictors and adaptive shrinkage.
method Normal Beta Prime (NBP) prior with tunable hyperparameters for flexible sparsity control.
result Empirical validation of robust and versatile GRASP across various sparsity and signal-to-noise ratios.
Hida-Matérn Kernelstat.ML
New Hida-Matérn kernels enable flexible process priors and efficient GP inference.
problem Flexible modeling of stationary processes with oscillatory components.
method Introducing a new class of covariance functions (Hida-Matérn kernels) and their state space representations.
result Efficient Gaussian Process inference and improved numerical stability.
DRMMs enable flexible conditional sampling for interactive machine learning.
problem Limited flexibility in conditional sampling for deep generative models.
method Proposes Deep Residual Mixture Models (DRMMs) that allow flexible conditional sampling.
result DRMMs enable sampling with arbitrary combinations of conditioning variables and priors.
A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Liftingcs.LG
SOBER framework optimizes Bayesian optimization tasks efficiently.
problem Challenges in parallel Bayesian optimization.
method Probabilistic Lifting with Kernel Quadrature.
result Versatile and flexible batch Bayesian optimization.
Although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors, time and other factors, relatively little has been done in the multivariate case. Our focus is on developing a class of nonparametric covariance regression models, which allow an unknown p …
Probabilistic neural networks are typically modeled with independent weight priors, which do not capture weight correlations in the prior and do not provide a parsimonious interface to express properties in function space. A desirable class of priors would represent weights compactly, capture correlations between weigh…
Improved Bayesian inference using power priors with historical data.
problem Improving Bayesian inference with historical data.
method Generalized power priors that adapt to the parameter of Amari's -divergence.
result Improved performance through appropriate choices of the parameter.
A simple, flexible approach to creating expressive priors in Gaussian process (GP) models makes new kernels from a combination of basic kernels, e.g. summing a periodic and linear kernel can capture seasonal variation with a long term trend. Despite a well-studied link between GPs and Bayesian neural networks (BNNs), t…
New LSTM scheme incorporates prior knowledge and measurement uncertainties.
problem Overfitting and insufficient data for accurate time-dependent solutions.
method Sparse Bayesian training algorithm for automatic connection determination.
result Less prone to overfitting, smaller data set required for satisfying accuracy.