We study convergence rates of variational posterior distributions for nonparametric and high-dimensional inference. We formulate general conditions on prior, likelihood, and variational class that characterize the convergence rates. Under similar "prior mass and testing" conditions considered in the literature, the rat…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “posterior convergence rate”
Rates of Convergence of Generalised Variational Inference Posteriors under Prior Misspecificationmath.ST
Study improves convergence rates for GVI under prior misspecification.
problem Improving convergence rates for GVI under prior misspecification.
method Proves rates of convergence and robustness to prior misspecification in GVI framework.
result Establishes sufficient conditions for existence and uniqueness of GVI posteriors.
Bayesian method with Gaussian process priors achieves optimal convergence rates for regression function and its derivatives.
problem Estimating the regression function and its derivatives in nonparametric regression.
method Bayesian approach with Gaussian process priors, focusing on convergence rates and plug-in property.
result Equivalence of convergence rates of posterior distributions and Bayes estimators for regression function and its derivatives.
Empirical Bayes rates via variational approximations and prior decomposition.
problem Nonparametric and high-dimensional inference convergence rates.
method Variational perspective and prior decomposition.
result Empirical Bayes posterior rates derived from variational Bayes.
Gaussian OBFS proves strong consistency in feature selection with correlations.
problem Feature selection consistency in the presence of correlations.
method Proves strong consistency of Gaussian OBFS under mild conditions.
result Identifies selected features and rates of convergence for different feature types.
Bayesian neural network achieves nearly optimal performance in Besov space.
problem Bayesian neural networks in Besov space.
method Spike-and-slab prior and shrinkage prior for posterior convergence rate.
result The posterior convergence rate is nearly minimax and adaptive to unknown smoothness.
Moving Target Monte Carlostat.CO
New algorithm MTMC reduces MCMC evaluation costs.
problem High-dimensional sampling with intractable posterior evaluations.
method Iteratively updated approximation of posterior distribution for acceptance rate.
result Approximation converges to true posterior as iterations increase.
One of the core problems in statistical models is the estimation of a posterior distribution. For topic models, the problem of posterior inference for individual texts is particularly important, especially when dealing with data streams, but is often intractable in the worst case. As a consequence, existing methods for…
Guarantees convergence for black-box variational inference without modifications.
problem Convergence guarantees for black-box variational inference.
method Analysis of log-smooth posterior densities, location-scale variational family, and convergence rates of algorithm design choices.
result Proximal stochastic gradient descent fixes suboptimal convergence rates and achieves strongest known guarantees.
Boosting Variational Inference improves posterior approximations with adaptive step-sizes.
problem Limited resources hinder the widespread adoption of Boosting Variational Inference.
method Characterized global curvature impact, introduced local curvature, and developed an approximate backtracking algorithm.
result New theoretical convergence rates and experimental validation demonstrate improved performance.
Improves understanding of stochastic NGVI convergence rates.
problem Lack of knowledge about non-asymptotic convergence rates in stochastic NGVI.
method Proved non-asymptotic convergence rates for conjugate likelihoods and showed implicit optimization for non-conjugate likelihoods.
result First non-asymptotic convergence rate for stochastic NGVI in conjugate likelihoods.
It is now known that an extended Gaussian process model equipped with rescaling can adapt to different smoothness levels of a function valued parameter in many nonparametric Bayesian analyses, offering a posterior convergence rate that is optimal (up to logarithmic factors) for the smoothness class the true function be…
Gaussian process (GP) regression is a powerful interpolation technique due to its flexibility in capturing non-linearity. In this paper, we provide a general framework for understanding the frequentist coverage of point-wise and simultaneous Bayesian credible sets in GP regression. As an intermediate result, we develop…
New bounds on neural network convergence using information theory.
problem Quantifying convergence rates of neural networks to Gaussian distributions.
method Entropic inequalities and Gaussian approximations.
result Improved convergence rates in various distances for neural networks.
Enhanced Gaussian process models accelerate optimization and posterior approximation.
problem Improving the accuracy and speed of Gaussian process models for optimization and inference.
method Introduces a random exploration step to classical GP-UCB algorithms, facilitating faster convergence.
result New algorithms achieve nearly optimal convergence rates and provide bounds for Hellinger distance.
Paper establishes statistical validity for variational Bayes in neural networks.
problem Lack of theoretical validity for Variational Bayes in Bayesian Neural Networks.
method Establishes posterior consistency for mean-field variational posterior in feed-forward neural networks.
result Proves VP concentrates around Hellinger neighborhoods of true density function under certain conditions.
New approach to quantify posterior concentration rates using Wasserstein dynamics.
problem Quantifying the speed of posterior distribution concentration in Bayesian statistics.
method Combining local Lipschitz-continuity with dynamic formulation of Wasserstein distance.
result Optimal posterior contraction rates in finite and infinite-dimensional models.
Enhances SGLD for log-concave posteriors with asynchronous computation.
problem Sampling log-concave posterior distributions efficiently.
method Integrates asynchronous computation into SGLD with delayed gradients.
result Convergence in measure is not significantly affected by delayed gradient information.
Bayesian approach learns linear operators from noisy data.
problem Learning linear operators from noisy data in infinite-dimensional spaces.
method Bayesian approach with Gaussian priors.
result Establishes posterior contraction rates and generalization error guarantees.
A recently-introduced class of probabilistic (uncertainty-aware) solvers for ordinary differential equations (ODEs) applies Gaussian (Kalman) filtering to initial value problems. These methods model the true solution and its first derivatives \emph{a priori} as a Gauss--Markov process , which is…
Improved Bayesian inference via variational approximations of generalized rho-posteriors.
problem Robust Bayesian inference under model misspecification and data contamination.
method Introducing a modified -posterior and using PAC-Bayesian analysis with variational approximations.
result Theoretical guarantees for tractable inference with competitive robustness and computational efficiency.
Approximate Bayesian Computation (ABC) is a method to obtain a posterior distribution without a likelihood function, using simulations and a set of distance metrics. For that reason, it has recently been gaining popularity as an analysis tool in cosmology and astrophysics. Its drawback, however, is a slow convergence r…
Variational inference is becoming more and more popular for approximating intractable posterior distributions in Bayesian statistics and machine learning. Meanwhile, a few recent works have provided theoretical justification and new insights on deep neural networks for estimating smooth functions in usual settings such…
ABI bypasses likelihood intractability with nonparametric distribution matching.
problem Approximate Bayesian computation's inefficiency in high-dimensional settings and under diffuse priors.
method Adaptive Bayesian Inference (ABI) compares posterior distributions directly using nonparametric distribution matching and MSW distance.
result ABI significantly outperforms other methods in high-dimensional or dependent observation regimes.
Proposes a Bayesian federated learning method for diverse tasks.
problem Current federated learning approaches focus on homogeneous tasks, ignoring task diversity.
method Integrates multi-task learning with MOGP at the local level and federated learning at the global level.
result Demonstrates superior predictive performance and uncertainty calibration on diverse tasks.
Bayesian neural networks approximate Student-t processes in the infinite-width limit.
problem Modeling uncertainty in neural networks with greater flexibility.
method Extending asymptotic properties of Gaussian processes to Student-t processes in the infinite-width limit of BNNs.
result Posterior BNNs converge to Student-t processes in the infinite-width limit.
New method uses public data to achieve optimal nonparametric classification with privacy constraints.
problem Optimal nonparametric classification under local differential privacy constraints.
method Locally differentially private classification tree with data-driven pruning.
result Achieves mini-max optimal convergence rate with public data.
Stochastic Gradient Descent (SGD) is an important algorithm in machine learning. With constant learning rates, it is a stochastic process that, after an initial phase of convergence, generates samples from a stationary distribution. We show that SGD with constant rates can be effectively used as an approximate posterio…
We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests. We provide a scalable algorithm for efficiently solving this problem with rigorous guarantees. In particular, we prove the convergence rate of our posterior update rule to the true probability function in…
We study the asymptotic consistency properties of -Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the -Rényi divergence from the true posterior. Unique to o…
Normalized random measures (NRMs) provide a broad class of discrete random measures that are often used as priors for Bayesian nonparametric models. Dirichlet process is a well-known example of NRMs. Most of posterior inference methods for NRM mixture models rely on MCMC methods since they are easy to implement and the…
This paper analyzes kNN convergence over feature transformations.
problem The curse of dimensionality affects kNN performance in transformed feature spaces.
method Developed a novel analysis on kNN convergence rates over transformed features, linking properties of the transformed space to raw feature space.
result Theoretical analysis explains why some feature transformations are better for kNN.
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
Oracle inequality for sparse neural nets adapts to unknown structure.
problem Sparse deep neural nets in nonparametric regression.
method Gibbs posterior distribution with Metropolis-adjusted Langevin algorithms and mixture of uniform priors.
result Oracle inequality showing adaptation to unknown regularity and structure, achieving minimax-optimal rate of convergence.
Meta-learning reformulated as Bayesian risk minimization.
problem Learning models to quickly adapt to new tasks from small datasets.
method Formalized meta-learning as Bayesian risk minimization, using a probabilistic framework to compute predictive distributions from posterior distributions of latent variables conditioned on contextual datasets.
result A novel Gaussian approximation for the posterior distribution that converges to maximum likelihood estimates and outperforms Neural Process on benchmark datasets.
Advocates for a new posterior that predicts better than classical and generalised Bayes.
problem Combining parameter inference and density estimation for better predictive models.
method Predictively Oriented (PrO) posterior using mean field Langevin dynamics.
result PrO posteriors converge to the predictively optimal model average, adapting to model misspecification.
This paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models, using transportation distances (i.e., Wasserstein metrics). The relationship between Wasserstein distances on the space of mixing measures and f-divergence functionals such as Hellinger and Kullback-Leibl…
We investigate a local reparameterizaton technique for greatly reducing the variance of stochastic gradients for variational Bayesian inference (SGVB) of a posterior over model parameters, while retaining parallelizability. This local reparameterization translates uncertainty about global parameters into local noise th…
Bayesian PINNs learn elliptic PDEs with near-minimax posterior contraction rate.
problem Learning elliptic PDEs with noisy data and non-homogeneous boundary conditions.
method Bayesian approach with a Hölder space prior on neural network weights.
result Posterior contracts at near-minimax rate without prior knowledge of solution smoothness.
High-dimensional VAEs inevitably collapse to prior, requiring large datasets for good performance.
problem Posterior collapse in VAEs leads to poor representation learning quality.
method Analyzed a minimal VAE in a high-dimensional limit, evaluating conditions for posterior collapse with respect to beta and dataset size.
result VAEs face 'inevitable posterior collapse' beyond a certain beta threshold, regardless of dataset size.
Variational inference is a popular technique to approximate a possibly intractable Bayesian posterior with a more tractable one. Recently, boosting variational inference has been proposed as a new paradigm to approximate the posterior by a mixture of densities by greedily adding components to the mixture. However, as i…
Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of th…
Hamiltonian Monte Carlo is a powerful algorithm for sampling from difficult-to-normalize posterior distributions. However, when the geometry of the posterior is unfavorable, it may take many expensive evaluations of the target distribution and its gradient to converge and mix. We propose neural transport (NeuTra) HMC, …
Statistical inference of analytically non-tractable posteriors is a difficult problem because of marginalization of correlated variables and stochastic methods such as MCMC and VI are commonly used. We argue that stochastic KL divergence minimization used by MCMC and VI is noisy, and we propose instead EL_2O, expectati…
The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow…
We prove exact BNN posterior convergence to GP limit and provide sampling methods.
problem Theoretical and empirical challenges in obtaining exact posterior distributions of wide BNNs.
method Theoretical proof and rejection sampling for generating exact samples.
result Exact BNN posterior converges to GP limit as width increases.
Develops a fast variational approximation for high-dimensional empirical Bayes posteriors.
problem Optimal posterior computation in high-dimensional settings with prior tails effect.
method Variational approximation of empirical Bayes posterior with data-driven centers and thin-tailed conjugate priors.
result Retains optimal concentration rate properties and superior performance compared to existing methods.
Paper proposes a second-order method for faster SVI convergence.
problem Poor convergence rate of first-order SVI algorithms.
method Derives Hessian matrix and implements two numerical schemes for efficient second-order SVI.
result Proposed approach achieves faster convergence compared to first-order SVI.