FTIP uses normalizing flows to improve posterior inference in function space.
problem Challenges in posterior inference with implicit-process priors.
method FTIP uses normalizing flows to define a richer variational distribution over combination weights.
result FTIP captures asymmetric and multimodal posterior structure better than Gaussian coefficient approximations.
This work explores function-space inference using KL divergence and proposes Bayesian linear regression as a benchmark.
problem Approximating the predictive posterior distribution of Bayesian models without parameter posterior approximation.
method Employing Kullback-Leibler divergence and proposing featurized Bayesian linear regression as a benchmark.
result Minimizing KL divergence leads to an ill-defined objective function, highlighting limitations of this approach.
Bayesian deep learning uses function-space priors to improve model uncertainty and robustness.
problem Bayesian deep learning struggles with model-specific weight-space priors that are hard to interpret and specify.
method Apply a Dirichlet prior in predictive space and perform approximate function-space variational inference.
result The approach improves uncertainty quantification, scalability, and adversarial robustness in large-scale image classification.
FHBI enhances generalization in Bayesian inference with iterative steps in functional spaces.
problem Improving generalization in Bayesian inference models.
method Iterative two-step procedure with adversarial and functional descent steps in a reproducing kernel Hilbert space.
result FHBI consistently outperforms nine baseline methods on the VTAB-1K benchmark.
GWI combines deep neural networks with Gaussian processes for better predictive performance and uncertainty quantification.
problem Combining deep learning with Gaussian process uncertainty quantification.
method Gaussian Wasserstein inference (GWI) using Wasserstein distance between Gaussian measures.
result GWI achieves state-of-the-art performance on benchmark datasets.
Develops theory for conditional optimal transport in infinite-dimensional spaces.
problem Bayesian inference with functional parameters in infinite-dimensional spaces.
method Theory of constrained optimal transport for block-triangular maps.
result Regularity estimates on conditioning maps from prior to posterior.
Proposes a new method for continual learning in neural networks.
problem Challenges in applying sequential Bayesian inference to neural networks.
method Sequential function-space variational inference.
result Neural networks trained with the proposed method achieve better predictive accuracy.
Paper addresses variational inference issues in Bayesian neural networks.
problem Negative infinite ELBO for function-space priors in BNNs.
method Regularized KL divergence for well-defined function-space variational inference.
result Method provides competitive uncertainty estimates for BNNs.
New method calibrates neural network uncertainty for medical images.
problem Uncalibrated probabilistic outputs from deep neural networks in medical diagnosis.
method Functional space variational inference for Bayesian neural networks.
result Better calibrated uncertainty estimates at lower computational cost.
The paper proposes a method for better uncertainty estimation in neural networks.
problem Estimating predictive uncertainty in neural networks is crucial but challenging.
method The paper proposes a function-space variational inference method to infer a posterior distribution over functions.
result The proposed method leads to state-of-the-art uncertainty estimation and predictive performance.
A new framework learns system design using neural features in function space.
problem Learning system design with neural feature extractors.
method Introduces feature geometry in function space, nesting technique for optimal feature approximation.
result Optimal features found from data samples using off-the-shelf architectures and optimizers.
Derives PF-ODE for infinite-dimensional functions, improving function generation tasks.
problem Efficient inference in infinite-dimensional diffusion models.
method Derives PF-ODE in infinite-dimensional function spaces.
result Reduces function evaluations while maintaining sample quality.
Flow Annealing Posterior Sampling unifies stochastic-process regression and PDE inverse problems.
problem Function-space posterior sampling for stochastic processes and inverse problems.
method Flow Annealing Posterior Sampling (FAPS) using pretrained function-space flow-matching priors.
result Coherent posterior samples with accurate uncertainty quantification.
Bayesian approach reduces FL communication cost by one-shot.
problem High communication cost in optimization-based FL for high-dimensional models.
method Bayesian pseudocoresets and function-space inference for one-shot FL.
result Achieves prediction performance competitive to state-of-the-art with up to 2 orders of magnitude reduction in communication cost.
Develops local elliptic regularity for geometrically-natural operators with low regularity coefficients.
problem Local elliptic regularity for operators with low regularity coefficients in Sobolev-type spaces.
method Rescaling estimates and multiplication results for function spaces.
result Unified set of interior estimates and regularity inference for operators with Sobolev-type coefficients.
New method tunes prior IP to data for flexible predictive distributions.
problem Challenges in approximate inference for large models with high parameter dependencies.
method Inducing-point representation of prior IP to approximate posterior process.
result Scalable method that tunes prior IP to data and provides accurate non-Gaussian predictive distributions.
Bayesian neural networks fail at out-of-distribution detection, revealing fundamental issues.
problem Out-of-distribution detection with Bayesian neural networks.
method Study of Bayesian inference with function space priors and comparison to Gaussian processes.
result Bayesian inference with function space priors does not lead to good OOD detection.
Efficiently quantifies uncertainty in DeepONets for function spaces.
problem Uncertainty quantification in deep operator networks.
method Randomized prior ensembles for frequentist inference.
result Improved robustness and accuracy, reliable uncertainty estimates, out-of-distribution detection, and model bias quantification.
Stein variational neural network ensembles improve diversity and uncertainty estimation.
problem Lack of proper Bayesian justification and diversity guarantees in deep neural network ensembles.
method Particle-based inference methods, specifically Stein variational gradient descent (SVGD), operating in weight space, function space, and hybrid settings.
result SVGD methods improve diversity and uncertainty estimation, approaching the true Bayesian posterior more closely.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
Study on Bayesian transformers finds issues with weight-space inference and prior specification.
problem Challenges in obtaining meaningful uncertainty estimates for transformer models.
method Proposed a novel method based on implicit reparameterization of the Dirichlet distribution for variational inference on attention weights.
result Proposed method performs competitively with baselines in estimating predictive uncertainty.
Gaussian processes are flexible function approximators, with inductive biases controlled by a covariance kernel. Learning the kernel is the key to representation learning and strong predictive performance. In this paper, we develop functional kernel learning (FKL) to directly infer functional posteriors over kernels. I…
Proposes adversarial method to estimate Riesz representer.
problem Estimating causal parameters as linear functionals of an underlying regression.
method Adversarial framework using general function spaces.
result Nonasymptotic mean square rate proved for neural networks, random forests, and RKHS.
New method guarantees global convergence in variational inference.
problem Limited convergence to local optima in variational inference.
method Minimizes inclusive KL divergence using neural networks and neural tangent kernel.
result Gradient descent dynamics converge to a unique solution in function space.
DVIP improves on IP-based methods by using IPs as priors over latent functions.
problem Limited expressiveness of IP-based models, especially in function space.
method Proposes DVIP, a multi-layer generalization of IPs, and scalable variational inference.
result DVIP outperforms previous IP-based methods and deep GPs in regression and classification tasks.
Bayesian neural networks ignore data in infinite units limit.
problem Pathological behavior of posterior in over-parameterized networks.
method Mean-field variational inference in infinite hidden units limit.
result Posterior mean converges to zero, ignoring data.
Develops methods to measure and set function-space learning rates in neural networks.
problem Measuring and optimizing changes in neural network output functions.
method Efficient methods to measure and set function-space learning rates, requiring minimal computational overhead.
result Demonstrates FLeRM (Function-space Learning Rate Matching) for hyperparameter transfer across model scales.
New method for Bayesian neural networks reduces inference difficulty.
problem Difficulty in sample-based inference for Bayesian neural networks.
method Embracing mode-connectedness to link overparameterization and sampling difficulty.
result Practical guidelines and deep ensemble approach for effective SBI.
Unified framework for Bayesian PDE-constrained inversion using physics-informed neural networks.
problem Incorporating prior distributions in function space into Bayesian PINN-based inversion.
method Functional-prior-based approaches (fpBPINN) to Bayesian PDE-constrained inversion using physics-informed neural networks (PINNs). Two complementary approaches: FPI-BPINN and fParVI-PINN.
result Accurate estimation of posterior distributions in seismic traveltime tomography and Darcy-flow permeability inversion.
Paper characterizes embeddability of function spaces into Lp-type RKBS via metric entropy.
problem Characterizing embeddability of function spaces into Lp-type RKBS. method Establishes a connection between metric entropy growth and embeddability.
result A bound on metric entropy growth allows embedding into Lp-type RKBS. Deep Gaussian Processes are reinterpreted as deep trigonometric networks for tractable inference.
problem Challenging inference in DGPs due to intractable marginalization in latent function space.
method Viewing DGPs as deep trigonometric networks with Bochner's theorem, and using the wide limit with a bottleneck to translate DGPs into deep trigonometric networks.
result The weight space view yields the same effective covariance functions as obtained in function space, and varying prior distributions over network parameters is equivalent to employing different kernels.
FunDPS improves PDE solution recovery from sparse data.
problem Recovering whole solutions from sparse or noisy measurements in PDEs.
method Function-space diffusion model with gradient-based guidance.
result FunDPS achieves 32% accuracy improvement over state-of-the-art methods.
MARS meta-learns function scores for improved predictive accuracy and uncertainty.
problem Difficulty in specifying expressive priors for Bayesian meta-learning.
method Meta-learning the score function of data-generating process marginals in the function space.
result State-of-the-art predictive accuracy and improved uncertainty estimates.
Gradient descent reshapes the function space of neural networks.
problem Understanding how feature learning affects the function space of neural networks.
method Characterized the evolution of the feature space during training using a two-layer neural network.
result Gradient descent induces a data-adaptive deformation that selectively enhances signal-aligned directions.
New method uses differential equations for better counterfactual analysis.
problem Estimating counterfactual outcomes for policy analysis.
method Continuous-time approach to synthetic controls using controlled differential equations.
result Improves counterfactual estimation for irregularly aligned multivariate time series.
Optimizes learning Hilbert-Schmidt operators between Sobolev spaces.
problem Statistical limits of learning mappings between infinite-dimensional function spaces.
method Minimax optimal regularization and multilevel training.
result Multilevel kernel operator learning achieves optimal learning rate.
The Brownian bridge serves as a physics-informed prior for solving the Poisson equation.
problem Reconstructing physical fields from limited and noisy data with known governing equations.
method Formalizing inverse problems via Bayesian inference in function spaces using a Brownian bridge Gaussian process.
result The Brownian bridge Gaussian process can be viewed as a physics-constrained prior for the Poisson equation, allowing for a fully Bayesian framework.
Method converts neural networks to function space for better uncertainty quantification.
problem Lack of uncertainty estimates and difficulty in incorporating new data in deep neural networks.
method Dual parameterization to convert from weight space to function space, enabling sparse representation.
result Compact and principled way to capture uncertainty and incorporate new data.
A new method converts neural networks to function space for scalable sequential learning.
problem Challenges in gradient-based deep learning for sequential data.
method Dual parameterization of neural networks from weight to function space.
result Efficient scaling, knowledge retention, and new data incorporation.
New embedding method in function spaces improves expressiveness.
problem Enhancing expressiveness in knowledge graph embeddings.
method Employing polynomial functions and neural networks with varying layer complexities.
result Improved expressiveness and more degrees of freedom in entity representation.
We extend diffusion models to function spaces and introduce a new method for sampling from posterior distributions.
problem Sampling from posterior distributions in infinite-dimensional function spaces using diffusion models.
method Infinite-dimensional extension of Doob's h-transform, Supervised Guidance Training for efficient sampling. result We prove that diffusion models can be conditioned to sample from posterior distributions and introduce a simulation-free score matching objective.
A new machine learning method for Bayesian inverse problems in function spaces.
problem Bayesian inverse problems in function spaces with incompatibility of white noise sources.
method One-step generative transport with amortized neural operator and prior-aligned Gaussian random field.
result Generative operator trained on prior samples and noisy observations generates posterior samples efficiently.
This work uses sampling theory to analyze smoothness and error bounds of finite neural networks.
problem Analyzing the function space of finite neural networks and providing error bounds.
method Applying sampling theory to finite neural networks with non-expansive activation functions, considering both deterministic and random sampling.
result Novel error bounds for univariate neural networks under band-limited input assumption, highlighting the advantage of deterministic uniform sampling.
Hi-fi priors enhance BNNs by learning flexible activations.
problem Challenging to impose function-space priors on BNNs.
method Optimization techniques to learn flexible activations.
result BNNs with flexible activations can achieve desired priors.
FFM generates functions between Gaussian and data distributions.
problem Generating functions between Gaussian and data distributions.
method Define a path of measures, learn a vector field to generate this path.
result FFM outperforms other function-space generative models.
Defines manifolds of mappings between function spaces and discusses their properties.
problem Defining smooth manifolds of mappings between function spaces.
method Defines a smooth manifold structure on sets of continuous mappings and discusses properties of natural mappings.
result Properties of spaces of sections and smoothness of natural mappings between spaces of mappings.
The paper defines function spaces on manifolds with bounded or singular geometries.
problem Defining function spaces on manifolds with various geometries.
method Introduces and analyzes Sobolev, Besov, and Bessel potential spaces on uniformly regular and singular Riemannian manifolds.
result Demonstrates maximal regularity for a linear parabolic problem on singular manifolds.
Modified training direction reduces generalization error in neural networks.
problem Reducing generalization error in neural networks.
method Theoretical analysis of modified natural gradient descent in function space.
result Modifying training direction in function space reduces total generalization error.