Study improves convergence rates for GVI under prior misspecification.
problem Improving convergence rates for GVI under prior misspecification.
method Proves rates of convergence and robustness to prior misspecification in GVI framework.
result Establishes sufficient conditions for existence and uniqueness of GVI posteriors.
Online DEM improves tracking of latent states in dynamic systems.
problem Tracking latent states in dynamic systems with online updates.
method Specializes DEM for online data assimilation, separating temporal scales.
result ODEM can track latent states of a non-linear generative model.
FedGVI improves FL robustness to model misspecification.
problem Limited robustness in FL approaches to model misspecification.
method Probabilistic Federated Learning framework that generalizes previous methods.
result FedGVI provides robust and calibrated predictions under model misspecification.
New algorithm minimizes FE objectives for synthetic AIF agents.
problem Minimizing Free Energy for synthetic active inference agents.
method Variational message passing on Constrained FFGs.
result Message passing algorithms induce epistemic behavior in AIF agents.
This work connects SAM to variational inference and evaluates its performance.
problem Improving generalization of gradient-based learning by finding flat minima.
method Establishes connections between SAM and Mean-Field Variational Inference (MFVI), and evaluates variational algorithms combining or interpolating between SAM and MFVI.
result SAM-like updates can be used as a drop-in replacement for the reparametrisation trick.
Gaussian processes (GPs) are a good choice for function approximation as they are flexible, robust to over-fitting, and provide well-calibrated predictive uncertainty. Deep Gaussian processes (DGPs) are multi-layer generalisations of GPs, but inference in these models has proved challenging. Existing approaches to infe…
New methods for scalable inference in modular models with misspecified sub-models.
problem Model misspecification in multi-modular models complicates evidence combination.
method Variational methods for approximating Cut and SMI posteriors, and Variational Meta-Posterior.
result Feasibility of analysis with multiple cuts using a single set of variational parameters.
A scalable GPLVM model using stochastic variational inference.
problem Scalable inference for Gaussian process latent variable models.
method Doubly stochastic formulation of Bayesian GPLVM with minibatch training.
result High-fidelity reconstructions in the presence of missing data.
A scalable MOGP model with stochastic variational inference for many outputs.
problem Efficiently modeling data from multiple sources with many outputs.
method Stochastic variational inference for Latent Variable MOGP (LV-MOGP).
result Computational complexity per iteration is independent of the number of outputs.
We consider a Gaussian process formulation of the multiple kernel learning problem. The goal is to select the convex combination of kernel matrices that best explains the data and by doing so improve the generalisation on unseen data. Sparsity in the kernel weights is obtained by adopting a hierarchical Bayesian approa…
Improved variational approximation for deep Wishart process models.
problem Improving predictive performance of deep Wishart process models.
method Generalizing the Bartlett decomposition of the Wishart distribution to allow linear combinations of rows and columns.
result Better predictive performance achieved with minimal additional computation cost.
This research improves multimodal systems by adding a second objective and regularisation methods.
problem Improving performance of multimodal systems with multiple objectives and regularisation.
method Introduces a second objective over multimodal fusion using variational inference and regularisation methods.
result Demonstrates potential for multiple objectives and probabilistic methods to lower variance and improve generalisation.
We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables. IPs are therefore highly flexible implicit priors over functions, with examples including data simulators, Bayesian neural networks and non-linear tr…
New insights into variational inference using Monte Carlo estimates.
problem Improving variational bounds in latent variable models.
method Analyzing properties of Monte Carlo estimates and their impact on variational gaps.
result Negative correlation reduces variational gaps, contrary to intuition.
Unified theory linking Bayesian and ensemble methods in deep learning.
problem Uncertainty quantification in deep learning.
method Reformulating optimisation as convex optimisation in probability measures, studying Wasserstein gradient flows.
result Unified theory explaining success of deep ensembles over variational inference.
Researchers develop methods to calibrate ABMs using Bayesian techniques.
problem Challenges in calibrating ABMs due to likelihood intractability and non-differentiability.
method Generalised variational inference for differentiable ABMs.
result Accurate Bayesian parameter inferences for differentiable ABMs demonstrated.
Robust Kalman filtering method for outlier detection.
problem Outliers and misspecified measurement models in state-space models.
method Combines generalised Bayesian inference with Kalman filters for robustness and efficiency.
result Matches or outperforms other robust filtering methods at lower computational cost.
Enhances robustness in experimental design through Generalised Bayesian inference.
problem Poor inference and estimates of information gain when statistical model is incorrectly specified.
method Generalised Bayesian (Gibbs) inference framework applied to experimental design.
result GBOED enhances robustness to outliers and incorrect assumptions about noise distribution.
Clarifies EM algorithm and variational Bayesian inference concepts.
problem Gaps in AI literature understanding of EM and variational concepts.
method Tutorial presentation of EM algorithm, variational Bayesian inference, and autoencoded variational Bayes.
result Establishes clear links between EM and variational methods.
Develops a new Bayesian inference method for discrete data.
problem Computational challenges in discrete state spaces, especially intractable likelihoods.
method Uses a discrete Fisher divergence to update beliefs about model parameters, circumventing the intractable normalising constant.
result Establishes statistical properties of the generalised posterior and proposes a calibration approach.
Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a re…
Bayesian inference uses Stein discrepancy for robustness in intractable likelihoods.
problem Intractable likelihoods in Bayesian inference.
method Generalised Bayesian inference with Stein discrepancy as the loss function.
result Robust generalised posteriors with closed form or accessible using MCMC.
Clarifies Einstein-Cartan gravitation with Dirac spinor on generalized frame bundle.
problem Formulating Einstein-Cartan gravitation on a frame bundle.
method Integrates Dirac spinor into the Einstein-Cartan spacetime structure.
result Variational equations imply standard field equations under standard frame bundle condition.
Enhanced DeepONet framework with uncertainty quantification for complex operators.
problem Learning complex operators with uncertainty quantification.
method Generalised variational inference (GVI) using Rényi's α-divergence.
result Superior predictive accuracy and uncertainty quantification.
Improved model for non-smooth signals with complex spectra.
problem Current models struggle with non-smooth signals and complex spectral structures.
method CGPCM and RGPCM models with causality and Bayesian nonparametric interpretations, improved variational inference.
result Proposed models show better performance on synthetic and real-world data.
We present a novel model architecture which leverages deep learning tools to perform exact Bayesian inference on sets of high dimensional, complex observations. Our model is provably exchangeable, meaning that the joint distribution over observations is invariant under permutation: this property lies at the heart of Ba…
Circular variables arise in a multitude of data-modelling contexts ranging from robotics to the social sciences, but they have been largely overlooked by the machine learning community. This paper partially redresses this imbalance by extending some standard probabilistic modelling tools to the circular domain. First w…
A Python package for GLHMM, a flexible HMM framework.
problem Handling diverse HMM applications in neuroscience.
method Stochastic variational inference for large datasets.
result Enables statistical testing and out-of-sample prediction.
Deep learning scheme identifies and reconstructs chaotic and stochastic systems from noisy data.
problem Challenging identification of governing equations from noisy and partial observations.
method Jointly learns inference model and governing laws using variational deep learning.
result Framework generalizes state-of-the-art methods and accounts for stochastic variabilities.
Generalizes beam models to include curvature and torsion.
problem Modeling curvature and torsion in unidimensional structures.
method Generalized Euler-Bernoulli and Timoshenko beam models using non-holonomic kinematics.
result Generalized beams exhibit curvature and torsion, with torsion in Timoshenko model.
Paper introduces robust Gaussian process regression without sacrificing computational efficiency.
problem Violation of independent and identically distributed Gaussian observation noise assumption in Gaussian process regression.
method Proves robust and conjugate Gaussian process regression (RCGP) at no additional cost using generalised Bayesian inference.
result RCGP enables exact conjugate closed form updates in all settings where standard GPs admit them.
We reformulate ten-dimensional type II supergravity as a generalised geometrical analogue of Einstein gravity, defined by an O(9,1)×O(1,9)⊂O(10,10)×R+ structure on the generalised tangent space. Using the notion of generalised connection and torsion, we introduce the analogue of the Levi-C…
Study introduces indecomposability for varifolds, leading to geometric consequences.
problem Understanding the structure of varifolds and their connectedness properties.
method Introducing indecomposability and related concepts for varifolds.
result Substantial geometric consequences derived from the connectedness properties of varifolds.
Gaussian processes (GPs) provide a powerful framework for extrapolation, interpolation, and noise removal in regression and classification. This paper considers constraining GPs to arbitrarily-shaped domains with boundary conditions. We solve a Fourier-like generalised harmonic feature representation of the GP prior in…
The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models th…
We address the problem of one-to-many mappings in supervised learning, where a single instance has many different solutions of possibly equal cost. The framework of conditional variational autoencoders describes a class of methods to tackle such structured-prediction tasks by means of latent variables. We propose to in…
The paper introduces a new divergence measure for variational autoencoders to improve reconstruction and generation.
problem Balancing reconstruction and generalizability in latent space of variational autoencoders.
method Presented a regularisation mechanism based on skew-geometric Jensen-Shannon divergence.
result The skew-geometric Jensen-Shannon divergence leads to better reconstruction and generation in variational autoencoders.
New scalable variational Bayes methods for Hawkes processes.
problem Computational intractability of Bayesian estimation for generalised nonlinear Hawkes processes.
method Unified variational Bayes framework, adaptive mean-field approximation, sparsity-inducing procedure.
result Adaptive mean-field variational algorithm for sigmoid Hawkes processes is scalable and robust.
We would like to learn a representation of the data which decomposes an observation into factors of variation which we can independently control. Specifically, we want to use minimal supervision to learn a latent representation that reflects the semantics behind a specific grouping of the data, where within a group the…
The variational auto-encoder (VAE) is a popular method for learning a generative model and embeddings of the data. Many real datasets are hierarchically structured. However, traditional VAEs map data in a Euclidean latent space which cannot efficiently embed tree-like structures. Hyperbolic spaces with negative curvatu…
CRL improves recommendation systems by reducing distribution shift.
problem Offline metrics fail to predict online performance due to distribution shift in recommender systems.
method Proposes an information-theoretic disentanglement criterion and a variational lower bound for better generalisation under distribution shift.
result CRL variants deliver substantial online gains in listener engagement compared to baseline models.
Proposes new methods for inference in GLMs without assuming model correctness.
problem Inference for GLMs assumes model correctness, leading to uncertainty and bias.
method Develops nonparametric estimands and uses influence curves with flexible procedures.
result Inference for GLM parameters is improved without model correctness assumptions.
Graph neural networks generalize well under certain conditions, explained by learning theory.
problem Understanding why graph neural networks generalize well in transductive inference.
method Analysis of transductive Rademacher complexity to explain generalization properties of graph convolutional networks.
result Transductive Rademacher complexity can explain the generalization of graph convolutional networks for node classification in stochastic block models.
A new method for deep Wishart processes improves kernel-based models.
problem Inference in deep Wishart processes is challenging due to the need for flexible distributions over positive semi-definite matrices.
method Developed a novel approach to flexible distributions over positive semi-definite matrices using the Bartlett decomposition of the Wishart probability density. Used this to create an approximate posterior for the DWP.
result Improved performance of inference in the DWP compared to DGP with equivalent prior.
We introduce Clique Matrices as an alternative representation of undirected graphs, being a generalisation of the incidence matrix representation. Here we use clique matrices to decompose a graph into a set of possibly overlapping clusters, de ned as well-connected subsets of vertices. The decomposition is based on a s…
This work frames reward modelling from preferences as a causal problem.
problem Reward modelling from preference data for AI alignment.
method Causal inference approach to identify challenges and assumptions.
result Causally-inspired approaches improve model robustness.
Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …
The paper improves generalization bounds using interpolation between various divergences.
problem Improving generalization bounds in machine learning.
method Derives new PAC-Bayes generalization bounds based on (f,Γ)-divergence and interpolates between various divergences. result Connects derived bounds to earlier statistical learning results and provides practical training objectives.