New distribution simplifies covariance matrix inference.
problem Efficient inference for covariance matrices in large models.
method Incorporates Inverse G-Wishart distribution for variational message passing.
result Elegant and succinct expression of variational message passing fragments.
New algorithm minimizes FE objectives for synthetic AIF agents.
problem Minimizing Free Energy for synthetic active inference agents.
method Variational message passing on Constrained FFGs.
result Message passing algorithms induce epistemic behavior in AIF agents.
Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret. We propose a variational message-passing algorithm for variational inference in such models. We make three contributions. First, we propose structured inference network…
A new algorithm improves topic model accuracy in small data sets.
problem Limited data leads to inaccurate topic model identification.
method Developed a novel variational message passing algorithm (ALBU) for LDA.
result ALBU learns latent distributions more accurately than Variational Bayes (VB) in small data sets.
ALBU improves LDA performance on small datasets.
problem Limited data impairs variational Bayes in LDA.
method Approximate Loopy Belief message passing algorithm.
result ALBU learns latent distributions more accurately than VB, especially for smaller data sets.
Bayesian method improves online NARMAX model identification.
problem Online identification of nonlinear systems with small sample sizes and low noise.
method Variational Bayesian inference using message passing algorithm for polynomial NARMAX models.
result Variational Bayesian estimator outperforms recursive and offline least-squares methods.
The Duffing oscillator's parameters are identified online using variational message passing.
problem Estimating parameters of a nonlinear Duffing oscillator in real-time.
method Variational message passing on a factor graph of the Duffing oscillator's generative model.
result The online inference procedure performs as well as offline methods.
Derives VMP for LDA, simplifying inference for topic modeling.
problem Manual derivation of VMP equations for LDA is challenging and time-consuming.
method Detailed derivation of VMP update equations for LDA.
result Enables easier implementation of VMP for LDA models.
AIF improves physical AI agents' performance in dynamic environments.
problem Physical AI agents are less capable than biological agents in open-ended real-world environments.
method Developed from probability theory, Bayesian machine learning, variational inference, and Active Inference (AIF), grounded in the Free Energy Principle.
result AIF minimizes variational free energy and is well-suited to physical constraints.
We analyze oversquashing in topological message-passing using relational structures.
problem Oversquashing in topological message-passing remains understudied.
method A unifying axiomatic framework that bridges graph and topological message-passing.
result Potential to advance topological deep learning.
Message passing is the key to graph neural networks, but new terms are needed to avoid confusion.
problem Current methods of graph neural networks cannot solve all problems over given input graphs.
method Demonstrates that any function of interest can be expressed using pairwise message passing over a modified graph.
result Message passing is the fundamental approach for graph neural networks, and new terms are needed to avoid confusion.
ADMP-GNN dynamically adjusts message-passing layers for better graph learning performance.
problem Fixed message-passing steps in GNNs do not account for nodes' varying computational needs.
method Proposes ADMP-GNN, which dynamically adjusts the number of message-passing layers for each node.
result Improves performance on node classification tasks compared to baseline GNN models.
A framework uses variational Bayes for solving inverse problems efficiently.
problem Solving inverse problems in various dimensions with flexibility and accuracy.
method Variational Bayes approximations with message passing and factor graph approach.
result Efficient algorithm updates for higher dimensions and computational advantage over MCMC.
A message passing algorithm is derived for recovering communities within a graph generated by a variation of the Barabási-Albert preferential attachment model. The estimator is assumed to know the arrival times, or order of attachment, of the vertices. The derivation of the algorithm is based on belief propagation unde…
We introduce an iterative optimization scheme for convex objectives consisting of a linear loss and a non-separable penalty, based on the expectation-consistent approximation and the vector approximate message-passing (VAMP) algorithm. Specifically, the penalties we approach are convex on a linear transformation of the…
Iterative information processing, either based on heuristics or analytical frameworks, has been shown to be a very powerful tool for the design of efficient, yet feasible, wireless receiver architectures. Within this context, algorithms performing message-passing on a probabilistic graph, such as the sum-product (SP) a…
We propose a novel distributed inference algorithm for continuous graphical models, by extending Stein variational gradient descent (SVGD) to leverage the Markov dependency structure of the distribution of interest. Our approach combines SVGD with a set of structured local kernel functions defined on the Markov blanket…
The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming para…
New graph coarsening method preserves GNN message-passing signals.
problem Preserving GNN message-passing on coarsened graphs.
method Proposed a new oriented message-passing operation for coarsened graphs.
result Improved node classification results compared to naive methods.
Study on convergence of graph neural networks on random graphs.
problem Convergence of message passing graph neural networks on large random graphs.
method Extended convergence results to a broad class of aggregation functions using McDiarmid inequality.
result Non-asymptotic bounds for convergence quantified with high probability.
MACE uses higher-order messages to create fast, accurate force fields.
problem Creating fast and accurate force fields in computational chemistry and materials science.
method Introducing MACE, an equivariant MPNN model that uses four-body messages.
result MACE reduces the required number of message passing iterations to just two, achieving state-of-the-art accuracy.
Factor graphs have recently gained increasing attention as a unified framework for representing and constructing algorithms for signal processing, estimation, and control. One capability that does not seem to be well explored within the factor graph tool kit is the ability to handle deterministic nonlinear transformati…
MPLP learns neural network weights by treating operations as message-passing agents.
problem Training neural networks using gradient-based methods.
method MPLP abstracts neural network operations as message-passing agents, updating internal states and passing messages.
result MPLP outperforms traditional gradient-based methods on simple feed-forward neural networks.
DimeNet uses directional message passing to improve molecular predictions.
problem Lack of directional information in graph neural networks for molecules.
method Directional message passing, rotationally equivariant embeddings, spherical functions.
result DimeNet outperforms previous GNNs by 76% on MD17 and 31% on QM9.
Topo-MLP learns network representations without message passing.
problem Lack of efficient higher-order network modeling methods.
method Proposes Topo-MLP, a simplicial neural network algorithm using MLP and HONC loss.
result Demonstrates improved robustness and efficiency in representation learning.
Introduces P-tensors for generalized higher-order message passing in graph neural networks.
problem Expanding the expressive power of graph neural networks through higher-order structures.
method Introduces P-tensors to define the most general form of permutation equivariant message passing.
result Achieves state-of-the-art performance on molecular datasets.
Improves SVGD for high-dimensional Bayesian inference by reducing variance collapse.
problem Variance collapse in SVGD reduces accuracy and diversity of estimation.
method Augmented Message Passing SVGD (AUMP-SVGD) method, a two-stage optimization procedure.
result AUMP-SVGD achieves satisfactory accuracy and overcomes variance collapse in various benchmark problems.
Paper interprets contrastive learning dynamics using message passing.
problem Lack of rigorous understanding of contrastive learning dynamics.
method Casts contrastive objective into message passing scheme on augmentation graph.
result The learning dynamics of contrastive learning can be theoretically characterized.
Stein variational gradient descent (SVGD) is a recently proposed particle-based Bayesian inference method, which has attracted a lot of interest due to its remarkable approximation ability and particle efficiency compared to traditional variational inference and Markov Chain Monte Carlo methods. However, we observed th…
Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.
problem Optimizing robot navigation in continuous-valued spaces with uncertainty.
method Autoregressive active inference agent using message passing on a factor graph.
result Agent modulates action based on predictive uncertainty, leading to better model of dynamics.
VFG model embeds flow-based models with hierarchical structures using variational inference.
problem Flow-based models struggle with high-dimensional latent spaces and lack of tractable inference for graphical structures.
method Integrates flow-based functions through variational inference with aggregation nodes for hierarchical information integration.
result VFG models achieve improved ELBO and likelihood values on multiple datasets.
Graph structured data are abundant in the real world. Among different graph types, directed acyclic graphs (DAGs) are of particular interest to machine learning researchers, as many machine learning models are realized as computations on DAGs, including neural networks and Bayesian networks. In this paper, we study dee…
New method explains GNNs using power iteration clustering.
problem Mysterious mechanism of message passing in GNNs.
method Subspace power iteration clustering (SPIC) models.
result Message passing in GNNs can be understood through power iteration.
Factorial Hidden Markov Models (FHMMs) are powerful models for sequential data but they do not scale well with long sequences. We propose a scalable inference and learning algorithm for FHMMs that draws on ideas from the stochastic variational inference, neural network and copula literatures. Unlike existing approaches…
BayesPy is an open-source Python software package for performing variational Bayesian inference. It is based on the variational message passing framework and supports conjugate exponential family models. By removing the tedious task of implementing the variational Bayesian update equations, the user can construct model…
Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. In this work, we propose a relational message passing method for knowledge graph completion. Different from existing embedding-based methods, relational message passing only considers edge features (i.e., relation types)…
Enhances graph neural networks with structural message-passing for better generalization.
problem Limited representation power and inability to learn basic graph topological properties.
method Proposes a framework that includes a one-hot encoding of nodes and parametrized message and update functions ensuring permutation equivariance.
result Achieves state-of-the-art results on molecular graph regression on the ZINC dataset.
Introduces Spectral Graph Network combining spatial and spectral message passing.
problem Relational reasoning in graph structured data.
method Applies message passing to both spatial and spectral domains of a graph.
result Promotes efficient training with fewer iterations and robustness to edge dropout.
Convex message passing algorithms converge to a fixed point.
problem Understanding convergence properties of convex message passing methods.
method Proving convergence of coordinate descent applied to piecewise-affine convex objectives, and showing this applies to various message passing methods.
result The iterates converge to a fixed point of the method, and the algorithm terminates in a known number of iterations.
New algorithms accelerate MAP inference in Markov fields with faster convergence.
problem Finding the most likely configuration in discrete-valued Markov random fields.
method Entropy-regularized linear programming with accelerated gradient methods.
result Accelerated algorithms find optimal solutions faster, especially when the LP is tight.
Proposes an alternative method to train RBMs with binary synapses using Bayesian learning rule.
problem Training RBMs with binary synapses is challenging due to discrete nature of synapses.
method Proposes an alternative optimization method using the Bayesian learning rule, updating natural parameters instead of expectation parameters.
result No additional clipping is needed as natural parameters take values in the entire real domain.
Develops a Gaussian-based message-passing algorithm for noisy matrix completion.
problem Noisy matrix completion problems.
method Gaussian parameterized belief propagation algorithm.
result Advantages over traditional methods in non-Gaussian noise scenarios.
Combines PCA and message passing for better graph node embeddings.
problem Challenges of learning long-range dependencies in graph data.
method PCAPass: PCA + message passing for unsupervised node embeddings.
result Empirically shows competitive performance on node classification benchmarks.
The paper sets limits for GNNs solving PDEs to avoid under-reaching phenomenon.
problem Under-reaching phenomenon in GNNs solving PDEs.
method Sharp lower bounds for message-passing iterations based on PDE characteristics.
result Proposed lower bounds ensure efficient information propagation in GNNs.
A new neural network model for molecular graphs that learns efficiently and accurately.
problem Learning on molecular graphs with cycles and complex structures.
method Hierarchical inter-message passing using raw graph and junction tree representations.
result The model outperforms classical GNNs in detecting cycles and is efficient to train.
DRew dynamically rewires message passing to improve long-range tasks.
problem Over-squashing in MPNNs due to local message passing.
method Layer-dependent rewiring and delay mechanism for gradual densification.
result DRew outperforms graph Transformers and multi-hop MPNNs on long-range tasks.
Improved SVAE models enhance sequential data prediction.
problem Challenges in implementing and using structured variational autoencoders.
method Modern machine learning tools, hardware acceleration, parallelization, automatic differentiation, exploiting structure in the prior.
result SVAEs outperform general alternatives in accuracy and efficiency.
Novel CG-EGNNs learn equivariant functions from Clifford algebras.
problem Lack of equivariance in high-order graph neural networks.
method Integrates high-order local structures with Clifford algebras for equivariant learning.
result CG-EGNNs outperform previous methods on various benchmarks.