BankGCN improves graph convolution networks by handling multi-channel signals with adaptive filter banks.
problem Handling multi-channel graph signals with limited architectures.
method BankGCN decomposes multi-channel signals into subspaces and uses adapted filters for each subspace.
result BankGCN achieves excellent performance in graph classification on benchmark datasets.
New graph coarsening method preserves GNN message-passing signals.
problem Preserving GNN message-passing on coarsened graphs.
method Proposed a new oriented message-passing operation for coarsened graphs.
result Improved node classification results compared to naive methods.
DG improves policy gradient efficiency by selectively backpropagating only valuable samples.
problem Expensive backward passes in policy gradient methods reduce efficiency.
method Introduces 'delight' as a forward-pass signal of learning value and a Kondo gate to selectively backpropagate.
result Selective backpropagation reduces backward pass costs without sacrificing learning quality.
A guide to using low-pass graph filters for network data.
problem Understanding and processing graph data with low-pass filters.
method Definition and application of low-pass graph filters to graph data.
result Low-pass filters effectively retain lower frequency graph data contents.
In this work, we consider compressed sensing reconstruction from M measurements of K-sparse structured signals which do not possess a writable correlation model. Assuming that a generative statistical model, such as a Boltzmann machine, can be trained in an unsupervised manner on example signals, we demonstrate how…
New algorithm improves signal reconstruction from noisy measurements with side information.
problem Reconstructing unknown signals from noisy linear measurements with side information.
method Integrates side information into approximate message passing (AMP) and tracks performance using state evolution.
result AMP-SI performance is accurately predicted by state evolution.
Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem. The AMP framework provides modularity in the choice of signal prior; here we propose a hierarchical form of the Gauss-Bernouilli prior which utilizes a Restricted Boltzmann Machin…
New algorithms improve rank one signal estimation from noisy data.
problem Estimating a rank one signal matrix from corrupted data with rotationally invariant noise.
method Developed approximate message-passing algorithms exploiting eigenvalues and iterates denoisers.
result Achieves optimal asymptotic estimation error among iterative algorithms.
New algorithm for signal estimation in noisy matrix models.
problem Signal estimation in rectangular spiked matrix models with rotationally invariant noise.
method Orthogonal Approximate Message Passing (OAMP) algorithm for signal estimation.
result Optimal OAMP algorithm minimizes mean-squared error and achieves Bayes-optimal performance.
Optimal graph classification uses message-passing neural networks.
problem Node classification on sparse graphs with fixed feature dimensions.
method Asymptotic local Bayes optimality, message-passing graph neural networks.
result Optimal message-passing architecture interpolates between MLP and convolution.
Backpropagation-free RL method trains layers using local signals.
problem Vanishing or exploding gradients in backpropagation-based RL.
method Local pairwise distance matching for layer-wise training without backpropagation.
result Backpropagation-free method achieves competitive performance and stability.
Paper develops an online EM algorithm for graph signal inference from streaming data.
problem Joint inference and clustering of graph signals with non-white excitation.
method Mixture model with low-rank plus sparse prior, online EM algorithm.
result Proposed online EM algorithm converges to MAP solution.
Study optimal algorithms for recovering signals through inhomogeneous low-rank channels.
problem Recovering signals through an inhomogeneous low-rank matrix channel.
method Derive and analyze an approximate message-passing algorithm (AMP) and a spectral method.
result The AMP iteration matches the conjectured optimal computational phase transition.
New AMP algorithms for rotationally invariant models with reduced complexity.
problem Signal estimation in generalized linear models with arbitrary spectral design matrices.
method Rotationally invariant approximate message passing (AMP) algorithms.
result Performance close to Vector AMP with significantly lower complexity.
New AMP algorithm estimates signals and latent variables in mixed regression models.
problem Estimating signals and latent variables in mixed regression models.
method Approximate Message Passing (AMP) algorithm for matrix GLM.
result State evolution recursion and optimal denoising functions for precise error minimization.
Factor graphs have recently gained increasing attention as a unified framework for representing and constructing algorithms for signal processing, estimation, and control. One capability that does not seem to be well explored within the factor graph tool kit is the ability to handle deterministic nonlinear transformati…
Method detects new physics signals without prior knowledge.
problem Selecting signal regions for novel particles.
method Model-agnostic approach using low-pass filtering and density estimation.
result Efficiently identifies data-driven signal regions in high-dimensional feature space.
MPNNs struggle with class-bottlenecks and heterophily, leading to performance limitations.
problem Performance limitations of MPNNs under heterophily and structural bottlenecks.
method A statistical framework decomposing model performance into SNR components and proving bounds on sensitivity.
result Optimal graph structures for maximizing higher-order homophily are disjoint unions of single-class and two-class-bipartite clusters.
The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming para…
New method infers viral load from pooled tests.
problem Inefficient viral load inference in pooled testing.
method Message passing algorithm with PCR noise function.
result Accurate viral load inference possible.
Proposes integrating global and local entropy for more reliable LLMs.
problem Uncertainty in large language models (LLMs) leads to unreliable predictions.
method Measures global uncertainty from hidden-state matrices and local uncertainty from tokens, combining them via a multiplicative gate.
result Global-Local Uncertainty (GLU) outperforms unsupervised baselines across multiple models and benchmarks.
Effective and powerful methods for denoising real electrocardiogram (ECG) signals are important for wearable sensors and devices. Deep Learning (DL) models have been used extensively in image processing and other domains with great success but only very recently have been used in processing ECG signals. This paper pres…
Approximate Message Passing (AMP) has been shown to be a superior method for inference problems, such as the recovery of signals from sets of noisy, lower-dimensionality measurements, both in terms of reconstruction accuracy and in computational efficiency. However, AMP suffers from serious convergence issues in contex…
An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small. This hard phase is typically understood as a metastable bran…
The presence of noise is common in signal processing regardless the signal type. Deep neural networks have shown good performance in noise removal, especially on the image domain. In this work, we consider deep neural networks as a denoising tool where our focus is on one dimensional signals. We introduce an encoder-de…
Proposes a new method to prevent forgetting in neural networks.
problem Catastrophic forgetting in neural networks.
method Total absolute signal passing through each connection.
result Simpler and closer to brain processes.
Model compresses event-like contexts using gated surprise signals.
problem Perceiving a dynamic world as organized events.
method Hierarchical, surprise-gated recurrent neural network architecture.
result Achieves best performance on multiple event processing tasks.
PASS protects private attributes by stochastically substituting data.
problem Protecting private attributes in ML services while maintaining data utility.
method PASS uses stochastic data substitution with a novel loss function derived from information theory.
result PASS effectively protects private attributes across various datasets.
Study phase transitions in shuffled regression problems.
problem Phase transitions in shuffled regression problems.
method Transformed permutation recovery into probabilistic graphical model, used message passing (MP) algorithm and branching random walk process.
result Characterized impact of signal-to-noise-ratio ($\snr$) on permutation recovery, proposed Gaussian approximation method.
The Duffing oscillator's parameters are identified online using variational message passing.
problem Estimating parameters of a nonlinear Duffing oscillator in real-time.
method Variational message passing on a factor graph of the Duffing oscillator's generative model.
result The online inference procedure performs as well as offline methods.
CW Networks leverage cell complexes to enhance GNNs, achieving state-of-the-art results on molecular datasets.
problem Graph Neural Networks struggle with long-range interactions and lack principled ways to model higher-order structures.
method CW Networks use cell complexes to decouple computational and input graph structures, enabling flexible hierarchical message passing.
result CW Networks achieve state-of-the-art results on molecular datasets.
AMP algorithm for matrix tensor product model provides recovery conditions.
problem Generalization of standard spiked matrix models with multiple pairwise observations.
method Approximate message passing with optimal weighing and combining of estimates.
result Asymptotically exact performance description and necessary/sufficient recovery conditions.
Paper proposes AMP with spectral initialization for robust signal estimation.
problem Signal estimation from generalized linear model measurements with correlated initialization.
method Approximate message passing (AMP) with spectral initialization.
result Characterization of AMP with spectral initialization in high-dimensional limit.
A new method for joint noise removal and trend estimation from sparse signals.
problem Jointly removing noise and estimating trends from sparse signals.
method PENDANTSS combines SOOT/SPOQ penalties with BEADS algorithm in a Trust-Region block alternating variable metric forward-backward approach.
result Outperforms comparable methods in deconvolving analytical chemistry signals.
Filter banks are a popular tool for the analysis of piecewise smooth signals such as natural images. Motivated by the empirically observed properties of scale and detail coefficients of images in the wavelet domain, we propose a hierarchical deep generative model of piecewise smooth signals that is a recursion across s…
Stabilizes deep Bayesian neural networks with self-stabilizing priors.
problem Brittleness and difficulty in training deep Bayesian neural networks.
method Signal propagation theory, reformulated ELBO, self-stabilizing priors.
result Improved convergence and robustness in training deeper networks and noisier settings.
Combines PCA and AMP for better signal estimation in noisy data.
problem Estimating a rank-1 signal in rotationally invariant noise.
method Combines PCA and AMP, with PCA initialization at the start of AMP.
result Rigorous asymptotic characterization of the new estimator's performance.
New AMP algorithm detects change points in high-dimensional GLMs.
problem Detecting change points in high-dimensional GLMs.
method Approximate Message Passing (AMP) algorithm for estimating signals and change points.
result Characterization of AMP algorithm's performance in high-dimensional limit.
New AMP algorithms improve multi-layer signal reconstruction.
problem Reconstructing signals and hidden variables from multi-layer networks with rotationally invariant weights.
method Developed multi-layer rotationally invariant generalized AMP (ML-RI-GAMP) algorithms and state evolution recursion.
result ML-RI-GAMP outperforms existing methods in terms of lower complexity and similar performance.
Adapts BP-based algorithms for deep learning, improving performance and accuracy.
problem Training deep neural networks with discrete weights and activations.
method Message-passing algorithms based on Belief Propagation, with reinforcement field.
result Comparable performance to SGD-inspired heuristics (BinaryNet) and higher accuracy in predictions.
In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model. We study a loss function that is the negative log-likelihood of the model. We analyse the number of local minima at a fixed distance …
Characterizing the phase transitions of convex optimizations in recovering structured signals or data is of central importance in compressed sensing, machine learning and statistics. The phase transitions of many convex optimization signal recovery methods such as ℓ1 minimization and nuclear norm minimization are…
Data coarse graining improves model performance by filtering out less relevant features.
problem Lossy data transformations lose information but can improve model generalization.
method Data coarse graining schemes that systematically discard features based on relevance to the learning task.
result A 'high-pass' scheme helps models generalize better by filtering out less relevant features.
This work proposes a novel method for semi-supervised learning from partially labeled massive network-structured datasets, i.e., big data over networks. We model the underlying hypothesis, which relates data points to labels, as a graph signal, defined over some graph (network) structure intrinsic to the dataset. Follo…
Vector approximate message passing (VAMP) is a computationally simple approach to the recovery of a signal x from noisy linear measurements y=Ax+w. Like the AMP proposed by Donoho, Maleki, and Montanari in 2009, VAMP is characterized by a rigorous state evolution (SE) that holds …
We present the Causal Gaussian Process Convolution Model (CGPCM), a doubly nonparametric model for causal, spectrally complex dynamical phenomena. The CGPCM is a generative model in which white noise is passed through a causal, nonparametric-window moving-average filter, a construction that we show to be equivalent to …
Robust ASR model removes fast-changing features to resist attacks.
problem Vulnerability of ASR systems to adversarial attacks.
method Removing fast-changing features using slow feature analysis or low-pass filtering.
result Hybrid ASR models are more than four times more robust against targeted attacks.
Iterative information processing, either based on heuristics or analytical frameworks, has been shown to be a very powerful tool for the design of efficient, yet feasible, wireless receiver architectures. Within this context, algorithms performing message-passing on a probabilistic graph, such as the sum-product (SP) a…