Paper proposes AMP with spectral initialization for robust signal estimation.
problem Signal estimation from generalized linear model measurements with correlated initialization.
method Approximate message passing (AMP) with spectral initialization.
result Characterization of AMP with spectral initialization in high-dimensional limit.
Combines PCA and AMP for better signal estimation in noisy data.
problem Estimating a rank-1 signal in rotationally invariant noise.
method Combines PCA and AMP, with PCA initialization at the start of AMP.
result Rigorous asymptotic characterization of the new estimator's performance.
This paper deals with chain graphs under the alternative Andersson-Madigan-Perlman (AMP) interpretation. In particular, we present a constraint based algorithm for learning an AMP chain graph a given probability distribution is faithful to. We also show that the extension of Meek's conjecture to AMP chain graphs does n…
Plug-and-play L-GM-AMP improves CS recovery for any i.i.d. source prior.
problem Efficiently recovering signals from compressed measurements with unknown priors.
method Deep learning with Gaussian-mixture model to approximate source prior, combined with learned denoising.
result L-GM-AMP achieves state-of-the-art performance without prior knowledge of source distribution.
This paper introduces SS-MAMP to address convergence issues in AMP algorithms.
problem Convergence issues in AMP algorithms for signal reconstruction.
method Proposes SS-MAMP algorithm framework for right-unitarily invariant sensing matrices and Lipschitz-continuous local processors.
result Covariance matrices of SS-MAMP are L-banded and convergent, ensuring optimal convergence.
Simple AMP algorithm robust to adversarial corruption.
problem Robust approximate message passing in spiked matrix models.
method Spectral pre-processing combined with robust spectral initialization.
result AMP output is close to correct for corrupted data.
AMP algorithms can be efficiently simulated by SDPs even with corrupted data.
problem Optimizing average-case optimization problems with corrupted data.
method Local statistics hierarchy semidefinite programs (SDPs) simulate AMP algorithms robustly.
result Robust guarantees for many AMP algorithms are offered, contrasting with strong lower bounds for SDPs.
New AMP algorithms for rotationally invariant models with reduced complexity.
problem Signal estimation in generalized linear models with arbitrary spectral design matrices.
method Rotationally invariant approximate message passing (AMP) algorithms.
result Performance close to Vector AMP with significantly lower complexity.
We solve minimal separator problems in AMP chain graphs and improve structure learning algorithms.
problem Finding minimal separators in AMP chain graphs and learning their structure from data.
method We analyze and solve several versions of the minimal separator problem. We propose modifications to the PC-like algorithm and extend a decomposition-based method for AMP CGs.
result Our modifications of the PC-like algorithm and the LCD-AMP method improve structure learning and are more accurate and stable, especially in high-dimensional settings.
New algorithm improves signal reconstruction from noisy measurements with side information.
problem Reconstructing unknown signals from noisy linear measurements with side information.
method Integrates side information into approximate message passing (AMP) and tracks performance using state evolution.
result AMP-SI performance is accurately predicted by state evolution.
Approximate Message Passing (AMP) has been shown to be a superior method for inference problems, such as the recovery of signals from sets of noisy, lower-dimensionality measurements, both in terms of reconstruction accuracy and in computational efficiency. However, AMP suffers from serious convergence issues in contex…
Approximate message passing (AMP) refers to a class of efficient algorithms for statistical estimation in high-dimensional problems such as compressed sensing and low-rank matrix estimation. This paper analyzes the performance of AMP in the regime where the problem dimension is large but finite. For concreteness, we co…
New framework predicts AMP behavior in spiked models for finite iterations.
problem Understanding AMP dynamics in high-dimensional spiked models.
method Developed a non-asymptotic framework for AMP in spiked matrix estimation.
result Predicted AMP behavior for up to O(polylognn) iterations in Z2 synchronization. A fast, robust AMP algorithm for quadratic optimization problems.
problem Implementing robust approximate-message passing algorithms for quadratic optimization problems.
method Spectral pre-processing and mild modification of AMP algorithm iterates.
result Output solution close to AMP algorithm output for perturbed inputs.
New AMP algorithms improve multi-layer signal reconstruction.
problem Reconstructing signals and hidden variables from multi-layer networks with rotationally invariant weights.
method Developed multi-layer rotationally invariant generalized AMP (ML-RI-GAMP) algorithms and state evolution recursion.
result ML-RI-GAMP outperforms existing methods in terms of lower complexity and similar performance.
A denoising algorithm seeks to remove noise, errors, or perturbations from a signal. Extensive research has been devoted to this arena over the last several decades, and as a result, today's denoisers can effectively remove large amounts of additive white Gaussian noise. A compressed sensing (CS) reconstruction algorit…
Unified tutorial on AMP for high-dimensional problems.
problem Structured high-dimensional statistical problems.
method Statistical perspective of AMP and its applications.
result Unified and strengthened results in AMP literature.
We address some computational issues that may hinder the use of AMP chain graphs in practice. Specifically, we show how a discrete probability distribution that satisfies all the independencies represented by an AMP chain graph factorizes according to it. We show how this factorization makes it possible to perform infe…
The paper develops AMP theory for sparse and robust regression with polynomial iterations.
problem Challenges in high-dimensional statistical estimation due to asymptotic theory breakdown.
method Non-asymptotic distributional theory of AMP for sparse and robust regression.
result First finite-sample non-asymptotic distributional theory of AMP for polynomial iterations.
Study on identifying AMP chain graph models under known and unknown component decompositions.
problem Identifying AMP chain graph models with known and unknown chain component decompositions.
method Analyzes conditions for identifiability of AMP models and proposes algorithms for structure recovery.
result Conditions for DAG identifiability in AMP models extend equal variance criteria for Bayes nets.
Compressive image recovery is a challenging problem that requires fast and accurate algorithms. Recently, neural networks have been applied to this problem with promising results. By exploiting massively parallel GPU processing architectures and oodles of training data, they can run orders of magnitude faster than exis…
We study optimal estimation for sparse principal component analysis when the number of non-zero elements is small but on the same order as the dimension of the data. We employ approximate message passing (AMP) algorithm and its state evolution to analyze what is the information theoretically minimal mean-squared error …
New method improves sparse signal reconstruction using 1RSB-AMP.
problem Sparse signal reconstruction with improved accuracy.
method Developed 1RSB-AMP and 1RSB-SE for SCAD penalty minimization.
result 1RSB-AMP achieves improved reconstruction compared to RS-AMP.
Novel AMP framework for multi-environment transfer learning.
problem Characterizing risk of Lasso-based transfer learning estimators.
method Multi-Environment Generalized Long AMP (multi-environment GLAMP) framework.
result Precise characterization of the risk of three Lasso-based transfer learning estimators.
Unified framework for AMP iterations using graph indexing.
problem Complex high-dimensional statistical inference problems.
method Graph-based indexing of AMP iterations, modular proof of state evolution.
result Unified SE equations for AMP iterations indexed by graphs.
A new AMP-based method speeds up conformal prediction intervals.
problem Computational cost in building high-dimensional prediction intervals.
method Approximate Message Passing (AMP) to accelerate full conformal prediction.
result AMP-based method produces valid prediction intervals orders of magnitude faster.
This paper deals with chain graphs under the Andersson-Madigan-Perlman (AMP) interpretation. In particular, we present a constraint based algorithm for learning an AMP chain graph a given probability distribution is faithful to. Moreover, we show that the extension of Meek's conjecture to AMP chain graphs does not hold…
AMP algorithm for matrix tensor product model provides recovery conditions.
problem Generalization of standard spiked matrix models with multiple pairwise observations.
method Approximate message passing with optimal weighing and combining of estimates.
result Asymptotically exact performance description and necessary/sufficient recovery conditions.
New AMP algorithm estimates signals and latent variables in mixed regression models.
problem Estimating signals and latent variables in mixed regression models.
method Approximate Message Passing (AMP) algorithm for matrix GLM.
result State evolution recursion and optimal denoising functions for precise error minimization.
SLOPE is a relatively new convex optimization procedure for high-dimensional linear regression via the sorted l1 penalty: the larger the rank of the fitted coefficient, the larger the penalty. This non-separable penalty renders many existing techniques invalid or inconclusive in analyzing the SLOPE solution. In this pa…
Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work we perform an analytic study of the performances of one of them, the Langevin algorithm, in the context of noisy high-dimensional inference. We employ the Langevin alg…
Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding …
AMP regularization improves deep learning models by favoring flat minima.
problem Improving deep learning model generalization and avoiding overfitting.
method AMP regularization uses adversarial model perturbation to minimize a norm-bounded perturbation of the empirical risk.
result AMP regularization leads to state-of-the-art performance across various deep architectures.
Paper tackles robust graph matching in dense graphs with AMP type algorithm.
problem Matching recovery between correlated Gaussian Wigner matrices with adversarial perturbations.
method Approximate Message Passing (AMP) type iterative algorithm with time-dependent matrix multiplication.
result Algorithm succeeds in polynomial time for non-vanishing correlation and small perturbations.
New AMP algorithm detects change points in high-dimensional GLMs.
problem Detecting change points in high-dimensional GLMs.
method Approximate Message Passing (AMP) algorithm for estimating signals and change points.
result Characterization of AMP algorithm's performance in high-dimensional limit.
This paper applies AMP theory to improve learning tasks.
problem Improving learning efficiency by optimizing task-specific models.
method Uses aggregated Markov processes to reduce model complexity and enhance learning.
result Demonstrates how AMP theory can be effectively applied to stochastic learning.
Tree-AMP simplifies inference in complex tree-structured models.
problem Inference in high-dimensional tree-structured models.
method Approximate Message Passing algorithms for various machine learning tasks.
result Theoretical performance predictions and automated entropy estimation.
This study optimizes multi-modal learning thresholds and algorithms in high dimensions.
problem Optimizing multi-modal learning performance in high-dimensional data.
method Analytical quantification and derivation of AMP algorithm with state evolution analysis.
result Bayes-optimal performance and recovery thresholds derived for multi-modal data.
Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem. The AMP framework provides modularity in the choice of signal prior; here we propose a hierarchical form of the Gauss-Bernouilli prior which utilizes a Restricted Boltzmann Machin…
We analyse a linear regression problem with nonconvex regularization called smoothly clipped absolute deviation (SCAD) under an overcomplete Gaussian basis for Gaussian random data. We propose an approximate message passing (AMP) algorithm considering nonconvex regularization, namely SCAD-AMP, and analytically show tha…
AMP method reconstructs rank-one matrices from noisy data efficiently.
problem Reconstructing rank-one matrices with prior structural information from noisy observations.
method Approximate Message Passing (AMP) with random initialization.
result AMP from random initialization converges rapidly and globally.
Unified bounds for iterative algorithms with Gaussian data matrices.
problem Establishing non-asymptotic bounds for iterative algorithms with Gaussian data.
method Explicit coupling between iterates and Gaussian process with deterministic covariance.
result Tight, dimension-free bounds for generalized first-order methods.
Consider the problem of estimating a low-rank matrix when its entries are perturbed by Gaussian noise. If the empirical distribution of the entries of the spikes is known, optimal estimators that exploit this knowledge can substantially outperform simple spectral approaches. Recent work characterizes the asymptotic acc…
Bayes-optimal limits in PCA with structured noise are determined.
problem Analyzing statistical dependencies in measurement noise for high-dimensional inference.
method Study of spiked matrix model with low-order polynomial orthogonal noise, providing Bayes-optimal limits and proposing a novel AMP.
result A novel AMP algorithm reaches the information-theoretic limits for more general priors.
EB-PCA reduces noise in high-dimensional PCA by estimating a joint prior distribution.
problem High-dimensional PCA noise in samples comparable to or larger than data.
method Empirical Bayes PCA using Kiefer-Wolfowitz MLE, random matrix theory, and AMP algorithm.
result EB-PCA achieves Bayes-optimal accuracy in spiked models and significantly improves over PCA in simulations and real data.
We consider compressed sensing formulated as a minimization problem of nonconvex sparse penalties, Smoothly Clipped Absolute deviation (SCAD) and Minimax Concave Penalty (MCP). The nonconvexity of these penalties is controlled by nonconvexity parameters, and L1 penalty is contained as a limit with respect to these para…
In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank…
We propose an estimator of prediction error using an approximate message passing (AMP) algorithm that can be applied to a broad range of sparse penalties. Following Stein's lemma, the estimator of the generalized degrees of freedom, which is a key quantity for the construction of the estimator of the prediction error, …