arXiv research

Unified tutorial on AMP for high-dimensional problems.

New algorithm improves signal reconstruction from noisy measurements with side information.

AMP method reconstructs rank-one matrices from noisy data efficiently.

New AMP algorithms for rotationally invariant models with reduced complexity.

Tree-AMP simplifies inference in complex tree-structured models.

New framework predicts AMP behavior in spiked models for finite iterations.

Approximate Message Passing (AMP) has been shown to be a superior method for inference problems, such as the recovery of signals from sets of noisy, lower-dimensionality measurements, both in terms of reconstruction accuracy and in computational efficiency. However, AMP suffers from serious convergence issues in contex…

A new AMP-based method speeds up conformal prediction intervals.

AMP algorithms can be efficiently simulated by SDPs even with corrupted data.

Novel AMP framework for multi-environment transfer learning.

The paper develops AMP theory for sparse and robust regression with polynomial iterations.

Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem. The AMP framework provides modularity in the choice of signal prior; here we propose a hierarchical form of the Gauss-Bernouilli prior which utilizes a Restricted Boltzmann Machin…

A fast, robust AMP algorithm for quadratic optimization problems.

Simple AMP algorithm robust to adversarial corruption.

Paper proposes AMP with spectral initialization for robust signal estimation.

Combines PCA and AMP for better signal estimation in noisy data.

Optimal spectral estimators and AMP combine for efficient weak recovery in orthogonally invariant GLMs.

New AMP algorithm estimates signals and latent variables in mixed regression models.

Unified framework for AMP iterations using graph indexing.

SLOPE is a relatively new convex optimization procedure for high-dimensional linear regression via the sorted l1 penalty: the larger the rank of the fitted coefficient, the larger the penalty. This non-separable penalty renders many existing techniques invalid or inconclusive in analyzing the SLOPE solution. In this pa…

New AMP algorithms improve multi-layer signal reconstruction.

In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank…

We propose an estimator of prediction error using an approximate message passing (AMP) algorithm that can be applied to a broad range of sparse penalties. Following Stein's lemma, the estimator of the generalized degrees of freedom, which is a key quantity for the construction of the estimator of the prediction error, …

AMP algorithm for matrix tensor product model provides recovery conditions.

Approximate message passing (AMP) refers to a class of efficient algorithms for statistical estimation in high-dimensional problems such as compressed sensing and low-rank matrix estimation. This paper analyzes the performance of AMP in the regime where the problem dimension is large but finite. For concreteness, we co…

We analyse a linear regression problem with nonconvex regularization called smoothly clipped absolute deviation (SCAD) under an overcomplete Gaussian basis for Gaussian random data. We propose an approximate message passing (AMP) algorithm considering nonconvex regularization, namely SCAD-AMP, and analytically show tha…

Study optimal algorithms for recovering signals through inhomogeneous low-rank channels.

New AMP algorithm detects change points in high-dimensional GLMs.

New algorithm for signal estimation in noisy matrix models.

New method improves sparse signal reconstruction using 1RSB-AMP.

Paper tackles robust graph matching in dense graphs with AMP type algorithm.

Plug-and-play L-GM-AMP improves CS recovery for any i.i.d. source prior.

Unrolled networks learn optimal Bayesian inference for unknown priors.

This paper introduces SS-MAMP to address convergence issues in AMP algorithms.

Vector approximate message passing (VAMP) is a computationally simple approach to the recovery of a signal $\mathbf{x}$ from noisy linear measurements $\mathbf{y}=\mathbf{Ax}+\mathbf{w}$. Like the AMP proposed by Donoho, Maleki, and Montanari in 2009, VAMP is characterized by a rigorous state evolution (SE) that holds …

We study optimal estimation for sparse principal component analysis when the number of non-zero elements is small but on the same order as the dimension of the data. We employ approximate message passing (AMP) algorithm and its state evolution to analyze what is the information theoretically minimal mean-squared error …

Iterative thresholding algorithms are well-suited for high-dimensional problems in sparse recovery and compressive sensing. The performance of this class of algorithms depends heavily on the tuning of certain threshold parameters. In particular, both the final reconstruction error and the convergence rate of the algori…

Study shows TAP free energy minimization provides better posterior inference in high-dimensional linear models.

The study shows how to accurately estimate embedding vectors in high dimensions.

Study of Bayes optimal learning in high-dimensional linear regression with network side information.

Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse problem encountered in compressive sensing, where one seeks to recover a sparse signal from a small number of noisy linear measurements. In this p…

We consider the problem of recovering a vector $β_o \in \mathbb{R}^p$ from $n$ random and noisy linear observations $y= Xβ_o + w$, where $X$ is the measurement matrix and $w$ is noise. The LASSO estimate is given by the solution to the optimization problem $\hatβ_λ = \arg \min_β \frac{1}{2} \|y-Xβ\|_2^2 + λ\| β\|_1$. A…

Consider the problem of estimating a low-rank matrix when its entries are perturbed by Gaussian noise. If the empirical distribution of the entries of the spikes is known, optimal estimators that exploit this knowledge can substantially outperform simple spectral approaches. Recent work characterizes the asymptotic acc…

This paper optimizes retraining models using their own predictions and noisy labels.

An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small. This hard phase is typically understood as a metastable bran…

Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding …

We consider compressed sensing formulated as a minimization problem of nonconvex sparse penalties, Smoothly Clipped Absolute deviation (SCAD) and Minimax Concave Penalty (MCP). The nonconvexity of these penalties is controlled by nonconvexity parameters, and L1 penalty is contained as a limit with respect to these para…

Various alignment problems arising in cryo-electron microscopy, community detection, time synchronization, computer vision, and other fields fall into a common framework of synchronization problems over compact groups such as Z/L, U(1), or SO(3). The goal of such problems is to estimate an unknown vector of group eleme…