arXiv research

Paper uses SGLD to recover signals from generative models, proving convergence under mild conditions.

We consider estimating a piecewise-constant image, or a gradient-sparse signal on a general graph, from noisy linear measurements. We propose and study an iterative algorithm to minimize a penalized least-squares objective, with a penalty given by the "l_0-norm" of the signal's discrete graph gradient. The method proce…

PEGR improves deep learning models' robustness against noisy data.

This paper reviews zeroth-order optimization in signal processing and machine learning.

Overview of non-stochastic-gradient SA algorithms in signal processing and ML.

Hadamard Wirtinger Flow recovers sparse signals from fewer measurements.

The natural gradient allows for more efficient gradient descent by removing dependencies and biases inherent in a function's parameterization. Several papers present the topic thoroughly and precisely. It remains a very difficult idea to get your head around however. The intent of this note is to provide simple intuiti…

We develop a projected Nesterov's proximal-gradient (PNPG) approach for sparse signal reconstruction that combines adaptive step size with Nesterov's momentum acceleration. The objective function that we wish to minimize is the sum of a convex differentiable data-fidelity (negative log-likelihood (NLL)) term and a conv…

We formulate the problem of sampling and recovering clustered graph signal as a multi-armed bandit (MAB) problem. This formulation lends naturally to learning sampling strategies using the well-known gradient MAB algorithm. In particular, the sampling strategy is represented as a probability distribution over the indiv…

AGNES accelerates gradient descent with noisy gradients.

DGPs with variational inference suffer from SNR issues that degrade gradient estimates, leading to unreliable training.

DG improves policy gradient efficiency by selectively backpropagating only valuable samples.

Enhances reinforcement learning with partial state information.

Classical stochastic gradient methods for optimization rely on noisy gradient approximations that become progressively less accurate as iterates approach a solution. The large noise and small signal in the resulting gradients makes it difficult to use them for adaptive stepsize selection and automatic stopping. We prop…

Random matrix theory explains transient signal detectability in early-stopped gradient flow.

A framework for decentralized optimization using first-order methods.

GD with large init shows incremental learning in matrix factorization.

FGD reduces noisy gradient variance in SGD for neural networks.

Backpropagation-free RL method trains layers using local signals.

We propose a training method for deep neural network (DNN)-based source enhancement to increase objective sound quality assessment (OSQA) scores such as the perceptual evaluation of speech quality (PESQ). In many conventional studies, DNNs have been used as a mapping function to estimate time-frequency masks and traine…

Proposes using equivariant generative models for compressed sensing with unknown orientations.

Paper presents a unique method to recover signals from their bispectrum.

New method shows random, diverse initializations are not essential for deep neural networks.

Studying a softmax-attention model, we show that the learned query converges to the latent signal subspace spanned by the informative direction.

Suppose that we observe noisy linear measurements of an unknown signal that can be modeled as the sum of two component signals, each of which arises from a nonlinear sub-manifold of a high dimensional ambient space. We introduce SPIN, a first order projected gradient method to recover the signal components. Despite the…

Flusion combines multiple data sources to improve flu forecasts.

Study shows overparameterization helps shallow neural networks recover signals in high dimensions.

DP-GD improves CNN training accuracy with privacy, especially with low signal-to-noise ratios.

SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.

WaveGrad generates high-fidelity audio using gradient estimation.

A method uses non-autonomous equations to classify time signals efficiently.

Paper tackles outlier detection in signals modeled by generative models with theoretical guarantees.

Gradient boosting detects insider purchases predicting abnormal returns in microcap stocks.

Faster reconstruction of compressed signals using conditional GAN and NPGD.

We develop a multi-kernel based regression method for graph signal processing where the target signal is assumed to be smooth over a graph. In multi-kernel regression, an effective kernel function is expressed as a linear combination of many basis kernel functions. We estimate the linear weights to learn the effective …

Paper learns dictionaries for sparse signal recovery using automatic differentiation.

LDA-GO improves LDA for high-dimensional data via gradient optimization.

Paper improves REINFORCE for VI without restrictive assumptions.

Untrained neural networks can recover natural images from few measurements.

We propose a new deep recurrent neural network (RNN) architecture for sequential signal reconstruction. Our network is designed by unfolding the iterations of the proximal gradient method that solves the l1-l1 minimization problem. As such, our network leverages by design that signals have a sparse representation and t…

We describe an approach for incorporating prior knowledge into machine learning algorithms. We aim at applications in physics and signal processing in which we know that certain operations must be embedded into the algorithm. Any operation that allows computation of a gradient or sub-gradient towards its inputs is suit…

We develop a theory of higher-order feature attribution for complex models.

A tuning-free method recovers jointly sparse signals in MMV using implicit regularization.

Paper improves variance control in importance weighted variational bounds.

In this work, we present direction-of-arrival (DoA) estimation algorithms based on the Krylov subspace that effectively exploit prior knowledge of the signals that impinge on a sensor array. The proposed multi-step knowledge-aided iterative conjugate gradient (CG) (MS-KAI-CG) algorithms perform subtraction of the unwan…

Attention models can overfit without harming test performance.

DDICA separates nonlinear mixed signals robustly.

EVA adapts LoRA for faster, more efficient fine-tuning.