arXiv research

This paper analyzes the generalization risk of unrolled neural networks using Stein's Unbiased Risk Estimator.

New neural network improves MRI reconstruction for non-Cartesian data.

Statistical analysis of algorithm unrolling for inverse problems.

Recurrent convolution (RC) shares the same convolutional kernels and unrolls them multiple steps, which is originally proposed to model time-space signals. We argue that RC can be viewed as a model compression strategy for deep convolutional neural networks. RC reduces the redundancy across layers. However, the perform…

While neural networks have achieved vastly enhanced performance over traditional iterative methods in many cases, they are generally empirically designed and the underlying structures are difficult to interpret. The algorithm unrolling approach has helped connect iterative algorithms to neural network architectures. Ho…

Blind image deblurring remains a topic of enduring interest. Learning based approaches, especially those that employ neural networks have emerged to complement traditional model based methods and in many cases achieve vastly enhanced performance. That said, neural network approaches are generally empirically designed a…

Unrolled networks learn optimal Bayesian inference for unknown priors.

The paper develops generalization bounds for deep compound Gaussian neural networks.

We introduce an architecture based on deep hierarchical decompositions to learn effective representations of large graphs. Our framework extends classic R-decompositions used in kernel methods, enabling nested part-of-part relations. Unlike recursive neural networks, which unroll a template on input graphs directly, we…

ES-Single uses ES to estimate gradients in unrolled graphs, reducing variance and improving performance.

LASE learns graph embeddings by unrolling GD iterations into a neural network.

PUDLE method analyzes and improves unrolled sparse coding networks for dictionary learning.

The paper compares unrolling and bilevel optimization for learning variational models.

PES method reduces bias in gradient estimation for unrolled graphs.

In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we…

This paper introduces Non-Autonomous Input-Output Stable Network(NAIS-Net), a very deep architecture where each stacked processing block is derived from a time-invariant non-autonomous dynamical system. Non-autonomy is implemented by skip connections from the block input to each of the unrolled processing stages and al…

We introduce a method to stabilize Generative Adversarial Networks (GANs) by defining the generator objective with respect to an unrolled optimization of the discriminator. This allows training to be adjusted between using the optimal discriminator in the generator's objective, which is ideal but infeasible in practice…

Bayesian neural networks learn graph structure with interpretable parameters.

uGLAD recovers sparse graphs from data using deep unrolled networks.

Improved hypernetwork for efficient neural network hyperparameter tuning.

This work analyzes the convergence rate of unrolling for optimizing quadratic objectives.

This work improves dictionary learning speed without sacrificing accuracy.

Deep network improves electrical tomography across multiple frequencies.

Deep learning speeds up MRI image reconstruction from sparse data.

Because of their effectiveness in broad practical applications, LSTM networks have received a wealth of coverage in scientific journals, technical blogs, and implementation guides. However, in most articles, the inference formulas for the LSTM network and its parent, RNN, are stated axiomatically, while the training fo…

Neurally Augmented ALISTA improves sparse reconstruction performance.

OPT framework improves neural network generalization by learning an orthogonal transformation.

LUTNet optimizes FPGA for neural network inference, achieving high efficiency.

Paper presents an ADMM-based approach to efficiently integrate quadratic programming layers into neural networks.

Modified Hennings invariant defined using quantum groups and integrals.

Framework for designing nonlinearities in neural networks with slope constraints.

Compressive image recovery is a challenging problem that requires fast and accurate algorithms. Recently, neural networks have been applied to this problem with promising results. By exploiting massively parallel GPU processing architectures and oodles of training data, they can run orders of magnitude faster than exis…

Careful tuning of the learning rate, or even schedules thereof, can be crucial to effective neural net training. There has been much recent interest in gradient-based meta-optimization, where one tunes hyperparameters, or even learns an optimizer, in order to minimize the expected loss when the training procedure is un…

E2Efold predicts RNA secondary structures better than previous methods.

Deep-neural-network-based image reconstruction has demonstrated promising performance in medical imaging for under-sampled and low-dose scenarios. However, it requires large amount of memory and extensive time for the training. It is especially challenging to train the reconstruction networks for three-dimensional comp…

New algorithm for online training of Spiking Neural Networks (SNNs).

HyperLISTA simplifies LISTA training with adaptive hyperparameters.

New deep-unfolded network improves video background separation.

IFR-Net improves MRI images with faster, better detail recovery.

L2GMOM learns financial networks and optimizes momentum strategies.

Many challenging image processing tasks can be described by an ill-posed linear inverse problem: deblurring, deconvolution, inpainting, compressed sensing, and superresolution all lie in this framework. Traditional inverse problem solvers minimize a cost function consisting of a data-fit term, which measures how well a…

Binarization of digital documents is the task of classifying each pixel in an image of the document as belonging to the background (parchment/paper) or foreground (text/ink). Historical documents are often subjected to degradations, that make the task challenging. In the current work a deep neural network architecture …

Unified framework for lifted training and inversion of neural networks.

We show that unrolled quantum groups at odd roots of unity give rise to relative modular categories. These are the main building blocks for the construction of 1+1+1-TQFTs extending CGP invariants, which are non-semisimple quantum invariants of closed 3-manifolds decorated with ribbon graphs and cohomology classes. Whe…

IDM learns clustered representations for complex FELs.

Quantum groups give lower genus bounds for links.

Hybrid deep architectures with reasoning layers show promising convergence and generalization properties.

Contemporary power grids are being challenged by rapid voltage fluctuations that are caused by large-scale deployment of renewable generation, electric vehicles, and demand response programs. In this context, monitoring the grid's operating conditions in real time becomes increasingly critical. With the emergent large …