arXiv research

New invertible transformations improve flow-based generative models.

We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation. Typically, enforcing invertibility requires partitioning dimensions or restricting network architectures. In contrast, our approach only requires adding a simple …

Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations where only Lipschitz conditions rather than strict architectural constraints are needed for enforci…

The paper explores how invertibility affects the complexity of encoder models in VAEs.

We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible architectures, including those similar to ResNets. Inversion is achieved with a locally convergent iterative procedure that is parallelizable …

Statistical generative models for molecular graphs attract attention from many researchers from the fields of bio- and chemo-informatics. Among these models, invertible flow-based approaches are not fully explored yet. In this paper, we propose a powerful invertible flow for molecular graphs, called graph residual flow…

Incorporates matrix exponential into generative flows for improved performance.

New model for simulating and inferring from inverse problems.

SGNs use Hamiltonian mechanics for invertible deep generative modeling.

Neural ODEs and i-ResNet are recently proposed methods for enforcing invertibility of residual neural models. Having a generic technique for constructing invertible models can open new avenues for advances in learning systems, but so far the question of whether Neural ODEs and i-ResNets can model any continuous inverti…

iGNN tackles inverse graph prediction using invertible neural networks.

This work tackles exploding inverses in INNs, revealing and mitigating their numerical non-invertibility.

CF-INNs can approximate any invertible function, resolving a long-standing problem.

Augmented KRnet improves flow-based generative modeling by maintaining exact invertibility.

Electronic power inverters are capable of quickly delivering reactive power to maintain customer voltages within operating tolerances and to reduce system losses in distribution grids. This paper proposes a systematic and data-driven approach to determine reactive power inverter output as a function of local measuremen…

This paper focuses on a novel generative approach for 3D point clouds that makes use of invertible flow-based models. The main idea of the method is to treat a point cloud as a probability density in 3D space that is modeled using a cloud-specific neural network. To capture the similarity between point clouds we rely o…

INNs can approximate diverse functions despite layer restrictions.

Study of strongly invertible Legendrian links in contact 3-space.

A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choic…

IZF uses flow-based models to overcome ZSL limitations.

By using parity arguments we prove that free knots are, generally, not invertible.

A normalizing flow models a complex probability density as an invertible transformation of a simple density. The invertibility means that we can evaluate densities and generate samples from a flow. In practice, autoregressive flow-based models are slow to invert, making either density estimation or sample generation sl…

We study invertible generating pairs of fundamental groups of graph manifolds, that is, pairs of elements (g,h) for which the map g --> g^{-1}, h --> h^{-1} extends to an automorphism. We show in particular that a graph manifold is of Heegaard genus 2 if and only if its fundamental group has an invertible generating pa…

Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier c…

Residual neural networks don't help overcome sampling complexity issues.

ISR creates analytical relationships from data via invertible maps.

Distribution grids are currently challenged by frequent voltage excursions induced by intermittent solar generation. Smart inverters have been advocated as a fast-responding means to regulate voltage and minimize ohmic losses. Since optimal inverter coordination may be computationally challenging and preset local contr…

We propose GraphNVP, the first invertible, normalizing flow-based molecular graph generation model. We decompose the generation of a graph into two steps: generation of (i) an adjacency tensor and (ii) node attributes. This decomposition yields the exact likelihood maximization on graph-structured data, combined with t…

i-DenseNets improve parameter efficiency and performance in density estimation.

For operators of many different kinds it has been proved that (generalized) Darboux transformations can be built using so called Wronskian formulae. Such Darboux transformations are not invertible in the sense that the corresponding mappings of the operator kernels are not invertible. The only known invertible ones wer…

Invertible DenseNets improve model efficiency and performance.

New analysis enables inversion of deep generative models with unique solutions.

ButterflyFlow uses butterfly matrices for efficient invertible layers in normalizing flows.

We consider so-called regular invertible Gaussian Volterra processes and derive a formula for their prediction laws. Examples of such processes include the fractional Brownian motions and the mixed fractional Brownian motions. As an application, we consider conditional-mean hedging under transaction costs in Black-Scho…

A new model decouples global and local image representations without supervision.

Study on estimating invertible functions with minimax analysis.

The paper stabilizes invertible neural networks by using Gaussian mixture models.

Normalizing flows attempt to model an arbitrary probability distribution through a set of invertible mappings. These transformations are required to achieve a tractable Jacobian determinant that can be used in high-dimensional scenarios. The first normalizing flow designs used coupling layer mappings built upon affine …

AIKAE enhances IKAE for long-term time series forecasting.

We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive property of our model is that both p(features), the density of the features, and p(targets | features), the predictive distribution, can be co…

Local invertibility of higher order tensor transforms on compact manifolds.

Paper develops a method to learn causal networks with non-invertible functions.

Flow-based generative models (Dinh et al., 2014) are conceptually attractive due to tractability of the exact log-likelihood, tractability of exact latent-variable inference, and parallelizability of both training and synthesis. In this paper we propose Glow, a simple type of generative flow using an invertible 1x1 con…

Study on equivariant Q-sliceness for strongly invertible knots.

We prove that for cobordant closed spin manifolds of dimension $n\geq 3$ the associated spaces of metrics with invertible Dirac operator are homotopy equivalent. This is the spinorial counterpart of a similar result on positive scalar curvature of Chernysh/Walsh and generalizes the surgery result of Ammann-Dahl-Humbert…

In this work, we investigate the use of normalizing flows to model conditional distributions. In particular, we use our proposed method to analyze inverse problems with invertible neural networks by maximizing the posterior likelihood. Our method uses only a single loss and is easy to train. This is an improvement on t…

The paper improves generative models to avoid replicating observed examples.

New CycleGAN uses invertible generator for faster, less resource-intensive CT denoising.