arXiv research

Augmented KRnet improves flow-based generative modeling by maintaining exact invertibility.

In this paper, we describe an integration of exact Courant algebroids to symplectic 2-groupoids, and we show that the differentiation procedure from [26] inverts our integration.

Let $f,g:{\Bbb R}\to{\Bbb R}$ be integrable functions, $f$ nowhere zero, and $φ(u)=\int du/f(u)$ be invertible. An exact solution to the generalized nonhomogeneous inviscid Burgers' equation $u_t+g(u).u_x=f(u)$ is given, by quadratures.

Flow-based generative models (Dinh et al., 2014) are conceptually attractive due to tractability of the exact log-likelihood, tractability of exact latent-variable inference, and parallelizability of both training and synthesis. In this paper we propose Glow, a simple type of generative flow using an invertible 1x1 con…

A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choic…

Incorporates matrix exponential into generative flows for improved performance.

A normalizing flow models a complex probability density as an invertible transformation of a simple density. The invertibility means that we can evaluate densities and generate samples from a flow. In practice, autoregressive flow-based models are slow to invert, making either density estimation or sample generation sl…

We propose GraphNVP, the first invertible, normalizing flow-based molecular graph generation model. We decompose the generation of a graph into two steps: generation of (i) an adjacency tensor and (ii) node attributes. This decomposition yields the exact likelihood maximization on graph-structured data, combined with t…

Several recent works have empirically observed that Convolutional Neural Nets (CNNs) are (approximately) invertible. To understand this approximate invertibility phenomenon and how to leverage it more effectively, we focus on a theoretical explanation and develop a mathematical model of sparse signal recovery that is c…

AIKAE enhances IKAE for long-term time series forecasting.

Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning. Specifically, designing models with tractable learning, sampling, inference and evaluation is crucial in solving this task. We extend the space of such models using real-valued non-volume preserving (real NVP) transf…

SGNs use Hamiltonian mechanics for invertible deep generative modeling.

An analytic index is defined for a family of cusp pseudodifferential operators, $P_b,$ on a fibration with fibres which are compact manifolds with boundaries, provided the family is elliptic and has invertible indicial family at the boundary. In fact there is always a perturbation $Q_b$ by a family of cusp operators of…

We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive property of our model is that both p(features), the density of the features, and p(targets | features), the predictive distribution, can be co…

Study SKK groups of manifolds to classify non-unitary TQFTs.

In this paper we investigate fiber-wise linear complex Banach sub-Poisson structures defined canonically by the structure of a W*-algebra M. In particular we show that these structures are arranged in the short exact sequence of complex Banach sub-Poisson VB-groupoids with the groupoid of partially invertible elements …

Generative flows are attractive because they admit exact likelihood optimization and efficient image synthesis. Recently, Kingma & Dhariwal (2018) demonstrated with Glow that generative flows are capable of generating high quality images. We generalize the 1 x 1 convolutions proposed in Glow to invertible d x d convolu…

A promising class of generative models maps points from a simple distribution to a complex distribution through an invertible neural network. Likelihood-based training of these models requires restricting their architectures to allow cheap computation of Jacobian determinants. Alternatively, the Jacobian trace can be u…

In this paper, we shall be concerned with a relation between TQFTs and cut and paste invariants introduced by Karras, Kreck, Neumann and Ossa. Cut and paste invariants, or SK invariants, are functions on the set of smooth manifolds that are invariant under the cutting and pasting operation. Central to the work in this …

Unified framework recovers exact input from SOM activation patterns.

Flow-based deep generative models learn data distributions by transforming a simple base distribution into a complex distribution via a set of invertible transformations. Due to the invertibility, such models can score unseen data samples by computing their exact likelihood under the learned distribution. This makes fl…

Deterministic training improves generative autoencoder performance.

Exact recovery method for community detection in Gaussian mixtures with dependent noise.

Flow based models such as Real NVP are an extremely powerful approach to density estimation. However, existing flow based models are restricted to transforming continuous densities over a continuous input space into similarly continuous distributions over continuous latent variables. This makes them poorly suited for m…

Proves spectral sequence for real Heegaard Floer homology.

We present a generative model that is defined on finite sets of exchangeable, potentially high dimensional, data. As the architecture is an extension of RealNVPs, it inherits all its favorable properties, such as being invertible and allowing for exact log-likelihood evaluation. We show that this architecture is able t…

FlowGMM uses normalizing flows for semi-supervised learning, showing promising results across various data types.

VAEs improve representation learning by inverting the data-generating process through self-consistency.

OT-Flow uses optimal transport to improve CNFs for faster and more accurate density estimation.

Study on estimating invertible functions with minimax analysis.

Local invertibility of higher order tensor transforms on compact manifolds.

Study of strongly invertible Legendrian links in contact 3-space.

New variational flows improve Monte Carlo and normalization tasks.

We study the problem of inverting a deep generative model with ReLU activations. Inversion corresponds to finding a latent code vector that explains observed measurements as much as possible. In most prior works this is performed by attempting to solve a non-convex optimization problem involving the generator. In this …

Study on invariant Seifert surfaces for strongly invertible knots, showing large gaps in genus.

Dirac operator invertibility proven for specific manifolds.

Table of symmetric diagrams for knots up to 10 crossings.

Global invertibility proven for orientation-preserving maps without homeomorphic extension.

ISR creates analytical relationships from data via invertible maps.

Local invertibility of ray transforms on convex manifolds.

This work tackles exploding inverses in INNs, revealing and mitigating their numerical non-invertibility.

CF-INNs can approximate any invertible function, resolving a long-standing problem.

This work presents a novel approach to train invertible linear layers by adding rank-one perturbations.

Develops equivariant grid homology for strongly invertible knots.

New findings on knot genera using advanced techniques.

Defines knot signature invariant using G-signature theorem.

In this paper, we study the space of metrics of positive scalar curvature using methods from coarse geometry. Given a closed spin manifold M with fundamental group G, Stephan Stolz introduced the positive scalar curvature exact sequence, in analogy to the surgery exact sequence in topology. It calculates a structure gr…

This paper explores the computational hardness of generating latent vectors for generative models.