New invertible transformations improve flow-based generative models.
problem Improving flow-based generative models for better performance.
method Proposed new invertible transformations and coupling layers.
result New coupling layers achieve better results in IDF.
Generative model for 3D point clouds using invertible flows.
problem Generating realistic 3D point clouds.
method Invertible flow-based models for point cloud generation with parameter sharing and embedding vectors.
result The model generates high-quality 3D point clouds with good similarity.
We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation. Typically, enforcing invertibility requires partitioning dimensions or restricting network architectures. In contrast, our approach only requires adding a simple …
Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations where only Lipschitz conditions rather than strict architectural constraints are needed for enforci…
The paper explores how invertibility affects the complexity of encoder models in VAEs.
problem The complexity of the encoder model in VAEs when the generative map is invertible.
method Formalizes the concept of strong invertibility and analyzes the complexity of the encoder model.
result Strongly invertible generative maps allow for simpler encoder models, while non-invertible maps require exponentially larger encoders.
We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible architectures, including those similar to ResNets. Inversion is achieved with a locally convergent iterative procedure that is parallelizable …
Statistical generative models for molecular graphs attract attention from many researchers from the fields of bio- and chemo-informatics. Among these models, invertible flow-based approaches are not fully explored yet. In this paper, we propose a powerful invertible flow for molecular graphs, called graph residual flow…
Incorporates matrix exponential into generative flows for improved performance.
problem Improving generative flow models for better density estimation.
method Integrates matrix exponential into generative flows, proposing new layers and modifying network architecture.
result The proposed model achieves great performance on density estimation.
New model for simulating and inferring from inverse problems.
problem Bayesian inverse problems in conditional sampling.
method Invertible generative model using triangular normalizing flows.
result Training loss for invertible map proposed.
SGNs use Hamiltonian mechanics for invertible deep generative modeling.
problem Efficient and exact likelihood evaluation for deep generative models.
method Symplectic structure in latent space, Hamiltonian dynamics for data generation.
result Exact likelihood evaluation without Jacobian calculations.
A new generative model relaxes the bijectivity requirement for invertible flows.
problem Challenges of invertible flow-based models in scaling to large datasets.
method Proposes a generative model based on relaxed injective probability flows.
result Improves sample quality over VAEs and AEs.
Neural ODEs and i-ResNet are recently proposed methods for enforcing invertibility of residual neural models. Having a generic technique for constructing invertible models can open new avenues for advances in learning systems, but so far the question of whether Neural ODEs and i-ResNets can model any continuous inverti…
iGNN tackles inverse graph prediction using invertible neural networks.
problem Inverse graph prediction problem in data analysis and machine learning.
method Developed invertible graph neural network (iGNN) to solve inverse prediction problem on graphs.
result iGNN model allows efficient generation from output labels and forward prediction.
This work tackles exploding inverses in INNs, revealing and mitigating their numerical non-invertibility.
problem Exploding inverses in INNs cause numerical non-invertibility, leading to failures in various tasks.
method Derived bi-Lipschitz properties of INN building blocks, proposed regularizers for local invertibility, and stable INN designs for global invertibility.
result Bi-Lipschitz properties and stable INN designs are crucial for addressing numerical non-invertibility.
CF-INNs can approximate any invertible function, resolving a long-standing problem.
problem Whether CF-INNs can approximate any invertible function.
method Demonstrated CF-INNs are universal approximators for invertible functions by showing a convenient criterion.
result CF-INNs are universal approximators for invertible functions.
Augmented KRnet improves flow-based generative modeling by maintaining exact invertibility.
problem Maintaining exact invertibility in flow-based generative models.
method Integrates augmented dimensions into KRnet to achieve full nonlinear updates in two iterations, keeping exact invertibility.
result Augmented KRnet achieves full nonlinear updates in two iterations, maintaining exact invertibility.
This paper uses linear rational splines for invertible modeling, offering a simpler inverse and similar costs.
problem Creating expressive invertible models with tractable Jacobian determinants.
method Replacing affine transformations with linear rational splines in coupling layers.
result Linear rational splines offer a simpler inverse and similar costs for inference and generation.
Electronic power inverters are capable of quickly delivering reactive power to maintain customer voltages within operating tolerances and to reduce system losses in distribution grids. This paper proposes a systematic and data-driven approach to determine reactive power inverter output as a function of local measuremen…
INNs can approximate diverse functions despite layer restrictions.
problem Can INNs approximate sufficiently diverse functions?
method Developed a theoretical framework based on differential geometry to simplify the approximation problem of diffeomorphisms.
result INNs have the universal approximation property.
Study of strongly invertible Legendrian links in contact 3-space.
problem Characterizing and understanding strongly invertible Legendrian links.
method Equivariant analogs of basic results for strongly invertible and Legendrian links.
result Existence of maximal equivariant Thurston-Bennequin number for strongly invertible links.
Improves conditional distribution modeling with simpler training.
problem Analyzing inverse problems with invertible neural networks.
method Uses normalizing flows to maximize posterior likelihood, incorporating conditioning.
result Easier training and natural framework for conditional generation.
A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choic…
IZF uses flow-based models to overcome ZSL limitations.
problem Hardness of training ZSL models and limited generation quality.
method IZF incorporates flow-based models to learn factorized data embeddings and generates samples.
result IZF significantly outperforms existing methods on ZSL benchmarks.
Proposes a constant memory iterative inverse model using invertible networks.
problem Memory limitations in iterative learning approaches for inverse problems.
method Invertible networks to avoid storing intermediate activations, constant memory model.
result Trains 400-layer models on 3D MRI volumes, achieving state-of-the-art image reconstruction.
By using parity arguments we prove that free knots are, generally, not invertible.
A normalizing flow models a complex probability density as an invertible transformation of a simple density. The invertibility means that we can evaluate densities and generate samples from a flow. In practice, autoregressive flow-based models are slow to invert, making either density estimation or sample generation sl…
We study invertible generating pairs of fundamental groups of graph manifolds, that is, pairs of elements (g,h) for which the map g --> g^{-1}, h --> h^{-1} extends to an automorphism. We show in particular that a graph manifold is of Heegaard genus 2 if and only if its fundamental group has an invertible generating pa…
Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier c…
Spaces of metrics with invertible Dirac operator are homotopy equivalent for cobordant manifolds.
problem Proving homotopy equivalence of spaces of metrics with invertible Dirac operator.
method Using cobordism theory and properties of Dirac operators.
result Spaces of metrics with invertible Dirac operator are homotopy equivalent for cobordant manifolds.
Woodbury transformations improve deep generative models with efficient invertibility and determinant calculation.
problem Efficiently invertible and determinant-calculable functions for deep generative models.
method Introducing Woodbury transformations that leverage matrix identities for efficient invertibility and determinant calculation.
result Woodbury transformations enable high-dimensional interactions, efficient sampling, and likelihood evaluation, outperforming other flow architectures.
Residual neural networks don't help overcome sampling complexity issues.
problem Learning invertible residual neural networks from samples is hard due to the curse of dimensionality.
method Investigated invertible residual neural networks and their sampling complexity.
result Invertible residual neural networks still suffer from the curse of dimensionality in sampling complexity.
ISR creates analytical relationships from data via invertible maps.
problem Creating analytical relationships from datasets.
method Combines INNs and EQL, using invertible maps and sparsity promoting regularization.
result ISR can serve as a normalizing flow for density estimation and solve inverse problems.
Distribution grids are currently challenged by frequent voltage excursions induced by intermittent solar generation. Smart inverters have been advocated as a fast-responding means to regulate voltage and minimize ohmic losses. Since optimal inverter coordination may be computationally challenging and preset local contr…
We propose GraphNVP, the first invertible, normalizing flow-based molecular graph generation model. We decompose the generation of a graph into two steps: generation of (i) an adjacency tensor and (ii) node attributes. This decomposition yields the exact likelihood maximization on graph-structured data, combined with t…
i-DenseNets improve parameter efficiency and performance in density estimation.
problem Improving parameter efficiency and performance in density estimation models.
method Invertible Dense Networks (i-DenseNets) with learnable weighted concatenation and Concatenated LipSwish activation function.
result i-DenseNets outperform Residual Flows and other flow-based models in bits per dimension.
For operators of many different kinds it has been proved that (generalized) Darboux transformations can be built using so called Wronskian formulae. Such Darboux transformations are not invertible in the sense that the corresponding mappings of the operator kernels are not invertible. The only known invertible ones wer…
Invertible DenseNets improve model efficiency and performance.
problem Improving model efficiency and performance in neural networks.
method Enforcing invertibility in DenseNets by satisfying the Lipschitz constraint and proposing a learnable concatenation.
result i-DenseNets outperform Residual Flows in negative log-likelihood on various datasets.
New analysis enables inversion of deep generative models with unique solutions.
problem Inverting deep generative models like GANs and VAEs.
method Sparse representation theory and layer-wise inversion pursuit algorithms.
result Invertible solutions for generative models with unique latent vectors.
ButterflyFlow uses butterfly matrices for efficient invertible layers in normalizing flows.
problem Building efficient invertible layers for complex probability distributions.
method Proposes butterfly layers for invertible linear layers, leveraging their ability to capture complex structures.
result ButterflyFlow achieves strong density estimation and significantly better log-likelihoods on various datasets.
We consider so-called regular invertible Gaussian Volterra processes and derive a formula for their prediction laws. Examples of such processes include the fractional Brownian motions and the mixed fractional Brownian motions. As an application, we consider conditional-mean hedging under transaction costs in Black-Scho…
A new model decouples global and local image representations without supervision.
problem Learning decoupled global and local image representations without supervision.
method Variational auto-encoding framework with invertible generative flow.
result The model effectively learns decoupled representations of images.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of L2-risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator. result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
The paper stabilizes invertible neural networks by using Gaussian mixture models.
problem Invertible neural networks can have exploding Lipschitz constants, leading to numerical errors.
method The authors use Gaussian mixture models to stabilize the latent distribution of invertible neural networks.
result Numerical simulations confirm that this modification improves sampling quality in multimodal applications.
AIKAE enhances IKAE for long-term time series forecasting.
problem Limitation of dimension conservation in IKAE models.
method Augmented with a non-invertible encoder network.
result AIKAE improves long-term forecasting accuracy.
We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive property of our model is that both p(features), the density of the features, and p(targets | features), the predictive distribution, can be co…
Local invertibility of higher order tensor transforms on compact manifolds.
problem Invertibility of higher order tensor transforms on compact manifolds.
method Local invertibility of transverse and mixed ray transforms of tensors on compact Riemannian manifolds.
result Local invertibility of transverse and mixed ray transforms of tensors for specific dimensions.
Paper develops a method to learn causal networks with non-invertible functions.
problem Identifying causal relationships from observational data with non-invertible functional relationships.
method Proposes a test for non-invertible bivariate causal models and develops a method to incorporate this test in structure learning of DAGs.
result Our algorithms outperform existing DAG learning methods in identifying causal graphical structures.
Flow-based generative models (Dinh et al., 2014) are conceptually attractive due to tractability of the exact log-likelihood, tractability of exact latent-variable inference, and parallelizability of both training and synthesis. In this paper we propose Glow, a simple type of generative flow using an invertible 1x1 con…