Invertible DenseNets improve model efficiency and performance.
problem Improving model efficiency and performance in neural networks.
method Enforcing invertibility in DenseNets by satisfying the Lipschitz constraint and proposing a learnable concatenation.
result i-DenseNets outperform Residual Flows in negative log-likelihood on various datasets.
ButterflyFlow uses butterfly matrices for efficient invertible layers in normalizing flows.
problem Building efficient invertible layers for complex probability distributions.
method Proposes butterfly layers for invertible linear layers, leveraging their ability to capture complex structures.
result ButterflyFlow achieves strong density estimation and significantly better log-likelihoods on various datasets.
Woodbury transformations improve deep generative models with efficient invertibility and determinant calculation.
problem Efficiently invertible and determinant-calculable functions for deep generative models.
method Introducing Woodbury transformations that leverage matrix identities for efficient invertibility and determinant calculation.
result Woodbury transformations enable high-dimensional interactions, efficient sampling, and likelihood evaluation, outperforming other flow architectures.
SPTN uses invertible transformations to improve sum-product networks.
problem Improving inference efficiency and tractability in sum-product networks.
method Integrates invertible transformations into sum-product networks (SPNs).
result SPTNs with Gaussian leaves and affine transformations are as tractable as SPNs.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of L2-risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator. result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
i-DenseNets improve parameter efficiency and performance in density estimation.
problem Improving parameter efficiency and performance in density estimation models.
method Invertible Dense Networks (i-DenseNets) with learnable weighted concatenation and Concatenated LipSwish activation function.
result i-DenseNets outperform Residual Flows and other flow-based models in bits per dimension.
ISR creates analytical relationships from data via invertible maps.
problem Creating analytical relationships from datasets.
method Combines INNs and EQL, using invertible maps and sparsity promoting regularization.
result ISR can serve as a normalizing flow for density estimation and solve inverse problems.
We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible architectures, including those similar to ResNets. Inversion is achieved with a locally convergent iterative procedure that is parallelizable …
This work tackles exploding inverses in INNs, revealing and mitigating their numerical non-invertibility.
problem Exploding inverses in INNs cause numerical non-invertibility, leading to failures in various tasks.
method Derived bi-Lipschitz properties of INN building blocks, proposed regularizers for local invertibility, and stable INN designs for global invertibility.
result Bi-Lipschitz properties and stable INN designs are crucial for addressing numerical non-invertibility.
Many recent invertible neural architectures are based on coupling block designs where variables are divided in two subsets which serve as inputs of an easily invertible (usually affine) triangular transformation. While such a transformation is invertible, its Jacobian is very sparse and thus may lack expressiveness. Th…
Electronic power inverters are capable of quickly delivering reactive power to maintain customer voltages within operating tolerances and to reduce system losses in distribution grids. This paper proposes a systematic and data-driven approach to determine reactive power inverter output as a function of local measuremen…
Incorporates matrix exponential into generative flows for improved performance.
problem Improving generative flow models for better density estimation.
method Integrates matrix exponential into generative flows, proposing new layers and modifying network architecture.
result The proposed model achieves great performance on density estimation.
ETGP improves multi-class classification efficiency.
problem Efficiently handling non-stationary, dependent multi-class classification problems.
method ETGP uses transformed Gaussian processes with efficient sparse variational inference.
result ETGPs outperform state-of-the-art methods in multi-class classification tasks.
iGNN tackles inverse graph prediction using invertible neural networks.
problem Inverse graph prediction problem in data analysis and machine learning.
method Developed invertible graph neural network (iGNN) to solve inverse prediction problem on graphs.
result iGNN model allows efficient generation from output labels and forward prediction.
Momentum ResNets improve ResNets' memory efficiency.
problem Memory inefficiency in deep residual neural networks (ResNets).
method Adding a momentum term to the forward rule of ResNets to make them invertible.
result Momentum ResNets can learn any linear mapping up to a multiplicative factor, improving memory efficiency.
Finet uses FBN for efficient, lightweight neural networks.
problem Building efficient neural networks with limited computational resources.
method Introduces Fine-grained Batch Normalization (FBN) and a novel light-weight network (Finet) that combines FBN with standard convolution.
result Finet achieves state-of-the-art performance on ImageNet classification with reduced computational complexity.
We propose GraphNVP, the first invertible, normalizing flow-based molecular graph generation model. We decompose the generation of a graph into two steps: generation of (i) an adjacency tensor and (ii) node attributes. This decomposition yields the exact likelihood maximization on graph-structured data, combined with t…
A new RNN model tackles long-time dependencies with fast, invertible, and memory-efficient hidden states.
problem Challenges in processing sequential inputs with long-time dependencies in RNNs.
method A novel RNN architecture based on a Hamiltonian system of oscillators.
result The proposed RNN mitigates exploding and vanishing gradient problems, providing state-of-the-art performance.
Hydrogen atom confined in an inverted-Gaussian potential, with detailed numerical methods and results.
problem Studying hydrogen atom in a specific potential.
method Three numerical methods: Lagrange-mesh, fourth order finite differences, and finite element method.
result Accurate numerical results for hydrogen atom energies and eigenfunctions, improving previous literature.
Flow-based generative models (Dinh et al., 2014) are conceptually attractive due to tractability of the exact log-likelihood, tractability of exact latent-variable inference, and parallelizability of both training and synthesis. In this paper we propose Glow, a simple type of generative flow using an invertible 1x1 con…
Local invertibility of higher order tensor transforms on compact manifolds.
problem Invertibility of higher order tensor transforms on compact manifolds.
method Local invertibility of transverse and mixed ray transforms of tensors on compact Riemannian manifolds.
result Local invertibility of transverse and mixed ray transforms of tensors for specific dimensions.
Study of strongly invertible Legendrian links in contact 3-space.
problem Characterizing and understanding strongly invertible Legendrian links.
method Equivariant analogs of basic results for strongly invertible and Legendrian links.
result Existence of maximal equivariant Thurston-Bennequin number for strongly invertible links.
IVF k-means algorithm improves performance on large sparse data sets.
problem Efficiently clustering large-scale sparse data sets with numerous classes.
method Sparse data representation and inverted-file structure for high-speed and low-memory clustering.
result IVF achieves better performance than other algorithms on real document data sets.
We develop and analyze efficient "coordinate-wise" methods for finding the leading eigenvector, where each step involves only a vector-vector product. We establish global convergence with overall runtime guarantees that are at least as good as Lanczos's method and dominate it for slowly decaying spectrum. Our methods a…
Kernelised flows improve density estimation and generation with fewer parameters.
problem Limited expressiveness of flow-based models due to invertibility constraints.
method Integrates kernels into normalising flows to enhance expressiveness and efficiency.
result Kernelised flows outperform neural network-based flows in parameter efficiency and low-data scenarios.
In this paper, we propose a novel lower dimensional representation of a shape sequence. The proposed dimension reduction is invertible and computationally more efficient in comparison to other related works. Theoretically, the differential geometry tools such as moving frame and parallel transportation are successfully…
Study on invariant Seifert surfaces for strongly invertible knots, showing large gaps in genus.
problem Understanding gaps in genus between strongly invertible knots and their invariant Seifert surfaces.
method Analysis of invariant Seifert surfaces and proof of genus gaps, with variants of Edmonds' theorem.
result Gap between equivariant genus and usual genus can be arbitrarily large for strongly invertible knots.
Dirac operator invertibility proven for specific manifolds.
problem Invertibility of twisted Dirac operator on manifolds.
method Closed connected spin manifold with non-negative scalar curvature, flat Hilbert module bundle.
result Dirac operator is invertible under given conditions.
Table of symmetric diagrams for knots up to 10 crossings.
problem Finding symmetric diagrams for strongly invertible knots.
method Compilation of symmetric diagrams for knots up to 10 crossings.
result Similarity of transversal diagrams to symmetric union diagrams for strongly invertible knots.
Local invertibility of ray transforms on convex manifolds.
problem Invertibility of ray transforms on compact Riemannian manifolds with strictly convex boundary.
method Local invertibility results for transverse and mixed ray transforms of 1 and 1+1 tensors.
result Local invertibility of ray transforms near boundary points, leading to global results.
Survival MDN uses invertible functions to speed up survival analysis models.
problem Training neural ODEs for survival analysis is computationally expensive.
method Survival MDN applies an invertible positive function to MDN outputs.
result Survival MDN outperforms or matches other models on concordance, Brier score, and log-likelihood.
Generative flows are attractive because they admit exact likelihood optimization and efficient image synthesis. Recently, Kingma & Dhariwal (2018) demonstrated with Glow that generative flows are capable of generating high quality images. We generalize the 1 x 1 convolutions proposed in Glow to invertible d x d convolu…
CF-INNs can approximate any invertible function, resolving a long-standing problem.
problem Whether CF-INNs can approximate any invertible function.
method Demonstrated CF-INNs are universal approximators for invertible functions by showing a convenient criterion.
result CF-INNs are universal approximators for invertible functions.
This work presents a novel approach to train invertible linear layers by adding rank-one perturbations.
problem Training invertible linear layers during optimization with gradient-based methods is challenging.
method Train rank-one perturbations and add them to weight matrices infrequently, keeping track of inverses and determinants.
result Invertible linear layers improve mixing and mode separation in normalizing flows.
Develops equivariant grid homology for strongly invertible knots.
problem Invariants of strongly invertible knots.
method Equivariant grid diagrams and mapping cones.
result Equivariant unknotting numbers and genus bounds.
New method solves high-dimensional Bayesian inverse problems efficiently.
problem Efficiently solving high-dimensional Bayesian inverse problems with limited data.
method Physics-informed Neural Operators with RealNVP architecture for invertibility and differentiability.
result Accurate approximations of the full posterior without additional forward solves or sampling.
New findings on knot genera using advanced techniques.
problem Understanding the 4-genus of knots, especially strongly invertible and periodic ones.
method Innovative concordance group invariants, Donaldson's theorem, and g-signature.
result Many new examples showing the equivariant 4-genus is larger than the 4-genus.
Defines knot signature invariant using G-signature theorem.
problem No specific problem stated; focuses on knot theory.
method Uses G-signature theorem to define knot invariant.
result Defines an invariant for strongly invertible knots.
Paper shows invertibility of tensor X-ray transform on certain manifolds.
problem Invertibility of tensor X-ray transform on asymptotically conic manifolds.
method Used 1-cusp pseudodifferential operator algebra and modified solenoidal gauge condition.
result Invertibility of tensor X-ray transform up to natural obstruction.
Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier c…
For operators of many different kinds it has been proved that (generalized) Darboux transformations can be built using so called Wronskian formulae. Such Darboux transformations are not invertible in the sense that the corresponding mappings of the operator kernels are not invertible. The only known invertible ones wer…
A promising class of generative models maps points from a simple distribution to a complex distribution through an invertible neural network. Likelihood-based training of these models requires restricting their architectures to allow cheap computation of Jacobian determinants. Alternatively, the Jacobian trace can be u…
New invertible transformations improve flow-based generative models.
problem Improving flow-based generative models for better performance.
method Proposed new invertible transformations and coupling layers.
result New coupling layers achieve better results in IDF.
We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation. Typically, enforcing invertibility requires partitioning dimensions or restricting network architectures. In contrast, our approach only requires adding a simple …
SIVF k-means algorithm speeds up sparse data clustering.
problem Efficiently clustering large-scale high-dimensional sparse data.
method Inverted-file structure for centroids, filter-based similarity reduction.
result SIVF achieves higher speed and lower memory consumption.
By using parity arguments we prove that free knots are, generally, not invertible.
A new method for aligning multiple distributions efficiently.
problem Aligning multiple distributions in a shared latent space.
method Iterative alignment of variational approximations of distribution divergences using invertible alignment maps.
result Our method achieves competitive distribution alignment at low computational cost.
While MCMC methods have become a main work-horse for Bayesian inference, scaling them to large distributed datasets is still a challenge. Embarrassingly parallel MCMC strategies take a divide-and-conquer stance to achieve this by writing the target posterior as a product of subposteriors, running MCMC for each of them …