INVERT connects neural representations to human-understandable concepts.
problem Lack of understanding and statistical significance in existing explainability methods.
method Inverse Recognition (INVERT) approach that connects learned representations to human-understandable concepts.
result INVERT provides interpretable metrics and statistical significance for representation alignment.
INNs can approximate diverse functions despite layer restrictions.
problem Can INNs approximate sufficiently diverse functions?
method Developed a theoretical framework based on differential geometry to simplify the approximation problem of diffeomorphisms.
result INNs have the universal approximation property.
CF-INNs can approximate any invertible function, resolving a long-standing problem.
problem Whether CF-INNs can approximate any invertible function.
method Demonstrated CF-INNs are universal approximators for invertible functions by showing a convenient criterion.
result CF-INNs are universal approximators for invertible functions.
A generalized bridge is the law of a stochastic process that is conditioned on N linear functionals of its path. We consider two types of representations of such bridges: orthogonal and canonical. The orthogonal representation is constructed from the entire path of the underlying process. Thus, future knowledge of the …
A new model decouples global and local image representations without supervision.
problem Learning decoupled global and local image representations without supervision.
method Variational auto-encoding framework with invertible generative flow.
result The model effectively learns decoupled representations of images.
Develops equivariant grid homology for strongly invertible knots.
problem Invariants of strongly invertible knots.
method Equivariant grid diagrams and mapping cones.
result Equivariant unknotting numbers and genus bounds.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of L2-risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator. result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
It is known, since works of Burde and de Rham, that one can detect the roots of the Alexander polynomial of a knot by the study of the representations of the knot group into the group of the invertible upper triangular 2x2 matrices. In this work, we propose to generalize this result by considering the representations…
Learning domain-invariant representations has become a popular approach to unsupervised domain adaptation and is often justified by invoking a particular suite of theoretical results. We argue that there are two significant flaws in such arguments. First, the results in question hold only for a fixed representation and…
Invertible DenseNets improve model efficiency and performance.
problem Improving model efficiency and performance in neural networks.
method Enforcing invertibility in DenseNets by satisfying the Lipschitz constraint and proposing a learnable concatenation.
result i-DenseNets outperform Residual Flows in negative log-likelihood on various datasets.
It is widely believed that the success of deep convolutional networks is based on progressively discarding uninformative variability about the input with respect to the problem at hand. This is supported empirically by the difficulty of recovering images from their hidden representations, in most commonly used network …
Robust optimization improves deep learning feature representations.
problem Deep learning feature representations lack versatility and high-level encoding.
method Robust optimization as a prior for feature learning.
result Robust models learn approximately invertible, salient feature representations.
ButterflyFlow uses butterfly matrices for efficient invertible layers in normalizing flows.
problem Building efficient invertible layers for complex probability distributions.
method Proposes butterfly layers for invertible linear layers, leveraging their ability to capture complex structures.
result ButterflyFlow achieves strong density estimation and significantly better log-likelihoods on various datasets.
As early as 1972, Penrose - in a purely formal way - introduced a "discontinuous coordinate transformation", which relates a continuous representation of the metric of impulsive pp-waves to a discontinuous one. On the basis of the invertibility concept for generalized functions developed recently by the first author, w…
VAEs improve representation learning by inverting the data-generating process through self-consistency.
problem VAEs struggle to invert the data-generating process, yet often succeed in representation learning.
method Studied VAEs in the limit of near-deterministic decoders, proving self-consistency and showing ELBO convergence to a regularized log-likelihood.
result VAEs can perform independent mechanism analysis (IMA), recovering true latent factors under specific conditions.
IVF k-means algorithm improves performance on large sparse data sets.
problem Efficiently clustering large-scale sparse data sets with numerous classes.
method Sparse data representation and inverted-file structure for high-speed and low-memory clustering.
result IVF achieves better performance than other algorithms on real document data sets.
We present a method for feature interpretation that makes use of recent advances in autoregressive density estimation models to invert model representations. We train generative inversion models to express a distribution over input features conditioned on intermediate model representations. Insights into the invariance…
New analysis enables inversion of deep generative models with unique solutions.
problem Inverting deep generative models like GANs and VAEs.
method Sparse representation theory and layer-wise inversion pursuit algorithms.
result Invertible solutions for generative models with unique latent vectors.
In this work we compute lower Lipschitz bounds of ℓp pooling operators for p=1,2,∞ as well as ℓp pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm tha…
The paper explores how invertibility affects the complexity of encoder models in VAEs.
problem The complexity of the encoder model in VAEs when the generative map is invertible.
method Formalizes the concept of strong invertibility and analyzes the complexity of the encoder model.
result Strongly invertible generative maps allow for simpler encoder models, while non-invertible maps require exponentially larger encoders.
i-DenseNets improve parameter efficiency and performance in density estimation.
problem Improving parameter efficiency and performance in density estimation models.
method Invertible Dense Networks (i-DenseNets) with learnable weighted concatenation and Concatenated LipSwish activation function.
result i-DenseNets outperform Residual Flows and other flow-based models in bits per dimension.
dpVAEs improve VAEs by decoupling representation and generation.
problem VAEs struggle with both representation learning and sample generation.
method Introduce decoupled priors (dpVAEs) that separate representation and generation spaces.
result dpVAEs enable regularization without compromising sample generation.
Paper aims to find joint representation between vocal tract geometry and speech sound acoustics.
problem Finding a joint latent representation between articulatory and acoustic domains for vowel sounds.
method Invertible neural network models, convolutional autoencoder, normalizing flows, semi-supervised learning.
result Satisfactory performance in articulatory-to-acoustic and acoustic-to-articulatory mapping.
Adversarial training enhances model transferability without sacrificing accuracy.
problem The principle of minimal information in classification models is challenged by adversarial training.
method Investigation of the dual relationship between adversarial training and information theory.
result Adversarial training improves linear transferability and introduces a trade-off between transferability and source task accuracy.
A new method integrates autoencoders with geometry regularization for manifold learning.
problem Extracting simplified low-dimensional representations that capture intrinsic geometry in data.
method Integrates autoencoders with a geometric regularization term based on diffusion potential distances.
result The method preserves intrinsic structure, enables out-of-sample extension, and faithful reconstruction.
We address representational challenges in normalizing flows, particularly depth and conditioning issues.
problem Challenges in training normalizing flows, including vanishing/exploding gradients and poor conditioning.
method Analyzes representational aspects of depth and conditioning in normalizing flows, proving theoretical bounds and investigating phenomena.
result Proves that shallow affine coupling networks are universal approximators in Wasserstein distance if ill-conditioning is allowed.
We interpret an open orbit in a 32-dimensional representation space of Spin(9,1) x SL(2,R) as a substitute for the non-existent group of invertible 2x2 matrices over the octonions and study various natural homogeneous subspaces. The approach is via twistor geometry in eight dimensions.
In this paper, we propose a novel lower dimensional representation of a shape sequence. The proposed dimension reduction is invertible and computationally more efficient in comparison to other related works. Theoretically, the differential geometry tools such as moving frame and parallel transportation are successfully…
Momentum ResNets improve ResNets' memory efficiency.
problem Memory inefficiency in deep residual neural networks (ResNets).
method Adding a momentum term to the forward rule of ResNets to make them invertible.
result Momentum ResNets can learn any linear mapping up to a multiplicative factor, improving memory efficiency.
A group-category is an additively semisimple category with a monoidal product structure in which the simple objects are invertible. For example in the category of representations of a group, 1-dimensional representations are the invertible simple objects. This paper gives a detailed exploration of "topological quantum …
We present a signal representation framework called the sparse manifold transform that combines key ideas from sparse coding, manifold learning, and slow feature analysis. It turns non-linear transformations in the primary sensory signal space into linear interpolations in a representational embedding space while maint…
MoFlow generates chemically valid molecular graphs from latent representations.
problem Generating chemically valid molecular graphs from latent representations is challenging.
method MoFlow uses a flow-based approach with Glow for bond generation and a novel graph conditional flow for atom generation, ensuring chemical validity and efficiency.
result MoFlow achieves state-of-the-art performance in molecular graph generation and optimization.
Recent work has sought to understand the behavior of neural networks by comparing representations between layers and between different trained models. We examine methods for comparing neural network representations based on canonical correlation analysis (CCA). We show that CCA belongs to a family of statistics for mea…
Local invertibility of higher order tensor transforms on compact manifolds.
problem Invertibility of higher order tensor transforms on compact manifolds.
method Local invertibility of transverse and mixed ray transforms of tensors on compact Riemannian manifolds.
result Local invertibility of transverse and mixed ray transforms of tensors for specific dimensions.
Using the representation of the isometries as 2x2 invertible matrices over the division algebra $\H$ of quaternions, we give an algebraic characterization of the dynamical types of the orientation-preserving isometries of the hyperbolic 5-space. We also determine the conjugacy classes and the conjugacy classes of centr…
Study of strongly invertible Legendrian links in contact 3-space.
problem Characterizing and understanding strongly invertible Legendrian links.
method Equivariant analogs of basic results for strongly invertible and Legendrian links.
result Existence of maximal equivariant Thurston-Bennequin number for strongly invertible links.
Study fermionic theories, their anomalies, and modular transformations.
problem Understanding fermionic theories and their anomalies.
method Use spin-cobordisms, surgeries, and invertible topological quantum field theories.
result Explicit combinatorial expressions for spin-bordism invariants.
Study on invariant Seifert surfaces for strongly invertible knots, showing large gaps in genus.
problem Understanding gaps in genus between strongly invertible knots and their invariant Seifert surfaces.
method Analysis of invariant Seifert surfaces and proof of genus gaps, with variants of Edmonds' theorem.
result Gap between equivariant genus and usual genus can be arbitrarily large for strongly invertible knots.
Dirac operator invertibility proven for specific manifolds.
problem Invertibility of twisted Dirac operator on manifolds.
method Closed connected spin manifold with non-negative scalar curvature, flat Hilbert module bundle.
result Dirac operator is invertible under given conditions.
Invertible networks help explain decisions and identify important features.
problem Interpreting and explaining the decisions of black-box neural networks.
method Two-stage approach: invertible transformation to feature space and linear classifier. Determining decision boundaries and feature importance using local linear models.
result Ability to explain decisions and identify important features in neural networks.
Table of symmetric diagrams for knots up to 10 crossings.
problem Finding symmetric diagrams for strongly invertible knots.
method Compilation of symmetric diagrams for knots up to 10 crossings.
result Similarity of transversal diagrams to symmetric union diagrams for strongly invertible knots.
Global invertibility proven for orientation-preserving maps without homeomorphic extension.
problem Global invertibility of orientation-preserving Sobolev maps.
method Avoiding homeomorphic extension, study of strictly orientation-preserving maps.
result Global invertibility can be achieved without homeomorphic extension.
Invertible neural networks with masked convolutions improve classification and generative models.
problem Building robust invertible neural networks for better model interpretability and generative tasks.
method Combining masked convolutions and iterative inversion methods to create invertible architectures.
result Invertible neural networks achieve competitive performance in classification and generative tasks.
ISR creates analytical relationships from data via invertible maps.
problem Creating analytical relationships from datasets.
method Combines INNs and EQL, using invertible maps and sparsity promoting regularization.
result ISR can serve as a normalizing flow for density estimation and solve inverse problems.
Summarizes quantum field theories with discrete symmetry, classifying representations and anomalies.
problem Classifying representations and anomalies in quantum field theories with discrete symmetry.
method Classification of representations and anomalies using the ring of profinite integers.
result Rich and complex classification of representations and anomalies.
This paper shows cross-entropy can recover latent structures in supervised learning.
problem Understanding why supervised learning works well and how models learn interpretable factors of variation.
method Extending identifiability results to parametric instance discrimination, proving cross-entropy minimization can recover latent structures up to linear transformations.
result Models trained with cross-entropy can learn representations of ground-truth factors of variation up to a linear transformation.
Local invertibility of ray transforms on convex manifolds.
problem Invertibility of ray transforms on compact Riemannian manifolds with strictly convex boundary.
method Local invertibility results for transverse and mixed ray transforms of 1 and 1+1 tensors.
result Local invertibility of ray transforms near boundary points, leading to global results.
Deep invertible networks decode EEG signals better than chance.
problem Decoding brain signals from EEG data.
method Deep invertible networks for generating and classifying brain signals.
result Deep invertible networks generate realistic EEG signals and classify novel signals above chance.