Finet uses FBN for efficient, lightweight neural networks.
problem Building efficient neural networks with limited computational resources.
method Introduces Fine-grained Batch Normalization (FBN) and a novel light-weight network (Finet) that combines FBN with standard convolution.
result Finet achieves state-of-the-art performance on ImageNet classification with reduced computational complexity.
New method uses IB to train normalizing flows for better generative classification.
problem Training generative models with information theory for improved classification.
method Developed IB-INNs, a class of conditional normalizing flows trained with IB objective.
result IB-INNs offer improved uncertainty quantification and out-of-distribution detection compared to traditional generative classifiers.
We present a method for feature interpretation that makes use of recent advances in autoregressive density estimation models to invert model representations. We train generative inversion models to express a distribution over input features conditioned on intermediate model representations. Insights into the invariance…
Training Gaussian process-based models typically involves an O(N3) computational bottleneck due to inverting the covariance matrix. Popular methods for overcoming this matrix inversion problem cannot adequately model all types of latent functions, and are often not parallelizable. However, judicious choice of model…
We explore models for translating abstract musical ideas (scores, rhythms) into expressive performances using Seq2Seq and recurrent Variational Information Bottleneck (VIB) models. Though Seq2Seq models usually require painstakingly aligned corpora, we show that it is possible to adapt an approach from the Generative A…
A new method integrates autoencoders with geometry regularization for manifold learning.
problem Extracting simplified low-dimensional representations that capture intrinsic geometry in data.
method Integrates autoencoders with a geometric regularization term based on diffusion potential distances.
result The method preserves intrinsic structure, enables out-of-sample extension, and faithful reconstruction.
We address representational challenges in normalizing flows, particularly depth and conditioning issues.
problem Challenges in training normalizing flows, including vanishing/exploding gradients and poor conditioning.
method Analyzes representational aspects of depth and conditioning in normalizing flows, proving theoretical bounds and investigating phenomena.
result Proves that shallow affine coupling networks are universal approximators in Wasserstein distance if ill-conditioning is allowed.
New research shows non-bottlenecked autoencoders can outperform bottlenecked ones for anomaly detection.
problem The necessity of a bottleneck in autoencoders for anomaly detection.
method Investigated two ways to remove bottlenecks: overparameterising the latent layer and introducing skip connections. Carried out extensive experiments on various AE types and datasets.
result Non-bottlenecked autoencoders can outperform bottlenecked ones, improving anomaly detection performance.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of L2-risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator. result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
SCBMs model causal effects using low-dimensional bottlenecks.
problem Causal effect estimation in high-dimensional systems.
method Structural causal models with low-dimensional summary statistics.
result SCBMs provide a flexible framework for task-specific dimension reduction.
Wide neural networks with narrow bottlenecks behave like deep Gaussian processes.
problem Understanding the behavior of neural networks with narrow layers in the wide limit.
method Analyzing the wide limit of BNNs with narrow bottlenecks, showing they behave like a composition of GPs.
result Wide neural networks with narrow bottlenecks form a composition of GPs, termed a bottleneck NNGP.
Local invertibility of higher order tensor transforms on compact manifolds.
problem Invertibility of higher order tensor transforms on compact manifolds.
method Local invertibility of transverse and mixed ray transforms of tensors on compact Riemannian manifolds.
result Local invertibility of transverse and mixed ray transforms of tensors for specific dimensions.
Study of strongly invertible Legendrian links in contact 3-space.
problem Characterizing and understanding strongly invertible Legendrian links.
method Equivariant analogs of basic results for strongly invertible and Legendrian links.
result Existence of maximal equivariant Thurston-Bennequin number for strongly invertible links.
Softmax is an output activation function for modeling categorical probability distributions in many applications of deep learning. However, a recent study revealed that softmax can be a bottleneck of representational capacity of neural networks in language modeling (the softmax bottleneck). In this paper, we propose an…
Study on invariant Seifert surfaces for strongly invertible knots, showing large gaps in genus.
problem Understanding gaps in genus between strongly invertible knots and their invariant Seifert surfaces.
method Analysis of invariant Seifert surfaces and proof of genus gaps, with variants of Edmonds' theorem.
result Gap between equivariant genus and usual genus can be arbitrarily large for strongly invertible knots.
Dirac operator invertibility proven for specific manifolds.
problem Invertibility of twisted Dirac operator on manifolds.
method Closed connected spin manifold with non-negative scalar curvature, flat Hilbert module bundle.
result Dirac operator is invertible under given conditions.
Table of symmetric diagrams for knots up to 10 crossings.
problem Finding symmetric diagrams for strongly invertible knots.
method Compilation of symmetric diagrams for knots up to 10 crossings.
result Similarity of transversal diagrams to symmetric union diagrams for strongly invertible knots.
ISR creates analytical relationships from data via invertible maps.
problem Creating analytical relationships from datasets.
method Combines INNs and EQL, using invertible maps and sparsity promoting regularization.
result ISR can serve as a normalizing flow for density estimation and solve inverse problems.
Study shows bottlenecks improve image segmentation quality.
problem Robust object discovery in real-world images remains challenging.
method Empirical investigation of reconstruction bottlenecks in GENESIS model.
result Reconstruction bottlenecks determine reconstruction and segmentation quality.
Local invertibility of ray transforms on convex manifolds.
problem Invertibility of ray transforms on compact Riemannian manifolds with strictly convex boundary.
method Local invertibility results for transverse and mixed ray transforms of 1 and 1+1 tensors.
result Local invertibility of ray transforms near boundary points, leading to global results.
Deep ResNets favor low bottleneck rank with proper hyperparameters.
problem Understanding the inductive bias of deep neural networks.
method Computed minimum-norm weights of a deep linear ResNet.
result Deep nonlinear ResNets have an inductive bias towards minimizing bottleneck rank.
Paper revisits Deep Variational Information Bottleneck and proposes a new optimization approach.
problem Limitations of Deep Variational Information Bottleneck in optimizing mutual information.
method Proposes a new optimization approach by circumventing the limitation of requiring both Markov chains during optimisation.
result Shows how to optimise a lower bound for mutual information, circumventing the limitation of requiring both Markov chains.
This work tackles exploding inverses in INNs, revealing and mitigating their numerical non-invertibility.
problem Exploding inverses in INNs cause numerical non-invertibility, leading to failures in various tasks.
method Derived bi-Lipschitz properties of INN building blocks, proposed regularizers for local invertibility, and stable INN designs for global invertibility.
result Bi-Lipschitz properties and stable INN designs are crucial for addressing numerical non-invertibility.
CF-INNs can approximate any invertible function, resolving a long-standing problem.
problem Whether CF-INNs can approximate any invertible function.
method Demonstrated CF-INNs are universal approximators for invertible functions by showing a convenient criterion.
result CF-INNs are universal approximators for invertible functions.
This work presents a novel approach to train invertible linear layers by adding rank-one perturbations.
problem Training invertible linear layers during optimization with gradient-based methods is challenging.
method Train rank-one perturbations and add them to weight matrices infrequently, keeping track of inverses and determinants.
result Invertible linear layers improve mixing and mode separation in normalizing flows.
Develops equivariant grid homology for strongly invertible knots.
problem Invariants of strongly invertible knots.
method Equivariant grid diagrams and mapping cones.
result Equivariant unknotting numbers and genus bounds.
CB-APM uses analyst consensus as a bottleneck to interpret stock returns.
problem Tackles the challenge of understanding and predicting stock returns using professional beliefs.
method Embeds analyst consensus as a structural bottleneck, treating it as a sufficient statistic for market information.
result CB-APM portfolios exhibit strong monotonic return gradients and robust across different economic conditions.
New findings on knot genera using advanced techniques.
problem Understanding the 4-genus of knots, especially strongly invertible and periodic ones.
method Innovative concordance group invariants, Donaldson's theorem, and g-signature.
result Many new examples showing the equivariant 4-genus is larger than the 4-genus.
Defines knot signature invariant using G-signature theorem.
problem No specific problem stated; focuses on knot theory.
method Uses G-signature theorem to define knot invariant.
result Defines an invariant for strongly invertible knots.
Paper shows invertibility of tensor X-ray transform on certain manifolds.
problem Invertibility of tensor X-ray transform on asymptotically conic manifolds.
method Used 1-cusp pseudodifferential operator algebra and modified solenoidal gauge condition.
result Invertibility of tensor X-ray transform up to natural obstruction.
Wide neural networks become linear, but adding bottlenecks makes them bilinear or multilinear.
problem Understanding the transition of neural networks from linearity to higher-order functions.
method Analyzing the behavior of randomly initialized wide neural networks with and without bottleneck layers.
result Bottleneck layers transform the network's function from linear to bilinear or multilinear.
Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier c…
For operators of many different kinds it has been proved that (generalized) Darboux transformations can be built using so called Wronskian formulae. Such Darboux transformations are not invertible in the sense that the corresponding mappings of the operator kernels are not invertible. The only known invertible ones wer…
New invertible transformations improve flow-based generative models.
problem Improving flow-based generative models for better performance.
method Proposed new invertible transformations and coupling layers.
result New coupling layers achieve better results in IDF.
We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation. Typically, enforcing invertibility requires partitioning dimensions or restricting network architectures. In contrast, our approach only requires adding a simple …
New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
ButterflyFlow uses butterfly matrices for efficient invertible layers in normalizing flows.
problem Building efficient invertible layers for complex probability distributions.
method Proposes butterfly layers for invertible linear layers, leveraging their ability to capture complex structures.
result ButterflyFlow achieves strong density estimation and significantly better log-likelihoods on various datasets.
Electronic power inverters are capable of quickly delivering reactive power to maintain customer voltages within operating tolerances and to reduce system losses in distribution grids. This paper proposes a systematic and data-driven approach to determine reactive power inverter output as a function of local measuremen…
By using parity arguments we prove that free knots are, generally, not invertible.
Many recent invertible neural architectures are based on coupling block designs where variables are divided in two subsets which serve as inputs of an easily invertible (usually affine) triangular transformation. While such a transformation is invertible, its Jacobian is very sparse and thus may lack expressiveness. Th…
Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations where only Lipschitz conditions rather than strict architectural constraints are needed for enforci…
Invertible DenseNets improve model efficiency and performance.
problem Improving model efficiency and performance in neural networks.
method Enforcing invertibility in DenseNets by satisfying the Lipschitz constraint and proposing a learnable concatenation.
result i-DenseNets outperform Residual Flows in negative log-likelihood on various datasets.
Two knots with unique surgery properties.
problem Characterizing strongly invertible L-space knots.
method Examined surgeries and knot properties.
result Found knots whose surgeries are never Khovanov thin.
We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible architectures, including those similar to ResNets. Inversion is achieved with a locally convergent iterative procedure that is parallelizable …
By a result of John Ball (1981), a locally orientation preserving Sobolev map is almost everywhere globally invertible whenever its boundary values admit a homeomorphic extension. As shown here for any dimension, the conclusions of Ball's theorem and related results can be reached while completely avoiding the problem …
New spectral sequences define knot invariants.
problem Understanding strongly invertible knots.
method Two spectral sequences in knot Floer homology.
result Numerical invariant defined for strongly invertible knots.
The Mahler volume of a centrally symmetric convex body K is defined as M(K)= (Vol K)(Vol K^dual). Mahler conjectured that this volume is minimized when K is a cube. We introduce the bottleneck conjecture, which stipulates that a certain convex body K^diamond subset K X K^dual has least volume when K is an ellipsoid. If…
The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are informative about B. The information bottleneck has already been applied to doc…