arXiv research

Incorporates matrix exponential into generative flows for improved performance.

Normalizing flows attempt to model an arbitrary probability distribution through a set of invertible mappings. These transformations are required to achieve a tractable Jacobian determinant that can be used in high-dimensional scenarios. The first normalizing flow designs used coupling layer mappings built upon affine …

CF-INNs can approximate any invertible function, resolving a long-standing problem.

Paper proves CFlows can approximate any diffeomorphism and applies it in Bayesian optimization.

We address representational challenges in normalizing flows, particularly depth and conditioning issues.

This paper shows how to approximate any log-concave distribution using well-conditioned affine coupling flows.

Normalizing flows are shown to be equivalent to Bayesian networks, revealing new insights.

We study cascades on a two-layer multiplex network, with asymmetric feedback that depends on the coupling strength between the layers. Based on an analytical branching process approximation, we calculate the systemic risk measured by the final fraction of failed nodes on a reference layer. The results are compared with…

Current popular methods for Magnetic Resonance Fingerprint (MRF) recovery are bottlenecked by the heavy storage and computation requirements of a dictionary-matching (DM) step due to the growing size and complexity of the fingerprint dictionaries in multi-parametric quantitative MRI applications. In this paper we study…

Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.

New framework explains normalizing flows' power and limitations.

The paper calculates bounds on the local Lipschitz constants of neural network layers.

We introduce a class of objects which we call 'affine surfaces'. These provide families of foliations on surfaces whose dynamics we are interested in. We present and analyze a couple of examples, and we define concepts related to these in order to motivate several questions and open problems. In particular we generalis…

New invertible transformations improve flow-based generative models.

We investigate robustness of deep feed-forward neural networks when input data are subject to random uncertainties. More specifically, we consider regularization of the network by its Lipschitz constant and emphasize its role. We highlight the fact that this regularization is not only a way to control the magnitude of …

Equations link metrics with tensors, revealing curvature constraints.

Many recent invertible neural architectures are based on coupling block designs where variables are divided in two subsets which serve as inputs of an easily invertible (usually affine) triangular transformation. While such a transformation is invertible, its Jacobian is very sparse and thus may lack expressiveness. Th…

DS2CF-Net learns hierarchical representations with deep coupled factorization and enriched prior.

Graph-Coupled Oscillator Networks (GraphCON) tackles graph-based learning problems.

New bounds on optimal transport regularization show faster convergence rates than previously known.

We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be {\em max-affine spline operators} (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrat…

Recent work has shown how to embed differentiable optimization problems (that is, problems whose solutions can be backpropagated through) as layers within deep learning architectures. This method provides a useful inductive bias for certain problems, but existing software for differentiable optimization layers is rigid…

Flow-based generative models are powerful exact likelihood models with efficient sampling and inference. Despite their computational efficiency, flow-based models generally have much worse density modeling performance compared to state-of-the-art autoregressive models. In this paper, we investigate and improve upon thr…

Improved flow-based models capture dependencies better with multi-scale autoregressive priors.

Chaos in cerebellar cells enhances complexity of neural patterns.

Improved bounds on neural network regions using activation histograms.

Neural networks can represent complex piecewise functions efficiently.

Conditions for statistical structures on manifolds derived from solitons.

We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the global network level, with each branch being a standalone CNN. We show that this arrangement is an effici…

Dual Bayesian Affine Estimators for Wiener-type state-space models

Traditional convolutional layers extract features from patches of data by applying a non-linearity on an affine function of the input. We propose a model that enhances this feature extraction process for the case of sequential data, by feeding patches of the data into a recurrent neural network and using the outputs or…

Study reveals geometric context of second-order superintegrable systems.

We present convolutional neural network (CNN) based approaches for unsupervised multimodal subspace clustering. The proposed framework consists of three main stages - multimodal encoder, self-expressive layer, and multimodal decoder. The encoder takes multimodal data as input and fuses them to a latent space representa…

Clarifies relation for solving control-affine Schrödinger bridge problems.

Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.

This study compares different types of normalizing flows for generating complex distributions.

The paper designs neural networks with assurance for controlling nonlinear systems.

Framework uses optimal transport to quantify model risk in stochastic path laws.

This paper analyzes deep and wide transformer training dynamics.

Deep learning uses layers of transformations to predict structured data with uncertainty.

New NN design for nonlinear systems control with guarantees.

Many real-world complex systems across natural, social, and economical domains consist of manifold layers to form multiplex networks. The multiple network layers give rise to nonlinear effect for the emergent dynamics of systems. Especially, weak layers that can potentially play significant role in amplifying the vulne…

Fine-tuning normalization layers can reconstruct smaller networks.

Nonconvex and nonsmooth optimization problems are frequently encountered in much of statistics, business, science and engineering, but they are not yet widely recognized as a technology in the sense of scalability. A reason for this relatively low degree of popularity is the lack of a well developed system of theory an…

New algorithm solves Schrödinger bridge problem with mismatched channels.

Deep nonlinear models pose a challenge for fitting parameters due to lack of knowledge of the hidden layer and the potentially non-affine relation of the initial and observed layers. In the present work we investigate the use of information theoretic measures such as mutual information and Kullback-Leibler (KL) diverge…

We consider a stochastic factor financial model where the asset price process and the process for the stochastic factor depend on an observable Markov chain and exhibit an affine structure. We are faced with a finite time investment horizon and derive optimal dynamic investment strategies that maximize the investor's e…

We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - an…