arXiv research

This paper extends stability and generalization analysis of GD for multi-layer NNs.

A neural network derived from first principles using MaxEnt.

We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - an…

The rapid development of high-throughput technologies has enabled the generation of data from biological or disease processes that span multiple layers, like genomic, proteomic or metabolomic data, and further pertain to multiple sources, like disease subtypes or experimental conditions. In this work, we propose a gene…

Smectic liquid crystals are materials formed by stacking deformable, fluid layers. Though smectics prefer to have flat, uniformly-spaced layers, boundary conditions can impose curvature on the layers. Since the layer spacing and curvature are intertwined, the problem of finding minimal configurations for the layers bec…

Alternative to convolutions using decision trees for neural networks.

Paper learns DAGs with quadratic variance functions efficiently.

Improves conditions for mode connectivity in deep neural networks.

Gradient descent converges to minimum Bayes risk for two-layer ReLU networks in mean field regime.

Quantitatively assessing relationships between latent variables and observed variables is important for understanding and developing generative models and representation learning. In this paper, we propose latent-observed dissimilarity (LOD) to evaluate the dissimilarity between the probabilistic characteristics of lat…

The study characterizes sets for which one-layer neural networks are positive.

Study Transformer layers under cross-entropy training using mean field control.

Symmetric functions learn better with strategic initial conditions.

Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units. These networks define models of conditional probability distributions on the states of the output units given the states of the input units, parame…

Improved Q&A model with LSTM and bi-directional attention.

Two-layer CNNs can overfit well if initialized correctly.

Paper presents ML-VAMP for efficient multi-layer inference with exact performance analysis.

Study on functions computed by deep-layered machines finds same distribution in neural networks and Boolean circuits.

New method recovers signals from compressed measurements using generative networks with contractive layers.

Under several geometric conditions imposed below, the existence of the discrete spectrum below the essential spectrum is shown for the Dirichlet Laplacian on the quantum layer built over a spherically symmetric hypersurface with a pole embedded in the Euclidean space R4. At the end of this paper, we also show the advan…

Two-layer CNNs can overfit without harm under certain conditions.

Gradient descent with logistic loss can make two-layer networks interpolate binary classification data.

Batch normalization in the last layer reduces sharpness in wide neural networks.

Gradient descent proves global convergence for 4-layer matrix factorization.

Introduces Causal Energy Minimization to understand Transformer layers.

Integrates differentiable decision trees into neural networks for faster training and inference.

In this work we compute lower Lipschitz bounds of $\ell_p$ pooling operators for $p=1, 2, \infty$ as well as $\ell_p$ pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm tha…

Hybrid deep architectures with reasoning layers show promising convergence and generalization properties.

We analyze the loss landscape and expressiveness of practical deep convolutional neural networks (CNNs) with shared weights and max pooling layers. We show that such CNNs produce linearly independent features at a "wide" layer which has more neurons than the number of training samples. This condition holds e.g. for the…

A usual reinsurance policy for insurance companies admits one or two layers of the payment deductions. Under optimal criterion of minimizing the conditional tail expectation (CTE) risk measure of the insurer's total risk, this article generalized an optimal stop-loss reinsurance policy to an optimal multi-layer reinsur…

Deep neural networks converge to Gaussian mixtures as layer width increases.

Proves invertibility of layer potentials for generalized Stokes operators on smooth domains.

The muti-layer information bottleneck (IB) problem, where information is propagated (or successively refined) from layer to layer, is considered. Based on information forwarded by the preceding layer, each stage of the network is required to preserve a certain level of relevance with regards to a specific hidden variab…

We give the first provably efficient algorithm for learning a one hidden layer convolutional network with respect to a general class of (potentially overlapping) patches. Additionally, our algorithm requires only mild conditions on the underlying distribution. We prove that our framework captures commonly used schemes …

New model for multi-layer categorical data improves latent class analysis.

New model predicts radiative properties of nanoparticle layers with high accuracy and uncertainty.

Two-layer ReLU networks can overfit without harm, study finds.

In this work, we introduce a deep-structured conditional random field (DS-CRF) model for the purpose of state-based object silhouette tracking. The proposed DS-CRF model consists of a series of state layers, where each state layer spatially characterizes the object silhouette at a particular point in time. The interact…

Optimal scaling found to depend on operator norm across large models and datasets.

Paper perfect clusters sparse, diverse multilayer networks.

Paper uses SSC for identifying layers with identical community structures in DIMPLE networks.

New conditions ensure deep neural networks can approximate any function on non-Euclidean spaces.

Layer-wise preconditioning methods improve neural network optimization and feature learning.

DeepDrum generates drum rhythms under musical constraints.

Traditionally, when generative models of data are developed via deep architectures, greedy layer-wise pre-training is employed. In a well-trained model, the lower layer of the architecture models the data distribution conditional upon the hidden variables, while the higher layers model the hidden distribution prior. Bu…

Wide neural networks with weight decay exhibit neural collapse.

New methods estimate mixed memberships in multi-layer networks.

Paper presents an ADMM-based approach to efficiently integrate quadratic programming layers into neural networks.