arXiv research

HitNet uses a Hit-or-Miss layer to enhance feature interpretability in capsule networks.

3-layer NTK models generalize better than 2-layer models, especially with large input dimensions.

This paper presents two unsupervised learning layers (UL layers) for label-free video analysis: one for fully connected layers, and the other for convolutional ones. The proposed UL layers can play two roles: they can be the cost function layer for providing global training signal; meanwhile they can be added to any re…

Two-layer CNNs can overfit well if initialized correctly.

To infer a multilayer representation of high-dimensional count vectors, we propose the Poisson gamma belief network (PGBN) that factorizes each of its layers into the product of a connection weight matrix and the nonnegative real hidden units of the next layer. The PGBN's hidden layers are jointly trained with an upwar…

Network Implosion reduces ResNet layers without accuracy loss.

Proposes a fixed smooth convolutional layer to reduce checkerboard artifacts in CNNs.

Self-attention can replace convolutional layers in vision tasks.

Three-layer networks learn more complex features than two-layer networks.

To infer multilayer deep representations of high-dimensional discrete and nonnegative real vectors, we propose an augmentable gamma belief network (GBN) that factorizes each of its hidden layers into the product of a sparse connection weight matrix and the nonnegative real hidden units of the next layer. The GBN's hidd…

Study shows LLMs can remove half of layers without significant performance drop.

We improve neural network explainability by bypassing batch normalization.

We address the problem of learning hierarchical deep neural network policies for reinforcement learning. In contrast to methods that explicitly restrict or cripple lower layers of a hierarchy to force them to use higher-level modulating signals, each layer in our framework is trained to directly solve the task, but acq…

A robot learns to classify images with limited perception using a layered reinforcement learning approach.

Proposes continuous convolution layers for flexible feature map resizing.

One-layer transformers can't solve induction heads task efficiently.

Unified framework LPCD optimizes quantization of complex submodules.

AIBC uses AI and blockchain for efficient resource sharing.

A new neural network model MDRBM improves noise-robustness in classification.

Many real-world complex systems across natural, social, and economical domains consist of manifold layers to form multiplex networks. The multiple network layers give rise to nonlinear effect for the emergent dynamics of systems. Especially, weak layers that can potentially play significant role in amplifying the vulne…

The study shows removing fully connected output layers improves efficiency without sacrificing performance.

Our work presents extensive empirical evidence that layer rotation, i.e. the evolution across training of the cosine distance between each layer's weight vector and its initialization, constitutes an impressively consistent indicator of generalization performance. In particular, larger cosine distances between final an…

Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.

Introduces Causal Energy Minimization to understand Transformer layers.

A deep convolutional fuzzy system (DCFS) on a high-dimensional input space is a multi-layer connection of many low-dimensional fuzzy systems, where the input variables to the low-dimensional fuzzy systems are selected through a moving window across the input spaces of the layers. To design the DCFS based on input-outpu…

The rapid development of high-throughput technologies has enabled the generation of data from biological or disease processes that span multiple layers, like genomic, proteomic or metabolomic data, and further pertain to multiple sources, like disease subtypes or experimental conditions. In this work, we propose a gene…

PLN-Nets with two linear layers and parallel LN achieve universal approximation.

Smectic liquid crystals are materials formed by stacking deformable, fluid layers. Though smectics prefer to have flat, uniformly-spaced layers, boundary conditions can impose curvature on the layers. Since the layer spacing and curvature are intertwined, the problem of finding minimal configurations for the layers bec…

Improved recommendation systems using multi-layer embeddings reduce model size while maintaining accuracy.

Inferencing with network data necessitates the mapping of its nodes into a vector space, where the relationships are preserved. However, with multi-layered networks, where multiple types of relationships exist for the same set of nodes, it is crucial to exploit the information shared between layers, in addition to the …

New approach replaces BN layers with shifted-ReLU for embedded systems.

A method to analyze neural network performance by measuring layer saturation.

A new tensor-based layer reduces neural network dimensions without losing important features.

Proposes LML layer for multi-label predictions with k labels.

Graph embedding is an important approach for graph analysis tasks such as node classification and link prediction. The goal of graph embedding is to find a low dimensional representation of graph nodes that preserves the graph information. Recent methods like Graph Convolutional Network (GCN) try to consider node attri…

Understanding the representational power of Restricted Boltzmann Machines (RBMs) with multiple layers is an ill-understood problem and is an area of active research. Motivated from the approach of \emph{Inherent Structure formalism} (Stillinger & Weber, 1982), extensively used in analysing Spin Glasses, we propose a no…

Various forms of representations may arise in the many layers embedded in deep neural networks (DNNs). Of these, where can we find the most compact representation? We propose to use a pruning framework to answer this question: How compact can each layer be compressed, without losing performance? Most of the existing DN…

A new method for verifying deep learning architectures on FPGAs is proposed.

Given a complete non-compact surface embedded in R^3, we consider the Dirichlet Laplacian in a layer of constant width about the surface. Using an intrinsic approach to the layer geometry, we generalise the spectral results of an original paper by Duclos et al. to the situation when the surface does not possess poles. …

New model for multi-layer categorical data improves latent class analysis.

New loss functions reveal layer roles in deep neural networks.

A new histogram layer improves texture analysis performance.

Two spectral clustering methods for multi-layer networks are analyzed and compared.

Study reveals layers in deep networks can be robust or critical, affecting model performance.

This paper extends stability and generalization analysis of GD for multi-layer NNs.

A new unpooling layer enhances graph generation in molecular models.

The paper extends mean field results to three-layer neural networks using SGD.

This paper analyzes how normalization layers improve neural network training.