A deep convolutional fuzzy system (DCFS) on a high-dimensional input space is a multi-layer connection of many low-dimensional fuzzy systems, where the input variables to the low-dimensional fuzzy systems are selected through a moving window across the input spaces of the layers. To design the DCFS based on input-outpu…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “layer design”
Proposes continuous convolution layers for flexible feature map resizing.
problem Fixed stride limitations in discrete convolution layers.
method Introduces Continuous Convolution (CC) layers that use learned continuous functions.
result Dynamic and consistent resizing of feature maps at any scale, non-integer and axis-dependent.
Designs a neural network to reduce training cost by mapping to higher dimensions.
problem High training cost in neural networks.
method Maps feature vectors to higher dimensional space, designs weight matrices to reduce cost, uses convex constraints.
result Reduces training cost as the number of layers increases, without cross-validation.
Study on shortcuts in deep networks, revealing their layer-wise distribution and impact.
problem Understudied impact of shortcuts on feature representations in deep networks.
method Layer-wise localization through counterfactual training on clean and skewed datasets.
result Shortcuts are distributed throughout the network, not localized in specific layers.
Optimizes CNN architectures by analyzing receptive fields without training.
problem CNNs often have too many layers, making them resource-intensive and less effective.
method Layer-wise analysis of receptive fields to identify unproductive layers.
result Identifies and removes unproductive layers, improving CNN performance and efficiency.
Multi-layer optical film has been found to afford important applications in optical communication, optical absorbers, optical filters, etc. Different algorithms of multi-layer optical film design has been developed, as simplex method, colony algorithm, genetic algorithm. These algorithms rapidly promote the design and …
Automatically designs normalization and activation layers together.
problem Designing normalization and activation layers separately.
method Unified tensor-to-tensor computation graph, low-level mathematical functions, rejection protocols, multi-objective evolution.
result Discovery of EvoNorms with novel structures.
Develops a two-layer model to design mortgage assistance products.
problem Designing effective mortgage assistance products to improve household resilience.
method Two-layer approach: simulation and optimization.
result Shows how the approach can design and evaluate mortgage assistance products.
A new method for verifying deep learning architectures on FPGAs is proposed.
problem Design-time verification of deep learning architectures on FPGAs.
method 2-Level 3-Way (2L-3W) hardware-software co-verification methodology.
result Layer-by-layer similarity scores of 99% accuracy for successful mappings.
New layer improves neural networks' flexibility and speed.
problem Designing efficient neural networks for real-time learning.
method Sparse Associative Memories for incremental learning.
result Improved Neural Networks with faster training and real-time support.
A new framework for efficient sequence maps using Bayesian filtering and covariance.
problem Designing efficient recurrent sequence maps from explicit memory assumptions.
method Design-model framework, exact Bayesian filtering, query-dependent readout, linear-Gaussian instantiation.
result Improved robustness and retrieval performance across various benchmarks.
Designs an MLP from LDA for multi-Gaussian class classification.
problem Classifying inputs with multiple Gaussian distributions.
method Interprets MLP as generalized LDA, using LDAs for half-space partitioning, neurons for subspace isolation, and merging for class-wise representation.
result Automatic feedforward design for MLP architecture and weights.
This research explores the dynamics of linearised neural nets, revealing distinct learning phases and layer growth rates.
problem Understanding the fundamental mechanics of neural nets and their learning dynamics.
method Derivation of properties of learning dynamics in general multi-layer linear neural nets, including orthogonal networks.
result Linear multi-layer neural nets exhibit distinct phases of learning with different layer growth rates, and nonlinearity affects these dynamics.
Introduces Causal Energy Minimization to understand Transformer layers.
problem Empirical parameterization of Transformer blocks remains largely unexplored.
method Causal Energy Minimization framework that recasts Transformer layers as optimization steps on conditional energy functions.
result Identifies design space for Transformer layers including weight sharing and energy-based interpretations.
DeepAM optimizes deep neural networks for image super-resolution.
problem Image super-resolution using deep neural networks.
method L-layer analysis dictionary model with IPAD and CAD.
result DeepAM outperforms deep neural networks with back-propagation.
Automates graph convolutional network design for semi-supervised node classification.
problem Designing optimal graph convolutional network architectures for semi-supervised node classification.
method An automatic process to define a problem-specific architecture based on graph structure.
result The proposed method outperforms existing methods in classification performance and network compactness.
Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks. In this paper, we illustrate an advanced information theoretic methodology to understand the dynamics of learning and the design of autoencoders, a special type of deep lea…
We characterize convolutional neural networks with respect to the relative amount of features per layer. Using a skew normal distribution as a parametrized framework, we investigate the common assumption of monotonously increasing feature-counts with higher layers of architecture designs. Our evaluation on models with …
FPGA-based multi-layer equalizer adapts to changing channels.
problem Real-time adaptation to time-varying channel impairments.
method Multi-layer machine learning on FPGA with on-chip gradient backpropagation training.
result Real-time adaptation to changing channel conditions achieved.
We develop an algorithm for systematic design of a large artificial neural network using a progression property. We find that some non-linear functions, such as the rectifier linear unit and its derivatives, hold the property. The systematic design addresses the choice of network size and regularization of parameters. …
Enhanced ELM reduces randomness in neural network training.
problem Challenges in ELM architecture design and sensitivity to random weight initialization.
method Introduces Effective Non-Random ELM (ENR-ELM) incorporating signal processing concepts.
result ENR-ELM simplifies architecture design and eliminates random weight selection.
Deep Regression Ensemblesstat.ML
DRE combines DNN with random feature regression for efficient neural network design.
problem Designing and training deep neural networks (DNN) efficiently and effectively.
method DRE architecture with two-layer neural networks, randomly drawn input and output weights trained with linear ridge regression.
result DRE outperforms state-of-the-art DNN in many data sets with lower computational cost.
New AMP algorithms improve multi-layer signal reconstruction.
problem Reconstructing signals and hidden variables from multi-layer networks with rotationally invariant weights.
method Developed multi-layer rotationally invariant generalized AMP (ML-RI-GAMP) algorithms and state evolution recursion.
result ML-RI-GAMP outperforms existing methods in terms of lower complexity and similar performance.
New CNN layer selects important channels to improve model capacity.
problem Improving model capacity under resource constraints.
method Selective allocation of channels in convolutional layers.
result New layer allows new optima that generalize better.
Pruning the Pilots: Deep Learning-Based Pilot Design and Channel Estimation for MIMO-OFDM Systemscs.IT
Paper proposes neural network for efficient MIMO channel estimation and pilot reduction.
problem High overhead from pilot transmission in wideband MIMO systems.
method Neural network architecture for frequency-aware pilot design and channel estimation, with pruning technique.
result Neural network outperforms linear minimum mean square error (LMMSE) estimation.
An Infinite Parade of Giraffes: Expressive Augmentation and Complexity Layers for Cartoon Drawingcs.GR
In this paper, we explore creative image generation constrained by small data. To partially automate the creation of cartoon sketches consistent with a specific designer's style, where acquiring a very large original image set is impossible or cost prohibitive, we exploit domain specific knowledge for a huge reduction …
A Layered Architecture for Active Perception: Image Classification using Deep Reinforcement Learningcs.LG
A robot learns to classify images with limited perception using a layered reinforcement learning approach.
problem Image classification for robots with partial perception.
method Three-layer architecture using deep reinforcement learning, including meta-layer, action-layer, and classification-layer.
result The method achieves high accuracy on the MNIST dataset and provides explainability of the agent's decision-making process.
Kernel and neural embeddings improve optimization and generalization in deep networks.
problem Improving optimization and generalization in deep neural networks.
method Investigated three kernel representations and their neural network approximations, comparing their optimization and generalization properties.
result Kernel and neural embeddings enhance both optimization and generalization in deep networks.
In this work we compute lower Lipschitz bounds of pooling operators for as well as pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm tha…
Relational Networks (RN) as introduced by Santoro et al. (2017) have demonstrated strong relational reasoning capabilities with a rather shallow architecture. Its single-layer design, however, only considers pairs of information objects, making it unsuitable for problems requiring reasoning across a higher number of fa…
Layer normalization placement affects training stability and warm-up stage necessity.
problem Training instability and the necessity of a learning rate warm-up stage in Transformers.
method Theoretical analysis and mean field theory to prove gradient behavior at initialization.
result Removing the warm-up stage for Pre-LN Transformers can achieve comparable results with less time and tuning.
Improved GAN performance with a novel local attention mechanism.
problem Enhancing the performance of Generative Adversarial Networks (GANs).
method Introducing a two-dimensional local attention mechanism that preserves geometry and locality, and using information flow graphs for design.
result Significant improvements in FID and Inception scores, from 18.65 to 15.94 on ImageNet.
Strength of weak layers in cascading failures on multiplex networks: case of the international trade networkphysics.soc-ph
Many real-world complex systems across natural, social, and economical domains consist of manifold layers to form multiplex networks. The multiple network layers give rise to nonlinear effect for the emergent dynamics of systems. Especially, weak layers that can potentially play significant role in amplifying the vulne…
The performance of an organic photovoltaic device is intricately connected to its active layer morphology. This connection between the active layer and device performance is very expensive to evaluate, either experimentally or computationally. Hence, designing morphologies to achieve higher performances is non-trivial …
Law explains how deep networks separate data for classification.
problem Black-box nature of deep learning limits architecture design and interpretation.
method Studied how deep neural networks process data in intermediate layers.
result Law of geometric data separation emerges in various architectures and datasets.
A new tensor-based layer reduces neural network dimensions without losing important features.
problem Reducing dimensionality in tensor-structured feature data for deep neural networks.
method TensorProjection layer that projects input tensors into output tensors with reduced dimensions through mode-wise projections.
result The TensorProjection layer outperforms traditional downsampling methods in tasks like medical image classification and segmentation.
Two graph auto-encoders decouple feature propagation from graph convolution layers.
problem Designing efficient graph auto-encoders with fixed receptive fields.
method L-GAE and L-VGAE using linear matrix computation before auto-encoder input.
result Comparable performance to VGAEs with smaller, simpler networks.
We describe Bayesian Layers, a module designed for fast experimentation with neural network uncertainty. It extends neural network libraries with drop-in replacements for common layers. This enables composition via a unified abstraction over deterministic and stochastic functions and allows for scalability via the unde…
Hybrid deep architectures with reasoning layers show promising convergence and generalization properties.
problem Understanding the theoretical foundations of hybrid deep architectures with reasoning layers.
method Analyzing the interplay between algorithm layers and neural components in deep architectures.
result Properties of algorithm layers are closely related to the approximation and generalization abilities of end-to-end models.
This work optimizes neural network bit-width and layer-width for efficiency.
problem Efficient optimization of deep neural networks for reduced size and computational demands.
method Cluster-based tree-structured Parzen estimator for surrogate modeling, Hessian-based pruning for parameter reduction.
result 20% decrease in model size with 12x reduction in search time compared to existing methods.
Many activation functions have been proposed in the past, but selecting an adequate one requires trial and error. We propose a new methodology of designing activation functions within a neural network at each layer. We call this technique an "activation ensemble" because it allows the use of multiple activation functio…
The study shows removing fully connected output layers improves efficiency without sacrificing performance.
problem Large number of parameters in fully connected layers for high-category datasets.
method Examined architectures replacing fully connected output layers with fixed layers and compared performance.
result Fixed classifiers offer no additional benefit over removing the output layer and its parameters.
Optimizes crypto-oriented neural architectures for faster secure inference.
problem Privacy conflicts between model users and providers in neural network applications.
method Proposes a novel Partial Activation layer to optimize the initial design of crypto-oriented neural architectures.
result Significant improvement in the efficiency of secure inference on common evaluation metrics.
This paper extends the earlier work on an oscillating error correction technique. Specifically, it extends the design to include further corrections, by adding new layers to the classifier through a branching method. This technique is still consistent with earlier work and also neural networks in general. With this ext…
Layered graphical models improve discriminative learning efficiency.
problem Improving discriminative learning efficiency in graphical models.
method Designing layered graphical models (LGMs) in analogy to neural networks, using tensorized truncated variational inference and backpropagation.
result LGMs achieve competitive results in image classification, comparable to neural networks.
Understanding the representational power of Restricted Boltzmann Machines (RBMs) with multiple layers is an ill-understood problem and is an area of active research. Motivated from the approach of \emph{Inherent Structure formalism} (Stillinger & Weber, 1982), extensively used in analysing Spin Glasses, we propose a no…
When designing fully-convolutional neural network, there is a trade-off between receptive field size, number of parameters and spatial resolution of features in deeper layers of the network. In this work we present a novel network design based on combination of many convolutional and recurrent layers that solves these …
Decentralised fund framework allocates capital via tokenised vaults.
problem Traditional asset management's inefficiencies and centralisation.
method Permissionless, multi-strategy capital allocation through on-chain vaults.
result Self-regulating, cooperative optimisation across financial domains.