arXiv research

Three-layer networks learn more complex features than two-layer networks.

New proof shows neural networks can represent all multivariate functions.

Analytical solution found for a three-layer network with a specific activation function.

Three-layer neural networks learn hierarchical polynomial functions efficiently.

Global convergence proved for three-layer neural networks in mean field regime.

Deep ReLU networks can be simplified to a three-layer model.

The paper extends mean field results to three-layer neural networks using SGD.

The study analyzes a three-layer neural network's training dynamics using a functional-space mean-field theory.

Three-layer networks learn complex hierarchical polynomials of multiple nonlinear features.

This paper extends stability and generalization analysis of GD for multi-layer NNs.

Polynomial-time convex optimization for CNNs with ReLU activations.

Study analyzes error in neural network solving PDEs, providing convergence and error bounds.

Study clarifies Bayesian generalization error in CBM for 3-layered linear neural networks.

The paper sets limits on neural network sizes based on dataset shapes.

Global convergence of multilayer neural networks proven for any depth.

A new neural network model MDRBM improves noise-robustness in classification.

This informal technical report details the geometric illustration of decision boundaries for ReLU units in a three layer fully connected neural network. The network is designed and trained to predict pixel intensity from an (x, y) input location. The Geometric Illustration of Neural Networks (GINN) tool was built to vi…

Examines learning efficiency in neural networks and related models.

In this paper, we prove a conjecture published in 1989 and also partially address an open problem announced at the Conference on Learning Theory (COLT) 2015. With no unrealistic assumption, we first prove the following statements for the squared loss function of deep linear neural networks with any depth and any widths…

FluxLayer solves cross-chain liquidity fragmentation for better MEV capture.

The study explores the compressive power of Boolean threshold autoencoders, finding that seven layers are necessary but three are not.

This work explains neural collapse in shallow neural networks and its impact on generalization.

The fundamental learning theory behind neural networks remains largely open. What classes of functions can neural networks actually learn? Why doesn't the trained network overfit when it is overparameterized? In this work, we prove that overparameterized neural networks can learn some notable concept classes, including…

Adversarial attacks against machine learning models are a rather hefty obstacle to our increasing reliance on these models. Due to this, provably robust (certified) machine learning models are a major topic of interest. Lipschitz continuous models present a promising approach to solving this problem. By leveraging the …

3-layer NTK models generalize better than 2-layer models, especially with large input dimensions.

Deep learning can learn compositional functions more efficiently by breaking them into stages.

We analyzed optimism in linear and kernel regression models.

Sleep disorders are implicated in a growing number of health problems. In this paper, we present a signal-processing/machine learning approach to detecting arousals in the multi-channel polysomnographic recordings of the Physionet/CinC Challenge2018 dataset. Methods: Our network architecture consists of two components.…

The time it takes for a classifier to make an accurate prediction can be crucial in many behaviour recognition problems. For example, an autonomous vehicle should detect hazardous pedestrian behaviour early enough for it to take appropriate measures. In this context, we compare the switching linear dynamical system (SL…

Unified framework reveals regularization mechanism in deep ReLU networks via convex optimization.

Proves deep networks can learn hierarchical structures efficiently.

Simplifies deep learning scaling analysis without sacrificing accuracy.

PCBM improves neural network generalization by partially observing concepts.

Improved neural network training with ADMM for hardware compatibility.

New algorithm interpolates data with neural nets, independent of sample size.

Smart mobility management would be an important prerequisite for future fog computing systems. In this research, we propose a learning-based handover optimization for the Internet of Vehicles that would assist the smooth transition of device connections and offloaded tasks between fog nodes. To accomplish this, we make…

We consider a three-layer Sejnowski machine and show that features learnt via contrastive divergence have a dual representation as patterns in a dense associative memory of order P=4. The latter is known to be able to Hebbian-store an amount of patterns scaling as N^{P-1}, where N denotes the number of constituting bin…

How can neural networks such as ResNet efficiently learn CIFAR-10 with test accuracy more than 96%, while other methods, especially kernel methods, fall relatively behind? Can we more provide theoretical justifications for this gap? Recently, there is an influential line of work relating neural networks to kernels in t…

Paper analyzes SHB method for neural networks, proving stability, connectivity, and global convergence.

The paper demonstrates the use of the fully convolutional neural network for glioma segmentation on the BraTS 2019 dataset. Three-layers deep encoder-decoder architecture is used along with dense connection at encoder part to propagate the information from coarse layer to deep layers. This architecture is used to train…

We develop a mathematically rigorous framework for multilayer neural networks in the mean field regime. As the network's widths increase, the network's learning trajectory is shown to be well captured by a meaningful and dynamically nonlinear limit (the \textit{mean field} limit), which is characterized by a system of …

A new framework explains GNN predictions by simulating graph structure and feature changes.

Study improves chiral photonic metasurface design using neural networks and genetic algorithms.

System detects financial opportunities in tweets with high precision.

We develop a corrective mechanism for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second group approximates the error in approximation produced by the first group and corrects it, the third gro…

To address potential gaps noted in patient monitoring in the hospital, a novel patient behavior detection system using mmWave radar and deep convolution neural network (CNN), which supports the simultaneous recognition of multiple patients' behaviors in real-time, is proposed. In this study, we use an mmWave radar to t…

This paper proposes a new approach to Transformers by integrating hierarchical associative memory with MetaFormers.

Neural network solves inverse problem in multiscale mechanics.