arXiv research

Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.

New method reconstructs significant parts of training data from neural networks.

It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural networks are trained efficiently using SGD and a variety of tricks that include different activation functions (e.g. ReLU), over-specification (i.e., train networks which are larger than needed), a…

Neural nets trained with linear discriminant initialization converge faster and more accurately.

We propose \emph{Taylorized training} as an initiative towards better understanding neural network training at finite width. Taylorized training involves training the $k$-th order Taylor expansion of the neural network at initialization, and is a principled extension of linearized training---a recently proposed theory …

SMGD trains low-bit neural networks with memory constraints.

ReLU neural networks define piecewise linear functions of their inputs. However, initializing and training a neural network is very different from fitting a linear spline. In this paper, we expand empirically upon previous theoretical work to demonstrate features of trained neural networks. Standard network initializat…

New study shows neural networks need many samples for training.

The paper analyzes a neural network two-sample test using kernel analysis.

Multirate training speeds up neural network fine-tuning.

Optical co-processor speeds up neural network training.

This work theoretically investigates the performance of a composite neural network. A composite neural network is a rooted directed acyclic graph combining a set of pre-trained and non-instantiated neural network models, where a pre-trained neural network model is well-crafted for a specific task and targeted to approx…

Parallel neural network training yields better long-term prediction accuracy.

Un-trained neural networks outperform trained methods in MRI reconstruction.

Proposes dynamic channel pruning during neural network training.

This paper analyzes how normalization layers improve neural network training.

We present DANTE, a novel method for training neural networks using the alternating minimization principle. DANTE provides an alternate perspective to traditional gradient-based backpropagation techniques commonly used to train deep networks. It utilizes an adaptation of quasi-convexity to cast training a neural networ…

Paper benchmarks quantum neural networks against classical ones for binary classification tasks.

Study the effects of data parallelism and sparsity on neural network training.

learn2mix trains neural nets faster by adjusting class proportions dynamically.

Two preprocessing techniques reduce neural network training cost.

Quadratic models explain neural network behavior during training.

Training shapes the geometry of neural network feature maps, revealing local area magnification.

Alternative neural network training using monotone variational inequality.

SingleProp speeds up robust neural network training with minimal certification.

The neural tangent kernel equivalence theorem fails in practice.

BayesFlow trains neural networks for fast Bayesian inference.

A new faster neural network training method using backprojection.

Langevin algorithms improve training of very deep neural networks, especially for image classification.

This paper develops a method to train compact neural networks with reduced memory and computational costs.

Securely trains neural networks remotely with deep learning's flaws.

Survey on statistical theories of neural networks, focusing on approximation, training dynamics, and generative models.

ZORB speeds up neural network training without sacrificing accuracy.

Research on neural networks has gained significant momentum over the past few years. Because training is a resource-intensive process and training data cannot always be made available to everyone, there has been a trend to reuse pre-trained neural networks. As such, neural networks themselves have become research data.…

In domains such as health care and finance, shortage of labeled data and computational resources is a critical issue while developing machine learning algorithms. To address the issue of labeled data scarcity in training and deployment of neural network-based systems, we propose a new technique to train deep neural net…

Deep neural networks have shown remarkable performance across a wide range of vision-based tasks, particularly due to the availability of large-scale datasets for training and better architectures. However, data seen in the real world are often affected by distortions that not accounted for by the training datasets. In…

The use of artificial neural networks as models of chaotic dynamics has been rapidly expanding. Still, a theoretical understanding of how neural networks learn chaos is lacking. Here, we employ a geometric perspective to show that neural networks can efficiently model chaotic dynamics by becoming structurally chaotic t…

Speeds up deep neural networks training by 10x using GPU concurrency.

The paper trains neural networks with robustness guarantees using semidefinite constraints.

Study shows neural networks trained with GD converge to Gaussian processes with polynomial decay.

Neural nets learn simple distributions first, then more complex ones.

GOLS-I automatically determines learning rates for various neural network training algorithms.

The evolution of a deep neural network trained by the gradient descent can be described by its neural tangent kernel (NTK) as introduced in [20], where it was proven that in the infinite width limit the NTK converges to an explicit limiting kernel and it stays constant during training. The NTK was also implicit in some…

Langevin algorithms enhance training of deep neural networks for stochastic control problems.

A hybrid method improves Convolutional Neural Networks training.

Approach to verify neural network training integrity.

Training-free GNNs use labels as features to improve node classification.

Wide neural networks with weight decay exhibit neural collapse.