A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper proposes continuous residual layers for graph neural networks.
Residual Continual Learning prevents forgetting in sequential tasks.
Study Transformer layers under cross-entropy training using mean field control.
A beta function for double layers is defined and analyzed.
The Brylinski beta function is extended for coaxial layers on submanifolds.
Deep residual networks implicitly converge to neural ODEs.
Investigates the relationship between ResNets and Neural ODEs, quantifying their closeness and providing training methods.
Deep residual networks can approximate any continuous function using control theory.
A new model improves CT image quality from low-dose scans.
Neural ODEs and i-ResNet are recently proposed methods for enforcing invertibility of residual neural models. Having a generic technique for constructing invertible models can open new avenues for advances in learning systems, but so far the question of whether Neural ODEs and i-ResNets can model any continuous inverti…
Following the recent work on capacity allocation, we formulate the conjecture that the shattering problem in deep neural networks can only be avoided if the capacity propagation through layers has a non-degenerate continuous limit when the number of layers tends to infinity. This allows us to study a number of commonly…
Study shows how deep residual networks can be analyzed as shallow network ensembles for optimization.
DQNs can approximate optimal Q-functions with high accuracy on compact sets.
To have a superior generalization, a deep learning neural network often involves a large size of training sample. With increase of hidden layers in order to increase learning ability, neural network has potential degradation in accuracy. Both could seriously limit applicability of deep learning in some domains particul…
Algorithm learns two-layer residual units using ReLU activations from samples.
In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …
Continuous formulation of machine learning models and algorithms.
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constan…
Speeds up deep neural networks training by 10x using GPU concurrency.
Inverse depth scaling found in LLMs due to similar layers averaging error.
We present the Video Ladder Network (VLN) for efficiently generating future video frames. VLN is a neural encoder-decoder model augmented at all layers by both recurrent and feedforward lateral connections. At each layer, these connections form a lateral recurrent residual block, where the feedforward connection repres…
DLC enhances distillation-based continual learning with lightweight plugins.
We introduce a new deep convolutional neural network, CrescendoNet, by stacking simple building blocks without residual connections. Each Crescendo block contains independent convolution paths with increased depths. The numbers of convolution layers and parameters are only increased linearly in Crescendo blocks. In exp…
ELF simplifies normalizing flows, making them more efficient and universal.
Subspace clustering aims to cluster unlabeled data that lies in a union of low-dimensional linear subspaces. Deep subspace clustering approaches based on auto-encoders have become very popular to solve subspace clustering problems. However, the training of current deep methods converges slowly, which is much less effic…
Residual networks' depth is mathematically equivalent to expanding an implicit ensemble size.
Enhanced image denoising with MWRDCNN using residual dense blocks.
Generalization bounds derived for neural ODEs and deep residual networks.
Image super-resolution is a challenging task and has attracted increasing attention in research and industrial communities. In this paper, we propose a novel end-to-end Attention-based DenseNet with Residual Deconvolution named as ADRD. In our ADRD, a weighted dense block, in which the current layer receives weighted f…
Residual Networks with convolutional layers are widely used in the field of machine learning. Since they effectively extract features from input data by stacking multiple layers, they can achieve high accuracy in many applications. However, the stacking of many layers raises their computation costs. To address this pro…
STRIC detects anomalies in time series by analyzing residual signals.
Signal models based on sparsity, low-rank and other properties have been exploited for image reconstruction from limited and corrupted data in medical imaging and other computational imaging applications. In particular, sparsifying transform models have shown promise in various applications, and offer numerous advantag…
A residual network (or ResNet) is a standard deep neural net architecture, with state-of-the-art performance across numerous applications. The main premise of ResNets is that they allow the training of each layer to focus on fitting just the residual of the previous layer's output and the target output. Thus, we should…
Study on residual Monge-Ampère mass of complex functions with directional Lipschitz continuity.
Due to a resource-constrained environment, network compression has become an important part of deep neural networks research. In this paper, we propose a new compression method, \textit{Inter-Layer Weight Prediction} (ILWP) and quantization method which quantize the predicted residuals between the weights in all convol…
Gradient oversmoothing and expansion hinder deep GNN training, solved with normalization.
SPACE algorithm prevents forgetting in neural networks by partitioning learned knowledge.
RDL-Net improves speech enhancement with fewer parameters and better performance.
Residual Network (ResNet) is undoubtedly a milestone in deep learning. ResNet is equipped with shortcut connections between layers, and exhibits efficient training using simple first order algorithms. Despite of the great empirical success, the reason behind is far from being well understood. In this paper, we study a …
Simplifies residual flows to make flow-based modeling more practical.
Materials discovery is crucial for making scientific advances in many domains. Collections of data from experiments and first-principle computations have spurred interest in applying machine learning methods to create predictive models capable of mapping from composition and crystal structures to materials properties. …
Stacking improves deep neural network training efficiency.
An emerging design principle in deep learning is that each layer of a deep artificial neural network should be able to easily express the identity transformation. This idea not only motivated various normalization techniques, such as \emph{batch normalization}, but was also key to the immense success of \emph{residual …
New models explain residual and dilated dense neural networks using sparse coding.
RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.
Normalization layers are a staple in state-of-the-art deep neural network architectures. They are widely believed to stabilize training, enable higher learning rate, accelerate convergence and improve generalization, though the reason for their effectiveness is still an active research topic. In this work, we challenge…
Unified learning-rate scale for CNNs and ResNets, avoiding depth imbalance.
Highly distributed training of Deep Neural Networks (DNNs) on future compute platforms (offering 100 of TeraOps/s of computational capacity) is expected to be severely communication constrained. To overcome this limitation, new gradient compression techniques are needed that are computationally friendly, applicable to …