A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Neural nets trained with linear discriminant initialization converge faster and more accurately.
This work optimizes reservoir computing models by linking recurrence and non-linear dynamics.
Basic binary relations such as equality and inequality are fundamental to relational data structures. Neural networks should learn such relations and generalise to new unseen data. We show in this study, however, that this generalisation fails with standard feed-forward networks on binary vectors. Even when trained wit…
Adapting robust statistics to neural networks, researchers found neural networks can be more robust with certain loss functions.
Efficient neural network optimization reduces costs and improves model performance.
Transformer networks have lead to important progress in language modeling and machine translation. These models include two consecutive modules, a feed-forward layer and a self-attention layer. The latter allows the network to capture long term dependencies and are often regarded as the key ingredient in the success of…
We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.
New study shows neural networks need many samples for training.
We propose a feed-forward inference method applicable to belief and neural networks. In a belief network, the method estimates an approximate factorized posterior of all hidden units given the input. In neural networks the method propagates uncertainty of the input through all the layers. In neural networks with inject…
New insights into how encoder-decoder networks generate attention matrices.
Training deep recurrent neural network (RNN) architectures is complicated due to the increased network complexity. This disrupts the learning of higher order abstracts using deep RNN. In case of feed-forward networks training deep structures is simple and faster while learning long-term temporal information is not poss…
General lower bounds on neural network approximation in L^p norm.
Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically, we prove stable recurrent neural networks are well approximated by feed-forward …
Deep network learns Obstacle Tower challenge without human demonstrations.
When a feed-forward neural network (FNN) is trained for source ranging in an ocean waveguide, it is difficult evaluating the range accuracy of the FNN on unlabeled test data. A fitting-based early stopping (FEAST) method is introduced to evaluate the range error of the FNN on test data where the distance of source is u…
Improved bounds on neural network regions using activation histograms.
We design a self size-estimating feed-forward network (SSFN) using a joint optimization approach for estimation of number of layers, number of nodes and learning of weight matrices. The learning algorithm has a low computational complexity, preferably within few minutes using a laptop. In addition the algorithm has a l…
In this paper, we model the impact of oil price volatility on Tehranstock and industry indices in two periods of international sanctions and post-sanction. To analyse the purpose of study, we use Feed-forward neural net-works. The period of study is from 2008 to 2018 that is split in two periods during international en…
Proposes a neural network for contextual regression.
DeepPhaseCut uses neural networks to improve Fourier phase retrieval.
Proposes a new CG interpretation of neural networks for better theoretical analysis.
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
The paper develops a neural network-based method for detecting change points in large-scale time-evolving data.
Adaptive Group Lasso selects important features in neural networks.
It has been argued in the past that high-dimensional neural networks do not exhibit local minima capable of trapping an optimisation algorithm. However, the relationship between loss surface modality and the neural architecture parameters, such as the number of hidden neurons per layer and the number of hidden layers, …
Recurrent and convolutional neural networks are the most common architectures used for time series forecasting in deep learning literature. These networks use parameter sharing by repeating a set of fixed architectures with fixed parameters over time or space. The result is that the overall architecture is time-invaria…
Two of the most popular modelling paradigms in computer vision are feed-forward neural networks (FFNs) and probabilistic graphical models (GMs). Various connections between the two have been studied in recent works, such as e.g. expressing mean-field based inference in a GM as an FFN. This paper establishes a new conne…
Stable processes emerge as limits of deep neural networks with symmetric stable distributions.
We introduce Dynamic Deep Neural Networks (D2NN), a new type of feed-forward deep neural network that allows selective execution. Given an input, only a subset of D2NN neurons are executed, and the particular subset is determined by the D2NN itself. By pruning unnecessary computation depending on input, D2NNs provide a…
The paper sets information-theoretic lower bounds for neural networks' parameter recovery and excess risk.
Edge devices adapt pre-trained models to local data without backpropagation.
New methods for uncertainty in neural networks with leaky ReLU activations.
Learning representations that accurately capture long-range dependencies in sequential inputs -- including text, audio, and genomic data -- is a key problem in deep learning. Feed-forward convolutional models capture only feature interactions within finite receptive fields while recurrent architectures can be slow and …
The universal approximation property of various machine learning models is currently only understood on a case-by-case basis, limiting the rapid development of new theoretically justified neural network architectures and blurring our understanding of our current models' potential. This paper works towards overcoming th…
New method uses neural networks to forecast spatial-temporal data.
A new algorithm speeds up neural network derivative calculations.
Develops hyperparameter transfer methods for Dense Associative Memories.
We present experiments demonstrating that some other form of capacity control, different from network size, plays a central role in learning multilayer feed-forward networks. We argue, partially through analogy to matrix factorization, that this is an inductive bias that can help shed light on deep learning.
NN2Poly converts deep neural networks into polynomial models for better understanding.
New bound for neural nets on non-iid data.
New method uses MMAF-guided learning for spatio-temporal probabilistic forecasts.
The study compares feed-forward and attention layers in language models.
A reliable controller is critical and essential for the execution of safe and smooth maneuvers of an autonomous vehicle.The controller must be robust to external disturbances, such as road surface, weather, and wind conditions, and so on.It also needs to deal with the internal parametric variations of vehicle sub-syste…
This paper removes the finite variance assumption for deep convolutional neural networks.
MTL improves multi-dimensional regression in luminescence sensing.
Algorithm optimizes cascaded functions with known structure.
Field theory explains optimal scaling in ResNets for signal propagation.
Neural nets solve braid untangling up to length 20.