A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
learn2mix trains neural nets faster by adjusting class proportions dynamically.
FIRE PBT improves neural network training by focusing on long-term performance.
We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix multiplications and convolutions. We introduce new sampling techniques, study their theoretical properties, and prove that they provide the same…
The time complexity of support vector machines (SVMs) prohibits training on huge data sets with millions of data points. Recently, multilevel approaches to train SVMs have been developed to allow for time-efficient training on huge data sets. While regular SVMs perform the entire training in one -- time consuming -- op…
Faster training of neural ODEs using Gauß-Legendre quadrature.
Exploiting sparsity enables hardware systems to run neural networks faster and more energy-efficiently. However, most prior sparsity-centric optimization techniques only accelerate the forward pass of neural networks and usually require an even longer training process with iterative pruning and retraining. We observe t…
Deep learning methods are useful for high-dimensional data and are becoming widely used in many areas of software engineering. Deep learners utilizes extensive computational power and can take a long time to train-- making it difficult to widely validate and repeat and improve their results. Further, they are not the b…
We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the trainin…
We find faster-converging sub-networks that significantly reduce adversarial training time.
Unbalanced GANs stabilize GAN training by pre-training the generator with VAE.
Deeper neural networks learn lower frequency functions faster, according to a new principle.
Paper adapts diffusion sampler training for faster convergence and better sampling.
Framework improves gradient estimation for faster training convergence.
FROST speeds up and stabilizes one-shot semi-supervised learning.
Unified framework for training SNNs using EP, faster convergence.
We design a non-convex second-order optimization algorithm that is guaranteed to return an approximate local minimum in time which scales linearly in the underlying dimension and the number of training examples. The time complexity of our algorithm to find an approximate local minimum is even faster than that of gradie…
Batch Normalization (BatchNorm) is a widely adopted technique that enables faster and more stable training of deep neural networks (DNNs). Despite its pervasiveness, the exact reasons for BatchNorm's effectiveness are still poorly understood. The popular belief is that this effectiveness stems from controlling the chan…
Stagewise training strategy is widely used for learning neural networks, which runs a stochastic algorithm (e.g., SGD) starting with a relatively large step size (aka learning rate) and geometrically decreasing the step size after a number of iterations. It has been observed that the stagewise SGD has much faster conve…
Paper proposes a faster RAE with sequence-aware encoding.
Recently mean field theory has been successfully used to analyze properties of wide, random neural networks. It gave rise to a prescriptive theory for initializing feed-forward neural networks with orthogonal weights, which ensures that both the forward propagated activations and the backpropagated gradients are near $…
MACER trains robust models without adversarial training, faster and more effective.
Reservoir Memory Machines solve benchmark tasks faster than Neural Turing Machines.
Single neural network predicts ImageNet model parameters for faster training.
A new faster neural network training method using backprojection.
A new training method improves MLIPs for faster, lighter simulations.
Proposes a method to train neural networks that solve differential equations faster.
There is a general trend towards solving problems suited to deep learning with more complex deep learning architectures trained on larger training sets. This requires longer compute times and greater data parallelization or model parallelization. Both data and model parallelism have been historically faster in paramete…
Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…
New method trains neural ODEs faster with fewer layers.
Improved KL divergence estimators for normalizing flows lead to faster convergence and better approximations.
Faster convergence in federated learning for non-convex problems.
Paper explores EEG-based speech recognition using transformers, showing faster training and better performance for smaller vocabularies.
Deep learning has shown that learned functions can dramatically outperform hand-designed functions on perceptual tasks. Analogously, this suggests that learned optimizers may similarly outperform current hand-designed optimizers, especially for specific problems. However, learned optimizers are notoriously difficult to…
ImageNet pre-training has been regarded as essential for training accurate object detectors for a long time. Recently, it has been shown that object detectors trained from randomly initialized weights can be on par with those fine-tuned from ImageNet pre-trained models. However, the effects of pre-training and the diff…
Multi-headed ensembles boost model performance with faster training.
Training modern deep learning models requires large amounts of computation, often provided by GPUs. Scaling computation from one GPU to many can enable much faster training and research progress but entails two complications. First, the training library must support inter-GPU communication. Depending on the particular …
Compressive image recovery is a challenging problem that requires fast and accurate algorithms. Recently, neural networks have been applied to this problem with promising results. By exploiting massively parallel GPU processing architectures and oodles of training data, they can run orders of magnitude faster than exis…
ZORB speeds up neural network training without sacrificing accuracy.
SliceOut speeds up deep learning training without sacrificing accuracy.
Paper presents faster, robust adversarial training methods.
This work presents a novel objective function for the unsupervised training of neural network sentence encoders. It exploits signals from paragraph-level discourse coherence to train these models to understand text. Our objective is purely discriminative, allowing us to train models many times faster than was possible …
Transformer learns to estimate negative binomial parameters efficiently.
This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration. Selective-Backprop uses the output of a training example's forward pass to decide whether to use that example to compute gradients and update para…
A new method for graph neural networks speeds up inference and training.
A novel neural network training method reduces gradient variance for faster and better reinforcement learning.
We propose and evaluate new techniques for compressing and speeding up dense matrix multiplications as found in the fully connected and recurrent layers of neural networks for embedded large vocabulary continuous speech recognition (LVCSR). For compression, we introduce and study a trace norm regularization technique f…
New method speeds up lifelong learning of complex tasks.
Deep networks prioritize easier examples over harder ones, leading to faster training.