A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Two retraining techniques outperform fine-tuning in neural network pruning.
Data augmentation is a widely used technique in many machine learning tasks, such as image classification, to virtually enlarge the training dataset size and avoid overfitting. Traditional data augmentation techniques for image classification tasks create new samples from the original training data by, for example, fli…
Paper proposes a new technique to compress CNNs while maintaining accuracy.
New findings suggest adversarial training does not flatten loss landscapes, challenging current intuition.
Memory is increasingly often the bottleneck when training neural network models. Despite this, techniques to lower the overall memory requirements of training have been less widely studied compared to the extensive literature on reducing the memory requirements of inference. In this paper we study a fundamental questio…
Adversarial training was recently shown to be competitive against supervised learning methods on computer vision tasks, however, studies have mainly been confined to generative tasks such as image synthesis. In this paper, we apply adversarial training techniques to the discriminative task of learning a steganographic …
Improved GAN training speed and quality with FastGAN.
New technique trains deep neural networks without normalization or minibatch statistics.
One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to in…
Paper trains models to resist string transformations.
A simple regularization technique speeds up training of Neural ODEs.
TFCheck detects training issues in ML programs using TensorFlow.
Enhanced dropout technique improves training speed and generalization.
In this paper, we explore techniques centered around periodic sampling of model weights that provide convergence improvements on gradient update methods (vanilla \acs{SGD}, Momentum, Adam) for a variety of vision problems (classification, detection, segmentation). Importantly, our algorithms provide better, faster and …
Detects training data usage with radioactive data technique.
Meta-learning techniques like MAML and Reptile fail to generalize well to out-of-distribution tasks compared to simple finetuning.
Re-initializing neural networks improves generalization but not as much as other techniques.
Artificial Neural Networks (ANNs) have emerged as hot topics in the research community. Despite the success of ANNs, it is challenging to train and deploy modern ANNs on commodity hardware due to the ever-increasing model size and the unprecedented growth in the data volumes. Particularly for microarray data, the very-…
We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix multiplications and convolutions. We introduce new sampling techniques, study their theoretical properties, and prove that they provide the same…
WAFFLE embeds watermarks in federated learning models without access to training data.
Sourcerer uses deep learning to map land cover from limited labeled data.
As neural networks become widely deployed in different applications and on different hardware, it has become increasingly important to optimize inference time and model size along with model accuracy. Most current techniques optimize model size, model accuracy and inference time in different stages, resulting in subopt…
A technique to quickly fix mistakes in neural networks.
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
New technique reduces complexity of deep networks.
Lapse improves parameter servers by dynamically allocating parameters, achieving near-linear scaling.
XGBoost outperforms other boosting techniques in training speed and generalization performance.
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models also increases. We introduce a technique to train deep neural networks using hal…
New visualization techniques reveal GAN optimization landscapes.
Combining self-training and contrastive learning improves performance under distribution shift.
Improved DP models with active learning on public data.
New technique improves time series forecasting with less data.
We present DANTE, a novel method for training neural networks using the alternating minimization principle. DANTE provides an alternate perspective to traditional gradient-based backpropagation techniques commonly used to train deep networks. It utilizes an adaptation of quasi-convexity to cast training a neural networ…
This paper reviews normalization techniques for DNNs.
New techniques improve 16-bit training accuracy without 32-bit units.
New methods distill data for deep networks efficiently.
Across numerous applications, forecasting relies on numerical solvers for partial differential equations (PDEs). Although the use of deep-learning techniques has been proposed, actual applications have been restricted by the fact the training data are obtained using traditional PDE solvers. Thereby, the uses of deep-le…
An important problem in training deep networks with high capacity is to ensure that the trained network works well when presented with new inputs outside the training dataset. Dropout is an effective regularization technique to boost the network generalization in which a random subset of the elements of the given data …
User interaction with voice-powered agents generates large amounts of unlabeled utterances. In this paper, we explore techniques to efficiently transfer the knowledge from these unlabeled utterances to improve model performance on Spoken Language Understanding (SLU) tasks. We use Embeddings from Language Model (ELMo) t…
HYDRA prunes robust neural networks to improve both benign and adversarial robustness.
Trained neural networks perform Bayesian reasoning for tasks beyond their initial scope.
New method closes certification gap for adversarially trained models.
Classifiers used in the wild, in particular for safety-critical systems, should not only have good generalization properties but also should know when they don't know, in particular make low confidence predictions far away from the training data. We show that ReLU type neural networks which yield a piecewise linear cla…
Study evaluates federated learning with differential privacy on MIMIC-III, improving model performance with careful parameter tuning.
Modern deep neural networks are typically highly overparameterized. Pruning techniques are able to remove a significant fraction of network parameters with little loss in accuracy. Recently, techniques based on dynamic reallocation of non-zero parameters have emerged, allowing direct training of sparse networks without…
Power meters are becoming a widely used tool for measuring training and racing effort in cycling, and are now spreading also to other sports. This means that increasing volumes of data can be collected from athletes, with the aim of helping coaches and athletes analyse and understanding training load, racing efforts, t…
Simplifies RL training with fewer techniques, reducing bias and instability.
Framework combines adversarial training and provable robustness for neural networks.