A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Proposes model-based robust deep learning to handle natural variation in data.
New method reduces adversarial training time by up to 2.5x with comparable accuracy.
New approach improves adversarial robustness without sacrificing natural generalization.
AOPU stabilizes NN training by approximating natural gradient, improving stability and convergence.
NRWS improves training of SBNs and HMs using natural gradient.
Language models learn from training data and can leak private information.
The paper proposes a method to train time-varying generative models using natural gradients.
Natural gradient descent is a robust optimization method for machine learning.
We present a novel algorithm to train a deep Q-learning agent using natural-gradient techniques. We compare the original deep Q-network (DQN) algorithm to its natural-gradient counterpart, which we refer to as NGDQN, on a collection of classic control domains. Without employing target networks, NGDQN significantly outp…
We show new connections between adversarial learning and explainability for deep neural networks (DNNs). One form of explanation of the output of a neural network model in terms of its input features, is a vector of feature-attributions. Two desirable characteristics of an attribution-based explanation are: (1) $\texti…
SONet stabilizes ODE networks for robustness without adversarial training.
Natural gradient has been recently introduced to the field of boosting to enable the generic probabilistic predication capability. Natural gradient boosting shows promising performance improvements on small datasets due to better training dynamics, but it suffers from slow training speed overhead especially for large d…
Recent work on adversarial attack and defense suggests that PGD is a universal first-order attack, and PGD adversarial training can significantly improve network robustness against a wide range of first-order -bounded attacks, represented as the state-of-the-art defense method. However, an obvious …
RoBERTa outperforms other pre-trained models in NER tasks.
We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix. In particular, we show a specific example that employs a simple and efficient reparametrization of the neural network weigh…
We consider the problem of making machine translation more robust to character-level variation at the source side, such as typos. Existing methods achieve greater coverage by applying subword models such as byte-pair encoding (BPE) and character-level encoders, but these methods are highly sensitive to spelling mistake…
Proposes an alternative method to train RBMs with binary synapses using Bayesian learning rule.
Training neural networks on image datasets generally require extensive experimentation to find the optimal learning rate regime. Especially, for the cases of adversarial training or for training a newly synthesized model, one would not know the best learning rate regime beforehand. We propose an automated algorithm for…
Untrained neural networks can recover natural images from few measurements.
This paper introduces the Metric-Free Natural Gradient (MFNG) algorithm for training Boltzmann Machines. Similar in spirit to the Hessian-Free method of Martens [8], our algorithm belongs to the family of truncated Newton methods and exploits an efficient matrix-vector product to avoid explicitely storing the natural g…
Stochastic defense improves natural classifiers against adversarial attacks.
Deep Neural Network (DNN) acoustic models often use discriminative sequence training that optimises an objective function that better approximates the word error rate (WER) than frame-based training. Sequence training is normally implemented using Stochastic Gradient Descent (SGD) or Hessian Free (HF) training. This pa…
Today's state-of-the-art image classifiers fail to correctly classify carefully manipulated adversarial images. In this work, we develop a new, localized adversarial attack that generates adversarial examples by imperceptibly altering the backgrounds of normal images. We first use this attack to highlight the unnecessa…
Paper improves natural language understanding with less data using a new training method.
Dropout training is shown to be optimal for adversarial covariate corruption.
Improves natural accuracy of deep learning models by combining robust predictions and features.
In this work, we introduce a two-step framework for generative modeling of temporal data. Specifically, the generative adversarial networks (GANs) setting is employed to generate synthetic scenes of moving objects. To do so, we propose a two-step training scheme within which: a generator of static frames is trained fir…
Study improves breast lesion segmentation with limited in vivo data using simulated and natural images.
Modified training direction reduces generalization error in neural networks.
New approach shows backdoor attacks are indistinguishable from natural data features.
We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating). These toolkits provide state-of-the-art pre-trained models, training scripts, and training logs, to facilitate rapid prototyping and promote reproducible research. We also …
We describe the neural-network training framework used in the Kaldi speech recognition toolkit, which is geared towards training DNNs with large amounts of training data using multiple GPU-equipped or multi-core machines. In order to be as hardware-agnostic as possible, we needed a way to use multiple machines without …
Associating image regions with text queries has been recently explored as a new way to bridge visual and linguistic representations. A few pioneering approaches have been proposed based on recurrent neural language models trained generatively (e.g., generating captions), but achieving somewhat limited localization accu…
Empirical adversarial risk minimization (EARM) is a widely used mathematical framework to robustly train deep neural nets (DNNs) that are resistant to adversarial attacks. However, both natural and robust accuracies, in classifying clean and adversarial images, respectively, of the trained robust models are far from sa…
Our work proves convergence to low robust training loss for polynomial width ReLU networks.
The paper explores how neural networks generalize differently from natural and medical images.
New method reduces bias in NLI models using ensemble adversarial training.
SP-NGD improves deep learning models' generalization with large mini-batch sizes.
The study enhances financial rule matching using NLP without datasets.
Reconstructing observed images from fMRI brain recordings is challenging. Unfortunately, acquiring sufficient "labeled" pairs of {Image, fMRI} (i.e., images with their corresponding fMRI responses) to span the huge space of natural images is prohibitive for many reasons. We present a novel approach which, in addition t…
We present a theoretical analysis of Gaussian-binary restricted Boltzmann machines (GRBMs) from the perspective of density models. The key aspect of this analysis is to show that GRBMs can be formulated as a constrained mixture of Gaussians, which gives a much better insight into the model's capabilities and limitation…
New analysis shows diverse classes in pre-training boost NLP performance.
Language models can answer questions without external knowledge.
Gaudy images help train deep neural networks with less data.
Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high dimensional interpolati…
SINGD improves KFAC for memory-efficiency and stability in low-precision training.
Recurrent neural networks (RNN) have been successfully applied to various sequential decision-making tasks, natural language processing applications, and time-series predictions. Such networks are usually trained through back-propagation through time (BPTT) which is prohibitively expensive, especially when the length o…
ETC improves Transformer models for long and structured inputs.