A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Training on mixed distributions improves test performance even when components are unrelated.
Paper optimizes training data distribution for better model performance across various deployment conditions.
Training machine learning (ML) models on large datasets requires considerable computing power. To speed up training, it is typical to distribute training across several machines, often with specialized hardware like GPUs or TPUs. Managing a distributed training job is complex and requires dealing with resource contenti…
Distributed Lion optimizes large model training by reducing communication costs.
In-N-Out improves model robustness to out-of-distribution data.
Deep learning has led to tremendous advancements in the field of Artificial Intelligence. One caveat however is the substantial amount of compute needed to train these deep learning models. Training a benchmark dataset like ImageNet on a single machine with a modern GPU can take upto a week, distributing training on mu…
Generative Adversarial Networks (GANs) are widely used models to learn complex real-world distributions. In GANs, the training of the generator usually stops when the discriminator can no longer distinguish the generator's output from the set of training examples. A central question of GANs is that when the training st…
New techniques improve distributed training with compressed gradients.
This work proposes efficient classical training protocols for IQP circuits to train quantum generative models.
We use GG distributions to optimize LLMs, reducing size and training time.
The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their…
A distributed algorithm for training graph convolutional networks.
The paper addresses the reliability of conformal prediction under covariate shift.
Method diagnoses model performance under distribution shifts.
Ensemble approaches for uncertainty estimation have recently been applied to the tasks of misclassification detection, out-of-distribution input detection and adversarial attack detection. Prior Networks have been proposed as an approach to efficiently \emph{emulate} an ensemble of models for classification by paramete…
New method predicts model output distributions to improve data attribution.
DistGP models multi-robot mapping with distributed Gaussian process learning.
Study on continuous sequence classification with distribution uncertainty.
Quantum models learn unitary actions on entangled states from product states.
Discriminatively trained neural classifiers can be trusted, only when the input data comes from the training distribution (in-distribution). Therefore, detecting out-of-distribution (OOD) samples is very important to avoid classification errors. In the context of OOD detection for image classification, one of the recen…
Public pretraining improves private model training even in extreme distribution shift scenarios.
Distributed machine learning (ML) can bring more computational resources to bear than single-machine learning, thus enabling reductions in training time. Distributed learning partitions models and data over many machines, allowing model and dataset sizes beyond the available compute power and memory of a single machine…
Neural networks are vulnerable to small adversarial perturbations. Existing literature largely focused on understanding and mitigating the vulnerability of learned models. In this paper, we demonstrate an intriguing phenomenon about the most popular robust training method in the literature, adversarial training: Advers…
Language models are generally trained on data spanning a wide range of topics (e.g., news, reviews, fiction), but they might be applied to an a priori unknown target distribution (e.g., restaurant reviews). In this paper, we first show that training on text outside the test distribution can degrade test performance whe…
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure. The situation gets even worse with distributed training on mobile devices (federated learning), which suff…
In this work, we propose a method to reject out-of-distribution samples which can be adapted to any network architecture and requires no additional training data. Publicly available chest x-ray data (38,353 images) is used to train a standard ResNet-50 model to detect emphysema. Feature activations of intermediate laye…
Causal influence measures for machine learnt classifiers shed light on the reasons behind classification, and aid in identifying influential input features and revealing their biases. However, such analyses involve evaluating the classifier using datapoints that may be atypical of its training distribution. Standard me…
A decision-theoretic bootstrapping method for robust uncertainty quantification.
New perspective on distribution shift helps make learning easier.
Combining self-training and contrastive learning improves performance under distribution shift.
SAPAG attacks distributed learning by reconstructing true training data from gradients.
Domain randomization (DR) is a successful technique for learning robust policies for robot systems, when the dynamics of the target robot system are unknown. The success of policies trained with domain randomization however, is highly dependent on the correct selection of the randomization distribution. The majority of…
New methods reduce communication in distributed training for variational inequalities.
Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands. Classical distributed approaches, synchronous or asynchronous, are based on the parameter server architecture, i.e., worker nodes compute gradients which…
RATIO improves neural network robustness and explainability.
The paper optimizes training samples for image denoising across different noise levels.
FIRE method improves model performance in federated learning by penalizing fragmentation-induced covariate shifts.
TAET tackles long-tailed distributions in adversarial robustness.
Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. However, most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks. Besides, a singl…
Deep learning models frequently make incorrect predictions with high confidence when presented with test examples that are not well represented in their training dataset. We propose a novel and straightforward approach to estimate prediction uncertainty in a pre-trained neural network model. Our method estimates the tr…
New method trains neural samplers to sample from multi-modal distributions efficiently.
LSAM optimizes deep learning training with improved efficiency.
Deep learning with medical data often requires larger samples sizes than are available at single providers. While data sharing among institutions is desirable to train more accurate and sophisticated models, it can lead to severe privacy concerns due the sensitive nature of the data. This problem has motivated a number…
Study optimal ridge regularization for out-of-distribution prediction.
ADB framework improves OOD generalization by increasing ID bias during training.
Reweighting training data to better represent new tasks.
Mobile edge learning is an emerging technique that enables distributed edge devices to collaborate in training shared machine learning models by exploiting their local data samples and communication and computation resources. To deal with the straggler dilemma issue faced in this technique, this paper proposes a new de…
SMT trains generative models by estimating mixture scores, outperforming existing methods.