A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method reveals how training data influence diffusion model outputs.
Pre-training is crucial for learning deep neural networks. Most of existing pre-training methods train simple models (e.g., restricted Boltzmann machines) and then stack them layer by layer to form the deep structure. This layer-wise pre-training has found strong theoretical foundation and broad empirical support. Howe…
Joint training improves model accuracy by selectively using privileged information.
Paper proposes a new method to measure model sensitivity using final model only.
Recent work on discriminative segmental models has shown that they can achieve competitive speech recognition performance, using features based on deep neural frame classifiers. However, segmental models can be more challenging to train than standard frame-based approaches. While some segmental models have been success…
The training of Deep Neural Networks usually needs tremendous computing resources. Therefore many deep models are trained in large cluster instead of single machine or GPU. Though major researchs at present try to run whole model on all machines by using asynchronous asynchronous stochastic gradient descent (ASGD), we …
With the rise of low power speech-enabled devices, there is a growing demand to quickly produce models for recognizing arbitrary sets of keywords. As with many machine learning tasks, one of the most challenging parts in the model creation process is obtaining a sufficient amount of training data. In this paper, we exp…
New framework explains how larger pre-trained models reduce downstream learning sample complexity.
Paper tackles model vulnerabilities by reconstructing training data.
This paper connects masked pre-training to Bayesian model selection.
WAFFLE embeds watermarks in federated learning models without access to training data.
In many applications, one works with neural network models trained by someone else. For such pretrained models, one may not have access to training data or test data. Moreover, one may not know details about the model, e.g., the specifics of the training data, the loss function, the hyperparameter values, etc. Given on…
GP-TS optimizes TLM pre-training hyperparameters efficiently.
New attack reveals memorization patterns in pre-trained LLMs.
Ensemble pre-trained models for low data transfer learning.
SMT trains generative models by estimating mixture scores, outperforming existing methods.
Survey on reducing deep neural network training complexity.
Predicts optimal training dataset sizes per class for machine learning models.
New method improves neural spike train models by minimizing divergence directly, leading to better performance.
We use GG distributions to optimize LLMs, reducing size and training time.
Machine learning models are vulnerable to adversarial examples. Iterative adversarial training has shown promising results against strong white-box attacks. However, adversarial training is very expensive, and every time a model needs to be protected, such expensive training scheme needs to be performed. In this paper,…
Gradient amplification boosts deep learning model performance without increasing training time.
MGD improves ML training efficiency and accuracy.
Model trains passing events on a bridge using multilevel Gaussian process.
Adversarial samples are perturbed inputs crafted to mislead the machine learning systems. A training mechanism, called adversarial training, which presents adversarial samples along with clean samples has been introduced to learn robust models. In order to scale adversarial training for large datasets, these perturbati…
The paper studies stability of generative models trained on mixed data.
Theoretical study shows adversarial training improves robustness in deep learning models.
Auxiliary Tuning adapts pre-trained models for novel tasks efficiently.
LEMON uses pre-trained models to scale neural networks efficiently.
In this work, three lattice-free (LF) discriminative training criteria for purely sequence-trained neural network acoustic models are compared on LVCSR tasks, namely maximum mutual information (MMI), boosted maximum mutual information (bMMI) and state-level minimum Bayes risk (sMBR). We demonstrate that, analogous to L…
Deep convolutional neural networks have achieved great success in various applications. However, training an effective DNN model for a specific task is rather challenging because it requires a prior knowledge or experience to design the network architecture, repeated trial-and-error process to tune the parameters, and …
For many large undirected models that arise in real-world applications, exact maximumlikelihood training is intractable, because it requires computing marginal distributions of the model. Conditional training is even more difficult, because the partition function depends not only on the parameters, but also on the obse…
End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wa…
Adversarially-trained models transfer better in limited data scenarios.
Pre-trained model from healthy ADLs improves gait pattern classification for Parkinson's disease.
This study explains RL training dynamics in LLMs, focusing on token-level optimization and reasoning pattern reshaping.
This paper proposes a model of information cascades as directed spanning trees (DSTs) over observed documents. In addition, we propose a contrastive training procedure that exploits partial temporal ordering of node infections in lieu of labeled training links. This combination of model and unsupervised training makes …
The paper analyzes how adding gradient data affects overparameterized models' performance.
Paper tackles model collapse in recursive generative models using a weighted training scheme.
Self-supervised skip-tree training improves mathematical reasoning in language models.
Surrogate models speed up RL training in dynamic systems.
Adversarial training was introduced as a way to improve the robustness of deep learning models to adversarial attacks. This training method improves robustness against adversarial attacks, but increases the models vulnerability to privacy attacks. In this work we demonstrate how model inversion attacks, extracting trai…
A novel solve-training framework is proposed to train neural network in representing low dimensional solution maps of physical models. Solve-training framework uses the neural network as the ansatz of the solution map and train the network variationally via loss functions from the underlying physical models. Solve-trai…
State-of-the art vision models can achieve superhuman performance on image classification tasks when testing and training data come from the same distribution. However, when models are tested on corrupted images (e.g. due to scale changes, translations, or shifts in brightness or contrast), performance degrades signifi…
Personalizes pre-trained models for nonparametric regression with limited data.
Traditionally, when generative models of data are developed via deep architectures, greedy layer-wise pre-training is employed. In a well-trained model, the lower layer of the architecture models the data distribution conditional upon the hidden variables, while the higher layers model the hidden distribution prior. Bu…
Paper proposes a new method for training nonconvex models.
Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific f-divergence between the model and data distribution. In light of recent successes in training Generative Adversarial Networks, alternative non-likelihood training criteria have been proposed. Whilst not necessarily…