A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The study finds a theoretical bound for pre-training iterations needed for pruning to yield good subnetwork performance.
Paper shows pre-training and transfer learning reduce sample complexity for neural networks.
GCC pre-trains graph neural networks to transfer across diverse datasets.
This work theoretically investigates the performance of a composite neural network. A composite neural network is a rooted directed acyclic graph combining a set of pre-trained and non-instantiated neural network models, where a pre-trained neural network model is well-crafted for a specific task and targeted to approx…
We present a novel approach to leverage large unlabeled datasets by pre-training state-of-the-art deep neural networks on randomly-labeled datasets. Specifically, we train the neural networks to memorize arbitrary labels for all the samples in a dataset and use these pre-trained networks as a starting point for regular…
Pre-training is crucial for learning deep neural networks. Most of existing pre-training methods train simple models (e.g., restricted Boltzmann machines) and then stack them layer by layer to form the deep structure. This layer-wise pre-training has found strong theoretical foundation and broad empirical support. Howe…
Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.
Survey on self-supervised pre-training for neural networks using unlabeled data.
SimCLR pre-training improves CNN performance with fewer labels.
Adaptive kernel density estimation improves accuracy in high dimensions.
GPT-GNN pre-trains GNNs on unlabeled graphs to improve downstream performance.
Many applications of machine learning require a model to make accurate pre-dictions on test examples that are distributionally different from training ones, while task-specific labels are scarce during training. An effective approach to this challenge is to pre-train a model on related tasks where data is abundant, and…
This work investigates the framework and performance issues of the composite neural network, which is composed of a collection of pre-trained and non-instantiated neural network models connected as a rooted directed acyclic graph for solving complicated applications. A pre-trained neural network model is generally well…
Graph neural networks (GNNs) are shown to be successful in modeling applications with graph structures. However, training an accurate GNN model requires a large collection of labeled data and expressive features, which might be inaccessible for some applications. To tackle this problem, we propose a pre-training framew…
We propose a novel learning method for multilayered neural networks which uses feedforward supervisory signal and associates classification of a new input with that of pre-trained input. The proposed method effectively uses rich input information in the earlier layer for robust leaning and revising internal representat…
This paper ranks pre-trained DNNs using a novel SI measure.
We consider unsupervised domain adaptation: given labelled examples from a source domain and unlabelled examples from a related target domain, the goal is to infer the labels of target examples. Under the assumption that features from pre-trained deep neural networks are transferable across related domains, domain adap…
Study compares deep feature methods for anomaly detection in limited data scenarios.
New method improves ensemble quality by exploring the pre-train basin more effectively.
SAINT improves neural networks for tabular data with row attention and contrastive pre-training.
Post-hoc uncertainty quantification improves on pre-trained neural networks without underfitting.
Theoretical study shows adversarial training improves robustness in deep learning models.
TCGPN improves stock forecasting by capturing temporal correlation patterns.
Unsupervised pre-training improves model generalization, but lacks theoretical understanding.
Deep Neural Networks (DNNs) have become increasingly popular in computer vision, natural language processing, and other areas. However, training and fine-tuning a deep learning model is computationally intensive and time-consuming. We propose a new method to improve the performance of nearly every model including pre-t…
Ensemble pre-trained models for low data transfer learning.
Improved LLM pre-training performance through better weight and variance control.
Personalizes pre-trained models for nonparametric regression with limited data.
The study uses pre-trained neural networks to adjust for confounding in non-tabular data.
Bridging the exponentially growing gap between the numbers of unlabeled and labeled protein sequences, several studies adopted semi-supervised learning for protein sequence modeling. In these studies, models were pre-trained with a substantial amount of unlabeled data, and the representations were transferred to variou…
PFNs pre-train models on simulated data to predict class probabilities.
Unbalanced GANs stabilize GAN training by pre-training the generator with VAE.
Deep convolutional neural networks have achieved great success in various applications. However, training an effective DNN model for a specific task is rather challenging because it requires a prior knowledge or experience to design the network architecture, repeated trial-and-error process to tune the parameters, and …
Deep neural networks have shown promising results for various clinical prediction tasks. However, training deep networks such as those based on Recurrent Neural Networks (RNNs) requires large labeled data, significant hyper-parameter tuning effort and expertise, and high computational resources. In this work, we invest…
Lottery tickets find good initializations for IMP with sparse training.
This paper aims to analyze knowledge consistency between pre-trained deep neural networks. We propose a generic definition for knowledge consistency between neural networks at different fuzziness levels. A task-agnostic method is designed to disentangle feature components, which represent the consistent knowledge, from…
Fine-tunes GNNs by preserving generative patterns to improve transferability.
Enhances deep neural networks with fixed-mean Gaussian processes for uncertainty estimation.
Study improves breast lesion segmentation with limited in vivo data using simulated and natural images.
New findings show BERT subnetworks can train independently and transfer to various tasks.
Deep neural networks have achieved impressive performance in many applications but their large number of parameters lead to significant computational and storage overheads. Several recent works attempt to mitigate these overheads by designing compact networks using pruning of connections. However, we observe that most …
LEMON uses pre-trained models to scale neural networks efficiently.
In this contribution, we present a novel approach for segmenting laser radar (lidar) imagery into geometric time-height cloud locations with a fully convolutional network (FCN). We describe a semi-supervised learning method to train the FCN by: pre-training the classification layers of the FCN with image-level annotati…
ABNN converts pre-trained DNNs into BNNs for reliable uncertainty quantification.
Transfer learning boosts chemically accurate neural network potentials for organic molecules.
Deep neural networks have shown promising results for various clinical prediction tasks such as diagnosis, mortality prediction, predicting duration of stay in hospital, etc. However, training deep networks -- such as those based on Recurrent Neural Networks (RNNs) -- requires large labeled data, high computational res…
Transfer learning have been frequently used to improve deep neural network training through incorporating weights of pre-trained networks as the starting-point of optimization for regularization. While deep transfer learning can usually boost the performance with better accuracy and faster convergence, transferring wei…
Capturing sentence semantics plays a vital role in a range of text mining applications. Despite continuous efforts on the development of related datasets and models in the general domain, both datasets and models are limited in biomedical and clinical domains. The BioCreative/OHNLP organizers have made the first attemp…