A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Unsupervised pre-training improves model generalization, but lacks theoretical understanding.
POUF fine-tunes large models without labeled data.
Deep Convolutional features extracted from a comprehensive labeled dataset, contain substantial representations which could be effectively used in a new domain. Despite the fact that generic features achieved good results in many visual tasks, fine-tuning is required for pretrained deep CNN models to be more effective …
Robust CLIP improves vision models' resistance to attacks.
Big models pretrain and fine-tune for semi-supervised learning on ImageNet.
This research shows unsupervised GANs can perform object segmentation without labels.
New method improves ABI for sequential data, reducing forgetting and improving accuracy.
PanRep learns universal node embeddings for heterogeneous graphs.
Novel approach trains ASR models with less supervision using bilevel optimization.
Universal Language Model for Fine-tuning [arXiv:1801.06146] (ULMFiT) is one of the first NLP methods for efficient inductive transfer learning. Unsupervised pretraining results in improvements on many NLP tasks for English. In this paper, we describe a new method that uses subword tokenization to adapt ULMFiT to langua…
This paper presents our contribution to PolEval 2019 Task 6: Hate speech and bullying detection. We describe three parallel approaches that we followed: fine-tuning a pre-trained ULMFiT model to our classification task, fine-tuning a pre-trained BERT model to our classification task, and using the TPOT library to find …
In this paper, we present a statistical-mechanical analysis of deep learning. We elucidate some of the essential components of deep learning---pre-training by unsupervised learning and fine tuning by supervised learning. We formulate the extraction of features from the training data as a margin criterion in a high-dime…
GRASP removes spurious correlations in fine-tuned models, improving task performance and reducing bias.
Recent state-of-the-art language models utilize a two-phase training procedure comprised of (i) unsupervised pre-training on unlabeled text, and (ii) fine-tuning for a specific supervised task. More recently, many studies have been focused on trying to improve these models by enhancing the pre-training phase, either vi…
In this study, we propose the integration of competitive learning into convolutional neural networks (CNNs) to improve the representation learning and efficiency of fine-tuning. Conventional CNNs use back propagation learning, and it enables powerful representation learning by a discrimination task. However, it require…
Multi-task learning shares information between related tasks, sometimes reducing the number of parameters required. State-of-the-art results across multiple natural language understanding tasks in the GLUE benchmark have previously used transfer from a single large task: unsupervised pre-training with BERT, where a sep…
Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.
Generative models predict page quality without training, useful for low-resource settings.
We present a novel framework to deal with relation extraction tasks in cases where there is complete lack of supervision, either in the form of gold annotations, or relations from a knowledge base. Our approach leverages syntactic parsing and pre-trained word embeddings to extract few but precise relations,which are th…
Effective modeling of electronic health records presents many challenges as they contain large amounts of irregularity most of which are due to the varying procedures and diagnosis a patient may have. Despite the recent progress in machine learning, unsupervised learning remains largely at open, especially in the healt…
RAFT fine-tunes models using high-quality samples to align them with human preferences.
The human brain is able to learn, generalize, and predict crossmodal stimuli. Learning by expectation fine-tunes crossmodal processing at different levels, thus enhancing our power of generalization and adaptation in highly dynamic environments. In this paper, we propose a deep neural architecture trained by using expe…
Mask-reconstruction pretraining helps in downstream tasks by capturing more semantic features.
This work addresses encoding biases in neural networks by tailoring models with unsupervised losses.
Program synthesis has emerged as a successful approach to the image parsing task. Most prior works rely on a two-step scheme involving supervised pretraining of a Seq2Seq model with synthetic programs followed by reinforcement learning (RL) for fine-tuning with real reference images. Fully unsupervised approaches promi…
With the proliferation of social media, fashion inspired from celebrities, reputed designers as well as fashion influencers has shortened the cycle of fashion design and manufacturing. However, with the explosion of fashion related content and large number of user generated fashion photos, it is an arduous task for fas…
AI model automates financial investment research tasks.
New method uses limited labeled data and multiple starts to adapt models across domains.
Pretraining method enhances dialogue representation learning across various tasks.
Feature learning and deep learning have drawn great attention in recent years as a way of transforming input data into more effective representations using learning algorithms. Such interest has grown in the area of music information retrieval (MIR) as well, particularly in music audio classification tasks such as auto…
The field of few-shot learning has been laboriously explored in the supervised setting, where per-class labels are available. On the other hand, the unsupervised few-shot learning setting, where no labels of any kind are required, has seen little investigation. We propose a method, named Assume, Augment and Learn or AA…
Recent advancements in language representation models such as BERT have led to a rapid improvement in numerous natural language processing tasks. However, language models usually consist of a few hundred million trainable parameters with embedding space distributed across multiple layers, thus making them challenging t…
We propose the application of a semi-supervised learning method to improve the performance of acoustic modelling for automatic speech recognition based on deep neural net- works. As opposed to unsupervised initialisation followed by supervised fine tuning, our method takes advantage of both unlabelled and labelled data…
Cancer is still one of the most devastating diseases of our time. One way of automatically classifying tumor samples is by analyzing its derived molecular information (i.e., its genes expression signatures). In this work, we aim to distinguish three different types of cancer: thyroid, skin, and stomach. For that, we co…
ProtoTransfer learns from unlabeled data to classify unseen tasks.
New measure FTC quantifies how much a ReLU network can fine-tune.
ECM uses class means for efficient early exits in neural networks.
This paper re-evaluates hyperparameters for fine-tuning pre-trained models.
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
Unsupervised near-duplicate detection has many practical applications ranging from social media analysis and web-scale retrieval, to digital image forensics. It entails running a threshold-limited query on a set of descriptors extracted from the images, with the goal of identifying all possible near-duplicates, while l…
BERT fine-tuning is unstable due to optimization issues, not forgetting or dataset size.
We present a representation learning method that learns features at multiple different levels of scale. Working within the unsupervised framework of denoising autoencoders, we observe that when the input is heavily corrupted during training, the network tends to learn coarse-grained features, whereas when the input is …
Improved code translation by preserving structure with composed fine-tuning.
BlosSOM improves data visualization for large datasets.
The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.
Fine-tuning LLMs improves capability but harms safety, study finds.
Optimizes sparse fine-tuning for privacy in neural networks.
Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.