A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This study explores how examples influence ICL in LLMs.
Language models learn automotive complaints, improving defect detection.
This paper aims to analyze knowledge consistency between pre-trained deep neural networks. We propose a generic definition for knowledge consistency between neural networks at different fuzziness levels. A task-agnostic method is designed to disentangle feature components, which represent the consistent knowledge, from…
Pre-training is crucial for learning deep neural networks. Most of existing pre-training methods train simple models (e.g., restricted Boltzmann machines) and then stack them layer by layer to form the deep structure. This layer-wise pre-training has found strong theoretical foundation and broad empirical support. Howe…
Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.
Pre-trained GPs improve Bayesian optimization efficiency.
Deep Reinforcement Learning (DRL) algorithms are known to be data inefficient. One reason is that a DRL agent learns both the feature and the policy tabula rasa. Integrating prior knowledge into DRL algorithms is one way to improve learning efficiency since it helps to build helpful representations. In this work, we co…
Transfer learning does not improve character recognition performance.
Pre-training improves Bayesian optimization efficiency.
System helps engineers with concept recognition for SEVA.
Method transfers knowledge without label overlap, source data, or target architecture consistency.
Unsupervised pre-training improves model generalization, but lacks theoretical understanding.
Low-dimensional embeddings of knowledge graphs and behavior graphs have proved remarkably powerful in varieties of tasks, from predicting unobserved edges between entities to content recommendation. The two types of graphs can contain distinct and complementary information for the same entities/nodes. However, previous…
Fine-tuning neural networks is widely used to transfer valuable knowledge from high-resource to low-resource domains. In a standard fine-tuning scheme, source and target problems are trained using the same architecture. Although capable of adapting to new domains, pre-trained units struggle with learning uncommon targe…
Researchers develop method to protect against 'weight poisoning' attacks on pre-trained models.
Pre-trained language model boosts RL efficiency.
Study shows how a strong model can learn a task's feature while retaining other capabilities.
Improved sentiment analysis in Korean finance using masked PLMs.
ImageNet pre-training has been regarded as essential for training accurate object detectors for a long time. Recently, it has been shown that object detectors trained from randomly initialized weights can be on par with those fine-tuned from ImageNet pre-trained models. However, the effects of pre-training and the diff…
Enhances financial time-series prediction by adapting pre-trained models to new data.
Language models can answer questions without external knowledge.
Understanding learning materials (e.g. test questions) is a crucial issue in online learning systems, which can promote many applications in education domain. Unfortunately, many supervised approaches suffer from the problem of scarce human labeled data, whereas abundant unlabeled resources are highly underutilized. To…
Bayesian method transfers knowledge between brain tumor datasets.
GCTM integrates GCN into topic models for better topic learning from data streams.
Study compares deep feature methods for anomaly detection in limited data scenarios.
New framework transfers latent knowledge from weak to strong models.
Due to insufficient training data and the high computational cost to train a deep neural network from scratch, transfer learning has been extensively used in many deep-neural-network-based applications. A commonly used transfer learning approach involves taking a part of a pre-trained model, adding a few layers at the …
Deep convolutional neural networks have achieved great success in various applications. However, training an effective DNN model for a specific task is rather challenging because it requires a prior knowledge or experience to design the network architecture, repeated trial-and-error process to tune the parameters, and …
Improves synthetic-to-real generalization without real data.
This paper explores the connection between adversarial and knowledge transferability.
Automatic segmentation methods based on deep learning have recently demonstrated state-of-the-art performance, outperforming the ordinary methods. Nevertheless, these methods are inapplicable for small datasets, which are very common in medical problems. To this end, we propose a knowledge transfer method between disea…
With access to large datasets, deep neural networks (DNN) have achieved human-level accuracy in image and speech recognition tasks. However, in chemistry, data is inherently small and fragmented. In this work, we develop an approach of using rule-based knowledge for training ChemNet, a transferable and generalizable de…
Framework incorporates prior knowledge into Bayesian models for data streams.
We present a novel framework to deal with relation extraction tasks in cases where there is complete lack of supervision, either in the form of gold annotations, or relations from a knowledge base. Our approach leverages syntactic parsing and pre-trained word embeddings to extract few but precise relations,which are th…
Knowledge graphs are a versatile framework to encode richly structured data relationships, but it can be challenging to combine these graphs with unstructured data. Methods for retrofitting pre-trained entity representations to the structure of a knowledge graph typically assume that entities are embedded in a connecte…
Two things seem to be indisputable in the contemporary deep learning discourse: 1. The categorical cross-entropy loss after softmax activation is the method of choice for classification. 2. Training a CNN classifier from scratch on small datasets does not work well. In contrast to this, we show that the cosine loss fun…
LEMON uses pre-trained models to scale neural networks efficiently.
Transfer learning which aims at utilizing knowledge learned from one problem (source domain) to solve another different but related problem (target domain) has attracted wide research attentions. However, the current transfer learning methods are mostly uninterpretable, especially to people without ML expertise. In thi…
Weight Squeezing transfers knowledge from large models to smaller ones, improving performance and speed.
The paper studies estimation rates for MoE models with a new prompt.
Proposes efficient deep causal generative models for high-dimensional causal inference.
ConEx learns complex embeddings for knowledge graphs, improving link prediction.
In recent years, deep learning models have shown great potential in source code modeling and analysis. Generally, deep learning-based approaches are problem-specific and data-hungry. A challenging issue of these approaches is that they require training from starch for a different related problem. In this work, we propo…
DPTA improves CIL by adapting PTMs with dual prototypes.
This study investigates how much knowledge from natural images can be transferred to pathology images.
Deep neural networks have shown promising results for various clinical prediction tasks such as diagnosis, mortality prediction, predicting duration of stay in hospital, etc. However, training deep networks -- such as those based on Recurrent Neural Networks (RNNs) -- requires large labeled data, high computational res…
Transfer learning have been frequently used to improve deep neural network training through incorporating weights of pre-trained networks as the starting-point of optimization for regularization. While deep transfer learning can usually boost the performance with better accuracy and faster convergence, transferring wei…
Bayesian Power Steering fine-tunes large diffusion models for domain adaptation.