arXiv research

Elliptical Attention improves transformer performance by focusing on contextually relevant features.

Transformers model contextual relations using probabilistic measures, revealing their expressive power.

Study on how attention in prompt-tuning affects large language models.

Graph attention is not always beneficial; conditions for perfect node classification are identified.

Multi-head attention mechanism is capable of learning various representations from sequential data while paying attention to different subsequences, e.g., word-pieces or syllables in a spoken word. From the subsequences, it retrieves richer information than a single-head attention which only summarizes the whole sequen…

CATA++ improves scientific article recommendations by learning latent spaces with attention.

Paper proposes a method to control robots of different shapes efficiently.

Proposes a new algorithm for graph-based semi-parametric contextual bandits.

Method learns audio embeddings with contextualized tags.

MEANTIME improves sequential recommendation by using multi-temporal embeddings and attention mechanisms.

SLiCE learns contextual node embeddings for link prediction in heterogeneous networks.

We consider a novel formulation of the multi-armed bandit model, which we call the contextual bandit with restricted context, where only a limited number of features can be accessed by the learner at every iteration. This novel formulation is motivated by different online problems arising in clinical trials, recommende…

Neural networks using transformer-based architectures have recently demonstrated great power and flexibility in modeling sequences of many types. One of the core components of transformer networks is the attention layer, which allows contextual information to be exchanged among sequence elements. While many of the prev…

This paper introduces a new task to better understand Transformers in quantitative contexts.

CRAUM-Net improves salient object detection with context and uncertainty modeling.

Models use embeddings and attention for better claim severity prediction.

Recently, several studies have explored methods for using KG embedding to answer logical queries. These approaches either treat embedding learning and query answering as two separated learning tasks, or fail to deal with the variability of contributions from different query paths. We proposed to leverage a graph attent…

Transformers simplify modeling of small longitudinal cohort data by reducing parameters and incorporating attention mechanisms.

Home entertainment systems feature in a variety of usage scenarios with one or more simultaneous users, for whom the complexity of choosing media to consume has increased rapidly over the last decade. Users' decision processes are complex and highly influenced by contextual settings, but data supporting the development…

SARD improves deep learning clinical prediction performance.

There are two variants of the classical multi-armed bandit (MAB) problem that have received considerable attention from machine learning researchers in recent years: contextual bandits and simple regret minimization. Contextual bandits are a sub-class of MABs where, at every time step, the learner has access to side in…

Social bias in machine learning has drawn significant attention, with work ranging from demonstrations of bias in a multitude of applications, curating definitions of fairness for different contexts, to developing algorithms to mitigate bias. In natural language processing, gender bias has been shown to exist in contex…

RKT model improves knowledge tracing by considering exercise relations and student forget behavior.

AACC improves RL performance in changing environments.

GSANet improves semantic segmentation accuracy with selective and global attention.

Transformers predict price movements from limit order books.

FGTSVA improves Thompson Sampling for contextual bandits with optimal variance-aware regret.

This work bridges federated learning and contextual bandits, enhancing FL's utility.

Generative adversarial networks have been successfully applied to inpainting in natural images. However, the current state-of-the-art models have not yet been widely adopted in the medical imaging domain. In this paper, we investigate the performance of three recently published deep learning based inpainting models: co…

Unified framework for non-linear attention using modern Hopfield networks.

Fairness in AI decisions for users with varying performance.

In automatic speech recognition (ASR) what a user says depends on the particular context she is in. Typically, this context is represented as a set of word n-grams. In this work, we present a novel, all-neural, end-to-end (E2E) ASR sys- tem that utilizes such context. Our approach, which we re- fer to as Contextual Lis…

Text segmentation plays an important role in various Natural Language Processing (NLP) tasks like summarization, context understanding, document indexing and document noise removal. Previous methods for this task require manual feature engineering, huge memory requirements and large execution times. To the best of our …

The paper provides bounds on the CDF of a variable under nonstationary conditions.

In the classical contextual bandits problem, in each round $t$, a learner observes some context $c$, chooses some action $i$ to perform, and receives some reward $r_{i,t}(c)$. We consider the variant of this problem where in addition to receiving the reward $r_{i,t}(c)$, the learner also learns the values of $r_{i,t}(c…

Much effort has been devoted to understanding the decisions of deep neural networks in recent years. A number of model-aware saliency methods were proposed to explain individual classification decisions by creating saliency maps. However, they are not applicable when the parameters and the gradients of the underlying m…

Attention layers are sensitive to single words, improving generalization over random features.

Transformers can approximate any sequence-to-sequence function, surprising given their complexity.

Efficient algorithm for CLSBM reduces misclassification rate.

Unified framework for sequence models using test-time regression.

Outlier detection plays an essential role in many data-driven applications to identify isolated instances that are different from the majority. While many statistical learning and data mining techniques have been used for developing more effective outlier detection algorithms, the interpretation of detected outliers do…

Paper proposes a risk-aware decision-making framework for real-world sequential decisions.

Graph attention improves node classification by distinguishing important edges.

Cross-domain sentiment classification (CDSC) is an importance task in domain adaptation and sentiment classification. Due to the domain discrepancy, a sentiment classifier trained on source domain data may not works well on target domain data. In recent years, many researchers have used deep neural network models for c…

Combines BERT and GCN for better text classification.

The paper uses GRU and self-attention for SPY option pricing.

LNUCB-TA improves MAB performance by dynamically adjusting exploration rates and recognizing spatiotemporal patterns.

A new approach to hedging using contextual bandit models outperforms traditional methods.