This is a lecture note for the course DS-GA 3001 <Natural Language Understanding with Distributed Representation> at the Center for Data Science , New York University in Fall, 2015. As the name of the course suggests, this lecture note introduces readers to a neural network based approach to natural language understand…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “natural language understanding”
Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (vision) and prosodic cues (acoustic). Understanding humor from these three modalities falls within boundaries of multimodal language; a recent r…
ZeroSCROLLS benchmarks zero-shot natural language understanding over long texts.
problem Evaluate natural language understanding models over long texts without training data.
method Adapt six tasks from SCROLLS benchmark and add four new datasets, including novel aggregation tasks.
result Claude outperforms ChatGPT, and GPT-4 achieves highest average score.
Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret. Especially, little is known about how they represent language in their intermediate layers. In an attempt to understand the representatio…
Optimus pre-trains sentences in a latent space for various NLP tasks.
problem Training large-scale language models for diverse NLP tasks.
method Pre-trained Variational Autoencoder (VAE) on large text corpus, fine-tuned for various tasks.
result Optimus achieves state-of-the-art on VAE language modeling benchmarks.
The study enhances financial rule matching using NLP without datasets.
problem Performing semantic matching between financial rules and policies.
method Outperforming pre-trained models with NLP techniques using free resources.
result Improved semantic matching between financial rules and policies.
Paper improves natural language understanding with less data using a new training method.
problem Limited data hinders performance of small models in natural language tasks.
method Generation-Distillation: uses large finetuned models to generate new training data and distill knowledge into smaller models.
result Achieves comparable performance to BERT with 300x fewer parameters and outperforms prior distillation methods.
We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through read}, compose and write operations. NSE c…
CERT improves language understanding by contrastively learning sentence-level semantics.
problem Lack of sentence-level semantics in existing pretraining tasks.
method Contrastive self-supervised learning at the sentence level using back-translation augmentations.
result CERT outperforms BERT on 7 out of 11 GLUE benchmark tasks, achieving the same performance as BERT on 2 tasks.
KG-A2C agent learns natural language IF games by reasoning and constraining action spaces.
problem Challenges of natural language understanding, partial observability, and combinatorially large action spaces in IF games.
method Builds a dynamic knowledge graph while exploring, constraining actions using templates.
result Outperforms current IF agents across various games with larger action spaces.
Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence. Ideally, these tasks should test a plethora of capabilities that integrate computer vision, reasoning, and natural language understanding. However, rather than behaving as visual Turing t…
TextHide secures language understanding tasks by adding minimal encryption.
problem Mitigating privacy risks in natural language understanding tasks.
method TextHide adds a simple encryption step to prevent data recovery.
result TextHide reduces accuracy loss by only 1.9% on GLUE benchmark.
lamBERT learns language and actions using multimodal BERT.
problem Learning language and actions in complex environments.
method Extending BERT to multimodal representation and integrating with reinforcement learning.
result lamBERT model achieved higher rewards in multitask and transfer settings.
MAC Net is a compositional attention network designed for Visual Question Answering. We propose a modified MAC net architecture for Natural Language Question Answering. Question Answering typically requires Language Understanding and multi-step Reasoning. MAC net's unique architecture - the separation between memory an…
Genomic models learn DNA sequences to predict functions.
problem Understanding complex genetic interactions.
method Training LLMs on DNA sequences to predict functions.
result gLMs can predict functions of DNA elements.
Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languageq-bio.BM
Enhances drug discovery models by understanding human language.
problem Low predictive quality of activity prediction models in drug discovery.
method Proposes a novel architecture with separate chemical and natural language input modules and a contrastive pre-training objective.
result Improves predictive performance on few-shot and zero-shot learning benchmarks.
Language models learn from training data and can leak private information.
problem Language models lack context understanding and can expose private data.
method Discussing the limitations of current privacy protection methods for language models.
result Existing privacy protection methods are insufficient for language models.
A benchmark for NLP models trained on text datasets.
problem Limited access to high-performance clusters for NAS experiments.
method Created a search space for recurrent neural networks on text datasets and trained 14k architectures.
result Demonstrated the potential of precomputed NAS results for NLP.
iPrompt uses LLMs to generate natural-language explanations of data patterns.
problem Finding and explaining patterns in data using natural language.
method Interpretable autoprompting (iPrompt) that generates natural-language explanations based on LLMs.
result iPrompt can accurately find and explain data patterns, improving upon human-written prompts.
Zero-shot understanding of accidents from surveillance videos using vision-language models
problem Accident understanding from surveillance videos
method Three-stage pipeline with vision-language similarity, metadata-driven multi-prompt reasoning, and entropy-gated pairwise adjudicator
result Substantial improvement in harmonic-mean score over baseline
LLMs improve financial sentiment analysis in finance.
problem Defining and measuring financial sentiment.
method Investigation of sentiment measurement methods and LLMs.
result LLMs enhance financial sentiment analysis.
To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand. Recent advances in representation learning for language make it possible to build models that acquire world knowledge f…
MuLan links music audio to natural language tags.
problem Traditional music tagging systems use rigid attributes; MuLan aims to link audio directly to natural language.
method Joint audio-text embedding model trained on 44 million music recordings and text annotations.
result MuLan's embeddings enable zero-shot functionalities and transfer learning.
Learning a natural language interface for database tables is a challenging task that involves deep language understanding and multi-step reasoning. The task is often approached by mapping natural language queries to logical forms or programs that provide the desired response when executed on the database. To our knowle…
Improved financial sentiment analysis using simple instruction tuning of LLMs.
problem Lack of accurate financial sentiment analysis by large language models.
method Instruction tuning of general-purpose LLMs with a small portion of financial sentiment data.
result Significant improvement in financial sentiment analysis, especially in complex scenarios.
Paper introduces a specialized text classification system for French Open Banking transactions.
problem Classifying specialized banking text data with high accuracy and efficiency.
method Data collection, labeling, preprocessing, modeling, and evaluation stages with language-specific techniques.
result Enhanced performance and efficiency compared to generic approaches.
This paper explores NLP techniques for insurance, detailing methods and applications.
problem Extracting value from insurance reports using complex text data.
method Detailed explanation of NLP methods and their implementation in insurance.
result Enhanced risk monitoring and policyholder benefits through NLP.
Sentiment classification is an important process in understanding people's perception towards a product, service, or topic. Many natural language processing models have been proposed to solve the sentiment classification problem. However, most of them have focused on binary sentiment classification. In this paper, we u…
We create interpretable word embeddings through sparse coding.
problem Difficult to interpret word embeddings in natural language processing.
method Transform pretrained dense word embeddings into sparse embeddings through sparse coding.
result Sparse embeddings are more interpretable and achieve good performance.
Researchers analyze the structure of token spaces in large language models.
problem Understanding the geometric and topological structure of token spaces in large language models.
method Developed estimators for the dimension and Ricci scalar curvature of the token subspace and applied them to three large language models.
result The token subspace is not a manifold but a stratified manifold with significantly negative Ricci curvature.
With online calendar services gaining popularity worldwide, calendar data has become one of the richest context sources for understanding human behavior. However, event scheduling is still time-consuming even with the development of online calendars. Although machine learning based event scheduling models have automate…
End-to-end KBQA system learns from multiple reasoning paths without labeled paths.
problem Lack of labeled reasoning paths limits KBQA system performance.
method End-to-end KBQA system using multiple reasoning paths.
result Demonstrates strong performance on various KBQA datasets.
Simple model outperforms neural networks on language understanding tasks.
problem Neural networks struggle with creating novel expressions from familiar ones.
method Attention-inspired modification of a baseline model, focusing on sequential thinking and acting.
result Simple model achieves good performance on gSCAN tasks, validating the benchmark.
Improved text-to-image and multimodal understanding through adaptive generation order optimization.
problem Determining optimal generation sequences in text-to-image synthesis and multimodal understanding.
method Introduced a learnable control module trained via Group Relative Policy Optimization (GRPO) to determine the generation order.
result Learning the control block substantially improves text-to-image alignment and multimodal understanding in DLMs.
Many Machine Reading and Natural Language Understanding tasks require reading supporting text in order to answer questions. For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as question…
Generates biomedical abstracts from titles, years, and keywords.
problem Difficulties in understanding biomedical research papers due to specialized language.
method Conditional transformer-based language model with metadata conditioning.
result Generated abstracts are more relevant and non-trivial than GPT-2.
Many tasks in natural language understanding require learning relationships between two sequences for various tasks such as natural language inference, paraphrasing and entailment. These aforementioned tasks are similar in nature, yet they are often modeled individually. Knowledge transfer can be effective for closely …
Automatic summarization of natural language is a current topic in computer science research and industry, studied for decades because of its usefulness across multiple domains. For example, summarization is necessary to create reviews such as this one. Research and applications have achieved some success in extractive …
As modern deep networks become more complex, and get closer to human-like capabilities in certain domains, the question arises of how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language pr…
Sequence models assign probabilities to variable-length sequences such as natural language texts. The ability of sequence models to capture temporal dependence can be characterized by the temporal scaling of correlation and mutual information. In this paper, we study the mutual information of recurrent neural networks …
This study analyzes how RNNs process context in sentiment analysis.
problem Understanding how recurrent neural networks process context in sentiment analysis.
method Developed methods to reverse engineer RNNs, identifying contextual effects and quantifying their strength and timescale.
result Identified inputs that induce contextual effects and quantified their properties.
Paper uses NLP and IRT to score ESG factors from news articles.
problem Lack of precise ESG metrics in finance.
method Combines NLP and IRT models on ESG-related news data.
result Method offers more precise ESG metrics with temporal dynamics.
Analyzes quantitative finance papers from arXiv using text mining and NLP.
problem Understanding trends and insights in quantitative finance research.
method Text mining, natural language processing, topic modeling.
result Identified most cited researchers and journals in quantitative finance.
Program comprehension is a fundamental task in software development and maintenance processes. Software developers often need to understand a large amount of existing code before they can develop new features or fix bugs in existing programs. Being able to process programming language code automatically and provide sum…
Study shows tweets about COVID-19 can predict stock market performance.
problem Understanding the impact of COVID-19 on stock markets.
method Text sentiment analysis of Twitter data to correlate tweets about COVID-19 with stock market performance.
result Strong relationship between COVID-19 sentiment and stock market performance can be predicted.
Paper proposes a deep learning model for understanding e-commerce addresses.
problem Challenges in parsing shipping addresses with no fixed format.
method Combines NLP techniques with pre-processing steps for addresses, uses RoBERTa for vector representations.
result RoBERTa model achieves 90% accuracy in sub-region classification for North and South Indian cities.
Recently, pre-trained language representation flourishes as the mainstay of the natural language understanding community, e.g., BERT. These pre-trained language representations can create state-of-the-art results on a wide range of downstream tasks. Along with continuous significant performance improvement, the size an…
FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasetscs.CL
This paper benchmarks FinGPT for financial datasets using open-source large language models.
problem Challenges in integrating GPT-based models with financial datasets.
method Instruction Tuning paradigm for open-source large language models adapted for financial contexts.
result Demonstrates the effectiveness and adaptability of FinGPT in financial tasks.