LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
Improved robot navigation using multi-head attention for natural language instructions.
problem Improving robot navigation in unfamiliar environments.
method Proposes a multi-head attention mechanism blending layer in a neural network model.
result Significant performance gains in translating instructions for unseen environments.
Interactive Fiction games are text-based simulations in which an agent interacts with the world purely through natural language. They are ideal environments for studying how to extend reinforcement learning agents to meet the challenges of natural language understanding, partial observability, and action generation in …
iPrompt uses LLMs to generate natural-language explanations of data patterns.
problem Finding and explaining patterns in data using natural language.
method Interpretable autoprompting (iPrompt) that generates natural-language explanations based on LLMs.
result iPrompt can accurately find and explain data patterns, improving upon human-written prompts.
New model shows natural language exhibits phase transition similar to physics.
problem Understanding critical properties in natural language models.
method Created a context-sensitive random language model.
result Demonstrated a Berezinskii--Kosterlitz--Thouless phase transition.
Paper introduces TrufLL for language model training without labeled data.
problem Training language models from scratch with reinforcement learning.
method Dynamic vocabulary truncation in RL for language generation.
result TrufLL successfully learns language generation policies from scratch.
LLMs are compared to Markov chains for natural language processing.
problem Theoretical analysis of LLMs' generalization capabilities.
method Equivalence between LLMs and Markov chains, studying multi-step inference.
result Derives generalization bounds for LLMs, capturing their behavior in practice.
Language models learn from training data and can leak private information.
problem Language models lack context understanding and can expose private data.
method Discussing the limitations of current privacy protection methods for language models.
result Existing privacy protection methods are insufficient for language models.
Linking human whole-body motion and natural language is of great interest for the generation of semantic representations of observed human behaviors as well as for the generation of robot behaviors based on natural language input. While there has been a large body of research in this area, most approaches that exist to…
This work investigates uncertainty quantification for black-box large language models in natural language generation.
problem Lack of trustworthiness in responses generated by black-box large language models.
method Differentiated uncertainty vs confidence, proposed and compared several confidence/uncertainty measures, applied to selective NLG.
result A simple measure for semantic dispersion can predict the quality of LLM responses.
Generates natural product-like compounds using GPT models.
problem Challenges in generating and evaluating natural product-like compounds.
method Trained GPT-based chemical language models on natural product dataset.
result Generated compounds have similar distribution to natural products.
Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far …
Dynamic meta-learning improves multi-agent communication with natural language.
problem Static population-based approaches struggle with adapting to human partners using natural language.
method Dynamic population-based meta-learning approach.
result Agents outperform all prior work in communicating with seen partners and humans.
This is a lecture note for the course DS-GA 3001 <Natural Language Understanding with Distributed Representation> at the Center for Data Science , New York University in Fall, 2015. As the name of the course suggests, this lecture note introduces readers to a neural network based approach to natural language understand…
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
Optimus pre-trains sentences in a latent space for various NLP tasks.
problem Training large-scale language models for diverse NLP tasks.
method Pre-trained Variational Autoencoder (VAE) on large text corpus, fine-tuned for various tasks.
result Optimus achieves state-of-the-art on VAE language modeling benchmarks.
Paper reviews neurolinguistics and language technologies, emphasizing mutual enrichment.
problem Understanding brain activity during language processing.
method Brain imaging studies and natural language representations.
result Development of brain-aware natural language representations.
This research explores discrete diffusion models for natural language generation.
problem Challenges in applying diffusion models to discrete data, especially natural language.
method Investigates Discrete Denoising Diffusion Probabilistic Model (D3PM) and compares it with autoregressive models.
result Discrete diffusion models achieve better processing speed than autoregressive models.
New method boosts performance of diffusion models on discrete data like natural language.
problem Performance of diffusion models on discrete data like natural language is poor.
method Proposes score entropy, a novel loss that extends score matching to discrete spaces.
result Significantly boosts performance on language modeling tasks.
PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.
problem Sparse reward settings in RL limit applicability to complex problems.
method Directly maps natural language descriptions to pixel-based rewards for guiding RL.
result Language-based rewards significantly improve sample efficiency in policy learning.
Automates translating natural language to Verilog for digital design.
problem Manual translation of natural language specifications to Verilog is time-consuming and error-prone.
method Fine-tuned GPT-2 to derive Verilog from English, using a dataset of design tasks.
result GPT-2 achieved 94.8% correct translation across simple and abstract design tasks.
Study quantifies gender bias in language models across 7 languages.
problem Measuring gender bias in language models across multiple languages.
method Curated dataset of politicians, multilingual language models, probing language models.
result Larger language models do not show significant gender bias compared to smaller ones.
MuLan links music audio to natural language tags.
problem Traditional music tagging systems use rigid attributes; MuLan aims to link audio directly to natural language.
method Joint audio-text embedding model trained on 44 million music recordings and text annotations.
result MuLan's embeddings enable zero-shot functionalities and transfer learning.
Generates biomedical abstracts from titles, years, and keywords.
problem Difficulties in understanding biomedical research papers due to specialized language.
method Conditional transformer-based language model with metadata conditioning.
result Generated abstracts are more relevant and non-trivial than GPT-2.
Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten ta…
Semantic code search is the task of retrieving relevant code given a natural language query. While related to other information retrieval tasks, it requires bridging the gap between the language used in code (often abbreviated and highly technical) and natural language more suitable to describe vague concepts and ideas…
The study enhances financial rule matching using NLP without datasets.
problem Performing semantic matching between financial rules and policies.
method Outperforming pre-trained models with NLP techniques using free resources.
result Improved semantic matching between financial rules and policies.
This study applies neural models to automatically recognize medical entities from natural language.
problem Automated recognition of medical entities from natural language is complex and time-consuming.
method Utilizes deep neural sequence models trained on a large dataset of death certificates.
result Deep neural models can efficiently recognize medical entities from natural language.
APo-VAE generates text in hyperbolic space for better hierarchical representation.
problem Lack of hierarchical structure in Euclidean embeddings for natural language.
method Adversarial Poincare Variational Autoencoder (APo-VAE) in hyperbolic latent space.
result APo-VAE outperforms Euclidean VAEs in capturing latent language hierarchies.
Recent advances in multi-modal vision and language tasks enable a new set of applications. In this paper, we consider the task of generating natural language fashion feedback on outfit images. We collect a unique dataset, which contains outfit images and corresponding positive and constructive fashion feedback. We trea…
Associating image regions with text queries has been recently explored as a new way to bridge visual and linguistic representations. A few pioneering approaches have been proposed based on recurrent neural language models trained generatively (e.g., generating captions), but achieving somewhat limited localization accu…
The paper proves impossibilities and positive results for universal machine translation.
problem Learning shared sentence representations across multiple language pairs.
method Formal proofs and analysis of natural generative processes.
result Lower bound on translation error and positive results under natural structure.
We conduct a large-scale, systematic study to evaluate the existing evaluation methods for natural language generation in the context of generating online product reviews. We compare human-based evaluators with a variety of automated evaluation procedures, including discriminative evaluators that measure how well machi…
In this work we present a technique to use natural language to help reinforcement learning generalize to unseen environments. This technique uses neural machine translation, specifically the use of encoder-decoder networks, to learn associations between natural language behavior descriptions and state-action informatio…
A new model tackles language generation issues by using discrete variational attention.
problem Information under-representation and posterior collapse in variational autoencoders.
method Proposes a discrete variational attention model with categorical distribution over attention mechanism.
result Enhances latent space for language generation and avoids posterior collapse.
Deep Sets improve reinforcement learning agent's object-centered navigation and generalization.
problem Improving reinforcement learning agents' ability to generalize to unseen objects and goals.
method Combining object-wise permutation invariant networks (Deep Sets) and gated-attention mechanisms.
result Agent demonstrates strong generalization to out-of-distribution goals in a procedurally-generated 2D world.
Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (vision) and prosodic cues (acoustic). Understanding humor from these three modalities falls within boundaries of multimodal language; a recent r…
MAC Net is a compositional attention network designed for Visual Question Answering. We propose a modified MAC net architecture for Natural Language Question Answering. Question Answering typically requires Language Understanding and multi-step Reasoning. MAC net's unique architecture - the separation between memory an…
We present a method of generating high resolution 3D shapes from natural language descriptions. To achieve this goal, we propose two steps that generating low resolution shapes which roughly reflect texts and generating high resolution shapes which reflect the detail of texts. In a previous paper, the authors have show…
NLP techniques improve drug discovery by analyzing chemical and protein text.
problem Improving drug discovery through better analysis of chemical and protein text.
method Natural language processing techniques applied to biochemical entities.
result Enhanced prediction of molecular properties and design of novel molecules.
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the …
Study reveals neural scaling laws in random graphs and natural language models.
problem Understanding the origin of neural scaling laws in complex systems.
method Examined scaling laws in transformers trained on random walks and simplified natural language models.
result Neural scaling laws emerge in the absence of power law structure in data correlations.
A method for disentangling text representations without supervision.
problem Challenges in learning disentangled representations of natural language.
method Information-theoretic guidance to induce independent style and content embeddings.
result High quality disentangled representations in terms of content and style preservation.
Seq2Seq models perform well in generating If-Then programs from natural language.
problem Creating If-Then programs for business process automation without technical expertise.
method Modeling If-Then programs as a sequence learning task using Seq2Seq approaches.
result Seq2Seq models can effectively generate If-Then programs from natural language.
Learning a natural language interface for database tables is a challenging task that involves deep language understanding and multi-step reasoning. The task is often approached by mapping natural language queries to logical forms or programs that provide the desired response when executed on the database. To our knowle…
ExpBERT uses natural language explanations to improve text interpretation.
problem Improving text interpretation for relation extraction tasks.
method Fine-tuning BERT on MultiNLI to interpret natural language explanations.
result ExpBERT matches a BERT baseline but requires less labeled data and improves performance.
Natural language processing often involves computations with semantic or syntactic graphs to facilitate sophisticated reasoning based on structural relationships. While convolution kernels provide a powerful tool for comparing graph structure based on node (word) level relationships, they are difficult to customize and…