The study enhances financial rule matching using NLP without datasets.
problem Performing semantic matching between financial rules and policies.
method Outperforming pre-trained models with NLP techniques using free resources.
result Improved semantic matching between financial rules and policies.
Automates translating natural language to Verilog for digital design.
problem Manual translation of natural language specifications to Verilog is time-consuming and error-prone.
method Fine-tuned GPT-2 to derive Verilog from English, using a dataset of design tasks.
result GPT-2 achieved 94.8% correct translation across simple and abstract design tasks.
LLMs are compared to Markov chains for natural language processing.
problem Theoretical analysis of LLMs' generalization capabilities.
method Equivalence between LLMs and Markov chains, studying multi-step inference.
result Derives generalization bounds for LLMs, capturing their behavior in practice.
Text-based representations of chemicals and proteins can be thought of as unstructured languages codified by humans to describe domain-specific knowledge. Advances in natural language processing (NLP) methodologies in the processing of spoken languages accelerated the application of NLP to elucidate hidden knowledge in…
This is a lecture note for the course DS-GA 3001 <Natural Language Understanding with Distributed Representation> at the Center for Data Science , New York University in Fall, 2015. As the name of the course suggests, this lecture note introduces readers to a neural network based approach to natural language understand…
Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (vision) and prosodic cues (acoustic). Understanding humor from these three modalities falls within boundaries of multimodal language; a recent r…
This study applies neural models to automatically recognize medical entities from natural language.
problem Automated recognition of medical entities from natural language is complex and time-consuming.
method Utilizes deep neural sequence models trained on a large dataset of death certificates.
result Deep neural models can efficiently recognize medical entities from natural language.
Improved robot navigation using multi-head attention for natural language instructions.
problem Improving robot navigation in unfamiliar environments.
method Proposes a multi-head attention mechanism blending layer in a neural network model.
result Significant performance gains in translating instructions for unseen environments.
Natural language processing (NLP) can be done using either top-down (theory driven) and bottom-up (data driven) approaches, which we call mechanistic and phenomenological respectively. The approaches are frequently considered to stand in opposition to each other. Examining some recent approaches in deep learning we arg…
Model uses LLMs to process numerical data guided by natural language descriptions.
problem Challenges in integrating prior knowledge into probabilistic models.
method Developed LLM Processes to condition numerical predictive distributions on natural language.
result Improved predictive performance and structured qualitative descriptions.
Study categorizes mutual funds using natural language processing from unstructured data.
problem Categorizing mutual funds using unstructured data for financial analysis.
method Used natural language processing models to classify mutual funds from their investment strategy descriptions.
result High accuracy in categorizing mutual funds using NLP from unstructured data.
Study compares LLMs vs classical models for financial sentiment analysis.
problem Improving sentiment analysis in financial market news.
method Comparative analysis of LLMs and classical models.
result LLMs outperform classical models in sentiment analysis of financial news.
LLMs translate natural language trading intents into correct option strategies using a domain-specific language.
problem Challenges in translating natural language trading intents into correct option strategies due to the complexity of option chain data.
method Introduce Option Query Language (OQL) as a domain-specific intermediate representation to abstract option markets into high-level primitives under grammatical rules. Use LLMs as semantic parsers and validate queries by an engine.
result Significantly improves execution accuracy and logical consistency over direct baselines.
ICU readmission is associated with longer hospitalization, mortality and adverse outcomes. An early recognition of ICU re-admission can help prevent patients from worse situation and lower treatment cost. As the abundance of Electronics Health Records (EHR), it is popular to design clinical decision tools with machine …
Code-switching, the alternation of languages within a conversation or utterance, is a common communicative phenomenon that occurs in multilingual communities across the world. This survey reviews computational approaches for code-switched Speech and Natural Language Processing. We motivate why processing code-switched …
This research explores discrete diffusion models for natural language generation.
problem Challenges in applying diffusion models to discrete data, especially natural language.
method Investigates Discrete Denoising Diffusion Probabilistic Model (D3PM) and compares it with autoregressive models.
result Discrete diffusion models achieve better processing speed than autoregressive models.
VALC provides concept-level interpretations of FLMs, overcoming word-level limitations.
problem Lack of higher-level structure interpretation in FLMs' attention weights.
method Formal definition of conceptual interpretation, variational Bayesian framework (VALC).
result VALC finds optimal language concepts for FLM predictions, providing concept-level interpretations.
Sentiment classification is an important process in understanding people's perception towards a product, service, or topic. Many natural language processing models have been proposed to solve the sentiment classification problem. However, most of them have focused on binary sentiment classification. In this paper, we u…
FairLangProc simplifies fairness in NLP models for Python users.
problem Addressing bias in NLP models for decision-making contexts.
method Develops a Python package for implementing fairness metrics and algorithms.
result Promotes the use of bias mitigation techniques in NLP.
This study uses NLP to detect financial risks from documents.
problem Detecting and predicting financial risks in documents.
method NLP model design, text preprocessing, feature extraction, machine learning.
result NLP model effectively identifies and predicts financial risks.
We create interpretable word embeddings through sparse coding.
problem Difficult to interpret word embeddings in natural language processing.
method Transform pretrained dense word embeddings into sparse embeddings through sparse coding.
result Sparse embeddings are more interpretable and achieve good performance.
A benchmark for NLP models trained on text datasets.
problem Limited access to high-performance clusters for NAS experiments.
method Created a search space for recurrent neural networks on text datasets and trained 14k architectures.
result Demonstrated the potential of precomputed NAS results for NLP.
We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating). These toolkits provide state-of-the-art pre-trained models, training scripts, and training logs, to facilitate rapid prototyping and promote reproducible research. We also …
Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten ta…
This paper surveys NLP techniques for predicting stock market movements.
problem Predicting stock market movements with volatility, seasonality, and time dependence.
method Survey of recent literature in NLP and ML for stock market prediction.
result Illustrates recent trends in stock market prediction research.
Natural language processing improves COVID-19 hospitalization identification.
problem Identifying patients hospitalized due to COVID-19 among those with positive SARS-CoV-2 tests.
method Used natural language processing on provider notes and structured EHR data elements to create classification algorithms.
result Classification algorithms using provider notes outperformed those using only structured EHR data elements, with AUROC of 0.894 compared to 0.841.
ETC improves Transformer models for long and structured inputs.
problem Scaling input length and encoding structured inputs in Transformers.
method Introduces global-local attention, relative position encodings, and CPC pre-training.
result Achieves state-of-the-art results on four natural language datasets.
Reduces gender bias in patient notes while maintaining medical classification accuracy.
problem Bias in natural language processing of patient notes.
method Identifying and removing gendered language using BERT-based classifiers, then augmenting data to maintain performance.
result Minimal degradation in health condition classification tasks with data augmentation.
This paper presents the beginnings of an automatic statistician, focusing on regression problems. Our system explores an open-ended space of statistical models to discover a good explanation of a data set, and then produces a detailed report with figures and natural-language text. Our approach treats unknown regression…
HuSpaCy offers an industrial-grade Hungarian NLP toolkit.
problem Lack of suitable open-source Hungarian NLP pipelines.
method Built on spaCy, HuSpaCy includes lemmatization, morphosyntactic analysis, entity recognition, and word embeddings.
result HuSpaCy achieves high accuracy with resource-efficient prediction.
The paper analyzes sports commentary to automatically recognize events and extract insights.
problem Automatically recognizing and categorizing major actions in sports events from commentary.
method Used multiple Natural Language Processing techniques for classification and sentiment analysis.
result Identified insights from analyzing live sport commentaries and classifying major actions.
Natural Language Processing (NLP) and especially natural language text analysis have seen great advances in recent times. Usage of deep learning in text processing has revolutionized the techniques for text processing and achieved remarkable results. Different deep learning architectures like CNN, LSTM, and very recent…
Chronologically consistent models maintain accuracy with time-restricted data.
problem Training data introduces lookahead bias and training leakage in large language models.
method ChronoBERT and ChronoGPT trained with only available data at each time point.
result Models achieve strong performance and competitive with larger models, mitigating lookahead bias.
Charmer improves character-level adversarial attacks for language models.
problem Efficiency and effectiveness of character-level adversarial attacks for language models.
method Query-based adversarial attack method that maintains semantic similarity.
result Charmer achieves high attack success rate and similar adversarial examples.
The goal of this research was to find a way to extend the capabilities of computers through the processing of language in a more human way, and present applications which demonstrate the power of this method. This research presents a novel approach, Rhetorical Analysis, to solving problems in Natural Language Processin…
This paper explores NLP techniques for insurance, detailing methods and applications.
problem Extracting value from insurance reports using complex text data.
method Detailed explanation of NLP methods and their implementation in insurance.
result Enhanced risk monitoring and policyholder benefits through NLP.
To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand. Recent advances in representation learning for language make it possible to build models that acquire world knowledge f…
This paper explores how NLP enhances insurance data analysis.
problem Traditional insurance data limitations and need for alternative data.
method Application of NLP techniques to transform and analyze unstructured text data.
result NLP techniques improve insurance data analysis and risk assessment.
FINCH dataset enables financial Text-to-SQL tasks, improving model evaluation.
problem Lack of a large-scale financial dataset for Text-to-SQL tasks.
method Curated financial dataset (FINCH), benchmarking reasoning and language models.
result Proposed FINCH Score for more accurate financial model evaluation.
Word vectors are at the core of many natural language processing tasks. Recently, there has been interest in post-processing word vectors to enrich their semantic information. In this paper, we introduce a novel word vector post-processing technique based on matrix conceptors (Jaeger2014), a family of regularized ident…
The paper evaluates machine learning cyber defenses using log data against adversarial attacks.
problem Evaluating the robustness of machine learning cyber defenses against adversarial attacks.
method Developed a testing framework using deep reinforcement learning and adversarial natural language processing.
result Higher dropout levels increase robustness, with 90% dropout probability showing the highest robustness.
The paper teaches machines to ask clarifying questions by applying NLI.
problem Teaching machines to ask clarifying questions for better comprehension.
method The approach is to use NLI to determine if a clarification question is about a missing piece of information in a post.
result The best model improved precision@1 by 40% and 60% over state-of-the-art baselines.
LLMs improve financial sentiment analysis in finance.
problem Defining and measuring financial sentiment.
method Investigation of sentiment measurement methods and LLMs.
result LLMs enhance financial sentiment analysis.
Survey of financial LLMs in finance.
problem Limited research on Financial LLMs (FinLLMs).
method Chronological overview of PLMs, comparison of techniques, performance evaluations, and advanced tasks.
result Compilation of accessible datasets and benchmarks for AI research in finance.
Stack Overflow is the most popular Q&A website among software developers. As a platform for knowledge sharing and acquisition, the questions posted in Stack Overflow usually contain a code snippet. Stack Overflow relies on users to properly tag the programming language of a question and it simply assumes that the progr…
Gaussian processes are conditioned on various types of data.
problem Exact inference in Gaussian processes is limited to linear-Gaussian settings.
method Established an equivalence between GPs and linear diffusion models, allowing for approximate inference in non-linear settings.
result A general-purpose GP inference scheme that handles various conditioning statements, including non-linear physics and natural language.
In natural language processing, a lot of the tasks are successfully solved with recurrent neural networks, but such models have a huge number of parameters. The majority of these parameters are often concentrated in the embedding layer, which size grows proportionally to the vocabulary length. We propose a Bayesian spa…
Detecting depression early from social media texts.
problem Early diagnosis and prevention of depression.
method Topic analysis and learned confidence scores.
result Achieved good results compared to state of the art.