arXiv research

This paper proposes a new method to connect language and physical actions in reinforcement learning.

Model learns image-word associations from captions using contrastive learning.

Proposes a method to align language and image data.

New approach decouples skill learning and language grounding for autonomous agents.

New method improves RL/IL agents' adaptability to unseen environments.

Unified model learns concepts across domains like left and right.

We are increasingly surrounded by artificially intelligent technology that takes decisions and executes actions on our behalf. This creates a pressing need for general means to communicate with, instruct and guide artificial agents, with human language the most compelling means for such communication. To achieve this i…

Developing an AI economist agent using RAG, knowledge graphs, and LLMs for economic scenario analysis.

Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward function…

Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasoning, e.g., they often contain sentences deprived of world knowledge. We propose a simple multi-task learning scheme to achieve quantitatively …

FUSE improves verification quality without ground truth labels.

Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence. Ideally, these tasks should test a plethora of capabilities that integrate computer vision, reasoning, and natural language understanding. However, rather than behaving as visual Turing t…

The study categorizes reward errors in reinforcement learning, finding some can be beneficial.

Autonomous reinforcement learning agents, like children, do not have access to predefined goals and reward functions. They must discover potential goals, learn their own reward functions and engage in their own learning trajectory. Children, however, benefit from exposure to language, helping to organize and mediate th…

We build deep RL agents that execute declarative programs expressed in formal language. The agents learn to ground the terms in this language in their environment, and can generalize their behavior at test time to execute new programs that refer to objects that were not referenced during training. The agents develop di…

Simple model outperforms neural networks on language understanding tasks.

A new framework evaluates LLMs by considering judge reliability.

StakeBench evaluates language understanding by linking comments to market commitments, improving model alignment with real-world outcomes.

Cosmos models scenes using neural encodings and symbolic attributes for compositional generalization.

VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation…

Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dropout shown to fail when applied to recurrent layers. Recent results at the intersection of Bayesian modelling and deep learning offer a Bay…

We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments. The task emulates a real-world scenario in that (a) the requester may not know how to navigate to …

MoA framework improves financial LLM responses at low cost.

Language models can predict numeric values as strings.

Enhances trading metrics with financially grounded loss functions.

New method evaluates language model forecasters by checking consistency of predictions.

Proposes a method to choose thresholds for LLM evaluation metrics.

DGDS uses documents to center conversations, promising broader AI understanding.

ClauseLens uses reinforcement learning to price reinsurance treaties transparently and auditably.

This paper uses LLMs and cycle consistency for better machine translation evaluation.

Paper addresses online alignment of large language models under uncertain preference feedback.

FinBloom enhances LLMs for real-time financial queries.

LLMs fail to match statistical ground truth despite stable run-to-run performance.

We present a recurrent encoder-decoder deep neural network architecture that directly translates speech in one language into text in another. The model does not explicitly transcribe the speech into text in the source language, nor does it require supervision from the ground truth source language transcription during t…

LLMs struggle with financial reasoning but can outperform the market with human oversight.

Zero-shot understanding of accidents from surveillance videos using vision-language models

Fine-grained event tagging system for SEC 8-K filings improves precision to 96%.

Survey of large language models in financial prediction and trading.

Triangulation filters spurious circuits in multilingual models.

This paper addresses anisotropy in Transformer models, providing geometric insights and empirical support.

A method for predicting credal sets in classification tasks using conformal prediction.

FinHEAR combines LLMs with human expertise for better financial decision-making.

Model uses LLMs to process numerical data guided by natural language descriptions.

Over the last few years, there has been growing interest in learning models for physically grounded language understanding tasks, such as the popular blocks world domain. These works typically view this problem as a single-step process, in which a human operator gives an instruction and an automated agent is evaluated …

Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for different multimodal tasks, such as semantic goal navigation and embodied question answering. In this paper, we propose a multitask model capable of jointly learnin…

TFB simplifies Bayesian LLM uncertainty estimation without extra training.

FLARKO uses LLMs, KGs, and KTO to generate profitable, behaviorally aligned financial recommendations.

In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is introduced to assist the training of the sequence prediction model (the generator network). Unlike MLE di…