Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

3672108144 · Jun 202019922001200920172026
48 results for open-ended question answering

A new framework evaluates LLM calibration in open-ended QA.

problem Evaluating LLM calibration in open-ended QA settings.
method Sem-ECE framework: sampling answers, grouping by semantic classes, and using frequencies as confidence.
result Sem-ECE estimators are unbiased and Sem2_2 achieves smaller calibration error.

Quriosity analyzes curiosity-driven questions from diverse sources.

problem Understanding and analyzing human curiosity-driven questions.
method Collection of 13.5K questions from various sources, development of an iterative prompt improvement framework.
result 42% of questions are causal, revealing unique linguistic properties.

FSPO optimizes synthetic preferences for LLM personalization.

problem Personalizing large language models for diverse users.
method FSPO reframes reward modeling as a meta-learning problem, using few labeled preferences and synthetic data.
result FSPO achieves high winrates in personalized responses, both synthetic and real.

PerSense assesses personality traits from text for commonsense reasoning.

problem Estimating human personality traits from text for mental health analysis.
method Aggregated Probability Density Functions (PDF) and Machine Learning (ML) models.
result PerSense algorithms achieve comparable results to ground truth data, with high accuracy for personality assessment and commonsense prediction.

A new approach reduces the size of prediction sets in CP, making it more efficient.

problem Large and costly prediction sets in standard CP.
method Expand correctness criterion to include inferred admissible answers, and use prediction cascades to prune implausible labels.
result Empirically effective in natural language processing and drug discovery.

CausalEvolve improves efficiency and discovery in open-ended scientific tasks.

problem Lack of targeted guidance and knowledge organization in evolve-based agents.
method Causal scratchpad that identifies and reasons about guiding factors for evolution.
result Effective improvement in evolutionary efficiency and better solutions.

Paper analyzes biases in video QA datasets, showing models can answer 37-48% questions correctly without multimodal context.

problem Question answering biases in video QA datasets can lead to model overfitting and poor generalization.
method Analyzed popular video question answering datasets, conducted ablation studies on biases from annotators and question types.
result Pretrained language models can answer 37-48% questions correctly without multimodal context, far exceeding random guess baseline.

Paper reviews intrinsic motivations and their role in open-ended learning.

problem Understanding intrinsic motivations and their role in open-ended learning.
method Defining intrinsic motivations, presenting psychological/neuroscientific and computational models.
result Links between psychological/neuroscientific and computational models of intrinsic motivations.

A novel method for visual question answering using scene graphs and reinforcement learning.

problem Answering free-form questions about images with deep linguistic and visual understanding.
method Context-driven, sequential reasoning based on scene graphs and reinforcement learning.
result Our method almost reaches human performance on the GQA dataset.

Autonomous multiple tasks learning is a fundamental capability to develop versatile artificial agents that can act in complex environments. In real-world scenarios, tasks may be interrelated (or "hierarchical") so that a robot has to first learn to achieve some of them to set the preconditions for learning other ones. …

2019-06-04abs ↗pdf ↗

GSR optimizes tasks in scientific workflows, improving performance across diverse applications.

problem Uncertainty in task selection and evaluation in scientific workflow optimization.
method Generate-Select-Refine (GSR) framework that alternates between task generation and optimization.
result GSR outperforms existing LLM-based optimizers in various scientific applications.

We introduce the new task of Acoustic Question Answering (AQA) to promote research in acoustic reasoning. The AQA task consists of analyzing an acoustic scene composed by a combination of elementary sounds and answering questions that relate the position and properties of these sounds. The kind of relational questions …

2019-02-28abs ↗pdf ↗

AI tested on 10 math questions from research.

problem Assessing AI's ability to solve research-level math problems.
method Shared 10 math questions not previously publicly available.
result Answers to questions are known to authors but encrypted.

FinanceBench benchmarks LLMs on financial QA, revealing limitations.

problem Evaluating LLMs' performance on financial question answering.
method Developed a comprehensive test suite (FinanceBench) with 10,231 questions, tested 16 models, and manually reviewed answers.
result Existing LLMs have significant limitations for financial QA, especially GPT-4-Turbo.

Proposes a methodology to improve data science ROI by addressing key business questions.

problem Companies often fail to maximize data science value, focusing on basic analysis.
method Categorizes and answers 'The Big Three' questions using data science methods.
result Shows how to apply the methodology to real business use cases.

Model improves BERT for answering multiple-choice questions in large texts.

problem Improving machine comprehension of large text corpora for question answering.
method Developed a model using BERT with a semantic similarity attention layer to extract key sentences.
result Outperforms leading models in MovieQA challenge with 87.79% test accuracy.

Introduces Relational Privacy (RP) to control relation memorization in question answering models.

problem Relation memorization in question answering models can lead to privacy issues.
method Formalizes Relational Privacy (RP) and Differential Relational Privacy (DrP), providing bounds on relation memorization.
result DrP allows effective learning of general properties of underlying concepts while preventing relation memorization.

The emergence of structured databases for Question Answering (QA) systems has led to developing methods, in which the problem of learning the correct answer efficiently is based on a linking task between the constituents of the question and the corresponding entries in the database. As a result, parsing the questions i…

2019-09-27abs ↗pdf ↗

The rate at which medical questions are asked online far exceeds the capacity of qualified people to answer them, and many of these questions are not unique. Identifying same-question pairs could enable questions to be answered more effectively. While many research efforts have focused on the problem of general questio…

2019-10-09abs ↗pdf ↗

Study evaluates how well question-answering models generalize to new data types.

problem Generalization of question-answering models to new data types.
method Constructed new test sets from different domains and evaluated models' performance.
result Models show significant performance drops when tested on new data types.

Knowledge graph based simple question answering (KBSQA) is a major area of research within question answering. Although only dealing with simple questions, i.e., questions that can be answered through a single knowledge base (KB) fact, this task is neither simple nor close to being solved. Targeting on the two main ste…

2019-04-04abs ↗pdf ↗

Study examines how decoding algorithms affect fairness in language generation models.

problem Impact of decoding algorithms on fairness in open-ended language generation.
method Systematic analysis of top-pp, top-kk, and temperature decoding algorithms.
result Decoding algorithms significantly impact fairness across demographic groups.

SpanishTinyRoBERTa distills large Spanish models into efficient question-answering models.

problem Efficient Spanish question-answering models for resource-constrained environments.
method Knowledge distillation from large Spanish language models onto a smaller model.
result SpanishTinyRoBERTa achieves comparable performance to large models with faster inference.

Machine Reading Comprehension (MRC) is an important topic in the domain of automated question answering and in natural language processing more generally. Since the release of the SQuAD 1.1 and SQuAD 2 datasets, progress in the field has been particularly significant, with current state-of-the-art models now exhibiting…

2019-06-24abs ↗pdf ↗