MathChat uses LLM agents to solve challenging math problems through conversational problem-solving.
problem Solving math problems expressed in natural language.
method MathChat is a conversational framework combining an LLM agent and a user proxy agent for collaborative problem-solving.
result MathChat improves tool-using prompting methods by 6% on difficult math problems.
This paper explores how LLMs can improve pipeline-based conversational agents.
problem Limitations of pipeline-based conversational agents in human-like conversations.
method Investigated LLMs' capabilities in two phases: design and development, and operations.
result LLMs can enhance pipeline-based agents in various tasks like data generation, intent classification, and auto-correction.
Paper presents LLM-enhanced contract metadata extraction.
problem Automatic detection and annotation of legal clauses in contracts.
method Integration of publicly available and proprietary datasets with advanced LLM methodologies.
result Substantial improvements in clause identification accuracy and efficiency.
Model user preferences for conversational LLMs using weak rewards.
problem Lack of persistent user models in conversational LLMs leading to repeated user restatements.
method Vector-Adapted Retrieval Scoring (VARS) framework that updates user vectors online from weak scalar rewards.
result Full VARS agent achieves strongest overall performance, matches strong Reflection baseline in task success, and reduces user effort.
Dual active learning improves RLHF by selecting optimal conversations and teachers.
problem Efficiently aligning LLMs with human preferences using RLHF from feedback.
method Offline RL for conversation and teacher selection, dual active reward learning, pessimistic RL.
result The proposed algorithm achieves minimal generalized variance and outperforms state-of-the-arts.
TraderTalk uses LLMs to simulate human trading interactions in financial markets.
problem Simulating realistic human trading interactions in financial markets.
method Hybrid ABM with LLM-generated behaviors for detailed conversations.
result Successfully replicates trade-to-order volume ratios in financial markets.
BED-LLM uses Bayesian experimental design to improve LLMs' information gathering.
problem Improving LLMs' ability to gather information adaptively.
method Iteratively choosing questions to maximize expected information gain using a probabilistic model.
result BED-LLM achieves substantial performance gains compared to other adaptive design strategies.
JoLT uses LLMs to make probabilistic predictions on tabular data.
problem Making probabilistic predictions on tabular data efficiently and without preprocessing.
method JoLT leverages LLMs' in-context learning to define joint distributions over tabular data.
result JoLT outperforms other methods on tabular classification and regression tasks.
Study proposes a multi-agent framework to mitigate bias in sentiment analysis.
problem Bias in sentiment analysis models.
method Integrates multiple LLMs, incorporates dialogue sessions, and uses probabilistic prediction.
result KCS+IBC reduces entropy and increases variance, suggesting improved balance between aggregation and diversity.
LoLCATs improves linearized LLM quality with less memory and compute.
problem Linearizing large language models (LLMs) often degrades model quality and requires expensive training.
method Two-step method: attention transfer and low-rank adaptation.
result Significant improvement in linearizing quality with 20+ points on 5-shot MMLU.
Study shows market volatility affects optimal communication design for trading strategies.
problem Investigating how communication impacts trading strategy performance in multi-agent systems.
method 5-agent LLM-based trading systems across 450 experiments spanning 21 months, comparing 5 organizational structures.
result Communication improves performance but depends on market characteristics, with competitive conversation excelling in volatile tech stocks.
Statistical framework improves LLM chatbot ranking.
problem Improving evaluation of LLM-based chatbots through pairwise comparisons.
method Factored tie model, covariance modeling, and parameter constraints.
result Substantial improvements in modeling pairwise comparison data.
AI bias arises from human-defined goals, not algorithmic flaws.
problem AI bias due to human-defined goals in LLMs.
method Purpose-conditioned cognition and revealing downstream use of LLM outputs.
result AI bias can be reduced by purpose-aware prompting but not fully by regularization.
Study shows LLM-advisors match human performance in eliciting preferences but struggle with conflicting needs and trust.
problem How do LLM-advisors perform in complex financial domains where domain expertise is crucial?
method Lab-based user study with 64 participants, focusing on three challenges: preference elicitation, personalized guidance, and relationship building.
result LLM-advisors can match human performance in preference elicitation but struggle with conflicting needs and trust issues.
UQE uses LLMs to analyze unstructured data efficiently.
problem Efficient analytics on unstructured data.
method Proposes UQE, a query engine that uses LLMs to interpret UQL queries.
result Demonstrates efficient analytics on various unstructured data types.
DUET optimizes LLM training data mixtures using feedback from unseen tasks.
problem Optimizing LLM performance on unseen tasks without prior data knowledge.
method Interleaves influence function with Bayesian optimization for data selection.
result DUET converges to optimal training data mixture for unseen tasks.
MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.
problem Dequantization bottleneck in LLM inference on modern AI accelerators.
method MSD decomposes high-precision activations into multiple low-precision components for direct multiplication with quantized weights.
result MSD avoids INT8-to-BF16 weight conversion, reducing dequantization cycles and HBM traffic.
LATM framework uses LLMs to create and reuse tools for efficient problem-solving.
problem Improving problem-solving capabilities of LLMs with external tools.
method Closed-loop framework with two phases: tool making and tool using. LLMs act as both tool makers and users. Resource-intensive model for tool making, lightweight model for tool using.
result LATM framework achieves performance equivalent to using a powerful LLM for both roles but with significantly reduced costs.
FinDPO uses preference optimization to improve financial sentiment analysis models.
problem Financial sentiment analysis models often fail to generalize to unseen data.
method FinDPO uses Direct Preference Optimization (DPO) to align LLMs with human preferences.
result FinDPO achieves state-of-the-art performance and maintains positive returns under realistic trading conditions.
Work shows hallucination detection by LLMs is impossible without expert feedback.
problem Detecting hallucinations in LLMs is theoretically impossible without expert-labeled feedback.
method Investigated hallucination detection using a theoretical framework inspired by language identification.
result Automated hallucination detection is impossible for most language collections without expert-labeled feedback.
Conf-Gen applies uncertainty quantification to generative models.
problem Uncertainty quantification for unsupervised generative models.
method Adapts CRC to generative tasks, relaxing theoretical assumptions.
result Demonstrates flexibility and correctness of AI agent outputs.
C-IP improves LLMs' query selection for interactive tasks by estimating uncertainty robustly.
problem Minimizing the number of queries for interactive LLMs.
method Conformal Information Pursuit (C-IP) using conformal prediction sets.
result C-IP achieves better predictive performance and shorter query-answer chains.
Quriosity analyzes curiosity-driven questions from diverse sources.
problem Understanding and analyzing human curiosity-driven questions.
method Collection of 13.5K questions from various sources, development of an iterative prompt improvement framework.
result 42% of questions are causal, revealing unique linguistic properties.
Fine-tuning LLMs with observational data can lead to spurious correlations, but DeconfoundLM can mitigate this.
problem Aligning LLMs with human preferences and business objectives using observational data.
method DeconfoundLM, a method that removes confounders from reward signals.
result DeconfoundLM improves recovery of causal relationships and mitigates spurious correlations.
Enhances anomaly detection in financial markets using AI agents.
problem Manual verification of financial market anomalies is time-consuming and error-prone.
method A multi-agent LLM framework for automated anomaly detection.
result Framework reduces human intervention and improves efficiency and accuracy.
Proposes an iterative algorithm for optimizing attention mechanisms in large language models.
problem Optimizing attention mechanisms in large language models.
method Iterative algorithm for rescaled hyperbolic functions regression.
result Efficiency and generalizability of the rescaled softmax regression framework.
Traditional methods outperform LLMs in forecasting corporate credit ratings.
problem Forecasting corporate credit ratings using LLMs.
method Comparison of traditional methods (XGBoost) and LLMs (LLaMA) on credit rating forecasting.
result XGBoost outperforms LLMs in forecasting corporate credit ratings.
Unified framework to bridge human and LLM judgments.
problem Systematic discrepancies between human and LLM evaluations.
method Latent human preference score and linear transformations of covariates.
result Higher agreement with human ratings and exposure of systematic gaps.
LLM-Lasso uses LLMs to improve feature selection in Lasso regression.
problem Improving feature selection in Lasso regression with domain-specific knowledge.
method Combines LLMs with Lasso regularization to generate feature weights.
result Outperforms standard Lasso and feature selection baselines in biomedical studies.
THERMOMETER calibrates LLMs efficiently for diverse tasks.
problem Calibrating large language models is challenging due to computational and versatility issues.
method THERMOMETER learns an auxiliary model for calibrating a LLM using data from multiple tasks.
result THERMOMETER produces better-calibrated responses for new tasks.
Regurgitative training with synthetic data harms LLM performance.
problem The impact of training LLMs with synthetic data generated by other LLMs.
method Fine-tuning GPT-3.5 with synthetic and real data in a machine translation task.
result Regurgitative training significantly reduces LLM performance.
FLUID-LLM uses LLMs to predict fluid dynamics with improved accuracy.
problem Leveraging LLMs for CFD due to their pattern recognition abilities but struggles with fluid dynamics complexities.
method Combines pre-trained LLMs with spatiotemporal-aware encoding to predict unsteady fluid dynamics.
result Significant performance improvements in CFD predictions across various datasets.
This paper improves continuous adversarial training for LLMs using in-context learning theory.
problem Efficiently defending large language models (LLMs) against jailbreak attacks.
method The paper presents a theoretical analysis of continuous adversarial training (CAT) for LLMs based on in-context learning (ICL) theory, proving a robust generalization bound and proposing an improved regularization term.
result The robust generalization bound explains why CAT can defend against jailbreak prompts and shows that LLM robustness is related to embedding matrix singular values.
LLMs mimic human traders in finance, but not as much as expected.
problem Evaluating how LLMs behave in financial markets.
method Adapted experimental design with LLMs and human traders, analyzed in single and mixed model settings.
result LLMs tend to price assets near their fundamental value, but not as much as humans, and show less trading strategy variance.
Paper presents a probabilistic model to improve LLM cascade performance.
problem Complexity of LLM cascades and their interaction error rates.
method Probabilistic model for joint performance distribution of LLMs.
result Improves area under the error-cost curve by 4.3% on average for cascades with k≥3 models.
LLMs struggle to optimize hyperparameters efficiently, but hybrid methods can improve performance.
problem Optimizing hyperparameters of small language models using LLMs.
method Comparison of classical HPO algorithms and LLM-based methods, introducing Centaur hybrid approach.
result Hybrid Centaur approach achieves best results, outperforming classical and pure LLM methods.
Improves feature selection in high-dimensional data using LLM-generated weights.
problem Inaccurate LLM-generated weights degrade feature selection performance.
method Integrates LLM-generated weights into prior inclusion probabilities using LLM Sparsity Prior (LSP).
result Improves prediction accuracy and identifies clinically relevant features.
Telescope detects LLM generated text by measuring token repetition probability.
problem Distinguishing LLM generated text from human writing.
method Telescope Perplexity, evaluating token repetition probability.
result Telescope Perplexity enables effective zero-shot LLM detection.
New benchmarks show LLMs struggle with causal discovery.
problem Leveraging LLMs for causal discovery is unreliable due to dataset leakage.
method Developing science-grounded benchmarks and hybrid methods combining LLM predictions with statistical analysis.
result LLMs perform poorly on novel, real-world scientific studies compared to classical methods.
Tiny benchmarks reduce LLM evaluation costs by using fewer examples.
problem Expensive evaluation of LLMs with tens of thousands of examples.
method Developed evaluation tools and tiny versions of popular benchmarks.
result Accurately estimate LLM performance with just 100 curated examples.
SelfReflect measures LLM uncertainty by summarizing belief distribution.
problem LLMs lack transparency in expressing uncertainty.
method Developed SelfReflect metric to quantify uncertainty.
result Modern LLMs cannot reflect their uncertainty.
LLMs overestimate stock returns and are less accurate at predicting extreme outcomes.
problem Behavioral biases in LLMs' stock return forecasts.
method Comparison of LLM forecasts with crowd-sourced estimates and historical data.
result LLMs overestimate stock returns and are less accurate at predicting extreme outcomes.
Survey of LLMs in finance tasks, including adoption and performance.
problem Utilizing large language models in financial tasks.
method Review of current approaches, decision framework for adoption.
result Synthesizes state-of-the-art for LLMs in finance.
Statsformer validates and adapts LLM-derived semantic priors for improved supervised learning.
problem Unreliable semantic priors from LLMs can degrade supervised learning performance.
method Adapts LLM-derived feature scores into a family of learner-specific prior-injection mechanisms, calibrating their influence using out-of-fold validation.
result Improves prediction performance by adaptively downweighting unreliable LLM priors, ensuring a guardrailed statistical learning system.
StealthRank subtly boosts LLM rankings without detectable anomalies.
problem Adversarial manipulation of LLM-driven ranking systems.
method Energy-based optimization with Langevin dynamics for stealthy prompt generation.
result StealthRank outperforms existing methods in covertly boosting rankings.
No-knowledge alarms detect misaligned LLM judges without trusting them.
problem Monitoring LLM judges for consistency when ground truth is unknown.
method Formal logic and Linear Programming to detect disagreements.
result No false positives in detecting misaligned judges.
Fine-tunes LLMs to correct bias in predictions.
problem LLMs exhibit bias in predictions from data.
method Supervised fine-tuning with Low-Rank Adaptation (LoRA).
result Fine-tuning corrects bias in both controlled and real-world settings.
AlphaPruning optimizes LLM pruning using HT-SR theory for better performance.
problem Improving pruning of large language models to reduce size without sacrificing performance.
method AlphaPruning uses HT-SR theory to allocate layerwise sparsity ratios more theoretically.
result AlphaPruning prunes LLaMA-7B to 80% sparsity with reasonable perplexity.