LIDS assesses LLM summaries with interpretable key words.
problem Challenges in evaluating the quality of LLM summaries.
method BERT-SVD-based direction metric and SOFARI for key word extraction.
result LIDS provides interpretable key words for layered themes.
SelfReflect measures LLM uncertainty by summarizing belief distribution.
problem LLMs lack transparency in expressing uncertainty.
method Developed SelfReflect metric to quantify uncertainty.
result Modern LLMs cannot reflect their uncertainty.
Text clustering method replaces centroids with summaries for interpretability and scalability.
problem Efficiently clustering text data while maintaining interpretability and scalability.
method k-NLPmeans and k-LLMmeans, which periodically replace numeric centroids with textual summaries.
result Consistently outperforms classical baselines and recent LLM-based clustering methods.
LLMs excel at summarizing and repairing complex models without needing full models.
problem Understanding and repairing complex models like GAMs.
method Hierarchical reasoning and extensive background knowledge.
result LLMs can detect anomalies, describe reasons, and suggest repairs.
Study evaluates LLMs for predicting Chinese stock movements using financial news sentiments.
problem Evaluating LLMs' ability to predict stock price movements using financial news sentiments.
method Standardized experimental procedure with three LLMs, each with unique performance enhancement methods.
result Developed quantitative trading strategies and conducted back-tests to assess LLMs' performance.
Novel method diagnoses large language models' reasoning abilities.
problem Fine-grained evaluation of large language models' reasoning abilities.
method Adapting cognitive diagnosis models to LLMs, estimating mastery profiles and Q-matrix, incorporating textual information.
result Accurate parameter recovery and insights into LLMs' capabilities.
LLM forecasting benchmarks suffer from information leakage, which confounds model performance.
problem LLM forecasting benchmarks suffer from information leakage.
method A retrieval-augmented LLM forecaster observes only decision-time information.
result The full pipeline obtains a median monthly Spearman rank IC of +0.154.
LLMs compress financial texts, but distort decision-making.
problem LLMs compress financial texts, altering decision-making.
method Analyzed two diagnostic patterns: decontextualization and model dependency. Proposed Agentic Context Compression.
result LLM-compressed financial texts alter decision-making.
Framework evaluates AI proposals for drug discovery, finds no LLM advantage.
problem No principled framework exists for evaluating AI-guided scientific selection under budget constraints.
method Formally verified metric (BSDS/DQS) penalizes false discoveries and excessive abstention.
result LLMs provide no marginal value over existing classifiers in drug discovery.
This paper uses LLMs to improve equity stock ratings by ingesting diverse financial and news data.
problem Challenges in traditional stock rating methods, including data overload, inconsistencies, and delayed reactions.
method Application of LLMs to generate multi-horizon stock ratings using various datasets.
result LLMs enhance the accuracy and consistency of stock ratings, outperforming traditional methods in forward returns.
ChatGPT snapshots predict future stock returns.
problem Predicting future stock returns using pre-cutoff text.
method Extracted LLM outlook scores from OpenAI snapshots.
result Outlook scores positively correlate with future stock returns.
GPT model shows promise in financial market predictions.
problem Challenges in predicting stock market movements due to noisy data.
method Comparing GPT model with BERT model using Federal Reserve Beige Book data.
result GPT model has look-ahead bias, traditional models still perform better.
Top-H decoding improves text generation by balancing creativity and coherence.
problem Balancing creativity and coherence in text generation.
method Entropy-constrained minimum divergence problem, NP-hard ECMM problem, top-H decoding.
result Top-H decoding outperforms min-p sampling by up to 25.63% on creative writing benchmarks.
New mechanism for pure differential privacy on functional summaries using Laplace-like process.
problem Challenges in achieving differential privacy for complex, structured functional summaries.
method Independent Component Laplace Process (ICLP) mechanism for infinite-dimensional Hilbert space.
result Effective enhancement of utility of private summaries through oversmoothing.
Approximate Bayesian Computation (ABC) methods are used to approximate posterior distributions in models with unknown or computationally intractable likelihoods. Both the accuracy and computational efficiency of ABC depend on the choice of summary statistic, but outside of special cases where the optimal summary statis…
Unified approach for selecting summary statistics in ABC.
problem Efficient inference from large datasets in likelihood-free methods.
method Characterizing and unifying three classes of summary statistics, minimizing expected posterior entropy.
result EPE-minimizing summaries lead to competitive posterior inference.
Enhances weak lensing inference with neural summaries.
problem Extracting additional information from weak lensing convergence maps.
method Hybrid approach combining physics-based and neural summaries.
result Neural summaries extract up to 8 times more information than angular power spectra.
Approximate Bayesian computation is an established and popular method for likelihood-free inference with applications in many disciplines. The effectiveness of the method depends critically on the availability of well performing summary statistics. Summary statistic selection relies heavily on domain knowledge and care…
Paper presents ECL dataset for multi-modal bankruptcy prediction.
problem Developing models to predict corporate bankruptcy using textual and numerical data.
method Used ECL dataset to develop and evaluate classical and neural models.
result Textual and numerical data modalities complement each other for bankruptcy prediction.
Traditional methods outperform LLMs in forecasting corporate credit ratings.
problem Forecasting corporate credit ratings using LLMs.
method Comparison of traditional methods (XGBoost) and LLMs (LLaMA) on credit rating forecasting.
result XGBoost outperforms LLMs in forecasting corporate credit ratings.
Unified framework to bridge human and LLM judgments.
problem Systematic discrepancies between human and LLM evaluations.
method Latent human preference score and linear transformations of covariates.
result Higher agreement with human ratings and exposure of systematic gaps.
LLM-Lasso uses LLMs to improve feature selection in Lasso regression.
problem Improving feature selection in Lasso regression with domain-specific knowledge.
method Combines LLMs with Lasso regularization to generate feature weights.
result Outperforms standard Lasso and feature selection baselines in biomedical studies.
THERMOMETER calibrates LLMs efficiently for diverse tasks.
problem Calibrating large language models is challenging due to computational and versatility issues.
method THERMOMETER learns an auxiliary model for calibrating a LLM using data from multiple tasks.
result THERMOMETER produces better-calibrated responses for new tasks.
Regurgitative training with synthetic data harms LLM performance.
problem The impact of training LLMs with synthetic data generated by other LLMs.
method Fine-tuning GPT-3.5 with synthetic and real data in a machine translation task.
result Regurgitative training significantly reduces LLM performance.
FLUID-LLM uses LLMs to predict fluid dynamics with improved accuracy.
problem Leveraging LLMs for CFD due to their pattern recognition abilities but struggles with fluid dynamics complexities.
method Combines pre-trained LLMs with spatiotemporal-aware encoding to predict unsteady fluid dynamics.
result Significant performance improvements in CFD predictions across various datasets.
Improves inference from sparse data with hybrid summary statistics.
problem Robust simulation-based inference from limited data.
method Augment traditional summary statistics with neural network outputs to maximize mutual information.
result Improves information extraction and makes inference robust in low-data settings.
Automatically learns summary features from time series data for likelihood-free inference.
problem Necessity of hand-tailored summary features for time series data in likelihood-free inference.
method Data-driven approach to automatically learn summary features.
result Learning summary features from data can outperform hand-crafted values in likelihood-free inference.
This paper improves continuous adversarial training for LLMs using in-context learning theory.
problem Efficiently defending large language models (LLMs) against jailbreak attacks.
method The paper presents a theoretical analysis of continuous adversarial training (CAT) for LLMs based on in-context learning (ICL) theory, proving a robust generalization bound and proposing an improved regularization term.
result The robust generalization bound explains why CAT can defend against jailbreak prompts and shows that LLM robustness is related to embedding matrix singular values.
LLMs mimic human traders in finance, but not as much as expected.
problem Evaluating how LLMs behave in financial markets.
method Adapted experimental design with LLMs and human traders, analyzed in single and mixed model settings.
result LLMs tend to price assets near their fundamental value, but not as much as humans, and show less trading strategy variance.
Paper presents a probabilistic model to improve LLM cascade performance.
problem Complexity of LLM cascades and their interaction error rates.
method Probabilistic model for joint performance distribution of LLMs.
result Improves area under the error-cost curve by 4.3% on average for cascades with k≥3 models.
LLMs struggle to optimize hyperparameters efficiently, but hybrid methods can improve performance.
problem Optimizing hyperparameters of small language models using LLMs.
method Comparison of classical HPO algorithms and LLM-based methods, introducing Centaur hybrid approach.
result Hybrid Centaur approach achieves best results, outperforming classical and pure LLM methods.
Bayesian neural networks improve with summary information and Dirichlet process.
problem Lack of prior knowledge in BNNs for complex architectures.
method Incorporates external summary information about predicted probabilities using a Dirichlet process.
result Improves model accuracy, uncertainty calibration, and robustness.
Improves feature selection in high-dimensional data using LLM-generated weights.
problem Inaccurate LLM-generated weights degrade feature selection performance.
method Integrates LLM-generated weights into prior inclusion probabilities using LLM Sparsity Prior (LSP).
result Improves prediction accuracy and identifies clinically relevant features.
Telescope detects LLM generated text by measuring token repetition probability.
problem Distinguishing LLM generated text from human writing.
method Telescope Perplexity, evaluating token repetition probability.
result Telescope Perplexity enables effective zero-shot LLM detection.
Plug-in robust NPE method adapts summaries independently of pretrained NPE.
problem Misspecification of neural posterior estimators under test data distribution.
method Minimum-distance summaries using maximum mean discrepancy (MMD).
result Substantial robustness gains with minimal additional overhead.
BED-LLM uses Bayesian experimental design to improve LLMs' information gathering.
problem Improving LLMs' ability to gather information adaptively.
method Iteratively choosing questions to maximize expected information gain using a probabilistic model.
result BED-LLM achieves substantial performance gains compared to other adaptive design strategies.
Few summaries enable automatic summarization of product reviews.
problem Lack of large labeled datasets for training supervised models in opinion summarization.
method Conditional Transformer model trained to generate summaries given other reviews, fine-tuned to predict summary properties.
result Few summaries (5-10) are sufficient to generate fluent, informative, and sentiment-preserving summaries.
New benchmarks show LLMs struggle with causal discovery.
problem Leveraging LLMs for causal discovery is unreliable due to dataset leakage.
method Developing science-grounded benchmarks and hybrid methods combining LLM predictions with statistical analysis.
result LLMs perform poorly on novel, real-world scientific studies compared to classical methods.
Tiny benchmarks reduce LLM evaluation costs by using fewer examples.
problem Expensive evaluation of LLMs with tens of thousands of examples.
method Developed evaluation tools and tiny versions of popular benchmarks.
result Accurately estimate LLM performance with just 100 curated examples.
Improved likelihood-free inference by localizing and refining low-dimensional approximations.
problem Poor performance of common likelihood-free methods in high-dimensional models.
method Localisation followed by refinement of low-dimensional summaries.
result Improved accuracy in marginal posteriors through localized and refined approximations.
LLMs overestimate stock returns and are less accurate at predicting extreme outcomes.
problem Behavioral biases in LLMs' stock return forecasts.
method Comparison of LLM forecasts with crowd-sourced estimates and historical data.
result LLMs overestimate stock returns and are less accurate at predicting extreme outcomes.
Survey of LLMs in finance tasks, including adoption and performance.
problem Utilizing large language models in financial tasks.
method Review of current approaches, decision framework for adoption.
result Synthesizes state-of-the-art for LLMs in finance.
Unified definition of hallucinations in language models.
problem Persistent hallucinations despite mitigation efforts.
method Unified definition of hallucination as inaccurate world modeling.
result Unified framework distinguishes hallucinations from other errors.
Statsformer validates and adapts LLM-derived semantic priors for improved supervised learning.
problem Unreliable semantic priors from LLMs can degrade supervised learning performance.
method Adapts LLM-derived feature scores into a family of learner-specific prior-injection mechanisms, calibrating their influence using out-of-fold validation.
result Improves prediction performance by adaptively downweighting unreliable LLM priors, ensuring a guardrailed statistical learning system.
StealthRank subtly boosts LLM rankings without detectable anomalies.
problem Adversarial manipulation of LLM-driven ranking systems.
method Energy-based optimization with Langevin dynamics for stealthy prompt generation.
result StealthRank outperforms existing methods in covertly boosting rankings.
No-knowledge alarms detect misaligned LLM judges without trusting them.
problem Monitoring LLM judges for consistency when ground truth is unknown.
method Formal logic and Linear Programming to detect disagreements.
result No false positives in detecting misaligned judges.
Fine-tunes LLMs to correct bias in predictions.
problem LLMs exhibit bias in predictions from data.
method Supervised fine-tuning with Low-Rank Adaptation (LoRA).
result Fine-tuning corrects bias in both controlled and real-world settings.
New summary measures reveal geometric structure in weighted measures on manifolds.
problem Lack of geometric information in standard weight-only summaries.
method Heat-kernel entropy profiles, tracking nonuniformity across scales.
result Geometric effective sample size discounts nearby or duplicate particles.