LLM-Lasso uses LLMs to improve feature selection in Lasso regression.
problem Improving feature selection in Lasso regression with domain-specific knowledge.
method Combines LLMs with Lasso regularization to generate feature weights.
result Outperforms standard Lasso and feature selection baselines in biomedical studies.
Proposes a new algorithm for efficient online model selection of LLMs considering the increasing-then-converging trend.
problem Balancing cost and performance in choosing the best LLM among a diverse set of models.
method Introduces a time-increasing bandit algorithm (TI-UCB) that predicts model performance increases and balances exploration and exploitation.
result Achieves a logarithmic regret upper bound, indicating a fast convergence rate in model selection.
Improves feature selection in high-dimensional data using LLM-generated weights.
problem Inaccurate LLM-generated weights degrade feature selection performance.
method Integrates LLM-generated weights into prior inclusion probabilities using LLM Sparsity Prior (LSP).
result Improves prediction accuracy and identifies clinically relevant features.
LLMs learn to recommend models and hyperparameters from dataset metadata.
problem Model and hyperparameter selection in machine learning is challenging and resource-intensive.
method Converted datasets into metadata and prompted LLMs to recommend models and hyperparameters.
result LLMs can recommend competitive models and hyperparameters without search.
LMoE uses LLMs to improve stock trading by selecting experts based on textual and price data.
problem Traditional neural network-based router selection in MoE models is suboptimal and ignores textual data.
method Proposes LLMoE, using LLMs as routers to select experts based on historical price data and stock news.
result LLoM outperforms state-of-the-art MoE models and other deep neural network approaches.
LLMs can select predictive features without seeing training data.
problem Selecting the best features for prediction tasks.
method Zero-shot prompting LLMs with feature names and task descriptions.
result LLMs consistently identify predictive features across various mechanisms.
Generative AI models enhance sector-based investment portfolios, but performance varies by market conditions.
problem Improving investment performance through better stock selection in volatile markets.
method Applied LLMs from OpenAI, Google, Anthropic, DeepSeek, and xAI to select and weight stocks within S&P 500 sectors.
result LLM-weighted portfolios outperform sector indices in stable markets but underperform in volatile ones.
CausalGame benchmarks LLM agents' causal thinking in games.
problem Evaluating causal thinking in AI Scientists with LLMs.
method Interactive games with 14 scenarios incorporating selection bias, measurement error, and hidden confounders.
result None of the 30 LLM agents demonstrated reliable causal thinking, with the best model achieving only 68.0% survival.
New framework selects high-quality pretraining data without training LLMs.
problem Slow progress in understanding pretraining data due to costly experiments.
method Statistical framework based on perplexity-benchmark correlations.
result Approach outperforms existing methods on multiple benchmarks.
FisherSFT selects informative examples to fine-tune LLMs efficiently.
problem Adapting large language models to new domains efficiently.
method Selects examples maximizing information gain using Hessian of log-likelihood.
result Empirically demonstrates improved performance with reduced computational cost.
Framework evaluates AI proposals for drug discovery, finds no LLM advantage.
problem No principled framework exists for evaluating AI-guided scientific selection under budget constraints.
method Formally verified metric (BSDS/DQS) penalizes false discoveries and excessive abstention.
result LLMs provide no marginal value over existing classifiers in drug discovery.
LLMs can be influenced by unseen dataset subtexts, revealing new ways to select data subsets.
problem Understanding how datasets subtly influence LLMs and their properties.
method Logit-Linear-Selection (LLS) method to select subsets of datasets.
result LLS reveals hidden effects in LLMs that persist across different models and architectures.
LLM evaluation suffers from systematic biases and lacks reliable positive judgments.
problem LLM evaluation suffers from systematic biases and lacks reliable positive judgments.
method Formulate LLM evaluation as a positive-unlabelled learning problem and propose a geometric auditing framework based on Partial Optimal Transport.
result Improved alignment with human preferences, increased robustness to presentation biases, and interpretable confidence estimates.
Physics-inspired methods optimize SVD compression of LLMs.
problem Efficiently compressing large language models (LLMs) using SVD.
method FermiGrad for globally optimal rank selection and PivGa for lossless compression.
result Global optimization of SVD ranks and lossless compression of low-rank factors.
Benchmark evaluates LLM trading agents by masking identifiers to prevent memory leaks.
problem Evaluate LLM trading agents without relying on market memory or noise.
method Data-side masking protocol, Barra-style performance attribution framework.
result LLM agents' returns are largely explained by market and style exposure, not stock selection.
RACER optimizes LLM-as-judge accuracy with dynamic reasoning selection.
problem Balancing reasoning accuracy with computational cost in LLM-as-judge settings.
method Formulates routing as a constrained distributionally robust optimization problem, accounting for distribution shift via KL-divergence uncertainty set.
result RACER achieves superior accuracy-cost trade-offs under distribution shift.
SIREN protocol corrects optimistic winner's scores in LLM evaluation.
problem Optimistic winner's scores in LLM evaluation due to adaptive benchmarking.
method SIREN protocol that freezes post-search shortlist, separates selection and evaluation, and uses bootstrap for uncertainty quantification.
result SIREN provides valid confidence intervals for procedure-performance curves and deployment conclusions.
Best-of- ∞ \infty ∞ improves LLM performance by efficiently allocating inference-time computation.
problem Achieving optimal performance in test-time LLM ensembling with infinite budget.
method Adaptive generation scheme and weighted ensembles of LLMs, formulated as mixed-integer linear program.
result Optimal ensemble weighting improves performance over individual models.
Dynamic abstention improves LLM accuracy by selectively terminating unpromising reasoning.
problem LLMs waste compute on incorrect responses, leading to inefficiency.
method Formal reinforcement learning framework with abstention reward parameter.
result Dynamic abstention outperforms natural baselines in selective accuracy.
3S-Trader uses LLMs to optimize stock portfolios by scoring, strategizing, and selecting stocks.
problem Lack of multi-LLM frameworks for adaptive stock scoring, strategy, and selection in portfolio optimization.
method 3S-Trader incorporates scoring, strategy, and selection modules for stock portfolio construction, using historical strategies and market conditions to generate optimized selections.
result 3S-Trader achieves the highest accumulated return of 131.83% on DJIA constituents with a Sharpe ratio of 0.31 and Calmar ratio of 11.84.
TRIPLE efficiently optimizes prompts with a budget constraint.
problem Efficiently selecting good prompts from a pool of candidates.
method TRIPLE connects prompt optimization to best arm identification in MAB, leveraging BAI-FB tools.
result TRIPLE outperforms baselines on multiple tasks with limited budget constraints.
Develops a power-calibrated framework for LLM watermarking, optimizing tradeoffs between detectability and distortion.
problem The trade-off between detectability and semantic distortion in logit-based watermarking.
method Power-calibrated statistical framework for watermark hyperparameters, establishing explicit relationships.
result Derives practical parameter selection procedures achieving optimal tradeoffs under constraints.
A new method selects models for ensemble learning to maximize mutual information, outperforming existing approaches.
problem Selecting models for ensemble learning to improve performance and reduce correlation issues.
method Formulate budgeted ensemble selection as maximizing mutual information, use Gaussian-copula to model correlated errors, propose a greedy mutual-information selection algorithm.
result Our method consistently outperforms strong baselines across multiple datasets.
DUET optimizes LLM training data mixtures using feedback from unseen tasks.
problem Optimizing LLM performance on unseen tasks without prior data knowledge.
method Interleaves influence function with Bayesian optimization for data selection.
result DUET converges to optimal training data mixture for unseen tasks.
CSA fills a gap in RLVR-trained LLM deployment by providing anytime-valid selective risk control.
problem Deployment of RLVR-trained LLMs in regulated organizations requires a safety certificate for every round without waiting for long-run averages.
method CSA uses a (test statistic, validity guarantee, deployment rule) framework to fill the gap, maintaining a Ville-type e-process per threshold on a Bonferroni grid.
result CSA provides the first anytime-valid selective risk control for RLVR-trained LLMs, matching the long-run average certification rate and satisfying pathwise validity and non-refusing deployment on every cell.
Hybrid model uses LLM to build transparent Bayesian networks for trading decisions.
problem Rigorous and transparent reasoning required in financial trading, especially for options strategies.
method Combines LLM strengths with Bayesian Networks, using LLM to construct context-specific networks and select relevant data.
result Empirically, the hybrid system outperforms market benchmarks with superior risk-adjusted performance.
Proposes a method to choose thresholds for LLM evaluation metrics.
problem Ensuring reliable large language models (LLMs) with correct threshold selection.
method Identify risks, stakeholders' risk tolerance, and use ground-truth data to determine thresholds.
result Demonstrates a concrete example with the Faithfulness metric and HaluBench dataset.
Efficiently predict LLM benchmarks using feature selection and regression.
problem Predicting full benchmark scores with minimal question subsets.
method Multiple regression with feature selection, using kernel ridge regression and mRMR.
result Improved prediction accuracy and ranking correlation across various benchmarks.
Benchmark assesses LLMs' causal inference skills, revealing significant limitations.
problem Lack of rigorous evaluation of LLMs' causal inference capabilities.
method CausalPitfalls benchmark with structured challenges and grading rubrics.
result Significant limitations in current LLMs' statistical causal inference.
Survey of LLMs in finance tasks, including adoption and performance.
problem Utilizing large language models in financial tasks.
method Review of current approaches, decision framework for adoption.
result Synthesizes state-of-the-art for LLMs in finance.
EASE optimizes exemplar selection for ICL in LLMs efficiently.
problem Efficiently selecting exemplars for in-context learning (ICL) in large language models (LLMs).
method EASE uses neural bandit algorithms to optimize exemplar sets, considering both exemplar ordering and instruction.
result EASE outperforms existing methods in finding optimal exemplar sets for all test queries.
Proposes an online model for LLM cascading with adaptive API selection.
problem Adaptive querying and selection of LLM APIs in a context-dependent environment.
method Develops a learning approach combining GMM estimation and UCB-style bounds.
result Achieves cumulative regret of O ~ ( T ) \widetilde O(\sqrt T) O ( T ) over T T T periods. Cer-Eval saves LLM evaluation costs while maintaining accuracy.
problem Challenges in evaluating large language models due to large dataset requirements.
method Adapts to different evaluation objectives, uses test sample complexity, and develops a partition-based algorithm.
result Cer-Eval can save 20-40% test points with comparable accuracy and 95% confidence guarantee.
LLMs improve financial analysis by processing large data sets.
problem Traditional financial analysis methods struggle with large data volumes.
method Integrating LLMs for enhanced data processing and analysis.
result LLMs offer new capabilities for real-time financial decision-making.
CARROT optimizes LLM routing by choosing the cheapest and most accurate model.
problem Optimizing query routing to the most cost-effective LLM for a given task.
method CARROT uses cost and performance estimates to select the best LLM for each query.
result CARROT is minimax optimal, selecting the best LLM for any query.
Paper presents a probabilistic model to improve LLM cascade performance.
problem Complexity of LLM cascades and their interaction error rates.
method Probabilistic model for joint performance distribution of LLMs.
result Improves area under the error-cost curve by 4.3% on average for cascades with k≥3 models.
Improved selection of best outputs from LLMs for better accuracy.
problem BoN fails to reliably find the correct answer with imperfect rewards.
method Majority-of-the-Bests (MoB) selects the mode of a bootstrapped output distribution of BoN.
result MoB consistently improves over BoN in 25 out of 30 setups.
Spectrum selectively trains LLMs based on SNR to save resources.
problem Efficiently training large language models with limited resources.
method Targeting layer modules based on SNR for selective training.
result Spectrum achieves similar performance to full fine-tuning but with reduced VRAM usage.
R-AutoEval+ improves model evaluation efficiency and reliability using adaptive synthetic data.
problem Accurate model selection from AI candidates using real-world data is costly and impractical at scale.
method R-AutoEval+ uses adaptive prediction-powered inference to correct bias in autoevaluators while maintaining or improving sample efficiency.
result R-AutoEval+ provides finite-sample reliability guarantees and enhanced sample efficiency compared to conventional methods.
Training-free source selection for LLM families with shared vocabularies
problem Source selection for LLM families with shared vocabularies
method Fisher alignment at vocabulary scale
result Fisher alignment is a cosine between kernel mean embeddings in the joint activation-error space
SOCRATES uses LLMs to automate simulation optimization of complex systems.
problem Optimizing complex, expensive-to-sample stochastic systems.
method Two-stage procedure: replica construction and meta-optimization.
result Adaptive hybrid optimization schedule for real systems.
MarketSenseAI uses LLMs to improve stock analysis and outperforms benchmarks.
problem Improving accuracy in stock analysis and selection.
method Combining LLMs with SEC filings, earnings calls, and macroeconomic reports.
result Significant improvement in fundamental analysis accuracy and outperformance of benchmarks.
FAQ efficiently evaluates LLMs with statistical guarantees using adaptive query selection.
problem Efficiently evaluating many LLMs on a large suite of benchmarks is expensive.
method FAQ uses Bayesian factor models, adaptive sampling, and proactive active inference to select queries.
result FAQ delivers up to 5x effective sample size gains over baselines, matching CI width with fewer queries.
PDO optimizes LLM prompts without labels, improving performance.
problem Optimizing prompts for LLMs without access to labeled data.
method Pairwise preference feedback, dueling bandits, Thompson Sampling, mutation.
result PDO identifies stronger prompts than label-free methods.
Meta-Router optimizes LLM selection using gold-standard and preference-based data.
problem Training a high-quality LLM router with combined data sources is challenging due to bias and scarcity.
method Developed an integrative causal router training framework to correct bias and improve routing accuracy.
result Our approach delivers more accurate routing and improves the trade-off between cost and quality.
Study uses LLMs to improve Black-Litterman portfolio optimization.
problem Systematically generating investor views for Black-Litterman model.
method Translates LLM return forecasts and uncertainty into Black-Litterman inputs.
result LLM-driven portfolios outperform traditional baselines.
Algorithm reduces audit costs by identifying best service configurations from biased textual evidence.
problem Designing service systems from textual evidence requires accurate selection despite biased automated scoring.
method Developed PP-LUCB algorithm combining LLM scores and selective audits to minimize costs.
result Correctly identified the best model in 40/40 trials with 90% cost reduction.
In2Core selects a coreset for efficient LLM fine-tuning with reduced data.
problem Costly fine-tuning of large language models due to extensive parameters and data requirements.
method Analyzes model gradients to estimate training sample influence, optimizing for efficiency.
result Achieves similar performance with 50% of training data using In2Core.