New AI error correctors improve classifier performance with provable guarantees.
problem Improving AI classifier performance with scarce training data.
method Weakly supervised AI error correctors with performance guarantees.
result Provable performance guarantees for AI error correction.
Generative AI boosts analyst reports but increases forecast errors.
problem Improving financial analyst reports with AI.
method Natural experiment using FactSet's AI platform.
result AI-assisted reports are more comprehensive but lead to higher forecast errors.
Artificial Intelligence (AI) systems sometimes make errors and will make errors in the future, from time to time. These errors are usually unexpected, and can lead to dramatic consequences. Intensive development of AI and its practical applications makes the problem of errors more important. Total re-engineering of the…
This work proposes external correctors for quick AI error corrections without system modification.
problem Quick corrections of AI errors without modifying legacy systems.
method Special `external` devices with classifiers for high-dimensional data.
result Simple classifiers can correct small errors in high-dimensional data.
Experiment shows cognitive biases impact human-AI collaboration, highlighting the need for diverse evaluator samples.
problem Cognitive biases affect human-AI collaboration, leading to suboptimal outcomes.
method Randomized experiment with 2,784 participants, manipulating AI suggestion quality, task burden, and financial incentives.
result Individual attitudes toward AI are the strongest predictor of performance, influencing accuracy and overcorrection.
Improves AI system's understanding of human inputs by creating better examples.
problem AI systems misinterpret human inputs, leading to inefficiencies.
method Developed a conditional convolutional autoencoder (CCAE) to generate better examples.
result Generated examples lead to lower error rates and require less effort to create.
The paper solves optimal bounds for separating data points in high dimensions.
problem Correcting AI errors and analyzing vulnerabilities in high-dimensional data.
method General stochastic separation theorems with optimal probability estimates.
result Explicit and optimal estimates of separation probabilities for important classes of distributions.
AI detects 38% NFT trades likely manipulated, improving on indirect methods.
problem Detecting crypto wash trading using indirect methods and leaked data.
method Public NFT data analysis, direct estimation, AI-based estimator.
result AI reduces estimation errors in NFT markets, improving on indirect methods.
The paper studies how to allocate human validation in AI-assisted tasks to minimize errors.
problem Heterogeneous reliability of AI-generated signals across tasks, products, and customer segments.
method Tuned prediction-powered inference, upper confidence bounds policy, Neyman square-root rule.
result The proposed policy outperforms uniform and epsilon-greedy allocation, closing most of the gap to the oracle when reliability is heterogeneous.
AI systems are being deployed to support human decision making in high-stakes domains. In many cases, the human and AI form a team, in which the human makes decisions after reviewing the AI's inferences. A successful partnership requires that the human develops insights into the performance of the AI system, including …
A method compares AI corrections to a base model for explaining predictions.
problem Creating explanations for AI predictions.
method Introduces a surrogate model to correct a simpler base model and provides criteria for accuracy and fidelity.
result Induces neighborhoods of instances with ideal accuracy and fidelity.
PBC improves AI and dynamical subseasonal forecasts by reducing biases.
problem Subseasonal forecast accuracy drops due to model biases and compounding errors.
method Probabilistic bias correction (PBC) using machine learning to correct historical forecasts.
result PBC doubles AI Forecasting System's subseasonal skill and improves dynamical model skill.
Improves AI-prior reliability for Bayesian inference.
problem Error propagation from predictive models into posterior inference.
method Rectified AI-informed prior elicitation framework.
result Significant reduction in bias and improvement in predictive performance.
Generative Augmented Inference improves AI-generated data for causal inference.
problem Challenges in using AI-generated annotations for reliable causal inference.
method Generative Augmented Inference (GAI) treats AI outputs as informative features for learning true labels, flexibly modeling the relationship using nonparametric methods.
result GAI significantly reduces estimation error and improves confidence interval quality compared to human-only and PPI-based methods.
Annealed importance sampling (AIS) is a common algorithm to estimate partition functions of useful stochastic models. One important problem for obtaining accurate AIS estimates is the selection of an annealing schedule. Conventionally, an annealing schedule is often determined heuristically or is simply set as a linear…
Deep ReLU networks can approximate matrix-vector products with error bounds.
problem Can deep ReLU networks accurately approximate matrix-vector products?
method Derived error bounds in Lebesgue and Sobolev norms for deep ReLU FNNs.
result Developed deep approximation theory with successful applications.
Algorithmic insurance tackles financial risks from AI errors, proving CVaR-optimal thresholds reduce tail risk.
problem High-stakes AI errors lead to heterogeneous losses, challenging traditional insurance assumptions.
method Analyzed binary classification performance to tail risk exposure, using CVaR to quantify extreme losses.
result CVaR-optimal thresholds reduce tail risk up to 13-fold compared to accuracy maximization.
Paper uses AI to predict stock market volatility with neural networks and genetic algorithms.
problem Traditional methods for predicting stock market volatility have high errors.
method Back-propagation neural network and genetic algorithm integrated model.
result The model predicts future volatility with low errors and high accuracy.
Method curates cost-effective, high-quality datasets using AI models.
problem Costly manual labeling of datasets.
method Probably Approximately Correct Labels (PACL) method.
result Curates high-quality datasets with low overall labeling error.
VERAFI improves financial AI by verifying calculations and compliance.
problem Financial AI systems generate errors and violations during reasoning.
method VERAFI combines dense retrieval, reranking, and automated reasoning policies.
result VERAFI achieves 94.7% factual correctness, 81% relative improvement.
Examines AI regulation in finance, highlighting risks and gaps in current laws.
problem Rapid AI adoption in finance introduces risks and compliance challenges.
method Reviews current legislation, industry guidelines, and real-world use cases.
result Need for adaptive, technology-neutral policies to balance innovation and consumer protection.
Redundancy in AI perception systems doesn't guarantee independent error occurrences.
problem Lack of direct statistical evidence of super-human automated driving performance.
method Investigated the effectiveness of redundancy in neural networks for independent error occurrences.
result Errors in neural networks for computer vision tasks are correlated, not independent.
This paper maps the insurability of AI risks across various insurance products.
problem Emerging AI risks and their implications for insurance coverage.
method Coding 55 AI threat classes against 26 insurance products using public carrier materials and threat catalogs.
result Identification of a four-tier insurability frontier: affirmatively insured, silent-AI exposures, actively excluded, and unstructured perils.
We consider the problem of efficient "on the fly" tuning of existing, or {\it legacy}, Artificial Intelligence (AI) systems. The legacy AI systems are allowed to be of arbitrary class, albeit the data they are using for computing interim or final decision responses should posses an underlying structure of a high-dimens…
CausalGame benchmarks LLM agents' causal thinking in games.
problem Evaluating causal thinking in AI Scientists with LLMs.
method Interactive games with 14 scenarios incorporating selection bias, measurement error, and hidden confounders.
result None of the 30 LLM agents demonstrated reliable causal thinking, with the best model achieving only 68.0% survival.
AI-assisted interviews allow respondents to describe experiences naturally, but mapping those accounts into structured survey variables is fallible.
problem Mapping AI-assisted interview responses into structured survey variables is fallible.
method Adaptive Matrix Validation (AMV) is proposed, which involves mapping responses into tabular data and using a small set of structured questions for statistical adjustment.
result The estimator calibrates mapped values using validation answers from other respondents and corrects remaining error with validation answers observed for the target respondent.
si4onnx enables selective inference on deep learning models.
problem Establishing the reliability of AI systems through statistical significance of identified regions.
method Selective inference techniques implemented through a Python package.
result Controlled type I error rates for hypothesis testing on deep learning models.
Generative AI agents improve ERP systems by automating complex financial tasks.
problem Static, rule-based workflows limit adaptability and intelligence in ERP systems.
method Introducing Generative Business Process AI Agents (GBPAs) that integrate generative AI with business process modeling and multi-agent orchestration.
result GBPAs achieve up to 40% reduction in processing time and 94% drop in error rate.
AI task delegation faces incentive collapse with unbounded payments as AI accuracy rises.
problem Incentive collapse in AI-assisted task delegation schemes.
method General impossibility result and sentinel-auditing payment mechanism.
result Sentinel-auditing mechanism enforces positive human effort at finite cost, independent of AI accuracy.
Extends post-prediction inference method for more accurate AI/ML data analysis.
problem Naively using AI/ML predictions as true observations leads to biased results.
method Extends Wang et al. method to relax assumptions and incorporate a scaling factor.
result Yields unbiased point estimates and proper coverage in simulations.
This paper analyzes errors in Shapley value-based model explanations.
problem Biased or unreliable explanations from existing SVA methods.
method Error theoretical analysis framework decomposing errors into observation and structural biases.
result Trade-off between observation and structural biases in SVA explanations.
Develops a framework for quantifying agentic AI model risk using LLM-inferred Bayesian state filters.
problem Quantifying the risk of agentic AI systems due to uncertain beliefs and actions.
method Representing the system as a partially observed Markov decision process with latent states, Bayesian belief updates, control-dependent losses, and tail-risk functionals.
result Develops a rigorous framework for separating uncertainty quantification from risk measurement.
AI enhances refinery optimization by detecting data errors and improving decision-making.
problem Interpreting and applying LP solutions for refinery optimization is challenging due to simplifications and data errors.
method Transformed ECOD methodology, Anomaly Detection tools, and high-dimensional data analysis.
result Identifies data supply errors and reveals business opportunities in refinery scheduling and planning.
Study improves prediction of UK road accidents' severity using AI.
problem Improving prediction of UK road traffic accident severity.
method Combination of machine learning, econometric, and statistical methods on historical data.
result XGBoost model with RMSE of 0.176 and MAE of 0.087 outperforms naive forecasting.
New AI platform screens portfolios for desirable firms and news.
problem Optimizing portfolio selection with AI.
method Two LLM agents screen for firm fundamentals and news sentiment. Agents deliberate to generate buy/sell signals. High-dimensional estimation determines optimal weights.
result Screened portfolio's Sharpe ratio consistently estimates target, superior to baseline and conventional approaches.
ProEval efficiently estimates AI performance and discovers failures using pre-trained Gaussian Processes.
problem Resource-intensive evaluation of generative AI models.
method ProEval uses pre-trained Gaussian Processes and Bayesian quadrature to estimate performance and discover failures.
result ProEval requires significantly fewer samples to achieve accurate performance estimates and reveals more diverse failure cases.
The paper certifies AI reliability via sampling and calibration, providing exact guarantees.
problem Ensuring trust in black-box AI systems' outputs.
method Self-consistency sampling and conformal calibration.
result Reliability levels derived from these methods offer finite-sample guarantees.
Automates investor/company matching with AI, explaining decisions.
problem Matching companies and investors is hard due to limited data and need for explanations.
method Representation learning for small datasets + parameterized explanation generation.
result System performs well on matching task and explains decisions.
QCircuitBench provides a dataset for evaluating AI's ability to design quantum algorithms.
problem Lack of datasets for evaluating AI's capability in designing quantum algorithms.
method Developed a comprehensive benchmark dataset with 120,290 data points, including 25 algorithms and 3 task suites.
result LLMs exhibit consistent error patterns and fine-tuning does not always outperform few-shot learning.
QTAML models quantum tunneling errors for AI robustness.
problem Quantum tunneling errors in AI inference.
method Derives weight-error distribution using WKB approximation, introduces TAC algorithm.
result TAC achieves 95% clean accuracy with 3.4-33.6x less ECC overhead.
U-Cast simplifies AI weather forecasting with a standard U-Net and efficient training.
problem Complex AI models limit accessibility and cost for weather forecasting.
method Simple U-Net backbone, deterministic pre-training, and probabilistic fine-tuning with Monte Carlo Dropout.
result U-Cast matches or exceeds state-of-the-art models in accuracy while reducing training and inference costs.
Deep-learning based classification algorithms have been shown to be susceptible to adversarial attacks: minor changes to the input of classifiers can dramatically change their outputs, while being imperceptible to humans. In this paper, we present a simple hypothesis about a feature compression property of artificial i…
New calibration measure SCDL improves trust in AI predictions.
problem Improving trust in AI predictions by ensuring they are both actionable and testable.
method Introducing SCDL, a new calibration measure that is fully actionable and testable.
result SCDL is the first calibration measure that is fully actionable and testable.
Study finds price-based clustering outperforms AI and human methods in stock market analysis.
problem Investigates if AI can improve stock clustering compared to traditional methods.
method Compares price-based, human-informed, and AI-driven clustering methods using synthetic factor models.
result Price-based clustering reduces RMSE by 15.9% relative to GICS and 14.7% relative to LLM embeddings.
The paper studies how to use AI-generated labels in econometrics to avoid bias.
problem Small misclassification errors in AI-generated labels can lead to large biases in econometric estimators.
method The paper proposes a coupled-label bootstrap method to correct bias and deliver valid inference.
result The coupled-label bootstrap method is valid without the strong independence condition between true and imputed labels.
Survey of AI-based outlier detection methods for various domains.
problem Detecting unusual events to prevent infrastructure damage.
method Categorizes six major OD methods: Statistical, Distance, Density, Clustering, Learning, and Ensemble.
result Advances and challenges of each method discussed.
Adaptive auditing improves AI robustness testing with anytime-valid guarantees.
problem Cost and time of annotation limit rigorous AI failure mode characterization.
method Introduces hypothesis testing framework for adaptive audits using SAVI.
result Proves anytime-valid type-I error control and robustness certification.
AI stocks hedge against AI singularity's economic impact.
problem AI singularity's displacement of consumption.
method Developed an asset pricing model with incomplete markets.
result AI stocks command a premium due to market incompleteness.