Paper tackles AI risks by customizing metrics and models.
problem AI risks are multidimensional and immaturely managed.
method Decomposes AI risks into data protection, fairness, etc., and develops metrics and models.
result Customized metrics and models reduce AI risk uncertainty.
CAT framework improves AI medical screening fairness and reliability.
problem Imbalanced data, varying performance across cohorts, and patient-level inconsistencies in traditional metrics.
method CAT framework introduces patient-level assessment, entropy-based distribution weighting, and cohort-weighted sensitivity and specificity.
result Enhanced predictive reliability, fairness, and interpretability of AI-driven medical screening models.
New AI stock indices classify firms' AI engagement using 10-K filings.
problem Opaque AI selection criteria in existing ETFs.
method NLP analysis of 10-K filings to classify AI stocks.
result Companies with higher AI engagement have greater positive returns.
Experiment shows cognitive biases impact human-AI collaboration, highlighting the need for diverse evaluator samples.
problem Cognitive biases affect human-AI collaboration, leading to suboptimal outcomes.
method Randomized experiment with 2,784 participants, manipulating AI suggestion quality, task burden, and financial incentives.
result Individual attitudes toward AI are the strongest predictor of performance, influencing accuracy and overcorrection.
Paper proposes metrics to evaluate AI explanations without ground truth.
problem Challenges in evaluating neural network explanations without ground truth.
method Designs four metrics to evaluate explanation results.
result New insights into neural network interpretation methods.
Survey reviews explainability in AI for healthcare, emphasizing trust and transparency.
problem Lack of transparency hinders AI adoption in healthcare.
method Comprehensive literature review to guide explainable AI design.
result Quantitative evaluation metrics are needed for some explainability properties.
This paper reviews metrics to assess AI model calibration accuracy.
problem AI model probabilities do not always match their true accuracy.
method Comprehensive review of 82 probability calibration metrics.
result Identified 4 classifier families and 1 object detection family of metrics.
Paper analyzes uncertainty metrics in ensemble learning for healthcare AI.
problem Selecting appropriate uncertainty metrics for ensemble learners in healthcare AI.
method Rigorous analysis of two uncertainty metrics: ensemble mean and variance.
result Ensemble mean is preferable to ensemble variance for decision making in healthcare AI.
Enhanced AI analysis predicts S&P 500 stock dynamics using various financial metrics.
problem Predicting S&P 500 stock performance with complex interplay of factors.
method Advanced financial metrics, machine learning, and integration of traditional and modern analytics.
result Enhanced predictive accuracy in market behavior and investment strategies.
The paper assesses fairness in AI for financial services, using statistical methods.
problem Unintentional bias and insufficient model validation in AI applications.
method Statistical methods for imbalanced data treatment and bias mitigation.
result Fairness evaluation metrics applied to a credit card default payment example.
This paper examines AI and ML bias and fairness issues.
problem Bias and unfairness in AI and ML algorithms.
method Overview of bias and fairness issues, types and sources of data bias, algorithmic unfairness, fairness metrics, and de-biasing techniques.
result Discussion of the limitations of fairness metrics and de-biasing techniques.
New accuracy measure Ha improves AI system assessment in clinical practice.
problem Inadequate metrics for assessing AI system performance in clinical settings.
method Introducing H-accuracy (Ha) as a more informative measure.
result H-accuracy is a generalization of balanced accuracy and related to Net Benefit.
This paper ranks Latin American countries based on AI potential.
problem Identifying emerging AI powers in Latin America.
method Ranking based on infrastructure, education, and finance.
result Argentina, Colombia, Uruguay, Costa Rica, and Ecuador are new emerging powers in AI.
Study examines risks of using non-verified open data for AI in India.
problem Risks of using non-verified open data for AI in resource-poor settings.
method Exploring challenges of using an open dataset from India to predict pregnancy outcomes.
result AI applications without proper understanding of metrics can lead to erroneous conclusions.
Proposes a new trust framework for AI models to maximize utility.
problem Concerns over bias and discrimination in predictive models.
method Introduces a novel trust framework inspired by philosophy, focusing on maximizing Bayes utility.
result Properly-ranked models are inherently U-trustworthy. AI techniques explain synthetic tabular data weaknesses.
problem Challenges in evaluating synthetic tabular data quality.
method Apply explainable AI to a binary detection classifier.
result Reveals inconsistencies, unrealistic dependencies, or missing patterns in synthetic data.
New metric assesses reliability of AI explanations.
problem Unreliable AI explanations under realistic conditions.
method Explanation Reliability Index (ERI) metrics quantifying stability under four axioms.
result Widespread reliability failures in popular explanation methods.
New method accounts for uncertainty in medical AI evaluations.
problem Uncertainty in ground truth affects AI model performance estimates.
method Statistical aggregation approach to infer probabilities of medical conditions.
result Performance estimates are significantly lower when uncertainty is accounted for.
New risk metric for AI systems reduces safety risks with minimal data.
problem Risk assessment in multi-agent AI systems.
method Free Energy Principle applied to risk metrics, introducing Cumulative Risk Exposure.
result Gatekeepers improve system safety in autonomous vehicle fleets.
This paper evaluates investment risks in LATAM AI startups using DCF method.
problem Unique challenges and risks faced by LATAM tech startups.
method Total Addressable Market (TAM), Serviceable Available Market (SAM), and Serviceable Obtainable Market (SOM) metrics; Discounted Cash Flow (DCF) method.
result Developed a ranking of emerging powers in Latin America for tech startup investment.
Rigorous uncertainty quantification of probabilistic AI weather forecasts with conformal prediction
problem Calibrated uncertainty quantification in probabilistic weather forecasts
method Conformal prediction
result Calibrated uncertainty at no expense to other probabilistic metrics
Survey of determinism issues in financial AI systems.
problem Vulnerabilities in reproducibility of financial AI systems.
method Literature review and first-party experiments on public financial datasets.
result Proposed a layered evaluation framework linking modality-specific metrics to audit readiness.
CAI automates extraction and validation of corporate GHG emission metrics.
problem Manual extraction of corporate GHG emission metrics is labor-intensive and error-prone.
method CAI uses LLMs to automate extraction and validation of metrics from corporate disclosures.
result CAI improves data collection efficiency and accuracy by automating the process.
Paper uses DBSCAN variation to detect ship anomalies.
problem Detecting anomalous ship behavior.
method Variation of DBSCAN algorithm applied to AIS data.
result Alternative anomaly metric is more statistically informative.
This research develops a new framework to measure AI investment returns considering both gains and risks.
problem Traditional ROI calculations fail to account for AI's dual impact on risk reduction and new exposures.
method Integrates ISO 42001 and regulatory exposure into a comprehensive financial framework using risk quantification methods.
result Accurate AI investment evaluation requires modeling both productivity gains and risk exposures.
New method reduces copyright risks in AI-generated images.
problem Copyright issues in AI-generated images.
method Genericization method using originality estimation and PREGen technique.
result PREGen reduces likelihood of generating copyrighted characters by over half.
This paper presents a computational model for conceptual shifts, based on a novelty metric applied to a vector representation generated through deep learning. This model is integrated into a co-creative design system, which enables a partnership between an AI agent and a human designer interacting through a sketching c…
Transforms web content for better visibility in AI-driven search engines.
problem Disruption of traditional SEO by generative AI search engines.
method Fine-tunes a BART-base transformer on synthetically generated training data.
result Significant improvements in ROUGE-L and BLEU scores, and substantial visibility gains in generative search responses.
This paper evaluates and improves metrics for identifying important features in machine learning models.
problem Evaluation metrics for explainable AI are limited by multicollinearity and model accuracy.
method Proposes Expected Accuracy Interval (EAI) to predict model accuracy with multicollinearity.
result EAI is a useful metric for identifying important features in models with multicollinearity.
A new metric MSD detects bias in datasets efficiently.
problem Detecting bias in AI systems and datasets.
method Introduced Maximum Subgroup Discrepancy (MSD) metric and a practical algorithm based on MIO.
result MSD provides a linear sample complexity for practical applications, distinguishing biases effectively.
AI models assess psychological risks in currency trading.
problem Identifying psychological risks in currency traders.
method Developed a decision tree model to identify patterns in historical data.
result Enhanced decision-making through real-time alerts.
AI benchmarks evaluate football team performance using generative models.
problem Evaluating human performance in complex interactive tasks is error-prone and unreliable.
method Trained Conditional VRNN Model on player and ball tracking data to imitate and predict team interactions.
result Trained model as a useful benchmark for evaluating team performance in football.
FinSphere improves stock analysis quality with AI and expert-curated data.
problem Lack of objective evaluation metrics and depth in stock analysis by FinLLMs.
method Developed AnalyScore, curated Stocksis dataset, and FinSphere AI agent.
result FinSphere outperforms general and domain-specific LLMs in generating high-quality stock analysis reports.
ProEval efficiently estimates AI performance and discovers failures using pre-trained Gaussian Processes.
problem Resource-intensive evaluation of generative AI models.
method ProEval uses pre-trained Gaussian Processes and Bayesian quadrature to estimate performance and discover failures.
result ProEval requires significantly fewer samples to achieve accurate performance estimates and reveals more diverse failure cases.
timeXplain bridges AI and time series, making predictions understandable.
problem Making time series classifier predictions interpretable.
method Developed a framework that combines time series data with model-agnostic explainers.
result timeXplain improves the interpretability of time series classifiers.
FinRobot AI agent for equity research provides comprehensive insights.
problem Narrow focus and limited discretion in AI solutions for equity research.
method Multi-agent Chain of Thought system integrating quantitative and qualitative analyses.
result FinRobot delivers insights comparable to major brokerage firms.
Platform combines RL and language models to study narrative influence on AI decisions.
problem Understanding how narrative elements shape AI decision-making.
method Dual-system architecture with reinforcement learning and language model integration.
result Initial experiments show narrative frameworks can influence AI decision-making.
AI stocks hedge against AI singularity's economic impact.
problem AI singularity's displacement of consumption.
method Developed an asset pricing model with incomplete markets.
result AI stocks command a premium due to market incompleteness.
AI system predicts acute critical illness from EHRs with explainability.
problem Lack of clinical interpretability in AI predictions for acute critical illness.
method Developed an explainable AI early warning score (xAI-EWS) system.
result System provides clinicians with insights into EHR data explaining predictions.
Study analyzes AI's impact on firms, markets, and workers using large language model data.
problem Understanding AI's effect on firms, markets, and workers.
method Used 380 trillion tokens from 400+ large language models to analyze AI's impact.
result Firms with higher AI exposure earn higher returns, creating an AI premium.
Study compares AI models for stock price prediction using financial news.
problem Predicting stock price movements using financial news.
method Used FinBERT, GPT-4, and Logistic Regression for sentiment analysis and prediction.
result Logistic Regression outperformed FinBERT and GPT-4, achieving 81.83% accuracy.
The paper analyzes risk spillovers between AI ETFs, AI tokens, and green markets.
problem Risk spillovers among AI ETFs, AI tokens, and green markets.
method R2 decomposition method
result AI ETFs and clean energy act as risk transmitters, while AI tokens and green assets act as receivers.
PDD detects concept drift using explainable AI, improving model performance in dynamic environments.
problem Detecting and adapting to concept drift in predictive models.
method Profile Drift Detection (PDD) using Partial Dependence Profiles (PDPs).
result PDD outperforms existing methods in detecting concept drift and maintaining high predictive performance.
As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developer…
This review covers AI in finance, challenges, techniques, and opportunities.
problem Challenges and opportunities in AI applications in finance.
method Comprehensive categorization and overview of AI research in finance over decades.
result A dense roadmap of AI challenges, techniques, and opportunities in finance.
Explainable AI improves human decision accuracy but does not enhance it significantly.
problem Improving human decision-making through explainable AI.
method Comparing human decision accuracy with and without AI predictions, including or excluding explanations.
result Providing AI predictions improves human decision accuracy, but explanations do not significantly enhance it.
The paper explores AI in finance, focusing on XAI's role in enhancing interpretability and trust.
problem The need for AI in finance and the importance of XAI for better decision-making.
method Tracing AI's evolution in finance, highlighting XAI's role, and demonstrating through simulations.
result XAI enhances trust in AI systems, leading to more responsible decision-making.
A new method improves AI fairness assessment by estimating performance across intersectional subgroups.
problem Limited evaluation of AI systems across intersectional subgroups due to small sample sizes.
method Structured regression approach to disaggregated evaluation.
result Our method yields more accurate performance estimates, especially for small subgroups.