Major advances have been made regarding the utilization of artificial intelligence in health care. In particular, deep learning approaches have been successfully applied for automated and assisted disease diagnosis and prognosis based on complex and high-dimensional data. However, despite all justified enthusiasm, over…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “diagnostic evaluation”
Proposes a diagnostic method to evaluate factor models using cap-axis integrals.
problem Improving factor model evaluation in low-dimensional spaces.
method Lifts pricing errors into a bridge-alpha curve along the market-capitalization rank axis.
result The cap-axis norm is distinct from Sharpe gain and size exposure.
Proposes a diagnostic method to evaluate factor models using cap-axis integrals.
problem Improving factor model evaluation for low-dimensional models.
method Lifts pricing errors into a bridge-alpha curve along the market-capitalization rank axis.
result The cap-axis norm is distinct from Sharpe gain and size exposure.
Paper proposes a diagnostic tool for evaluating model performance out-of-sample.
problem Evaluating model performance on unseen data.
method Uses a finite calibration dataset to assess future losses.
result Provides guarantees under weak assumptions and quantifies distribution shifts.
The receiver operating characteristic (ROC) curve is a very useful tool for analyzing the diagnostic/classification power of instruments/classification schemes as long as a binary-scale gold standard is available. When the gold standard is continuous and there is no confirmative threshold, ROC curve becomes less useful…
L-C2ST improves local diagnostics for SBI approximations.
problem Evaluating trustworthiness of posterior approximations in SBI.
method Local evaluation of posterior estimators at any observation.
result Offers better statistical power and interpretability.
Online symptom checkers have significant potential to improve patient care, however their reliability and accuracy remain variable. We hypothesised that an artificial intelligence (AI) powered triage and diagnostic system would compare favourably with human doctors with respect to triage and diagnostic accuracy. We per…
New diagnostics detect variability in individual risk estimates from machine learning models in healthcare.
problem Variability in individual risk estimates from machine learning models in healthcare, leading to unreliable treatment decisions.
method Proposed evaluation framework using empirical prediction interval width and empirical decision flip rate diagnostics.
result Randomness in optimization and initialization can lead to substantial individual-level variability in risk estimates, affecting clinical decisions.
While it's always possible to compute a variational approximation to a posterior distribution, it can be difficult to discover problems with this approximation. We propose two diagnostic algorithms to alleviate this problem. The Pareto-smoothed importance sampling (PSIS) diagnostic gives a goodness of fit measurement f…
The high-dimensionality and volume of large scale multistream data has inhibited significant research progress in developing an integrated monitoring and diagnostics (M&D) approach. This data, also categorized as big data, is becoming common in manufacturing plants. In this paper, we propose an integrated M\&D approach…
Efficient event generation for collider phenomenology using parallel Langevin sampling and learned Stein diagnostics.
problem Event generation for precision collider phenomenology.
method Parallel Langevin sampling with learned Stein diagnostics.
result Relaxation time is estimated using a data-driven approach.
Graph-based approach repairs programs from diagnostic feedback.
problem Learning to repair programs from limited labeled data and compiler error messages.
method Introduces program-feedback graph and graph neural network for reasoning, and self-supervised learning with unlabeled programs.
result DrRepair significantly outperforms prior work, achieving high repair rates.
TADDAA improves accuracy diagnostics for variational approximations.
problem Challenges in evaluating the accuracy of variational approximations.
method Uses many short parallel MCMC chains to obtain lower bounds on the error of each posterior functional of interest.
result Validates the practical utility and computational efficiency of TADDAA on various models.
DECAT framework evaluates multimodal models for shared biology, detecting confounders and false positives.
problem Determining if multimodal models learn shared biology or just confounders.
method DECAT framework classifies multimodal representations into four diagnostic scenarios using null-referenced metrics.
result DECAT detects confounders and false positives in multimodal models, improving with larger cohorts and stronger representations.
Improves reliability of medical diagnosis uncertainty estimates.
problem Label uncertainty in medical diagnosis.
method Post-hoc -calibration method for neural network classifiers.
result Significantly enhances reliability of uncertainty estimates.
The paper analyzes numerical instability in variational flows and proposes a diagnostic method.
problem Numerical instability in variational flows affects sampling, density evaluation, and ELBO estimation.
method Treated variational flows as dynamical systems, used shadowing theory for theoretical guarantees, and developed a diagnostic procedure.
result Despite numerical instability, results from variational flows can be accurate enough for practical applications.
The paper breaks down AUC into cluster-level components for better model diagnostics.
problem Global AUC masks weaknesses in specific subpopulations, leading to financial or operational risks.
method Formal decomposition of AUC into intra- and inter-cluster components, comparing with other performance metrics.
result Allows practitioners to evaluate and diagnose model performance within and across clusters.
Proposes and evaluates three diagnostic graphics for probabilistic classifiers.
problem Evaluating and comparing probabilistic classifiers.
method Triptych of diagnostic graphics: reliability diagram, ROC curve, Murphy diagram.
result Visual diagnostics reveal distinct aspects of forecast performance.
Machine learning classifies colorectal tissue using photoacoustic microscopy.
problem Traditional diagnostic methods for colorectal cancer are limited in detail and painful.
method Machine learning applied to acoustic resolution photoacoustic microscopy.
result Machine learning accurately classified benign and malignant tissue.
Paper presents a light-weight CNN model for Parkinson's disease diagnosis.
problem Diagnosing Parkinson's disease efficiently and accurately.
method A light-weight CNN model cascading to LSTM for time-series signals.
result The model achieves high-quality results with fewer parameters and operations.
Language models fail to process hallucinated responses, and this study diagnoses the failure.
problem Language models fail to process hallucinated responses, leading to over-concentration or diffuse attention.
method The study uses forced scoring of benchmark-labeled responses to compute attention shapes and analyze the symmetric component of the degree-normalized attention operator.
result The study proves that every transpose-invariant spectral diagnostic of the attention operator is orientation-blind and bounds the sensitivity of any Lipschitz diagnostic by the asymmetry coefficient \(G\).
Study evaluates five LLMs for financial report analysis, revealing performance differences and variability.
problem Lack of understanding in reliability, consistency, and transparency of LLMs in financial analysis.
method Human evaluation, automated similarity metrics, and behavioral diagnostics applied to five transformer-based LLMs over U.S. 10-K filings.
result No single LLM consistently dominates across all evaluation perspectives, highlighting variability and need for interpretability.
ABC improves uncertainty quantification in LLMs for clinical diagnostics.
problem Overconfident and poorly calibrated estimates of LLMs in clinical domains.
method Approximate Bayesian Computation (ABC) for likelihood-free inference.
result Improves accuracy by up to 46.9%, reduces Brier scores by 74.4%, and enhances calibration.
Paper evaluates whether AI is a bubble or a productivity revolution.
problem Determining if AI investments are a bubble or a sustainable technology.
method Hybrid review and diagnostic framework combining asset pricing foundations and modern econometric methods.
result AI investments show both genuine fundamentals and bubble-like fragilities.
CJE calibrates cheap LLM judges against an oracle, achieving high accuracy at a fraction of the cost.
problem Inexpensive LLM judges can produce biased rankings, leading to unreliable outcomes.
method CJE uses a small oracle to calibrate cheap scores, then evaluates at scale with valid uncertainty.
result CJE achieves 99% pairwise ranking accuracy at 14x lower cost compared to a 16x oracle/judge cost ratio.
The recent success of natural language understanding (NLU) systems has been troubled by results highlighting the failure of these models to generalize in a systematic and robust way. In this work, we introduce a diagnostic benchmark suite, named CLUTRR, to clarify some key issues related to the robustness and systemati…
Applying machine learning in the health care domain has shown promising results in recent years. Interpretable outputs from learning algorithms are desirable for decision making by health care personnel. In this work, we explore the possibility of utilizing causal relationships to refine diagnostic prediction. We focus…
Establishes statistical and computational bounds for influence diagnostics.
problem Identifying influential datapoints or subsets in machine learning models.
method Finite-sample statistical bounds and computational complexity for influence functions and approximate maximum influence perturbations.
result Established statistical and computational guarantees for influence diagnostics.
Using Apple Machine Learning Algorithms to Detect and Subclassify Non-Small Cell Lung Cancerq-bio.QM
Lung cancer continues to be a major healthcare challenge with high morbidity and mortality rates among both men and women worldwide. The majority of lung cancer cases are of non-small cell lung cancer type. With the advent of targeted cancer therapy, it is imperative not only to properly diagnose but also sub-classify …
The article explains how to estimate confusion matrices for classifiers using unlabeled data.
problem Estimating sensitivity and specificity of binary medical diagnostic tests without gold standard tests.
method Modifying diagnostic test solutions to estimate confusion matrices for classifiers on unlabeled data.
result The approach can be used to estimate accuracy statistics for supervised or unsupervised binary classifiers on unlabeled data.
Two new feature selection methods improve breast cancer diagnosis accuracy.
problem Improving breast cancer diagnosis accuracy and reducing dataset dimensions.
method Imperialist Competitive Algorithm (ICA) and Bat Algorithm (BA) integrated with ML algorithms.
result BA-based feature selection method outperforms other methods with 99.12% accuracy.
AI enhances cancer diagnostics using spectroscopy.
problem Early and accurate cancer diagnosis.
method Combining AI with spectroscopy-based techniques.
result AI improves cancer diagnostics speed and safety.
Safety evaluation of self-driving technologies has been extensively studied. One recent approach uses Monte Carlo based evaluation to estimate the occurrence probabilities of safety-critical events as safety measures. These Monte Carlo samples are generated from stochastic input models constructed based on real-world d…
An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulationscs.LG
Paper introduces a diagnostic for approximate inference methods.
problem Estimating errors in probabilistic inference algorithms, especially for approximate methods.
method Repeatedly simulate datasets from the prior and perform inference on each, estimating a symmetric KL-divergence.
result A diagnostic for approximate inference methods can be estimated using symmetric KL-divergence.
Machine learning augmented diagnostic testing to identify sources of variability in test performancecs.LG
Machine learning improves diagnostic test accuracy for bovine tuberculosis.
problem Improving diagnostic test sensitivity for bovine tuberculosis.
method Machine learning to assess risk landscapes and predict infection.
result Test sensitivity improved, detecting 240 more infected herds per year.
AI predicts medical specialty diagnostic choices from EHR records.
problem Predicting timely medical specialty diagnostic workups for patients.
method Ensemble of feed-forward neural networks trained on EHR data.
result Significantly higher accuracy compared to traditional checklists.
Deep chest X-ray classifiers show bias in predicting diagnoses.
problem Bias in deep learning classifiers predicting diagnoses from chest X-rays.
method Trained convolutional neural networks on multiple public datasets to predict 14 diagnostic labels.
result True positive rates vary significantly among different protected attributes, indicating bias.
We propose a fast, model agnostic method for finding interpretable counterfactual explanations of classifier predictions by using class prototypes. We show that class prototypes, obtained using either an encoder or through class specific k-d trees, significantly speed up the the search for counterfactual instances and …
New estimator uses clustering to improve off-policy evaluation accuracy.
problem Improving off-policy evaluation accuracy when logging and evaluation policies differ.
method Proposes an estimator that shares information across similar contexts using clustering.
result Clustering contexts improves estimation accuracy, especially in deficient information settings.
Generative models often fail to preserve joint structure despite matching marginals.
problem Generative models fail to capture complex dependencies beyond univariate marginals.
method Introduced D_Sigma(P,Q) = ||Sigma_P - Sigma_Q||_F to measure covariance-level dependence fidelity.
result Covariance-level divergence can lead to structural instability in downstream inference.
Magnetic resonance image (MRI) reconstruction is a severely ill-posed linear inverse task demanding time and resource intensive computations that can substantially trade off {\it accuracy} for {\it speed} in real-time imaging. In addition, state-of-the-art compressed sensing (CS) analytics are not cognizant of the imag…
Beyond Accuracy: A Unified Random Matrix Theory Diagnostic Framework for Crash Classification Modelscs.LG
Unified framework detects overfitting in crash classification models.
problem Evaluation metrics fail to detect overfitting in crash classification models.
method Random Matrix Theory and Heavy-Tailed Self-Regularization framework applied to various model types.
result Power-law exponent α reliably distinguishes well-regularized from overfit models.
Proposes method for eliciting non-parametric joint priors using normalizing flows.
problem Learning complex non-parametric joint priors for model parameters.
method Expert elicitation combined with normalizing flows for generative modeling.
result Framework supports elicitation of both parametric and non-parametric priors.
The paper analyzes convergence in SGD with momentum and proposes a diagnostic test.
problem Detecting convergence in stochastic gradient descent with momentum.
method Analyzes the transient and stationary phases of SGD with momentum, constructs a statistical diagnostic test.
result The proposed diagnostic test effectively detects convergence in the stationary phase of SGD with momentum.
We propose a novel adaptive importance sampling algorithm which incorporates Stein variational gradient decent algorithm (SVGD) with importance sampling (IS). Our algorithm leverages the nonparametric transforms in SVGD to iteratively decrease the KL divergence between our importance proposal and the target distributio…
Paper proposes a coupling-based diagnostic for SGD stepsize optimization.
problem Optimizing stepsize for SGD convergence.
method Coupling-based convergence diagnostic for monitoring stationarity.
result Proposed stepsize scheme achieves superior performance across convex and non-convex problems.
Simulation study evaluates causal ML models under confounding violations.
problem Assessing conditional exchangeability in causal machine learning models.
method Simulation study with varying confounding, sample size, and NCO structures.
result Causal ML models fail to recover true treatment effect heterogeneity under violations of conditional exchangeability.
Bayesian design improves accuracy without extra cost.
problem Nested inference in complex systems limits BED accuracy and efficiency.
method Grouped geometric pooled posterior with EKI formulation.
result Improved accuracy and stable estimators at comparable cost.