XLabel tool reduces medical experts' workload by 40% and explains its decisions.
problem Efficiently labeling large electronic health records.
method Visual-interactive tool using Explainable Boosting Machine (EBM) for classification and explanation.
result EBM achieves high accuracy and explainability, even with mislabeled data.
System uses conformal prediction to help experts make accurate decisions without understanding when to trust it.
problem Helping experts make accurate decisions in multiclass classification tasks.
method Develops an automated decision support system using conformal prediction to provide precise prediction sets and an efficient search method.
result System improves expert predictions by providing precise prediction sets and forcing experts to predict from these sets.
Automated generation of medical reports from chest x-rays using expert annotations.
problem Generating long, unstructured text from medical images with context and consistency.
method First learn visually-informative medical concepts from raw reports, then use these concepts to auto-generate structured reports from images.
result Validation on OpenI dataset shows auto-generated reports are consistent with manual annotations.
Online health communities are a valuable source of information for patients and physicians. However, such user-generated resources are often plagued by inaccuracies and misinformation. In this work we propose a method for automatically establishing the credibility of user-generated medical statements and the trustworth…
This research improves uncertainty estimation for medical predictions, enhancing model trust and decision support.
problem Improving model uncertainty estimation for rare medical conditions.
method Developed and refined heuristics for selecting uncertainty estimation techniques, distinguishing them by clinical use-case. Also, compared ensembles vs. auto-encoders for detecting out-of-domain examples.
result Auto-encoders outperform ensembles in detecting out-of-domain examples, highlighting their importance for medical tabular data.
Multi-expert L2D underfits more severely, requiring new methods.
problem Underfitting in multi-expert L2D settings.
method PiCCE (Pick the Confident and Correct Expert), a surrogate-based method.
result PiCCE effectively reduces multi-expert L2D to a single-expert-like problem, resolving underfitting.
The issue of disagreements amongst human experts is a ubiquitous one in both machine learning and medicine. In medicine, this often corresponds to doctor disagreements on a patient diagnosis. In this work, we show that machine learning models can be trained to give uncertainty scores to data instances that might result…
New method accounts for uncertainty in medical AI evaluations.
problem Uncertainty in ground truth affects AI model performance estimates.
method Statistical aggregation approach to infer probabilities of medical conditions.
result Performance estimates are significantly lower when uncertainty is accounted for.
New research finds uncertainty estimation techniques fail to reliably detect abnormal medical cases.
problem Uncertainty estimation does not reliably detect out-of-distribution patients in medical tabular data.
method A series of tests on various uncertainty estimation techniques on real-world medical data.
result Almost all techniques fail to identify out-of-distribution patients, contradicting earlier findings.
LHM integrates expert ODEs with neural ODEs for disease progression prediction.
problem Predicting disease progression under medications using limited data.
method Integrating expert-designed ODEs with machine-learned Neural ODEs.
result LHM consistently outperforms previous methods, especially in small sample regimes.
Improves reliability of medical diagnosis uncertainty estimates.
problem Label uncertainty in medical diagnosis.
method Post-hoc alpha-calibration method for neural network classifiers. result Significantly enhances reliability of uncertainty estimates.
Machine Learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption by the level of trust that models afford users. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be per…
PRCD-MAP learns to trust imperfect priors in causal discovery, improving accuracy and robustness.
problem Tackles the brittle trade-off between blind trust and rejection of external priors in causal discovery.
method Proposes PRCD-MAP, a soft prior-consumption layer that assigns per-edge trust to imperfect priors and modulates regularization in a MAP objective.
result Enjoys a population-level safety guarantee and outperforms existing methods on real-world causal discovery tasks.
Training set bugs are flaws in the data that adversely affect machine learning. The training set is usually too large for man- ual inspection, but one may have the resources to verify a few trusted items. The set of trusted items may not by itself be adequate for learning, so we propose an algorithm that uses these ite…
New monitoring method detects ML risk models' performance changes in medical interventions.
problem Monitoring ML risk models in healthcare is complicated by confounding medical interventions.
method Developed a new score-based CUSUM monitoring procedure with dynamic control limits.
result Valid inference is possible if conditional exchangeability or time-constant selection bias hold.
ExpCLR uses expert features to improve time-series representation learning.
problem Current representation learning approaches fail to ensure useful properties for time-series data.
method ExpCLR employs expert features to replace data transformations in contrastive learning, ensuring two useful properties for time-series representations.
result ExpCLR outperforms state-of-the-art methods on three real-world time-series datasets.
Many predictive tasks, such as diagnosing a patient based on their medical chart, are ultimately defined by the decisions of human experts. Unfortunately, encoding experts' knowledge is often time consuming and expensive. We propose a simple way to use fuzzy and informal knowledge from experts to guide discovery of int…
Deep learning predicts ICU mortality with enhanced interpretability.
problem Improving mortality prediction accuracy and clinician trust in AI.
method Trained a deep learning model on MIMIC-III to interpret nursing notes.
result Model reaches ROC of 0.8629, outperforming SAPS-II.
Deep learning algorithms produces state-of-the-art results for different machine learning and computer vision tasks. To perform well on a given task, these algorithms require large dataset for training. However, deep learning algorithms lack generalization and suffer from over-fitting whenever trained on small dataset,…
No-knowledge alarms detect misaligned LLM judges without trusting them.
problem Monitoring LLM judges for consistency when ground truth is unknown.
method Formal logic and Linear Programming to detect disagreements.
result No false positives in detecting misaligned judges.
This study applies neural models to automatically recognize medical entities from natural language.
problem Automated recognition of medical entities from natural language is complex and time-consuming.
method Utilizes deep neural sequence models trained on a large dataset of death certificates.
result Deep neural models can efficiently recognize medical entities from natural language.
We have recently seen many successful applications of recurrent neural networks (RNNs) on electronic medical records (EMRs), which contain histories of patients' diagnoses, medications, and other various events, in order to predict the current and future states of patients. Despite the strong performance of RNNs, it is…
Learning from electronic medical records (EMR) is challenging due to their relational nature and the uncertain dependence between a patient's past and future health status. Statistical relational learning is a natural fit for analyzing EMRs but is less adept at handling their inherent latent structure, such as connecti…
This paper improves privacy in federated learning without a trusted server.
problem Privacy in federated learning with silos that distrust each other.
method Introduces Inter-Silo Record-Level Differential Privacy (ISRL-DP) and accelerated algorithms for convex and smooth losses.
result Achieves optimal privacy and accuracy tradeoffs in federated learning.
Deep learning improves brain tumor detection with few data.
problem Scarce training data and data corruption in medical diagnosis.
method Data distillation and augmentation to improve deep neural network training.
result Reaches human expert-level accuracy with just a few thousand training examples.
TRM improves long-horizon LLM RL by masking divergent sequences.
problem Long-horizon reinforcement learning with LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
Risk adjustment has become an increasingly important tool in healthcare. It has been extensively applied to payment adjustment for health plans to reflect the expected cost of providing coverage for members. Risk adjustment models are typically estimated using linear regression, which does not fully exploit the informa…
PLIs improve classifier performance by fine-tuning latent representations.
problem Difficult interpretation of high-dimensional latent representations in neural networks.
method Back-propagation of manual changes to low-dimensional embeddings using t-distributed stochastic neighbourhood embeddings.
result Manual separation of class clusters in latent space enhances classifier performance.
Despite significant advances in artificial intelligence (AI) for computer vision, its application in medical imaging has been limited by the burden and limits of expert-generated labels. We used images from optical coherence tomography angiography (OCTA), a relatively new imaging modality that measures perfusion of the…
TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.
problem Long-horizon reinforcement learning for LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
Machine learning constructs problem-based medical records from electronic health records.
problem Difficulty in finding relevant medical information for clinical questions.
method Knowledge base completion using machine learning on electronic health records.
result Automatic construction of problem-based groupings of medications, procedures, and lab tests.
MCU-Net combines U-Net and Monte Carlo Dropout for uncertainty in medical image segmentation.
problem Lack of uncertainty representation in deep learning methods for patient-centered healthcare decisions.
method MCU-Net framework using U-Net and Monte Carlo Dropout with four uncertainty metrics.
result MCU-Net maximizes automated performance and refers truly uncertain cases.
Develops a two-stage conformal prediction method for Parkinson's disease medication needs.
problem Heterogeneous disease progression and treatment response in Parkinson's Disease.
method Two-stage conformal prediction framework with statistical guarantees.
result Quantifies uncertainty in medication needs predictions, improving clinical trust and quality of life.
Recent efforts in Machine Learning (ML) interpretability have focused on creating methods for explaining black-box ML models. However, these methods rely on the assumption that simple approximations, such as linear models or decision-trees, are inherently human-interpretable, which has not been empirically tested. Addi…
New benchmark evaluates BDL methods in medical retinopathy diagnosis.
problem Evaluate robustness and scalability of BDL methods in medical applications.
method Developed a new benchmark with real-world diabetic retinopathy tasks.
result Some BDL techniques overfit uncertainty to datasets, underperforming on new benchmark.
In many contexts, it can be useful for domain experts to understand to what extent predictions made by a machine learning model can be trusted. In particular, estimates of trustworthiness can be useful for fraud analysts who process machine learning-generated alerts of fraudulent transactions. In this work, we present …
Segmentation maps of medical images annotated by medical experts contain rich spatial information. In this paper, we propose to decompose annotation maps to learn disentangled and richer feature transforms for segmentation problems in medical images. Our new scheme consists of two main stages: decompose and integrate. …
Distributed learning across a coalition of organizations allows the members of the coalition to train and share a model without sharing the data used to optimize this model. In this paper, we propose new secure architectures that guarantee preservation of data privacy, trustworthy sequence of iterative learning and equ…
The paper addresses the difficulty of decision makers trusting AI-assisted predictions and proposes a method to improve confidence values.
problem Decision makers struggle to trust AI-assisted predictions based on confidence values.
method The paper investigates why decision makers have difficulties and proposes a method to construct more useful confidence values.
result Multicalibration with respect to the decision maker's confidence on her own predictions is a sufficient condition for alignment, leading to better decisions.
The paper investigates deep neural networks for medical imaging applications, providing interpretable results.
problem Uninterpretable decisions made by deep neural networks in medical imaging applications.
method Investigation of deep neural networks for malaria, diabetic retinopathy, brain tumor, and tuberculosis detection in various imaging modalities. Visualization of class activation mappings provided.
result Visualization of class activation mappings enhances understanding of deep neural networks and aids doctors in decision-making.
Optimized deferral improves accuracy in imbalanced settings.
problem Imbalance in expert predictions leads to suboptimal performance in two-stage learning to defer.
method Developed novel cost-sensitive learning algorithms and margin-based loss functions tailored for expert imbalance.
result MILD algorithm shows clear improvements over baselines in image classification and LLM routing tasks.
Deep learning models for semantic segmentation of images require large amounts of data. In the medical imaging domain, acquiring sufficient data is a significant challenge. Labeling medical image data requires expert knowledge. Collaboration between institutions could address this challenge, but sharing medical data to…
Framework for deferring decisions to experts in sequential medical settings.
problem Myopic and non-adaptive decision-making by ML models in sequential medical contexts.
method Sequential Learning-to-Defer (SLTD) framework using model-based reinforcement learning.
result Adaptive deferral policy improves trade-off between long-term outcomes and deferral frequency.
SurvMixClust clusters survival data and predicts individual survival curves.
problem Integrating clustering into survival analysis for precision medicine.
method SurvMixClust learns latent representations for clustering and predicts survival functions using a mixture of non-parametric experts.
result SurvMixClust creates balanced clusters with distinct survival curves, outperforming clustering baselines and competing with non-clustering models in predictive accuracy.
Adherence can be defined as "the extent to which patients take their medications as prescribed by their healthcare providers"[Osterberg and Blaschke, 2005]. World Health Organization's reports point out that, in developed countries, only about 50% of patients with chronic diseases correctly follow their treatments. Thi…
This paper tackles deferral learning with multiple experts, providing strong theoretical guarantees.
problem Optimizing input assignment to experts balancing accuracy and computational cost.
method Introducing new surrogate loss functions and efficient algorithms with strong theoretical learning guarantees.
result Realizable H-consistency, H-consistency bounds, and Bayes-consistency for deferral learning. Human-in-the-loop data analysis applications necessitate greater transparency in machine learning models for experts to understand and trust their decisions. To this end, we propose a visual analytics workflow to help data scientists and domain experts explore, diagnose, and understand the decisions made by a binary cl…
This paper presents the first deep reinforcement learning (DRL) framework to estimate the optimal Dynamic Treatment Regimes from observational medical data. This framework is more flexible and adaptive for high dimensional action and state spaces than existing reinforcement learning methods to model real-life complexit…