Develops methods for valid and validated confidence sets in multiclass and multilabel prediction.
problem Challenges of typical conformal prediction methods in multiclass and multilabel problems, especially uneven coverage.
method Leverages quantile regression to build methods that always guarantee correct coverage and asymptotically optimal conditional coverage, addressing label interactions with tree-structured classifiers.
result Empirical evaluation suggests more robust coverage of confidence sets.
New method uses weak labels to create valid confidence sets for predictions.
problem Lack of labeled data in machine learning models.
method Developed a conformal prediction framework to provide valid predictive confidence sets using weakly labeled data.
result New coverage definition allows for tighter and more informative (but valid) confidence sets.
Proposes a test to ensure predictive algorithms predict intended outcomes better than unintended ones.
problem Unintended model behavior leading to prediction of unintended outcomes.
method Falsification framework using nonparametric hypothesis testing to compare prediction losses across outcomes.
result Establishes discriminant validity with respect to gender but not race in an admissions setting.
Develops CPL for optimal prediction set length and validity.
problem Balancing conditional validity and length efficiency in conformal prediction.
method Conformal Prediction with Length-Optimization (CPL).
result Achieves optimal prediction set length while maintaining conditional validity.
Generalizes conformal prediction to multiple learnable parameters for efficient prediction sets.
problem Learning valid and efficient prediction sets with low-capacity function classes.
method Constrained empirical risk minimization (ERM) with gradient-based optimization of differentiable surrogate losses and Lagrangians.
result Achieves approximate valid population coverage and near-optimal efficiency within class.
Proposes a falsification framework to test algorithmic discriminant validity.
problem Unintended model behavior in predictive algorithms.
method Falsification framework based on statistical tests comparing prediction losses across outcomes.
result Establishes discriminant validity for some outcomes but not others.
Valid prediction sets for dynamic graphs using conformal prediction.
problem Uncertainty quantification in dynamic graph neural networks.
method Dynamic graph unfolding for input to GNNs, combined with conformal prediction.
result Valid prediction sets with minimal assumptions, even in challenging scenarios.
Optimizes minimum-volume prediction sets for multivariate regression.
problem Lack of efficient methods for multivariate conformal prediction.
method Optimization-driven framework for minimum-volume covering sets.
result Efficient and informative prediction sets with tight coverage.
A stability-based method selects the most desirable conformal prediction set.
problem Selecting the most desirable conformal prediction set from multiple valid sets invalidates coverage guarantees.
method A stability-based approach that ensures coverage for the selected prediction set.
result The stability-based approach maintains coverage guarantees for the selected prediction set.
Meta-learning reduces set prediction size in conformal prediction for few-shot calibration.
problem Inefficient set prediction in conformal prediction for limited training data.
method Meta-learning approach using cross-validation-based conformal prediction.
result Meta-learning scheme reduces set prediction size and preserves formal guarantees.
The paper extends conformal risk control to be valid with high probability over a growing calibration dataset.
problem Valid risk control over a growing calibration dataset.
method Quantile-based arguments for anytime-valid control.
result Guarantees remain valid with high probability over a cumulatively growing calibration dataset.
COP improves online conformal prediction by incorporating data patterns, leading to tighter prediction sets.
problem Overly conservative prediction sets in online conformal prediction methods when data distribution shifts.
method Conformal Optimistic Prediction (COP) incorporating estimated cumulative distribution function of non-conformity scores.
result COP produces tighter prediction sets with valid coverage guarantees, outperforming other methods.
CPTD improves prediction intervals in time series regression with cross-sectional data.
problem Constructing valid prediction intervals in time series regression with a cross-section.
method Conformal Prediction with Temporal Dependence (CPTD) for post-hoc, light-weight approach.
result CPTD maintains cross-sectional validity while improving longitudinal coverage.
New algorithms for hierarchical classification using conformal prediction.
problem Valid prediction sets in hierarchical classification tasks.
method Extended split conformal prediction framework with two inference algorithms.
result Empirical evaluations show effectiveness in achieving nominal coverage.
New method for robust prediction valid under non-exchangeable data.
problem Non-exchangeability in data violates robustness of common CP methods.
method Introduces efficient CP approach for general non-exchangeable data.
result Produces provably valid confidence sets for non-exchangeable data.
Paper develops new conformal prediction methods for sum or average of unknown labels.
problem Uncertainty quantification in joint distributions of random variables.
method Introduces novel conformal prediction methods for sum or average of unknown labels.
result Validates the proposed method for sum or average of unknown labels under permutation invariant assumptions.
New CV method reduces bias in spatial prediction models.
problem Bias in standard cross-validation due to uneven sampling.
method Target-Weighted Cross-Validation (TWCV) framework.
result Weighted CV approaches reduce bias in prediction error.
While the use of deep learning in drug discovery is gaining increasing attention, the lack of methods to compute reliable errors in prediction for Neural Networks prevents their application to guide decision making in domains where identifying unreliable predictions is essential, e.g. precision medicine. Here, we prese…
Paper develops a new method for open-set and imbalanced classification with valid prediction sets.
problem Tackles open-set and imbalanced classification with new prediction methods.
method Develops a new family of conformal p-values and a selective sample splitting algorithm.
result Valid prediction sets with valid coverage in open-set scenarios and informative predictions under extreme class imbalance.
Develops anytime-valid conformal and PAC prediction for streaming data.
problem Lack of guarantee in traditional conformal methods for sequential settings.
method Extends conformal and PAC prediction frameworks to handle streaming data.
result Provides anytime-valid prediction sets for sequential settings.
Adapts conformal prediction for missing data, ensuring valid coverage.
problem Uncertainty quantification with missing covariates.
method Proposes a reweighted conformal prediction procedure for handling missing values.
result Guaranteed Marginal Coverage and Mask-Conditional Validity for general missing data mechanisms.
Backward Conformal Prediction offers flexible control over prediction set sizes while ensuring coverage guarantees.
problem Providing reliable prediction sets with controlled sizes in applications like medical diagnosis.
method Defines a rule that constrains prediction set sizes based on observed data, adapting coverage levels.
result Maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.
Proposes a new cross-validation method to estimate model performance.
problem The standard cross-validation method does not accurately estimate the performance of the recommended model.
method Develops a new random-effects model framework to improve naive cross-validation estimators.
result Proposed estimators outperform conventional and naive methods in estimating model performance.
ICP improves text infilling and POS tagging with valid confidence sets.
problem Statistical reliability of machine learning predictions.
method Inductive conformal prediction algorithms for text infilling and POS tagging.
result Valid set-valued predictions with small size for real-world applications.
New method improves spatial prediction validation accuracy.
problem Validation methods fail for spatial prediction tasks due to mismatch between validation and test locations.
method Proposes a new validation method that adapts existing covariate-shift ideas to spatial settings.
result Proves and demonstrates the new method's superiority in spatial prediction validation.
A method for making predictions with a reject option using conformal prediction.
problem Uncertainty in machine learning predictions, especially when models are unsure.
method Formalizing ML with reject option, using conformal prediction for distribution-free error guarantees.
result Theoretical guarantees on error rate for prediction sets with distribution-free validity.
The paper proposes a method for valid multi-target regression predictions.
problem Valid multi-variate predictions for multi-target regression.
method Copula functions applied to deep neural networks for inductive conformal prediction.
result The proposed method ensures efficiency and validity for multi-target regression problems.
New method for probabilistic prediction sets with conditional validity.
problem Marginal coverage guarantee of existing methods.
method Combines conformal methods with approximate conditional validity.
result Consistently outperforms existing approaches in conditional coverage.
Proposes PFWCP for multi-agent tasks with privacy and validity guarantees.
problem Challenges in uncertainty quantification for multi-agent settings.
method Personalized federated weighted conformal prediction (PFWCP) combining local density ratio weighting and weighted quantile aggregation.
result Asymptotically valid coverage guarantees for each agent in heterogeneous settings.
Improved TreNet for trend prediction in time series data.
problem Validation method for TreNet did not account for time series data's sequential nature.
method Walk-forward validation method and multiple independent runs to evaluate model stability.
result TreNet still performs better than vanilla DNN models but not on all data sets.
Develop conformal prediction for dyadic regression under complex missingness.
problem Conformal prediction for dyadic regression under complex missingness mechanisms.
method Developing general technical tools and conformal prediction procedures for dyadic regression under complex missingness.
result Establishing asymptotic validity of weighted conformal prediction under a nonparametric graphon model for missingness mechanism.
Extends PPI to sequential setting, improving inference over time.
problem Sequential data growth with unlabelled data.
method Prediction-powered confidence sequence procedures using Ville's inequality and the method of mixtures.
result Asymptotically valid uniformly over time, accommodating prior knowledge.
A new method controls risk for set predictors using cross-validation.
problem Inefficient set predictors when data limited.
method Cross-validation conformal risk control (CV-CRC).
result CV-CRC offers theoretical guarantees and reduces set size.
The paper presents a method to make predictions robust to distributional shifts.
problem Making predictions robust to distributional shifts in machine learning.
method The method uses prediction sets and conformal inference to provide valid coverage under shifts.
result The method achieves nearly valid coverage in finite samples under exchangeable training data.
Unified review of Conformal Prediction theory and applications.
problem Distribution-free, non-parametric forecasting method for valid prediction sets.
method Minimal assumptions, straightforward predictions sets valid in finite sample cases.
result Unified review of Conformal Prediction theory and applications.
The paper assesses quality measures for machine learning models using cross-validation.
problem Evaluating the accuracy and robustness of quality measures for machine learning models.
method Cross-validation approach to estimate prediction error and quantify explained variation. Confidence bounds and local quality measures derived from residuals.
result The reliability and robustness of quality measures are assessed through numerical examples and confidence bounds.
Validates conformal prediction for network data under non-uniform sampling.
problem Validity of conformal prediction for network data under non-representative sampling.
method Interprets sampling mechanisms as selection rules, studies validity conditional on selection events, uses permutation invariance and joint exchangeability.
result Finite-sample validity of conformal prediction for certain selection events and asymptotic validity for random walk sampling.
Paper extends prediction-powered inference using conformal prediction for robust and valid imputation.
problem Safe use of black-box ML models for imputing missing data with strong guarantees.
method Connecting prediction-powered inference with conformal prediction for valid and additional guarantees.
result First general prediction-powered procedure for e-values operating off-line.
Cross-validation estimates model performance on unseen data, not training data.
problem Understanding how cross-validation estimates prediction error and its limitations.
method Analyzing linear models and popular prediction error estimates, introducing nested cross-validation.
result Cross-validation estimates the average prediction error of models fit on other unseen training sets, not the model at hand.
Paper uses neural networks to predict NOx emissions from gas turbines.
problem Predicting NOx emissions from degrading gas turbines.
method Applied neural network algorithm to model NOx emissions from nine process variables.
result Neural network model optimizes process variables for minimal NOx emissions.
Paper discusses optimal CP for second-order predictions.
problem How to incorporate second-order predictions into conformal prediction.
method Introduces Bernoulli prediction sets (BPS) for second-order predictions and applies conformal risk control for compromised validity.
result BPS provides the smallest prediction sets with conditional coverage.
MOPI optimizes flexible set-valued mappings to achieve superior shape adaptivity in conformal prediction.
problem Challenges in achieving valid conditional coverage in conformal prediction.
method Minimax Optimization Predictive Inference (MOPI) framework that optimizes over a flexible class of set-valued mappings.
result MOPI achieves superior shape adaptivity and maintains a principled connection to mean squared coverage error.
Posterior conformal prediction improves prediction interval validity for subgroups.
problem Marginal and conditional prediction interval validity for subgroups.
method Modeling conditional nonconformity score distribution as a mixture of cluster distributions.
result PCP produces tighter prediction intervals, especially for well-represented clusters.
Improves conformal prediction by combining multiple score functions and optimizing weights.
problem Limitations of single-score conformal predictors in multi-class classification.
method Combines multiple score functions and optimizes weights to minimize prediction set size.
result Consistently outperforms single-score conformal predictors while maintaining valid coverage.
The paper develops distribution-free methods for ordinal classification.
problem Constructing valid prediction sets for ordinal classification problems.
method Leveraging conformal prediction and multiple testing with FWER control.
result The proposed methods achieve satisfactory levels of marginal and class-specific conditional coverages.
CoDrug uses KDE to create valid prediction sets for drug molecules under covariate shift.
problem Creating reliable uncertainty estimates for drug properties from computational models.
method CoDrug employs an energy-based model and KDE to assess and rectify distribution shift.
result CoDrug reduces the coverage gap by over 35% compared to non-adjusted conformal prediction sets.
Adapts self-supervised learning using probabilistic sets with validity guarantees.
problem Lack of validity guarantees in pseudo-labels from self-supervised learning.
method Uses conformal prediction to provide validity guarantees for probabilistic labels.
result Valid probabilistic labels improve calibration and performance.
Flexible framework integrates machine learning and DRO for uncertain parameter prediction.
problem Limited joint observations of uncertain parameters and covariates.
method Wasserstein, sample robust optimization, and phi-divergence-based ambiguity sets.
result Validation of theoretical and practical benefits in limited data scenarios.