Study robustness of split conformal prediction in data contamination setting.
problem Robustness of split conformal prediction under data contamination.
method Analyze split conformal prediction's performance in a contaminated data setting and propose a new method.
result Demonstrated the impact of corrupted data on prediction intervals' coverage and efficiency.
Combining Bayesian deep learning and split conformal prediction affects out-of-distribution coverage.
problem Improving out-of-distribution coverage in multiclass image classification.
method Combining Bayesian deep learning with split conformal prediction methods.
result Combining methods can reduce out-of-distribution coverage in some cases.
Optimizes data splitting for shorter conformal prediction intervals.
problem Minimizing prediction interval length while maintaining coverage.
method Theoretical framework for optimal data splitting in split conformal prediction.
result Analytical characterizations of length-optimal split ratios in various settings.
New privacy-preserving method for conformal prediction without splitting data.
problem Privacy and uncertainty quantification in data-driven decision making.
method Proposes a full-data privacy-preserving conformal prediction framework using differential privacy.
result Demonstrates improved prediction sets compared to split-based private baselines.
MCP extends conformal prediction to vector-valued score functions without data splitting.
problem Fixed prediction set shapes in scalar score functions limit coverage guarantees.
method MCP uses a single optimization problem for prediction set design and calibration, eliminating data splitting.
result RemMCP and RelMCP achieve target coverage with smaller or comparable prediction set sizes, reducing variance.
Split conformal prediction provides finite-sample guarantees for black-box models without distributional assumptions.
problem Weak performance guarantees for modern predictive models under minimal assumptions.
method Develops finite-sample guarantees for split conformal prediction, a method that uses nested prediction sets and order statistics.
result The coverage of prediction sets based on order statistics stochastically dominates the Beta distribution.
Exact distribution of split conformal prediction coverage found.
problem Determining the reliability of prediction sets in batch mode.
method Analysis of exchangeable data to find universal distribution of empirical coverage.
result Exact distribution of empirical coverage is universal and determined by nominal miscoverage level and calibration sample size.
Most existing examples of full conformal predictive systems, split-conformal predictive systems, and cross-conformal predictive systems impose severe restrictions on the adaptation of predictive distributions to the test object at hand. In this paper we develop split-conformal and cross-conformal predictive systems tha…
Conformal predictive systems are a recent modification of conformal predictors that output, in regression problems, probability distributions for labels of test observations rather than set predictions. The extra information provided by conformal predictive systems may be useful, e.g., in decision making problems. Conf…
Paper presents unsupervised calibration for split conformal classification.
problem Inconvenient requirement of labeled calibration samples.
method Uses unsupervised calibration samples alongside supervised training samples.
result Achieves comparable performance to supervised calibration methods.
Proposes SCD-split for CP to balance interpretability and efficiency.
problem Difficult interpretation of disconnected subintervals in CP prediction sets.
method Incorporates smoothing operations into CP framework.
result SCD-split balances interval length and subinterval number, theoretically provable.
LoBoost improves local conformal prediction for gradient-boosted trees without extra data splits.
problem Quantifying uncertainty in gradient-boosted tree predictions.
method Model-native local conformal prediction using leaf structure.
result Competitive interval quality and improved test MSE with large calibration speedups.
Study robustness of split conformal prediction under adversarial attacks.
problem Ensuring distribution-free coverage guarantees in CP under adversarial conditions.
method Theoretical analysis and extensive experiments on split conformal prediction robustness.
result Prediction coverage varies with calibration-time attack strength, enabling control over coverage under adversarial tests.
New methods improve prediction regions for high-dimensional data.
problem Creating effective prediction regions for high-dimensional data.
method CD-split and HPD-split methods that combine split method and data-driven partition.
result CD-split and HPD-split converge to oracle highest predictive density set and satisfy local and asymptotic conditional validity.
MD-split+ creates locally valid prediction regions for complex data.
problem Localized prediction regions for complex data.
method Localized model performance-based partitioning of feature space X.
result MD-split+ creates valid prediction regions that scale to high dimensions.
CAOS aggregates multiple one-shot predictors for efficient uncertainty quantification.
problem Lack of principled uncertainty quantification in one-shot prediction.
method CAOS, a conformal framework that aggregates multiple one-shot predictors and uses a leave-one-out calibration scheme.
result CAOS produces smaller prediction sets with reliable coverage compared to split conformal baselines.
This work approximates full conformal prediction for neural networks without sample splitting.
problem Uncertainty quantification for neural network regression models.
method Approximating full conformal prediction using Gauss-Newton influence for post-hoc uncertainty estimation.
result Locally-adaptive and often tighter prediction intervals compared to split-CP.
Proposes a new method to minimize non-singleton predictions in conformal prediction.
problem Large prediction sets in conformal prediction are costly and inefficient.
method Introduces a new nonconformity score to minimize non-singleton sets and provides an algorithm to compute it efficiently.
result The proposed Singleton-Optimized Conformal Prediction (SOCOP) method increases singleton frequency by over 20% compared to standard scores, with minimal impact on average set size.
Paper proposes a new method for conditional coverage in conformal prediction.
problem Lack of strong conditional coverage guarantees in existing conformal prediction methods.
method Modified non-conformity score using local approximation of conditional distribution.
result Unified framework and empirical evaluations show advantage of the new method.
A new method for predicting insurance claims with statistical guarantees.
problem Creating accurate prediction intervals for insurance claims.
method Model-agnostic framework using split conformal prediction for frequency-severity modeling.
result Shows effectiveness on simulated and real datasets using various models.
TA-CQR predicts regression intervals with exact coverage, splitting miscoverage between endpoints.
problem Predicting regression intervals with exact coverage under reporting constraints.
method TA-CQR uses tail allocation to parameterize the oracle, estimating the allocation by searching quantile cores and applying nonnegative additive split-conformal calibration.
result TA-CQR achieves exact finite-sample marginal coverage under exchangeability, with theoretical guarantees on calibration and length.
Differentially private conformal prediction improves statistical efficiency.
problem Quantifying uncertainty in private data analysis.
method Introducing differential conformal prediction and developing Differentially Private Conformal Prediction (DPCP).
result DPCP produces tighter prediction sets than existing private split conformal approaches.
Study optimizes prediction intervals in conformal regression.
problem Optimizing the length of prediction intervals in conformal regression.
method Introduces EffOrt and Ad-EffOrt methodologies to minimize interval length.
result Demonstrates theoretical and empirical improvements over classical methods.
Modified jackknife method improves predictive inference for time series data.
problem Lack of exchangeability and temporal dependence in time series data.
method Leave-a-window-out (LWO) method modification of the jackknife.
result LWO method achieves valid coverage in time series models with mild temporal dependence.
This paper improves prediction intervals for heteroskedastic regression.
problem Adaptive prediction intervals for heteroskedastic regression.
method Normalized and Mondrian conformal prediction methods.
result Conditional validity of chosen conformal predictors related to data-generating assumptions.
coverforest speeds up conformal predictions for random forests.
problem Efficient uncertainty quantification for random forest predictions.
method Optimized Python package leveraging random forest's out-of-bag scores for cross-conformal predictions.
result coverforest achieves desired coverage with faster training and prediction times.
Skew-adaptive method improves prediction intervals for regression.
problem Improving prediction intervals for regression models, especially in cases of skewness and varying scales.
method Develops a skew-adaptive extension of split conformal prediction using an asymmetric interval family and gauge approach.
result Preserves marginal validity and adapts to local scale and skewness, with efficiency gains over existing methods.
LOO-StabCP speeds up CP for multiple predictions.
problem Balancing computational efficiency and prediction accuracy in CP.
method Leave-One-Out Stable Conformal Prediction (LOO-StabCP) using algorithmic stability.
result LOO-StabCP is faster and more accurate than RO-StabCP.
Split conformal prediction works well for time series despite temporal dependence.
problem Uncertainty quantification for time series predictions with past data.
method Split conformal prediction method for time series data with predictors having memory.
result Theoretical bounds on coverage probability for split conformal prediction in time series with memory.
ICP improves prediction intervals for continuous outcomes at lower computational cost.
problem Systematic bias in point predictions that undermines their use in decision-making.
method Develops Isotonic Conformal Prediction (ICP) framework to decouple calibration from prediction-set construction.
result SICP and TICP procedures match SC-CP coverage at lower computational cost.
SConU improves uncertainty prediction for large models by testing for outliers and reducing miscoverage.
problem Real-world deployment of large language models requires reliable guarantees of task-specific metrics.
method SConU implements significance tests to identify and exclude outliers that violate exchangeability assumptions.
result SConU reduces miscoverage rates and enhances prediction efficiency in high-stakes tasks.
New method for conformal prediction under Markovian data reduces coverage gap.
problem Reducing coverage gap in conformal prediction for Markovian data.
method Split Conformal Prediction method adapted to Markovian data, with K-split CP for improved performance.
result Coverage gap typically scales as √(t_mix * ln(n) / n) for general Markov chains, and can be reduced to t_mix / (n * ln(n)) with K-split CP.
Extends conformal prediction for controlling expected risk of monotone loss functions.
problem Controlling expected risk of monotone loss functions.
method Generalizes split conformal prediction with coverage guarantee, extending to distribution shift, quantile risk, multiple, adversarial, and expectations of U-statistics.
result Tight up to an O(1/n) factor, with worked examples in computer vision and natural language processing. Adaptive uncertainty quantification improves black-box model predictions in generative AI.
problem Improving uncertainty quantification for black-box models in generative AI.
method Adaptive partitioning and local calibration of conformity scores.
result Local tightening of uncertainty sets with adaptive bands.
New methods improve anomaly detection with reduced false positives.
problem Effective anomaly detection with controlled error rates.
method Leave-one-out-, bootstrap-, and cross-conformal anomaly detection methods.
result Improved anomaly detection with reduced false positives.
We develop a general framework for distribution-free predictive inference in regression, using conformal inference. The proposed methodology allows for the construction of a prediction band for the response variable using any estimator of the regression function. The resulting prediction band preserves the consistency …
New algorithms for hierarchical classification using conformal prediction.
problem Valid prediction sets in hierarchical classification tasks.
method Extended split conformal prediction framework with two inference algorithms.
result Empirical evaluations show effectiveness in achieving nominal coverage.
This work introduces COLA, a strategy to aggregate conformal prediction sets efficiently.
problem Efficiently combining multiple conformity scores to reduce prediction set size.
method Introduces COnfidence-Level Allocation (COLA) to optimally allocate confidence levels across sets.
result COLA achieves smaller prediction sets than state-of-the-art methods while maintaining valid coverage.
Conformal methods create prediction bands that control average coverage under no assumptions besides i.i.d. data. Besides average coverage, one might also desire to control conditional coverage, that is, coverage for every new testing point. However, without strong assumptions, conditional coverage is unachievable. Giv…
The paper analyzes the expected size of conformal prediction sets.
problem Lack of finite-sample analysis and guarantees for prediction set sizes.
method Theoretical quantification and empirical computation of expected set size.
result Derives point estimates and high-probability interval bounds for prediction set size.
A method for non-parametric conditional distribution estimation using CRPS-optimal binning.
problem Non-parametric conditional distribution estimation.
method Partitioning covariate-sorted observations into bins to minimize LOO-CRPS, selecting K by K-fold cross-validation of test CRPS.
result Produces narrower prediction intervals with near-nominal coverage compared to split-conformal competitors.
Paper develops a new method for open-set and imbalanced classification with valid prediction sets.
problem Tackles open-set and imbalanced classification with new prediction methods.
method Develops a new family of conformal p-values and a selective sample splitting algorithm.
result Valid prediction sets with valid coverage in open-set scenarios and informative predictions under extreme class imbalance.
Paper proposes a new method for predicting DER adoption with hierarchical guarantees.
problem Accurately predicting DER adoption in electric grids with uncertainty and spatial disparity.
method Multivariate Hawkes process for modeling DER adoption dynamics and split conformal prediction algorithm for hierarchical validity.
result Empirical evaluation shows superior predictive accuracy and uncertainty calibration compared to existing methods.
New method improves predictive systems with better theoretical guarantees.
problem Constructing predictive systems with out-of-sample calibration guarantees.
method Residual Distribution Predictive Systems (RDPs) that nest conformal predictive systems and offer flexibility.
result Empirically, RDPs perform competitively with conformal predictive systems and can be implemented with various regression methods.
Improves conditional coverage of regression models using conformal prediction.
problem Lack of conditional coverage guarantees in conformal prediction methods.
method Proposes a novel algorithm to train a regression function to improve conditional coverage after split conformal prediction.
result Establishes an upper bound for miscoverage gap and proposes an end-to-end algorithm to control it.
New methods improve prediction intervals across multiple environments.
problem Valid confidence intervals and sets in multi-environment prediction.
method Extended jackknife and split-conformal methods, with resizing for problem difficulty.
result Distribution-free coverage achieved in non-traditional data scenarios.
New method improves conditional coverage of conformal prediction.
problem Improving conditional coverage in conformal prediction.
method Trainable transformation of conformity scores to improve conditional coverage.
result Highly adaptive to local data structure, outperforming existing methods.
Develop conformal prediction for dyadic regression under complex missingness.
problem Conformal prediction for dyadic regression under complex missingness mechanisms.
method Developing general technical tools and conformal prediction procedures for dyadic regression under complex missingness.
result Establishing asymptotic validity of weighted conformal prediction under a nonparametric graphon model for missingness mechanism.