The paper develops methods to predict the probability of achieving a user goal in a task, ensuring the system alerts when the probability falls below a threshold.
problem Ensuring an autonomous system achieves the user's goal with calibrated probability estimates.
method Invertible conformal prediction using Probability-space Conformalized Quantile Regression (PCQR) to produce well-calibrated conditional prediction intervals.
result The method produces well-calibrated probabilities that the cumulative reward will fall within a user-specified target interval, with finite-sample guarantees.
Debias recommender systems by accounting for hidden confounders using network information.
problem Debiased recommender systems to reduce bias caused by hidden confounders.
method Leverage network information to disentangle user conformity and item popularity, modeling exposure and ratings while controlling hidden confounders.
result The proposed method effectively debiases recommender systems, improving recommendation accuracy.
Proposes a method to improve class-conditional conformal prediction for many classes.
problem Weak guarantees for specific classes in classification problems.
method Clusters classes with similar conformal scores and performs conformal prediction at the cluster level.
result Clustered conformal typically outperforms existing methods in class-conditional coverage and set size metrics.
New methods protect privacy while providing accurate prediction sets.
problem Privacy-preserving conformal prediction for untrusted aggregators.
method Two LDP approaches: k-ary randomized response and binary search response.
result Finite-sample coverage guarantees and robust coverage under randomization.
New method calibrates probabilistic linear solver for online coverage guarantees.
problem Uncertainty in probabilistic linear solver solutions without coverage guarantees.
method Online conformal prediction-PLS (OCP-PLS) method to calibrate uncertainty thresholds.
result Validates online calibration of uncertainty thresholds via online conformal prediction.
Enhances conformal prediction for better uncertainty estimates in armed conflict fatalities.
problem Lack of individual-level uncertainty estimates in existing forecasting models.
method Introduces bin-conditional conformal prediction (BCCP) to improve coverage rates across subsets of the outcome variable.
result Demonstrates improved local coverage and well-calibrated uncertainty estimates across various ranges of fatalities.
Robot motions in the presence of humans should not only be feasible and safe, but also conform to human preferences. This, however, requires user feedback on the robot's behavior. In this work, we propose a novel approach to leverage the user's brain signals as a feedback modality in order to decode the judgment of rob…
PH-CS selects test inputs with reliability guarantees, adapting FDR to data.
problem Fixed FDR limits adaptability to downstream needs.
method Post-hoc conformal selection with e-variables, e-BH procedure.
result PH-CS provides reliable FDP estimates and competitive FDR control.
Adapts safe policies for exploration in high-risk settings.
problem Balancing safety and exploration in high-risk environments.
method Uses conformal calibration on a safe reference policy to determine aggressive action limits.
result Safe exploration improves performance without requiring model class identification or hyperparameter tuning.
Conformal prediction provides distribution-free uncertainty quantification for black-box models.
problem Uncertainty quantification for high-risk machine learning applications.
method Conformal prediction creates valid uncertainty sets without distributional assumptions.
result Sets contain the ground truth with a specified probability, e.g., 90%.
Method certifies edge predictions with cloud-level reliability.
problem Ensuring reliability of edge intelligence models.
method Conformal alignment-based cascading mechanism.
result Certifies conditional coverage with user control over risk level.
A new algorithm for time series prediction intervals.
problem Non-exchangeability in time series data.
method Adaptive re-estimation of non-conformity scores.
result Significant reduction in interval width compared to existing methods.
Adaptive Quantum Conformal Prediction improves reliability of quantum machine learning predictions.
problem Quantum machine learning lacks robust uncertainty quantification methods.
method Adaptive Conformal Inference applied to quantum conformal prediction to maintain validity over time.
result AQCP achieves target coverage levels and is more stable than standard quantum conformal prediction.
Extends conformal prediction to contrastive learning for better coverage of positive samples.
problem Lack of principled guarantees on coverage in contrastive learning.
method Introduces minimum-volume covering sets with learnable constraints.
result Improves inclusion-exclusion trade-offs in positive and negative samples.
Recently, how to expand data transmission to reduce cell data and repeated cell transmission has received more and more research attention. In mobile social networks, content popularity prediction has always been an important part of traffic offloading and expanding data dissemination. However, current mainstream conte…
Proposes a new method to unlearn from specific data points in conformal predictors.
problem Challenges of existing unlearning methods in conformal predictors.
method Formalizes conformal unlearning, introduces practical metrics, and presents an optimization algorithm.
result Demonstrates effective removal of targeted information while preserving utility.
SConU improves uncertainty prediction for large models by testing for outliers and reducing miscoverage.
problem Real-world deployment of large language models requires reliable guarantees of task-specific metrics.
method SConU implements significance tests to identify and exclude outliers that violate exchangeability assumptions.
result SConU reduces miscoverage rates and enhances prediction efficiency in high-stakes tasks.
This work connects conformal prediction to information theory for uncertainty estimation.
problem Uncertainty estimation in machine learning models.
method Information theory applied to conformal prediction.
result Proven ways to upper bound intrinsic uncertainty using information theory.
MACI improves LLM factuality inference with higher retention and lower time cost.
problem Ensuring factuality in LLM responses for high-stakes domains.
method Reformulated conformal inference in a multiplicative filtering setting, leveraging ensembles for more accurate factuality scores and group-conditional calibration.
result MACI achieves higher retention and lower time cost compared to baselines, preserving validity through group-conditional calibration.
New method calibrates uncertainty estimates for image classifiers without labeled data.
problem Uncertainty estimates for modern classifiers are unreliable without labeled calibration data.
method Calibrates uncertainty estimates using unlabeled examples for distribution shifts.
result Proposes a method that provides excellent uncertainty estimates under natural distribution shifts.
E-scores assess LLM outputs for correctness, addressing p-hacking issues.
problem Limited principled mechanisms to assess generative model correctness.
method Use e-values to complement LLM outputs with e-scores, providing flexibility in tolerance levels.
result Achieves guarantees of correctness assessment and upper bounds size distortion.
New conformal prediction methods for long-tailed classification problems.
problem Rare classes are systematically omitted in existing conformal prediction methods.
method Introduced a new conformal score function and a new interpolation procedure.
result Smoothly trade off set size and class-conditional coverage.
Novel framework for uncertainty quantification in neurosymbolic programs.
problem Lack of correctness guarantees in neurosymbolic programs due to machine learning model fallibility.
method Adapting conformal prediction to neurosymbolic programs using abstract interpretation.
result Framework provides probabilistic guarantees for correctness, compositionality, and structured values.
Study explores efficient data division for ICPs.
problem Efficiently dividing limited development data for ICPs.
method Experiments with training, calibration, and test data divisions.
result Allows overlap between training and calibration sets improves efficiency.
Adaptive coverage policies improve conformal prediction accuracy.
problem Fixed coverage levels in traditional conformal prediction lead to uninformative predictions.
method Optimizes adaptive coverage policy using a neural network trained on leave-one-out calibration.
result Adaptive coverage policies produce more informative and flexible prediction sets.
Posterior conformal prediction improves prediction interval validity for subgroups.
problem Marginal and conditional prediction interval validity for subgroups.
method Modeling conditional nonconformity score distribution as a mixture of cluster distributions.
result PCP produces tighter prediction intervals, especially for well-represented clusters.
This paper improves deep learning classifiers by integrating conformal prediction during training.
problem High-stake AI applications require reliable uncertainty estimates for safe deployment.
method Integrates conformal prediction (CP) during training of deep learning models.
result Reduces inefficiency and allows more control over confidence sets.
A framework for private prediction sets using conformal prediction and differential privacy.
problem Jointly addressing reliability and privacy in machine learning predictions.
method Split conformal prediction with privatized quantile subroutine.
result Private prediction sets can be generated from privately-trained models.
Community moderation drifts towards majority, study finds.
problem How to ensure crowd-sourced moderation systems trust and reward accurate evaluations.
method Consensus-based auditing with a two-stage algorithm that weights contributors by the stability of their past residuals.
result Minority contributors' evaluations drift towards the majority, and their participation share falls on controversial topics.
Framework controls uncertainty in LLMs without labels or probabilities.
problem Managing uncertainty in black-box LLMs without token-level probability or true labels.
method Integrates generative models, UCP, and conformal alignment to control uncertainty.
result Achieves close-to-nominal coverage and tighter thresholds than split UCP.
TS improves class coverage but reduces CP set size, offering a trade-off.
problem Combining temperature scaling with conformal prediction for deep classifiers.
method Empirical study and mathematical theory of TS's effect on CP.
result TS allows trading prediction set size and conditional coverage.
Autodock is a widely used molecular modeling tool which predicts how small molecules bind to a receptor of known 3D structure. The current version of AutoDock uses meta-heuristic algorithms in combination with local search methods for doing the conformation search. Appropriate settings of hyperparameters in these algor…
AutoCP automates the construction of accurate prediction intervals.
problem Creating valid and accurate prediction intervals for machine learning models.
method AutoML framework that optimizes prediction interval length for better accuracy and less conservatism.
result AutoCP significantly outperforms benchmark algorithms in constructing accurate prediction intervals.
COMA combines prediction sets from multiple models for online, adaptive prediction.
problem Combining multiple prediction models with uncertainty guarantees.
method Online model aggregation using weighted voting of conformal prediction sets.
result COMA retains coverage guarantees under negative correlation assumptions.
HCC extends conformal prediction to handle class hierarchies, improving prediction reliability.
problem Uncertainty quantification in classification models with class hierarchy considerations.
method Formulates HCC as a constrained optimization problem, ensuring coverage guarantees with a smaller subset of candidate solutions.
result HCC produces more reliable prediction sets by leveraging class hierarchy information.
The paper proposes a method to construct well-calibrated prediction sets for correlated target variables.
problem Constructing well-calibrated prediction sets for correlated target variables.
method The method uses vine copulas to estimate the joint cumulative distribution function of non-conformity scores and improves the asymptotic efficiency of the quantile estimate.
result The method guarantees asymptotically exact coverage and competitive efficiency on real-world regression problems.
Develops PromptShift-CRC for drift-aware conformal risk control in foundation models under prompt and domain shift.
problem Fixed calibration risk in foundation models due to prompt and domain shift.
method Embeds prompts and responses, measures drift, gives more weight to recent examples, and updates risk online.
result Develops method to control risk up to terms for distribution mismatch and weighted quantile uncertainty.
Proposes methods for online conformal prediction with nested prediction sets across multiple confidence levels.
problem Need for uncertainty quantification with multiple confidence levels in diverse applications.
method Online optimization perspective to enforce nestedness of prediction sets while controlling quantile estimation error.
result Achieves stable coverage across all levels, strictly nested prediction sets, and improved efficiency.
Paper discusses optimal CP for second-order predictions.
problem How to incorporate second-order predictions into conformal prediction.
method Introduces Bernoulli prediction sets (BPS) for second-order predictions and applies conformal risk control for compromised validity.
result BPS provides the smallest prediction sets with conditional coverage.
A method for making predictions with a reject option using conformal prediction.
problem Uncertainty in machine learning predictions, especially when models are unsure.
method Formalizing ML with reject option, using conformal prediction for distribution-free error guarantees.
result Theoretical guarantees on error rate for prediction sets with distribution-free validity.
Adaptive uncertainty quantification improves black-box model predictions in generative AI.
problem Improving uncertainty quantification for black-box models in generative AI.
method Adaptive partitioning and local calibration of conformity scores.
result Local tightening of uncertainty sets with adaptive bands.
This paper improves prediction accuracy for multi-input classification tasks using p-value aggregation.
problem Generating accurate predictive sets with guaranteed coverage for multi-input classification tasks.
method Integrates p-values from each observation to reduce the size of the predicted label set while maintaining class-conditional coverage.
result The method reduces the size of the predicted label set while preserving the required coverage guarantee.
A method selects candidates based on predictions with statistical control.
problem Screening candidates for resource-intensive steps like hiring or drug discovery.
method Wraps around any prediction model to produce a subset of candidates with controlled false selection rate.
result Empirically demonstrates selection of candidates whose predictions exceed a data-dependent threshold.
This paper improves conformal prediction to be robust to perturbations.
problem Ensuring robustness of conformal prediction to natural and adversarial perturbations.
method Probabilistically robust conformal prediction (PRCP) and its adaptive version (aPRCP).
result aPRCP achieves better trade-offs between nominal performance and robustness.
New method provides formal uncertainty guarantees for image classifiers.
problem Uncertainty quantification for image classifiers without formal guarantees.
method Adapts conformal prediction to give stable, formal coverage guarantees.
result Method outperforms existing approaches in coverage and set size.
NCP improves deep classifier uncertainty quantification efficiency.
problem Uncertainty quantification for deep classifiers in high-stake applications.
method Neighborhood Conformal Prediction (NCP) algorithm.
result NCP produces smaller prediction sets than traditional CP methods.
Method predicts biomarker trajectories with uncertainty bands for Alzheimer's disease.
problem Uncertainty in biomarker predictions poses risks in clinical deployment.
method Conformal prediction for randomly-timed biomarker trajectories.
result Conformal bands achieve desired coverage and are tighter than baseline.
Conformal Alignment ensures trustworthy outputs from foundation models.
problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.