Proposes methods for online conformal prediction with nested prediction sets across multiple confidence levels.
problem Need for uncertainty quantification with multiple confidence levels in diverse applications.
method Online optimization perspective to enforce nestedness of prediction sets while controlling quantile estimation error.
result Achieves stable coverage across all levels, strictly nested prediction sets, and improved efficiency.
Fuzzy prediction sets generalize binary predictions to include elements at varying confidence levels.
problem Binary prediction sets are limited; fuzzy prediction sets offer richer guarantees.
method Generalize prediction sets to fuzzy sets, showing they are e-values with merging properties.
result Optimal e-values lead to optimal fuzzy prediction sets, including optimal conformal prediction.
Develops methods to adjust prediction set coverage based on post-selection analysis.
problem Adjusting prediction set coverage after initial analysis to better fit specific needs.
method Post-selection conformal inference to adjust miscoverage levels.
result Allows for trade-off between coverage and prediction set quality.
Adaptive coverage policies improve conformal prediction accuracy.
problem Fixed coverage levels in traditional conformal prediction lead to uninformative predictions.
method Optimizes adaptive coverage policy using a neural network trained on leave-one-out calibration.
result Adaptive coverage policies produce more informative and flexible prediction sets.
The paper explores how neural networks make predictions using probabilistic programming.
problem Understanding how neural networks make individual predictions.
method Defining and sampling prediction level sets using probabilistic programming.
result The method can obtain examples that result in specified predictions by neural networks.
New method improves transductive learning predictions with multiplicative oracle inequalities.
problem Improving transductive learning predictions with known covariates.
method Median of Level-Set Aggregation (MLSA) for transductive LOO prediction.
result Proved multiplicative oracle inequality for LOO error.
This work introduces COLA, a strategy to aggregate conformal prediction sets efficiently.
problem Efficiently combining multiple conformity scores to reduce prediction set size.
method Introduces COnfidence-Level Allocation (COLA) to optimally allocate confidence levels across sets.
result COLA achieves smaller prediction sets than state-of-the-art methods while maintaining valid coverage.
This paper optimizes predicting support and resistance levels in financial markets.
problem Optimizing prediction of resistance and support levels in financial markets.
method Assuming a constant elasticity of variance process, the paper derives optimal trading boundaries using the aspiration level hypothesis.
result Optimal trading boundaries serve as predictors of resistance and support levels, located relative to the median interval of the hidden aspiration level.
LOO prediction method improves generalization guarantees for arbitrary datasets.
problem Understanding LOO error guarantees in fully transductive settings for arbitrary datasets.
method Median of Level-Set Aggregation (MLSA) for empirical-risk level sets.
result Multiplicative oracle inequality for LOO error with complexity scaling.
Deep learning predicts curvature of 2D interfaces in level-set method.
problem Estimating curvature in level-set method for complex interfaces.
method Deep learning using feed-forward neural networks trained on synthetic data.
result Deep learning models approximate curvature with comparable precision to traditional methods.
New bounds on efficiency for conformalized regression methods.
problem Efficiency of conformal prediction in regression models.
method Non-asymptotic bounds on prediction set length for conformalized quantile and median regression.
result Identifies phase transitions in convergence rates across different regimes of miscoverage level.
An automated metric to evaluate dialogue quality is vital for optimizing data driven dialogue management. The common approach of relying on explicit user feedback during a conversation is intrusive and sparse. Current models to estimate user satisfaction use limited feature sets and employ annotation schemes with limit…
DPSM minimizes prediction set size by integrating conformal principles into deep classifier training.
problem Large prediction sets from standard conformal methods are impractical.
method Formulates conformal training as bilevel optimization, proposing DPSM algorithm.
result Significantly reduces prediction set size compared to prior methods.
Backward Conformal Prediction offers flexible control over prediction set sizes while ensuring coverage guarantees.
problem Providing reliable prediction sets with controlled sizes in applications like medical diagnosis.
method Defines a rule that constrains prediction set sizes based on observed data, adapting coverage levels.
result Maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.
Method certifies edge predictions with cloud-level reliability.
problem Ensuring reliability of edge intelligence models.
method Conformal alignment-based cascading mechanism.
result Certifies conditional coverage with user control over risk level.
Study optimal ridge regularization for out-of-distribution prediction.
problem Optimal ridge regularization for predicting out-of-distribution data.
method Established conditions for optimal regularization under covariate and regression shifts, proving monotonic risk in data aspect ratio.
result Negative regularization can be optimal under shifts, even with isotropic or underparameterized training features.
MAPS algorithm creates reliable prediction intervals for high-dimensional data.
problem Computing reliable conditional prediction intervals in high-dimensional settings.
method Lifted predictive model (LPM) and MAPS algorithm for distribution-free intervals.
result MAPS algorithm produces valid prediction intervals for any trained model.
The paper finds a fundamental trade-off between confidence and efficiency in transductive conformal prediction.
problem The challenge is to balance confidence and efficiency in predicting multiple data points.
method The authors derive a strict finite-sample bound and introduce a practical algorithm to approach this bound.
result Any non-trivial confidence level leads to exponential growth in prediction set size, with a linear scaling in the number of samples.
Successful multimodal search and retrieval requires the automatic understanding of semantic cross-modal relations, which, however, is still an open research problem. Previous work has suggested the metrics cross-modal mutual information and semantic correlation to model and predict cross-modal semantic relations of ima…
Develops probabilistic forecasting for Sea Level Anomalies using Conformal Prediction on functional time series.
problem Forecasting and uncertainty quantification for Sea Level Anomalies.
method Functional data analysis, Conformal Prediction, Functional Autoregressive Processes.
result Proposed method provides accurate probabilistic predictions and uncertainty quantification for Sea Level Anomalies.
Most of the existing recommender systems use the ratings provided by users on individual items. An additional source of preference information is to use the ratings that users provide on sets of items. The advantages of using preferences on sets are two-fold. First, a rating provided on a set conveys some preference in…
Bayesian approach improves online prediction accuracy without distributional assumptions.
problem Online construction of confidence sets for black-box models.
method Combines empirical distribution with Bayesian regularization to predict quantiles.
result Adaptive algorithm with low regret and correct coverage probability for iid data.
Study explores efficient data division for ICPs.
problem Efficiently dividing limited development data for ICPs.
method Experiments with training, calibration, and test data divisions.
result Allows overlap between training and calibration sets improves efficiency.
Route Choice Models predict the route choices of travelers traversing an urban area. Most of the route choice models link route characteristics of alternative routes to those chosen by the drivers. The models play an important role in prediction of traffic levels on different routes and thus assist in development of ef…
New method uses label-weighted conformal prediction for macro-coverage guarantees in classification.
problem Finding a balance between class-conditional and marginal coverage in long-tailed datasets.
method Label-weighted conformal prediction for macro-coverage guarantees.
result Validated prediction sets with macro-coverage guarantees on large-scale image datasets.
CAP algorithm controls FCR in online selective prediction.
problem Online predictive tasks with temporal multiplicity and FCR control.
method CAP framework with adaptive pick rule and calibration set construction.
result CAP achieves exact selection-conditional coverage guarantee and FCR control.
Selective prediction framework reduces errors in molecular structure identification from MS/MS.
problem High-stakes applications require reliable molecular structure identification from MS/MS data.
method Selective prediction framework using risk-coverage tradeoff and uncertainty quantification.
result First-order confidence measures and retrieval-level aleatoric uncertainty achieve strong risk-coverage tradeoffs.
CP improves robustness against distribution shift using physics-informed structural causal models.
problem Uncertainty in machine learning predictions under distributional shift.
method Physics-informed structural causal model (PI-SCM) to upper bound coverage difference.
result PI-SCM improves coverage robustness across confidence levels and test domains.
Proposes a novel tensor-based approach for multi-level link prediction.
problem Inferring potential links from observed networks.
method Tensor-based joint network embedding capturing pairwise and hyperlinks.
result Improves hyperlink and pairwise link prediction accuracy.
Measuring the corporate default risk is broadly important in economics and finance. Quantitative methods have been developed to predictively assess future corporate default probabilities. However, as a more difficult yet crucial problem, evaluating the uncertainties associated with the default predictions remains littl…
Method predicts NAFLD risk with high accuracy and distribution-free coverage guarantees.
problem Insufficient population-level screening tools for NAFLD.
method Gradient-boosted decision trees with conformal prediction.
result Method achieves AUROC of 0.912 internally and 0.891 externally, superior to other models.
Adapting Hedge algorithm for semi-adversarial data with root-entropy regularization.
problem Minimizing regret in prediction with expert advice under varying distributions.
method Follow-the-Regularized-Leader (FTRL) with root-entropy regularization.
result Adaptive minimax optimal regret across all levels of constraint sets.
This research tackles uncertainty estimation in autoregressive structured prediction tasks.
problem Ensuring safety and robustness of AI systems through accurate uncertainty estimation.
method Develops a unified probabilistic ensemble-based framework for token-level and sequence-level uncertainty estimation.
result Provides baselines for error and out-of-domain detection on translation and speech recognition datasets.
Proposes a method to explain black-box models using causal learning.
problem Existing explainability methods focus on micro-level inputs, not interpretable features.
method Learns causal graphical representations to differentiate between causal and confounding influences.
result Graphs can differentiate between interpretable and confounding features.
\textit{Multiple Instance Learning} (MIL) is concerned with learning from bags of instances, where only bag labels are given and instance labels are unknown. Existent approaches in this field were mainly designed for the bag-level label prediction (predict labels for bags) but not the instance-level (predict labels for…
New diagnostics detect variability in individual risk estimates from machine learning models in healthcare.
problem Variability in individual risk estimates from machine learning models in healthcare, leading to unreliable treatment decisions.
method Proposed evaluation framework using empirical prediction interval width and empirical decision flip rate diagnostics.
result Randomness in optimization and initialization can lead to substantial individual-level variability in risk estimates, affecting clinical decisions.
Improves model classification accuracy in black-box settings.
problem Difficulty in inferring model properties due to limited query access.
method Introduces discriminative factorization to distinguish high-quality queries.
result Probability of chance-level classification decreases exponentially with query budget.
A new method for online prediction uncertainty quantification in non-exchangeable panel data.
problem Challenges in quantifying predictive uncertainty for non-exchangeable panel data.
method Online conformal prediction framework for non-exchangeable panel data, using similarity weights and adaptive miscoverage levels.
result Improves coverage on worst-covered target units through adaptive interval-width allocation.
A new DP approach for Conformal Prediction using quantile search.
problem Privacy leakage in uncertainty quantification methods like Conformal Prediction.
method Private Conformity via Quantile Search (P-COQS) using randomized binary search.
result The approach targets the desired (1−α)-level of coverage with slight under-covering. BILBO optimizes bilevel problems without repeated lower-level optimizations.
problem Challenges in bilevel optimization, especially in noisy, constrained, and derivative-free settings.
method BILevel Bayesian Optimization (BILBO) that optimizes both levels simultaneously, using confidence-bounds and function query selection.
result Theoretical and empirical evidence of BILBO's effectiveness on various problems.
Improved sales forecasting at various levels using ensemble methods.
problem Enhancing sales forecasting accuracy at different levels of e-commerce data.
method Hierarchical robust aggregation of sales forecasts using exponential smoothing and Holt's linear trend method.
result Better forecasts at subsubfamily, subfamily, and family levels compared to individual techniques.
New framework identifies worst-case shifts for predictive resource allocation models.
problem Identifying harmful shifts in predictive models for resource allocation.
method Hierarchical model structure and submodular optimization for worst-case loss.
result Empirical evidence shows divergent worst-case shifts identified by different metrics.
We are concerned with obtaining well-calibrated output distributions from regression models. Such distributions allow us to quantify the uncertainty that the model has regarding the predicted target value. We introduce the novel concept of distribution calibration, and demonstrate its advantages over the existing defin…
FRESH combines patient-level and aggregate-level data for better clinical decision making.
problem Combining patient-level and aggregate-level data for clinical decision making.
method FRESH method that re-calibrates a patient-level model to match specified aggregate statistics.
result Unified data-efficient model for clinical decision making.
In this work we present a clustering technique called \textit{multi-level conformal clustering (MLCC)}. The technique is hierarchical in nature because it can be performed at multiple significance levels which yields greater insight into the data than performing it at just one level. We describe the theoretical underpi…
Models predict fire and other emergencies in Edmonton.
problem Accurate prediction of emergency events for timely response.
method Data collection, descriptive analysis, feature selection, and negative binomial regression.
result Models perform well, with acceptable prediction errors for weekly and monthly periods.
GraphPPD models graph-level uncertainty using GNN embeddings.
problem Capturing uncertainty in graph-level predictions.
method Variational modelling for posterior predictive distribution.
result Effective uncertainty-aware predictions on graph-level tasks.
The paper develops distribution-free methods for ordinal classification.
problem Constructing valid prediction sets for ordinal classification problems.
method Leveraging conformal prediction and multiple testing with FWER control.
result The proposed methods achieve satisfactory levels of marginal and class-specific conditional coverages.