Study three types of uncertainty quantification for binary classification without distributional assumptions.
problem Uncertainty quantification for binary classification in a distribution-free setting.
method Established theorems connecting calibration, confidence intervals, and prediction sets for score-based classifiers.
result Distribution-free calibration is only possible using scoring functions that partition feature space into countably many sets.
The paper develops distribution-free methods for ordinal classification.
problem Constructing valid prediction sets for ordinal classification problems.
method Leveraging conformal prediction and multiple testing with FWER control.
result The proposed methods achieve satisfactory levels of marginal and class-specific conditional coverages.
The paper proposes a method for distribution-free prediction sets that adapt to unknown temporal changes.
problem Distribution-free prediction sets require reliable calibration data, which is often unavailable in real-world settings with temporal changes.
method The method selects an adaptive window to construct prediction sets, optimizing a bias-variance tradeoff.
result The method provides sharp coverage guarantees and is shown to be adaptive to temporal drift through numerical experiments.
This paper offers a distribution-free method for post-detection changepoint localization.
problem Locating the exact time of a change in distribution after a sequential detection procedure.
method A distribution-free framework using conformal test martingales for sequential change detection and post-detection inference.
result Valid post-detection coverage guarantees and non-asymptotic bounds on confidence set size.
New algorithm for precise changepoint localization without assumptions.
problem Offline changepoint localization in arbitrary distributions.
method Distribution-free algorithm CONformal CHangepoint localization (CONCH) using exchangeability arguments.
result Derives principled score functions for informative and small confidence sets with normalized length shrinking to zero.
Large language models can't efficiently reason conditionally in a distribution-free setting.
problem Impossibility of conditional PAC-efficient reasoning in large language models.
method Proof of impossibility in a distribution-free setting for non-atomic input spaces.
result Any algorithm achieving conditional PAC efficiency must defer to the expert model with high probability.
The paper tackles uncertainty quantification for classification under label shift without assuming i.i.d. data.
problem Uncertainty quantification for classification under label shift in non-i.i.d. settings.
method The paper uses conformal prediction and post-hoc binning for distribution-free UQ, and reweights these methods for label shift.
result The reweighted methods improve UQ performance under label shift, preserving coverage and calibration.
Characterizes distribution-free rates in unbalanced classification problems.
problem Minimizing error under two different distributions in unbalanced settings.
method Characterizes minimax rates over all pairs of distributions using a geometric condition.
result Identifies a dichotomy between hard and easy classes based on a three-points-separation condition.
Unified theoretical guarantees for distribution-free changepoint detection and testing.
problem Distribution-free changepoint inference with finite-sample validity and consistency.
method Distribution-free changepoint localization using conformal p-values with theoretical guarantees.
result Unified distribution-free guarantees for changepoint detection, localization, and testing.
Valid p-value for bounded random variables without distributional assumptions.
problem Calibration of predictive algorithms in a distribution-free setting.
method Built a super-uniform p-value based on a concentration inequality.
result Super-uniform p-value is tighter than existing alternatives.
Conformal prediction provides distribution-free uncertainty quantification for black-box models.
problem Uncertainty quantification for high-risk machine learning applications.
method Conformal prediction creates valid uncertainty sets without distributional assumptions.
result Sets contain the ground truth with a specified probability, e.g., 90%.
CROC identifies the earliest-changing stream as the root cause in multi-stream data.
problem Distribution-free root cause analysis in multi-stream data with unknown distributional changes.
method Conformal p-values and finite-sample valid confidence sets.
result CROC efficiently isolates the root cause under minimal assumptions.
We develop an online learning method for prediction, which is important in problems with large and/or streaming data sets. We formulate the learning approach using a covariance-fitting methodology, and show that the resulting predictor has desirable computational and distribution-free properties: It is implemented onli…
FedFaiREE addresses fairness in decentralized learning with small samples.
problem Ensuring fairness in decentralized federated learning with limited data.
method FedFaiREE is a post-processing algorithm for distribution-free fair learning in decentralized settings with small samples.
result FedFaiREE provides theoretical guarantees for both fairness and accuracy in decentralized environments.
Study robust linear regression without distributional assumptions for heavy-tailed responses.
problem Linear regression with heavy-tailed responses and no distributional assumptions.
method Combining truncated least squares, median-of-means, and aggregation theory to construct a non-linear estimator.
result Achieves excess risk of order d/n with optimal sub-exponential tail. Four geometries govern sequential and distribution-free inference.
problem Sequential and distribution-free inference challenges.
method Four distinct admissibility geometries.
result Four classes of admissible procedures are pairwise non-nested.
Unified review of Conformal Prediction theory and applications.
problem Distribution-free, non-parametric forecasting method for valid prediction sets.
method Minimal assumptions, straightforward predictions sets valid in finite sample cases.
result Unified review of Conformal Prediction theory and applications.
Split conformal prediction provides finite-sample guarantees for black-box models without distributional assumptions.
problem Weak performance guarantees for modern predictive models under minimal assumptions.
method Develops finite-sample guarantees for split conformal prediction, a method that uses nested prediction sets and order statistics.
result The coverage of prediction sets based on order statistics stochastically dominates the Beta distribution.
Method predicts NAFLD risk with high accuracy and distribution-free coverage guarantees.
problem Insufficient population-level screening tools for NAFLD.
method Gradient-boosted decision trees with conformal prediction.
result Method achieves AUROC of 0.912 internally and 0.891 externally, superior to other models.
ERAPS builds prediction sets for time-series data.
problem Uncertainty quantification in complex machine learning methods for time-series data.
method ERAPS is an ensemble-based framework for constructing prediction sets for time-series data, allowing unknown dependencies within features and responses.
result ERAPS demonstrates valid marginal and conditional coverage and yields smaller prediction sets than competing methods.
Mack's estimator improves chain ladder prediction for large exposure insurance models.
problem Uncertainty quantification in compound Poisson loss models.
method Large exposure asymptotics applied to Mack's estimator.
result Chain ladder prediction uncertainty can be quantified without model assumptions.
New findings on PAC learning and marginal distribution estimation.
problem Understanding how PAC learning relates to marginal distribution estimation under distributional constraints.
method Revisited the connection between PAC learning, uniform convergence, and density estimation, considering a known family of marginal distributions.
result PAC learning is sandwiched between two refined models of density estimation, differing only in whether the learner knows the set of well-estimated events in H.
New framework for robust uncertainty quantification in strategic settings.
problem Machine learning model predictions can be strategically altered by informed agents.
method Strategic Conformal Prediction framework
result Theoretical guarantees and experimental validation show remarkable effectiveness.
Locus scores predictions for risk, reducing large-loss events.
problem Deployment cost from inaccurate predictions, especially large losses.
method Distribution-free loss-scale reliability score using any predictive distribution.
result Reduces large-loss frequency compared to standard heuristics.
The intention of this paper is to estimate a Bayesian distribution-free chain ladder (DFCL) model using approximate Bayesian computation (ABC) methodology. We demonstrate how to estimate quantities of interest in claims reserving and compare the estimates to those obtained from classical and credibility approaches. In …
Algorithm learns without knowing distribution, reducing error.
problem Sequential prediction with adversarial injections and abstentions.
method Boosting procedure of weak learners for general VC classes.
result Sublinear error guarantees for general VC classes.
The paper explores how over-parameterized linear regression models generalize without violating learning theory principles.
problem Understanding how over-parameterized linear regression models generalize without violating learning theory principles.
method The paper uses the predictive normalized maximum likelihood (pNML) learner to investigate the minimum norm solution of over-parameterized linear regression models.
result The model generalizes well when the test sample lies in a subspace spanned by eigenvectors associated with large eigenvalues of the training data.
New method relaxes TV distance for two-sample testing without distributional assumptions.
problem Challenges in certifying equality or providing tight bounds on TV distance for two distributions.
method Examined blurred total variation distance, a relaxation of TV distance.
result Provided theoretical guarantees for upper and lower bounds on blurred TV distance.
MAPS algorithm creates reliable prediction intervals for high-dimensional data.
problem Computing reliable conditional prediction intervals in high-dimensional settings.
method Lifted predictive model (LPM) and MAPS algorithm for distribution-free intervals.
result MAPS algorithm produces valid prediction intervals for any trained model.
A method for making predictions with a reject option using conformal prediction.
problem Uncertainty in machine learning predictions, especially when models are unsure.
method Formalizing ML with reject option, using conformal prediction for distribution-free error guarantees.
result Theoretical guarantees on error rate for prediction sets with distribution-free validity.
CREDO assesses decision optimality under uncertainty without assuming a model.
problem Uncertainty in decision-making without reliable quantification of optimality.
method CREDO uses the inverse feasible region and conformal prediction balls to estimate decision optimality probability.
result CREDO provides accurate, efficient, and reliable evaluations of decision optimality.
A new method avoids quantile crossing in time series forecasting.
problem Quantile crossing in joint quantile regressions.
method Incremental (Spline) Quantile Functions (I(S)QF) with neural network.
result Improves consistency and accuracy in time series forecasting.
This work examines fundamental limits in model falsification without assuming specific distributions.
problem Establishing lower bounds on model class risk in distribution-free settings.
method Model-agnostic fundamental hardness result for constructing lower bounds on test error.
result No positive lower bound on model class risk is possible in certain settings.
E-values enhance conformal prediction methods.
problem Distribution-free uncertainty quantification.
method Reformulation of conformal prediction using e-values.
result E-values offer new theoretical and practical capabilities.
Characterizes sample complexity for outcome indistinguishability in machine learning.
problem Outcome indistinguishability in machine learning, focusing on distinguishers and predictors.
method Sample complexity characterized by metric entropy of predictor and distinguisher classes, using dual Minkowski norms.
result Equivalence and tightness of sample complexity characterizations in distribution-specific and distribution-free settings.
The paper offers a method to create prediction sets with uncertainty control.
problem Calibrating and communicating uncertainty in machine learning predictions.
method Distribution-free, risk-controlling prediction sets using a holdout set to calibrate set sizes.
result Explicit finite-sample guarantees for error control in various machine learning tasks.
Unified framework for reliable uncertainty quantification in RL.
problem Uncertainty quantification in high-stakes reinforcement learning.
method Unified conformal prediction framework integrating distributional RL and conformal calibration.
result Significantly improved coverage and reliability over standard methods.
New algorithm learns disjunctions faster than previous methods.
problem Learning Boolean disjunctions in the agnostic PAC model.
method Developed an agnostic learner with complexity 2ildeO(n1/3). result First separation between SQ and CSQ models in distribution-free agnostic learning.
ICP provides interval predictions with high confidence coverage.
problem High-risk settings where true output must be highly probable.
method ICP is a distribution-free, model-agnostic algorithm.
result ICP outputs prediction intervals with high coverage probability.
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
New method optimizes prediction set volume in conformal prediction.
problem Achieving volume optimality in conformal prediction without sacrificing coverage guarantees.
method Dynamic programming algorithm for finding near-optimal volume unions of k-intervals.
result Efficient algorithm finds unions of k-intervals with near-optimal volume for any distribution.
This paper develops a method for estimating animal motion submanifolds using distribution-free learning.
problem Estimating submanifold models of animal motion supported on a configuration manifold.
method Formulates a novel method based on distribution-free learning over the manifold of measurements.
result Derives approximations of the unknown mapping that converge with rates dependent on sample size and dimensionality.
Paper extends conformal prediction to complex survey data.
problem Applying distribution-free prediction intervals to complex survey data.
method Design-based conformal prediction for non-exchangeable data.
result Empirical guarantees of finite-sample coverage for complex survey data.
FaiREE provides fair classification with guarantees for small datasets.
problem Fairness in classification often requires large sample sizes and distributional assumptions.
method FaiREE offers finite-sample and distribution-free fairness guarantees.
result FaiREE achieves optimal accuracy and satisfies various fairness notions.
Conformal prediction offers distribution-free inference for complex models.
problem Traditional predictive inference methods are limited by assumptions about data distributions and model details.
method Conformal prediction uses symmetry assumptions and treats learning algorithms as black boxes.
result Conformal prediction provides exact finite-sample guarantees, even under limited assumptions.
This work presents the concept of kernel mean embedding and kernel probabilistic programming in the context of stochastic systems. We propose formulations to represent, compare, and propagate uncertainties for fairly general stochastic dynamics in a distribution-free manner. The new tools enjoy sound theory rooted in f…
Paper develops a neural network method for censored survival analysis.
problem Distribution-free quantile prediction for censored survival data.
method Develops a novel neural network algorithm for simultaneous quantile optimization.
result The algorithm produces better calibrated quantiles on real datasets.
A stability-based method selects the most desirable conformal prediction set.
problem Selecting the most desirable conformal prediction set from multiple valid sets invalidates coverage guarantees.
method A stability-based approach that ensures coverage for the selected prediction set.
result The stability-based approach maintains coverage guarantees for the selected prediction set.