Adaptive coverage policies improve conformal prediction accuracy.
problem Fixed coverage levels in traditional conformal prediction lead to uninformative predictions.
method Optimizes adaptive coverage policy using a neural network trained on leave-one-out calibration.
result Adaptive coverage policies produce more informative and flexible prediction sets.
Machine learning models are notoriously difficult to interpret and debug. This is particularly true of neural networks. In this work, we introduce automated software testing techniques for neural networks that are well-suited to discovering errors which occur only for rare inputs. Specifically, we develop coverage-guid…
Improves generative model coverage of underrepresented modes.
problem Generative models miss underrepresented modes in data.
method Leverage score sampling for complete mode coverage.
result Significantly improves mode coverage compared to standard methods.
Trading strategy uses analyst coverage network to outperform markets.
problem Leveraging spillover effects between firms through analyst network.
method Graph attention network to aggregate firm and network signals.
result Annualized returns of 29.44% and Sharpe ratio of 4.06.
Deep neural networks (DNN) are increasingly applied in safety-critical systems, e.g., for face recognition, autonomous car control and malware detection. It is also shown that DNNs are subject to attacks such as adversarial perturbation and thus must be properly tested. Many coverage criteria for DNN since have been pr…
CF-GNN provides reliable uncertainty estimates for graph data.
problem Lack of rigorous uncertainty estimates in GNNs.
method Conformal prediction extended to graph-based models (CF-GNN).
result CF-GNN produces prediction sets with provable coverage probability.
Unified neural network model for astro-particle physics predictions with coverage, systematics, and goodness-of-fit.
problem Lack of statistical uncertainties, coverage, systematic uncertainties, and goodness-of-fit in neural network predictions.
method KL-divergence objective for joint distribution of data and labels, conditional normalizing flows, amortized with neural networks.
result Unified supervised learning and VAEs under stochastic variational inference for event property predictions.
Concolic testing combines program execution and symbolic analysis to explore the execution paths of a software program. This paper presents the first concolic testing approach for Deep Neural Networks (DNNs). More specifically, we formalise coverage criteria for DNNs that have been studied in the literature, and then d…
Efficient PI for neural networks without distributional assumptions.
problem No distributional assumptions for efficient predictive inference.
method Differential privacy and linear approximation for leave-one-out models.
result Rigorous coverage guarantees with reduced computation.
T-SCI improves Cox-MLP's guaranteed coverage for censored data.
problem Losing guaranteed coverage when relaxing linear assumption with neural networks.
method Two-stage conformal inference algorithm with non-conformity score.
result T-SCI provides guaranteed coverage under milder assumptions.
Method constructs nonparametric prediction intervals with finite-sample guarantees.
problem Nonparametric instrumental variable regression with finite-sample coverage.
method Conformal inference framework applied to NPIV, combining with various estimators.
result Distribution-free, finite-sample coverage over chosen IV shifts.
C-SymmPI provides near-conditional coverage for structured data with group symmetries.
problem Establishing near-conditional coverage guarantees for structured data with group symmetries.
method Developed a framework C-SymmPI that achieves near-conditional coverage under general data structures with group symmetries.
result Near-conditional coverage guarantees for structured data with group symmetries.
Combining Bayesian deep learning and split conformal prediction affects out-of-distribution coverage.
problem Improving out-of-distribution coverage in multiclass image classification.
method Combining Bayesian deep learning with split conformal prediction methods.
result Combining methods can reduce out-of-distribution coverage in some cases.
This paper applies combinatorial testing to machine learning for robust model performance.
problem Identifying robust machine learning models using test and training sets.
method Adapting combinatorial interaction testing for machine learning, focusing on simple features.
result Combinatorial coverage can enhance model performance and robustness.
A third of adults in America use the Internet to diagnose medical concerns, and online symptom checkers are increasingly part of this process. These tools are powered by diagnosis models similar to clinical decision support systems, with the primary difference being the coverage of symptoms and diagnoses. To be useful …
Testing Deep Neural Network (DNN) models has become more important than ever with the increasing usage of DNN models in safety-critical domains such as autonomous cars. The traditional approach of testing DNNs is to create a test set, which is a random subset of the dataset about the problem of interest. This kind of a…
UAV enabled communications and networking can enhance wireless connectivity and support emerging services. However, this would require system-level understanding to modify and extend the existing terrestrial network infrastructure. In this paper, we integrate UAVs both as user equipment and base stations into existing …
Study uses neural networks to predict firm earnings, outperforming benchmarks and analysts.
problem Limited coverage and biased estimates by financial analysts.
method Developed a neural network model using 40 years of financial data.
result Model outperforms benchmarks and analysts' forecasts for fiscal-year-end earnings predictions.
Paper uses GNN and conformal prediction for accurate edge weight prediction.
problem Predicting edge weights on graphs for various applications.
method Graph Neural Network (GNN) with conformal prediction and error reweighting.
result Our method provides better coverage and efficiency than baselines.
Adaptive teacher improves sample efficiency and mode coverage in sampling tasks.
problem Efficient exploration and mode coverage in sampling tasks with amortized inference.
method An adaptive training distribution (teacher) guides the training of the primary sampler (student).
result Improves sample efficiency and mode coverage across various tasks.
Develops methods for valid and validated confidence sets in multiclass and multilabel prediction.
problem Challenges of typical conformal prediction methods in multiclass and multilabel problems, especially uneven coverage.
method Leverages quantile regression to build methods that always guarantee correct coverage and asymptotically optimal conditional coverage, addressing label interactions with tree-structured classifiers.
result Empirical evaluation suggests more robust coverage of confidence sets.
Locally Valid and Discriminative prediction intervals for deep learning models.
problem Efficient and theoretically sound uncertainty quantification for deep learning models.
method Locally Valid and Discriminative prediction intervals (LVD) using kernel regression.
result Locally Valid and Discriminative prediction intervals (LVD) offer better performance and scalability compared to existing methods.
The paper tackles temporal coverage bias in financial panel data, proposing a structuring framework to correct for incomplete histories.
problem Incomplete histories of financial instruments lead to biased panel data.
method Formalizes the problem and proposes a coverage-aware structuring framework using structured metadata and an availability matrix.
result The framework reveals substantial distortions in return dynamics and volatility when naive temporal alignment is used.
New method uses label-weighted conformal prediction for macro-coverage guarantees in classification.
problem Finding a balance between class-conditional and marginal coverage in long-tailed datasets.
method Label-weighted conformal prediction for macro-coverage guarantees.
result Validated prediction sets with macro-coverage guarantees on large-scale image datasets.
Boosted conformal procedure improves prediction intervals.
problem Enhancing prediction interval properties like coverage and length.
method Gradient boosting to optimize conformity score function.
result Significant improvements in interval length and coverage.
Algorithm ensures fair information spread in social networks with community structure.
problem Disparities in information coverage between communities in social networks.
method Fits a model to the social network, uses community structure, and determines optimal seed allocations for fair coverage.
result Empirical accuracy demonstrated on simulated and real networks.
Method predicts NAFLD risk with high accuracy and distribution-free coverage guarantees.
problem Insufficient population-level screening tools for NAFLD.
method Gradient-boosted decision trees with conformal prediction.
result Method achieves AUROC of 0.912 internally and 0.891 externally, superior to other models.
Optimizes ellipsoids for uncertainty regions in parameter estimation.
problem Learning minimal volume uncertainty ellipsoids for parameter estimation.
method Differentiable optimization approach using neural networks to approximate optimal ellipsoids.
result Approximately computed ellipsoids are smaller and more accurate than existing methods.
TS improves class coverage but reduces CP set size, offering a trade-off.
problem Combining temperature scaling with conformal prediction for deep classifiers.
method Empirical study and mathematical theory of TS's effect on CP.
result TS allows trading prediction set size and conditional coverage.
Algorithm balances learning and coverage for multi-robots over unknown fields.
problem Balancing learning and coverage for multi-robots over unknown, nonuniform sensory fields.
method DSLC algorithm that schedules learning and coverage epochs, using Gaussian Process modeling and coverage regret analysis.
result Upper bound on expected cumulative coverage regret provided for DSLC.
RR-GNN improves GNN prediction intervals by accounting for graph heteroscedasticity and structural biases.
problem Uncertainty quantification in GNNs for high-stakes domains.
method Graph-Structured Mondrian CP, Residual-Adaptive Nonconformity Scores, Cross-Training Protocol.
result Improved efficiency and no loss of coverage compared to CP baselines.
Improved conformal prediction for better conditional coverage of classifier predictions.
problem Achieving exact conditional coverage in finite samples for prediction sets.
method Developed a variant of conformal prediction targeting coverage conditional on confidence and trust score.
result Empirically improved conditional coverage properties compared to standard conformal prediction.
This paper improves conformal prediction to be robust to perturbations.
problem Ensuring robustness of conformal prediction to natural and adversarial perturbations.
method Probabilistically robust conformal prediction (PRCP) and its adaptive version (aPRCP).
result aPRCP achieves better trade-offs between nominal performance and robustness.
Develops a calibration prediction interval for non-parametric regression and neural networks.
problem Lack of accurate conditional prediction in regression settings.
method Calibration Prediction Interval (cPI) using Deep Neural Networks (DNN) or kernel methods.
result Asymptotically valid coverage rate and high probability of coverage rate with large sample sizes.
New metrics improve understanding of predictive system reliability.
problem Evaluating conditional coverage of predictive systems.
method Casting conditional coverage estimation as a classification problem, using excess risk of the target coverage (ERT) metrics.
result Modern classifiers provide higher statistical power for estimating conditional coverage.
Kandinsky conformal prediction expands conditional coverage guarantees.
problem Disparities in coverage guarantees across different subpopulations.
method Flexible handling of overlapping and fractional group memberships.
result Minimax-optimal high-probability conditional coverage bound.
ST-BCP narrows the coverage gap in BCP by transforming nonconformity scores.
problem The looseness in BCP's coverage guarantee due to Markov's inequality.
method Introduces a data-dependent transformation of nonconformity scores.
result Reduces the average coverage gap from 4.20% to 1.12% on benchmarks.
Improves conditional coverage of regression models using conformal prediction.
problem Lack of conditional coverage guarantees in conformal prediction methods.
method Proposes a novel algorithm to train a regression function to improve conditional coverage after split conformal prediction.
result Establishes an upper bound for miscoverage gap and proposes an end-to-end algorithm to control it.
This study uses AI to analyze financial market coverage from YouTube videos.
problem Challenges in analyzing a large number of financial market videos.
method Used Whisper model to generate text from videos, applied natural language processing.
result Highlights dynamics of financial market coverage and identifies trending topics.
Conformal methods create prediction bands that control average coverage under no assumptions besides i.i.d. data. Besides average coverage, one might also desire to control conditional coverage, that is, coverage for every new testing point. However, without strong assumptions, conditional coverage is unachievable. Giv…
TQA improves prediction intervals for time series data by adjusting quantiles for both cross-sectional and longitudinal coverage.
problem Constructing reliable prediction intervals for cross-sectional time series data.
method Temporal Quantile Adjustment (TQA) method that adjusts the quantile in Conformal Prediction to account for both cross-sectional and longitudinal coverage.
result TQA improves longitudinal coverage while preserving cross-sectional coverage, as validated through extensive experimentation.
New conformal prediction methods for long-tailed classification problems.
problem Rare classes are systematically omitted in existing conformal prediction methods.
method Introduced a new conformal score function and a new interpolation procedure.
result Smoothly trade off set size and class-conditional coverage.
New coverage conditions improve sample efficiency in online reinforcement learning.
problem Improving sample efficiency in online reinforcement learning with function approximation.
method Identifying and studying new coverage conditions for online reinforcement learning.
result Improved regret bounds achieved with new coverage conditions.
We introduce the concept of coverage risk as an error measure for density ridge estimation. The coverage risk generalizes the mean integrated square error to set estimation. We propose two risk estimators for the coverage risk and we show that we can select tuning parameters by minimizing the estimated risk. We study t…
Study robustness of split conformal prediction under adversarial attacks.
problem Ensuring distribution-free coverage guarantees in CP under adversarial conditions.
method Theoretical analysis and extensive experiments on split conformal prediction robustness.
result Prediction coverage varies with calibration-time attack strength, enabling control over coverage under adversarial tests.
Proposes a method to create shorter, more accurate prediction intervals.
problem Challenges in achieving both conditional validity and interval efficiency in complex settings.
method Uses a conformal-style calibration method for neural network responses, adjusting to empirical PIT distribution.
result Demonstrates better conditional calibration and shorter intervals than existing methods.
New method improves conditional coverage of conformal prediction.
problem Improving conditional coverage in conformal prediction.
method Trainable transformation of conformity scores to improve conditional coverage.
result Highly adaptive to local data structure, outperforming existing methods.
Audited Conformal Prediction improves conditional coverage in pretrained models under distribution shift.
problem Uncertainty quantification for pretrained models under unknown distribution shift
method Leverages a small labeled dataset to train an audit model for marginal coverage, integrates outputs into conformal prediction framework
result Significantly higher conditional coverage than existing approaches