Predictive sampling improves on Thompson sampling for non-stationary bandit environments.
problem Thompson sampling fails in non-stationary bandit environments.
method Proposes predictive sampling, which deprioritizes actions based on information loss rate.
result Predictive sampling outperforms Thompson sampling in all tested non-stationary environments.
Validates conformal prediction for network data under non-uniform sampling.
problem Validity of conformal prediction for network data under non-representative sampling.
method Interprets sampling mechanisms as selection rules, studies validity conditional on selection events, uses permutation invariance and joint exchangeability.
result Finite-sample validity of conformal prediction for certain selection events and asymptotic validity for random walk sampling.
Bayesian Attention Networks compress data by focusing on key training samples.
problem Lossless data compression for efficiency.
method Bayesian Attention Networks with attention factors and latent space.
result Efficient prediction using a few correlated training samples.
Develops a prediction method based on sampling design.
problem Creating accurate individual predictions.
method Design-based approach using expected cross-validation results.
result Valid inference of unobserved prediction errors defined with respect to sampling design.
Paper introduces PCP for efficient, reliable predictive inference.
problem Developing reliable predictive inference methods for target variables.
method Probabilistic conformal prediction using conditional random samples.
result PCP provides sharper predictive sets compared to existing methods.
The paper studies how more data affects prediction risk in high-dimensional models.
problem The impact of increasing data on prediction risk in high-dimensional models.
method Derives central limit theorem and provides finite-sample distribution and confidence interval for prediction risk.
result Demonstrates 'more data hurt' phenomenon in high-dimensional least squares estimation.
New sampling methods improve Shapley values for explaining machine learning predictions.
problem Computational limitations in calculating Shapley values for complex models.
method Asymptotic normality results and paired-sampling approximations (KernelSHAP and PermutationSHAP).
result Paired-sampling PermutationSHAP provides exact results for interactions of maximal order two and has the additive recovery property.
Link prediction in networks is typically accomplished by estimating or ranking the probabilities of edges for all pairs of nodes. In practice, especially for social networks, the data are often collected by egocentric sampling, which means selecting a subset of nodes and recording all of their edges. This sampling mech…
With a growing interest in using non-representative samples to train prediction models for numerous outcomes it is necessary to account for the sampling design that gives rise to the data in order to assess the generalized predictive utility of a proposed prediction rule. After learning a prediction rule based on a non…
Improved sampling for Bayesian neural networks reduces vanishing acceptance rates and increases predictive accuracy.
problem Sampling inefficiency in Bayesian neural networks, especially with deep architectures and large datasets.
method Approximate blocked Gibbs sampling to partition and sample subgroups of parameters.
result Increased predictive accuracy and quantification of predictive uncertainty in classification tasks.
Paper extends conformal prediction to complex survey data.
problem Applying distribution-free prediction intervals to complex survey data.
method Design-based conformal prediction for non-exchangeable data.
result Empirical guarantees of finite-sample coverage for complex survey data.
New algorithms test independence with fewer samples by using predictive information.
problem Testing independence of distributions with limited samples.
method Augmented distribution testing framework that incorporates predictive information.
result Optimal sample complexity achieved, matching lower bounds.
Efficiently samples conformal boundaries in high dimensions using flows.
problem Difficulty in interpreting and using prediction sets in high-dimensional or structured output spaces.
method Flow-based approach using differentiable nonconformity scores to induce deterministic flows on the output space.
result Sampling conformal boundaries in arbitrary dimensions becomes computationally efficient and training-free.
Unified framework for generalized Venn and Venn-Abers calibration for reliable prediction.
problem Asymptotic guarantees of popular distribution-free methods in model calibration.
method Unified framework extending Vovk's approach to generic loss functions, transforming predictors into set-valued predictions.
result Finite-sample set predictions shrink to a single conditionally calibrated prediction, capturing epistemic uncertainty.
The paper improves high-dimensional linear regression prediction and estimation using auxiliary samples.
problem Estimating and predicting high-dimensional linear regression models with auxiliary samples.
method Proposes Trans-Lasso for data-driven transfer learning, establishing optimality for prediction and estimation.
result Knowledge from auxiliary samples can improve learning performance in target problems.
New method for estimating out-of-sample R² from gene expression data.
problem Lack of a well-defined and unbiased estimator for out-of-sample R².
method Explicitly defined out-of-sample R², provided an unbiased estimator, and calculated standard error.
result Demonstrated improved model comparison for gene expression phenotypes.
Improved statistical inference for expensive data using machine learning predictions.
problem Statistical inference under adaptive two-phase multiwave sampling with expensive measurements.
method Multiwave Predict-Then-Debias estimator combining proxy information and expensive measurements.
result Valid estimators and confidence intervals for M-estimation under adaptive sampling.
Projected random forests improve circular data prediction with adaptive arc length and finite-sample coverage.
problem Regression with circular responses.
method Adapting linear-response models to circular data using projection and random forest out-of-bag mechanism.
result Projected random forest out-of-bag conformal prediction sets are more efficient and shorter than alternative methods.
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
Optimized testing of discrete distributions using predicted data.
problem Testing discrete distributions with reduced sample complexity.
method Adaptable algorithms that use a predicted distribution to reduce sample size.
result Optimal sample complexity improvements with self-adjusting algorithms.
The paper integrates multiple Gaussian process predictions using Monte Carlo sampling.
problem Accurate prediction of variables using multiple models.
method Log-linear pooling of Gaussian process predictions, combined with Monte Carlo sampling.
result The log-linear pooling method improves prediction accuracy compared to linear pooling.
PFN-TS uses Thompson sampling with PFNs to improve contextual bandit performance.
problem Improving contextual bandit performance using Thompson sampling with prior-data fitted networks.
method PFN-TS converts PFN posterior predictives into mean-reward samples using a subsampled predictive central limit theorem.
result PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks.
Improved preterm prediction using synthetic EHG signals.
problem Prediction bias towards term labor in preterm EHG data.
method Quantifying synthetic samples' effect, optimizing feature weights, and combining activation functions.
result Substantial improvement in prediction precision.
Study finds that only a fraction of data is needed for accurate patient-level prediction models.
problem Developing predictive models for patient-level outcomes using large observational data.
method Empirical assessment of sample size effects on model performance and complexity using learning curves.
result A median reduction of 9.5% to 78.5% in the number of observations and 8.6% to 68.3% in the number of predictors can be achieved with adequate sample size.
Develops methods to adjust prediction set coverage based on post-selection analysis.
problem Adjusting prediction set coverage after initial analysis to better fit specific needs.
method Post-selection conformal inference to adjust miscoverage levels.
result Allows for trade-off between coverage and prediction set quality.
Exact distribution of split conformal prediction coverage found.
problem Determining the reliability of prediction sets in batch mode.
method Analysis of exchangeable data to find universal distribution of empirical coverage.
result Exact distribution of empirical coverage is universal and determined by nominal miscoverage level and calibration sample size.
Paper presents unsupervised calibration for split conformal classification.
problem Inconvenient requirement of labeled calibration samples.
method Uses unsupervised calibration samples alongside supervised training samples.
result Achieves comparable performance to supervised calibration methods.
Paper develops a method to predict spatial point processes with guarantees.
problem Predicting the number of events in space with uncertainty.
method Regularized method to learn spatial models with out-of-sample guarantees.
result Method provides valid prediction intervals even when model is misspecified.
Study the impact of overfitting on linear predictive models' performance.
problem Overfitting reduces the out-of-sample performance of linear predictive trading strategies.
method Computed in- and out-of-sample means and variances of PnLs to derive replication ratios.
result Replication ratio diminishes for complex strategies with many assets.
Split conformal prediction provides finite-sample guarantees for black-box models without distributional assumptions.
problem Weak performance guarantees for modern predictive models under minimal assumptions.
method Develops finite-sample guarantees for split conformal prediction, a method that uses nested prediction sets and order statistics.
result The coverage of prediction sets based on order statistics stochastically dominates the Beta distribution.
New privacy-preserving method for conformal prediction without splitting data.
problem Privacy and uncertainty quantification in data-driven decision making.
method Proposes a full-data privacy-preserving conformal prediction framework using differential privacy.
result Demonstrates improved prediction sets compared to split-based private baselines.
Paper reduces uncertainty in predictive models using neural networks and Gaussian processes.
problem High epistemic uncertainty in predictive models.
method Adaptive sampling approach with prediction interval-generation neural networks and Gaussian processes.
result Method consistently converges faster to minimum epistemic uncertainty levels.
Improves sampling speed of ARM models without losing accuracy.
problem Slow sampling from ARMs.
method Predictive sampling algorithm using ARM fixed-point iteration and learned forecasting modules.
result Significant improvements in sampling speed and efficiency.
This work introduces COLA, a strategy to aggregate conformal prediction sets efficiently.
problem Efficiently combining multiple conformity scores to reduce prediction set size.
method Introduces COnfidence-Level Allocation (COLA) to optimally allocate confidence levels across sets.
result COLA achieves smaller prediction sets than state-of-the-art methods while maintaining valid coverage.
SPI uses synthetic data to improve predictive inference efficiency.
problem Inefficient predictive inference with scarce calibration data.
method Integrates synthetic data to align nonconformity scores and improve coverage guarantees.
result SPI yields substantially tighter and more informative prediction sets.
Improved predictive posterior density estimation through optimized importance sampling.
problem Low signal-to-noise ratio in posterior predictive densities.
method Optimized importance sampling using a test-time variational proxy.
result Significantly improved estimates of predictive posterior densities.
Paper presents a copula-based method to efficiently generate correlated sample paths from multi-step time series models.
problem Generating realistic correlation structures in multi-step forecast sample paths is expensive and time-consuming.
method Copula-based approach to generate correlated sample paths in one forward pass.
result Improved sample path quality and significant speedup over autoregressive sampling.
New method distinguishes predictive distribution estimators in high-dimensional inputs.
problem Difficulty in evaluating predictive distributions for high-dimensional inputs.
method Introduces dyadic sampling to focus on predictive distributions associated with pairs of inputs.
result Demonstrates efficient distinction of predictive distribution estimators in high-dimensional examples.
We consider the multi-class classification problem when the training data and the out-of-sample test data may have different distributions and propose a method called BCOPS (balanced and conformal optimized prediction sets). BCOPS constructs a prediction set C(x) as a subset of class labels, possibly empty. It tries …
Have you ever wondered how your feature space is impacting the prediction of a specific sample in your dataset? In this paper, we introduce Single Sample Feature Importance (SSFI), which is an interpretable feature importance algorithm that allows for the identification of the most important features that contribute to…
Mutual teaching improves graph models with less labeled data.
problem Training graph models with limited labeled data.
method Dual model training with mutual teaching strategy.
result Significant performance improvement with less labeled data.
Proposes SSL method for non-randomly sampled data.
problem Evaluation of prediction rules under non-random sampling.
method Two-step procedure with imputation and augmentation.
result Proposed method outperforms supervised methods in efficiency.
Prognostics and Health Management (PHM) is an emerging engineering discipline which is concerned with the analysis and prediction of equipment health and performance. One of the key challenges in PHM is to accurately predict impending failures in the equipment. In recent years, solutions for failure prediction have evo…
We develop a method to estimate the time to unsafe responses in LLMs.
problem Estimating the time to unsafe responses in large language models is challenging due to the rarity of unsafe outputs.
method We frame the problem as survival analysis and propose a calibration technique for constructing a lower predictive bound (LPB).
result Our method provides rigorous coverage guarantees and improves sample efficiency.
Though black-box predictors are state-of-the-art for many complex tasks, they often fail to properly quantify predictive uncertainty and may provide inappropriate predictions for unfamiliar data. Instead, we can learn more reliable models by letting them either output a prediction set or abstain when the uncertainty is…
New method identifies wrongly predicted samples for active learning.
problem Identifying important samples for machine learning models.
method A sample selection criterion based on model prediction and its effect on generalization error.
result State-of-the-art results and better rates at identifying wrongly predicted samples.
FMP sampling improves model calibration without sharing data.
problem Difficulties in specifying priors for modern neural networks.
method FMP sampling replaces prior and likelihood with a predictive distribution, running the sampler centrally.
result FMP sampling significantly improves calibration over baseline methods.
Skew-adaptive method improves prediction intervals for regression.
problem Improving prediction intervals for regression models, especially in cases of skewness and varying scales.
method Develops a skew-adaptive extension of split conformal prediction using an asymmetric interval family and gauge approach.
result Preserves marginal validity and adapts to local scale and skewness, with efficiency gains over existing methods.