Simple bounds show most cross-sectional predictability findings are likely true.
problem Determining the validity of cross-sectional return predictability findings.
method Developed simple and intuitive bounds on the false discovery rate (FDR).
result Bounds show the FDR is small, indicating most findings are likely true.
A new method for safer statistical inference after predictions.
problem Statistical inference with pseudo-outcomes from machine learning predictions.
method Prediction De-Correlated Inference (PDC) framework.
result PDC consistently outperforms supervised methods and can adapt to any model.
Paper establishes statistical inference for performative predictions.
problem Dynamic influence of predictions on their targets.
method End-to-end framework for estimation and inference under performativity.
result Established central limit theorem for performative settings.
Method controls extrapolation in prediction profiles for statistical and machine learning models.
problem Avoiding invalid predictions due to extrapolation in prediction profiles.
method Genetic algorithm optimization over constrained factor regions.
result Optimal factor settings without constraint are often invalid and extrapolated.
PAS improves estimation of multiple means using ML predictions and shrinkage.
problem Improving statistical estimates with limited gold-standard data and noisy ML predictions.
method Prediction-Powered Adaptive Shrinkage (PAS) that combines PPI with empirical Bayes shrinkage.
result PAS adapts to the reliability of ML predictions and outperforms traditional methods in large-scale applications.
New method for valid inference from ML-predicted data.
problem Invalid scientific conclusions from ML-predicted outcomes.
method Assumption-Lean and Data-Adaptive Post-Prediction Inference (PSPA)
result Valid and powerful inference based on ML-predicted data.
The study introduces backward baselines to distinguish past prediction from future prediction in machine learning models.
problem Differentiating between past and future prediction in machine learning models.
method Theoretical, empirical, and normative arguments support a family of simple and efficient statistical tests called backward baselines.
result The study provides a meaningful backward baseline for auditing black-box prediction systems.
FNNs can be made more interpretable with statistical methods.
problem FNNs lack interpretability and are often used as black-box models.
method Supplement FNNs with statistical inference and covariate-effect visualizations.
result FNNs can be made more like traditional statistical models.
A new method for predicting insurance claims with statistical guarantees.
problem Creating accurate prediction intervals for insurance claims.
method Model-agnostic framework using split conformal prediction for frequency-severity modeling.
result Shows effectiveness on simulated and real datasets using various models.
Using AI predictions as data can mislead inference, study shows.
problem Misleading inference when using AI predictions instead of real data.
method Characterized statistical challenges and reviewed methods for IPD.
result High predictive accuracy doesn't ensure valid inference.
In these notes we describe heuristics to predict computational-to-statistical gaps in certain statistical problems. These are regimes in which the underlying statistical problem is information-theoretically possible although no efficient algorithm exists, rendering the problem essentially unsolvable for large instances…
Novel framework for ML-assisted inference valid for any statistical task.
problem Limited validity of existing methods for post-prediction inference.
method Introduces PSPS framework for task-agnostic ML-assisted inference.
result Valid and efficient inference for arbitrary ML models.
New causal analysis reconciles predictive and statistical fairness.
problem Mutual exclusivity of predictive and statistical fairness notions.
method Derive a new causal decomposition formula for fairness measures.
result Predictive and statistical fairness are complementary, not mutually exclusive.
Paper compares neural networks and classical statistics for dementia prediction, highlighting interpretability of classical methods.
problem Tackles the challenge of interpreting risk factors for dementia prediction.
method Compares neural networks and classical statistics for dementia prediction.
result Classical statistics provide clearer interpretation of risk factors compared to neural networks.
Paper uses stats to predict treatment choice based on illness probability.
problem Improving treatment decision-making in personalized medicine.
method Statistical decision theory with maximum regret evaluation.
result Estimates illness probability for better treatment choice.
Neural network model predicts alternating event-free periods.
problem Dynamic prediction of alternating recurrent events with statistical nuance.
method Developed an online dynamic prediction framework using neural network theory.
result Outstanding performance in predicting alternating recurrent event-free time.
Improves statistical learning bounds with self-concordant losses.
problem Statistical prediction with nuisance components.
method Orthogonal statistical learning with self-concordant loss.
result Non-asymptotic bounds on excess risk improved by a dimension factor.
UnKGCP generates prediction intervals for uncertain knowledge graphs with statistical guarantees.
problem Lack of quantified predictive uncertainty in existing UnKGE methods.
method Proposes extsc{UnKGCP} framework using conformal prediction with a novel nonconformity measure.
result Sharp prediction intervals effectively capture predictive uncertainty in diverse UnKGE methods.
New method improves statistical inference using machine learning-imputed data.
problem Improving statistical inference with imputed data from machine learning.
method Two-phase sampling approach for Z-estimation with ML-imputed outcomes.
result Guaranteed efficiency matching or exceeding classical inference, regardless of prediction quality.
Proposes a framework for automated radiation therapy treatment planning with uncertainty quantification.
problem Quantifying uncertainties in dose-related quantities for automated treatment planning.
method Three-step pipeline: feature extraction, dose statistic prediction, and dose mimicking.
result Probabilistic treatment plans agree better with clinical counterparts than non-probabilistic ones.
Network-assisted regression uses conformal prediction for valid inference.
problem Predicting node attributes using network and conventional covariates with valid statistical inference.
method Network analog of conformal prediction under mild joint exchangeability assumption.
result Achieves finite sample validity and asymptotic conditional validity for various network covariates.
New methods improve cross-conformal prediction's prediction sets without sacrificing coverage guarantees.
problem Improving the width of prediction sets in cross-conformal prediction.
method Proposed new variants of existing methods based on recent results on more efficient combination of p-values.
result Smaller prediction sets achieved without compromising theoretical guarantees.
Proposes tests to control confounding bias in predictive models.
problem Lack of non-parametric tests for confounding bias in predictive modeling.
method Partial and full confounder tests for probing null hypotheses of unconfounded and fully confounded models.
result Reveals previously unreported or hard-to-correct confounders in machine learning models.
A new knockoff statistic using conditional prediction function improves variable selection in complex models.
problem Controlling false discovery rate in complex models with nonlinear relationships.
method Introducing a knockoff statistic based on the conditional prediction function for use with machine learning models.
result The CPF statistics provide superior power in detecting prognostic variables over existing knockoff statistics.
Measures three types of noise in LLM evaluations.
problem Separating signal from noise in LLM experiments.
method Defined and measured three types of noise: prediction, data, and total noise. Proposed the all-pairs paired method for statistical power.
result Total noise level is characteristic and predictable across all model pairs.
Paper provides statistical guarantees for GNNs in link prediction.
problem Link prediction accuracy in graph neural networks.
method Proposes a linear GNN architecture (LG-GNN) and derives statistical guarantees.
result LG-GNN produces consistent estimators for edge probabilities and has better detection of high-probability edges.
Paper proposes a new method for predicting DER adoption with hierarchical guarantees.
problem Accurately predicting DER adoption in electric grids with uncertainty and spatial disparity.
method Multivariate Hawkes process for modeling DER adoption dynamics and split conformal prediction algorithm for hierarchical validity.
result Empirical evaluation shows superior predictive accuracy and uncertainty calibration compared to existing methods.
Improves statistical inference using machine learning predictions with imputed data.
problem Invalid statistical inference due to machine learning prediction errors.
method Bootstrap confidence intervals for nonuniform samples and arbitrary imputed features.
result Valid confidence intervals without assumptions on machine learning model quality.
Discovering statistically significant patterns from databases is an important challenging problem. The main obstacle of this problem is in the difficulty of taking into account the selection bias, i.e., the bias arising from the fact that patterns are selected from extremely large number of candidates in databases. In …
Combines deep and statistical learning for structured data.
problem Structured high-dimensional data challenges.
method Generates nonlinear features via sparse regularization and stochastic optimisation, uses probabilistic output layer for uncertainty.
result Achieves best of scalability and uncertainty quantification.
Traffic prediction plays a vital role in efficient planning and usage of network resources in wireless networks. While traffic prediction in wired networks is an established field, there is a lack of research on the analysis of traffic in cellular networks, especially in a content-blind manner at the user level. Here, …
FPPI selectively uses predictions to improve inference efficiency.
problem Improving statistical inference with limited labeled data and heterogeneous prediction quality.
method Filtered Prediction-Powered Inference (FPPI) framework.
result FPPI achieves strictly improved asymptotic efficiency compared to existing methods.
The paper sets limits on the accuracy of macroeconomic forecasts based on statistical moments and trade volumes.
problem Uncertainty in predicting macroeconomic variables like prices and returns.
method Defines theoretical lower bounds of uncertainty and upper limits on forecast accuracy based on statistical moments and trade volumes.
result Accuracy of forecasts of probabilities of macroeconomic variables doesn't exceed Gaussian approximations.
The paper reviews exchangeability and its implications for conformal prediction and rank tests.
problem Ensuring distribution-free predictive inference in machine learning and statistics.
method Explains exchangeability and its role in conformal prediction and rank tests.
result Discovers similarities between conformal prediction and rank tests based on exchangeability.
Bayesian framework mixes imperfect models for improved predictions.
problem Improving predictions of complex computational models in unknown domains.
method Local Bayesian Dirichlet mixing of imperfect models using the Dirichlet distribution.
result Global and local mixtures of models achieve excellent performance in prediction accuracy and uncertainty quantification.
Paper introduces statistical learning for point processes.
problem Statistical learning for point processes in general spaces.
method Combines bivariate innovations and point process cross-validation.
result Statistical learning approach outperforms state of the art.
In this paper, we explore various statistical techniques for anomaly detection in conjunction with the popular Long Short-Term Memory (LSTM) deep learning model for transportation networks. We obtain the prediction errors from an LSTM model, and then apply three statistical models based on (i) the Gaussian distribution…
Raising statistical hurdles may not be justified due to data bias.
problem Data bias leads to unobserved results that weaken identification of revised hurdles.
method Theoretical and empirical analysis of statistical hurdles and data bias.
result Statistics targeting only published findings can be strongly identified.
Framework uses synthetic data from pretrained models to improve predictive modeling.
problem Limited effectiveness of synthetic data from generative models for improving predictive performance.
method Proposes an end-to-end framework that generates and filters synthetic data through domain-specific statistical methods.
result Consistent improvements in predictive performance across various settings.
Study uses complex networks and machine learning to predict soccer match outcomes.
problem Predicting soccer match outcomes with complex networks and machine learning.
method Complex network metrics and match statistics were used to build machine learning models.
result Models based on passing networks were as effective as traditional models using match statistics.
Enhanced TSFMs improve time series forecasting accuracy and reliability.
problem Variance, bias, and uncertainty in TSFMs' predictions on real data.
method Statistical and ensemble techniques including bagging, stacking, residual modeling, and prediction intervals.
result Hybrid models consistently outperform standalone TSFMs across multiple horizons.
Machine learning improves earnings forecasting accuracy and speed.
problem Improving earnings forecasting accuracy and speed.
method Adopting machine learning models for earnings forecasting.
result Machine learning model outperforms traditional models in accuracy and speed.
Differentially private conformal prediction improves statistical efficiency.
problem Quantifying uncertainty in private data analysis.
method Introducing differential conformal prediction and developing Differentially Private Conformal Prediction (DPCP).
result DPCP produces tighter prediction sets than existing private split conformal approaches.
Rigorous uncertainty quantification of probabilistic AI weather forecasts with conformal prediction
problem Calibrated uncertainty quantification in probabilistic weather forecasts
method Conformal prediction
result Calibrated uncertainty at no expense to other probabilistic metrics
This research improves model interpretability and uncertainty estimation for deep learning models on non-iid data.
problem Improving interpretability and uncertainty estimation for deep learning models on non-iid data.
method 4 UQ approaches (BNN, SWAG, MC dropout, ensemble) applied to ARMED MEDL models.
result Ensemble approaches, especially with 90% subsampling, provide best performance in prediction and uncertainty estimation.
Bayesian method predicts runtime metrics for fog manufacturing.
problem Accurate prediction of runtime performance metrics in fog manufacturing.
method Bayesian sparse regression for multivariate mixed responses.
result Enhanced prediction and statistical inferences of runtime metrics.
Extends conformal prediction for controlling expected risk of monotone loss functions.
problem Controlling expected risk of monotone loss functions.
method Generalizes split conformal prediction with coverage guarantee, extending to distribution shift, quantile risk, multiple, adversarial, and expectations of U-statistics.
result Tight up to an O(1/n) factor, with worked examples in computer vision and natural language processing. Theory predicts neural scaling exponents from language statistics.
problem No existing theory could quantitatively predict neural scaling exponents.
method Isolated two key statistical properties of language.
result Derives a simple formula predicting neural scaling exponents.