Binary classification models get more efficient predictive probabilities.
problem Computing predictive probabilities in Bayesian probit models is computationally challenging.
method Use of expectation propagation (EP) to find a closed-form expression for predictive probabilities.
result Closed-form predictive probabilities improve over existing methods.
In many real-world applications of machine learning classifiers, it is essential to predict the probability of an example belonging to a particular class. This paper proposes a simple technique for predicting probabilities based on optimizing a ranking loss, followed by isotonic regression. This semi-parametric techniq…
Develops a method to make predictions more informative without sacrificing accuracy.
problem Tension between well-calibrated and bold predictions.
method Bayesian model selection-based approach for assessing calibration and maximizing boldness.
result Maximally emboldening predictions while maintaining desired level of calibration.
Study optimizes tree-based models for better alignment of predicted scores and actual probabilities.
problem Traditional calibration metrics fail to align predicted scores with actual probabilities when score distributions deviate from the underlying data.
method Optimizes tree-based models (Random Forest, XGBoost) using Kullback-Leibler (KL) divergence to minimize the difference between predicted and true probability distributions.
result Optimized tree-based models yield superior alignment between predicted scores and actual probabilities without significant performance loss.
The paper develops methods to predict the probability of achieving a user goal in a task, ensuring the system alerts when the probability falls below a threshold.
problem Ensuring an autonomous system achieves the user's goal with calibrated probability estimates.
method Invertible conformal prediction using Probability-space Conformalized Quantile Regression (PCQR) to produce well-calibrated conditional prediction intervals.
result The method produces well-calibrated probabilities that the cumulative reward will fall within a user-specified target interval, with finite-sample guarantees.
PPI uses predictions and weighting to infer from partially labeled data.
problem Valid inference with partially labeled data.
method Combines model-based predictions with bias correction from labeled data, using Horvitz-Thompson and Hájek corrections.
result IPW-adjusted PPI with estimated propensities performs similarly to known-probability case.
New tools connect CP to GF inference for better probabilistic prediction.
problem Lack of versatility in conformal prediction for quantifying evidence.
method Imprecise probability theory and generalized fiducial inference.
result Establishes a formal connection between CP and GF inference.
The study predicts pass completion probability in NFL games.
problem Estimating the likelihood of NFL pass completion.
method Machine learning algorithm using distance measures and random forest model.
result Developed a method to predict pass completion probability.
New insights on active sequential prediction for mean estimation.
problem Active sequential prediction-powered mean estimation problem.
method Combining uncertainty-based suggestion with a constant probability, analyzing non-asymptotic bounds, and using no-regret learning.
result The optimal query probability is close to the constraint when using no-regret learning.
A method to monitor probability predictions for calibration loss in image classification models.
problem Maintaining calibration in machine learning predictions over time.
method Cumulative sum-based approach with dynamic limits for early detection of miscalibration.
result Early detection of operational context changes impacting image classification performance.
New method calibrates classifier probabilities with guaranteed coverage.
problem Inaccurate probability estimates by classifiers in high-risk applications.
method Adaptive temperature scaling algorithm for conformal prediction.
result Improves calibration error measures and standard metrics across various tasks.
Conformal Prediction Regions match Imprecise Highest Density Regions under consonance.
problem Matching conformal prediction regions with highest density regions.
method Using consonance and the Imprecise Probability theory of clouds.
result Imprecise Highest Density Regions are equivalent to Conformal Prediction Regions under consonance.
A Bayesian framework models dynamic probability predictions over time.
problem Dynamic probability predictions over time in various settings.
method Gaussian latent information martingale (GLIM) framework.
result GLIM outperforms baseline methods in predicting future uncertainties.
New acquisition function improves batch Bayesian active learning.
problem BatchBALD conflates epistemic and aleatoric uncertainty, leading to suboptimal performance.
method Focus on predictive probabilities to separate epistemic uncertainty, leading to better performance and faster evaluation.
result The new acquisition function performs better and allows for larger batches.
This work presents a new classifier that is specifically designed to be fully interpretable. This technique determines the probability of a class outcome, based directly on probability assignments measured from the training data. The accuracy of the predicted probability can be improved by measuring more probability es…
Midicoth compresses online probability estimates by correcting prior smoothing biases.
problem Compression inefficiency due to prior smoothing in adaptive models.
method Micro-diffusion denoising applied in a bitwise tree hierarchy.
result Significant compression improvement with reliable calibration.
This research improves demand forecasting by predicting complete probability density functions using machine learning.
problem Forecasting complete probability density functions for better operational decision making.
method Supervised machine learning method 'Cyclic Boosting' for explainable predictions.
result Predicted probability density functions are fully explainable and avoid 'black-box' models.
Two strategies extend multi-label chaining for imprecise probability estimates.
problem Handling imprecise probability estimates in multi-label classification.
method Adapting multi-label chaining to use convex sets of distributions (credal sets).
result Adapted approaches produce relevant cautiousness on hard-to-predict instances.
This paper compares ML algorithms for PD prediction, finding XGBoost to be the most effective.
problem Predicting the probability of default in loan portfolios.
method Comparison of five ML algorithms (Random Forests, Decision Trees, XGBoost, Gradient Boosting, AdaBoost) with logistic regression.
result XGBoost outperforms other ML algorithms for PD prediction.
RegFlow models future states with flexible probability distributions.
problem Predicting future states under complex, non-deterministic scenarios.
method Hypernetwork architecture and continuous normalizing flow model.
result RegFlow achieves state-of-the-art results on benchmark datasets.
The paper tackles multi-label ranking with uncertain probabilities.
problem Making skeptical inferences for multi-label ranking with sets of probabilities.
method Assumes a convex set of probabilities (credal set) over labels and seeks set-valued predictions.
result Developed methods for making skeptical inferences in multi-label ranking with uncertain probabilities.
This work improves deep neural network probability estimation methods.
problem Estimating probabilities from high-dimensional data with inherent uncertainty.
method Investigates and compares methods for probability estimation using deep neural networks, proposing a new method that promotes consistent probabilities.
result The new method outperforms existing approaches on most metrics on simulated and real-world data.
Paper provides statistical guarantees for GNNs in link prediction.
problem Link prediction accuracy in graph neural networks.
method Proposes a linear GNN architecture (LG-GNN) and derives statistical guarantees.
result LG-GNN produces consistent estimators for edge probabilities and has better detection of high-probability edges.
The article focuses on determining the predictive uncertainty of a model on the example of atrial fibrillation detection problem by a single-lead ECG signal. To this end, the model predicts parameters of the beta distribution over class probabilities instead of these probabilities themselves. It was shown that the desc…
Active Kriging Monte Carlo simulation method with conformal certification for failure probability estimation
problem Failure probability estimation in structural reliability analysis
method Active learning framework with conformal prediction
result Improved uncertainty quantification and reliability of failure probability estimates
KCal calibrates deep networks by embedding logits in a metric space.
problem Overconfident predictions from DNNs, especially in high-risk applications.
method KCal learns a metric space on the penultimate-layer latent embedding and generates predictions using kernel density estimates.
result KCal provides a provable full calibration guarantee and consistently outperforms baselines.
Proposes MCLLO for assessing and recalibrating multiclass probability predictions.
problem Limited multicategory recalibration methods for assessing and comparing model calibration.
method MCLLO recalibration method that assesses calibration without model access and is easy to interpret.
result MCLLO outperforms other methods in simulations and real-world case studies.
Study improves keyword forecasting in earnings-call prediction markets.
problem Accurately predicting future keyword mentions in earnings calls.
method Experiments on earnings-call mention markets, varying context and market probability, introducing MCP.
result Mixture of market probability and MCP yields the best forecasts.
Paper uses stats to predict treatment choice based on illness probability.
problem Improving treatment decision-making in personalized medicine.
method Statistical decision theory with maximum regret evaluation.
result Estimates illness probability for better treatment choice.
Proposes a new method to minimize non-singleton predictions in conformal prediction.
problem Large prediction sets in conformal prediction are costly and inefficient.
method Introduces a new nonconformity score to minimize non-singleton sets and provides an algorithm to compute it efficiently.
result The proposed Singleton-Optimized Conformal Prediction (SOCOP) method increases singleton frequency by over 20% compared to standard scores, with minimal impact on average set size.
A method for predicting credal sets in classification tasks using conformal prediction.
problem Designing methods for learning credal set predictors in machine learning.
method Incorporates conformal prediction for predicting credal sets in classification tasks.
result Conformal credal sets are guaranteed to be valid with high probability.
DynForest predicts event probabilities from longitudinal data, handling endogenous predictors.
problem Predicting individual risk using longitudinal patient history.
method Random survival forests with time-fixed features from longitudinal predictors.
result DynForest provides accurate individual event probability predictions.
Link prediction in networks is typically accomplished by estimating or ranking the probabilities of edges for all pairs of nodes. In practice, especially for social networks, the data are often collected by egocentric sampling, which means selecting a subset of nodes and recording all of their edges. This sampling mech…
Price and return predictions are limited by economic complexity, not just volatility.
problem Limited accuracy of price and return probability forecasts by Gaussian distributions.
method Analyzes economic reasons behind limitations in predicting price and return statistical moments.
result Predictions of price and return probabilities by Gaussian distributions are inaccurate due to economic complexity.
We extend Bayes' theorem for upper probabilities considering likelihood uncertainty.
problem Addressing uncertainty in likelihood for upper probability bounds.
method Generalization of Wasserman and Kadane's result, considering both prior and likelihood uncertainty.
result A sufficient condition for the upper bound to become an equality.
Gaussian Processes (GPs) are widely employed in control and learning because of their principled treatment of uncertainty. However, tracking uncertainty for iterative, multi-step predictions in general leads to an analytically intractable problem. While approximation methods exist, they do not come with guarantees, mak…
Paper introduces methods for more reliable probabilistic predictions with confidence intervals.
problem Inaccurate labeling of datasets due to unreliable probabilistic predictions from weak labeling functions.
method Proposes a methodology to provide confidence intervals for label probabilities using uncertainty sets of distributions.
result Improves reliability of probabilistic predictions and provides confidence intervals for label probabilities.
Boosted decision trees typically yield good accuracy, precision, and ROC area. However, because the outputs from boosting are not well calibrated posterior probabilities, boosting yields poor squared error and cross-entropy. We empirically demonstrate why AdaBoost predicts distorted probabilities and examine three cali…
Probabilistic forecasts in the form of probability distributions over future events have become popular in several fields of statistical science. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). Popular example of scoring rule for continuous outcomes is the …
SJS model predicts label shifts in multinomial datasets.
problem Predicting label shifts in multinomial datasets.
method Sparse joint shift model for dataset shift.
result Valid predictions and class prior probabilities estimates.
This paper establishes for the first time the predictive performance of speed priors and their computational complexity. A speed prior is essentially a probability distribution that puts low probability on strings that are not efficiently computable. We propose a variant to the original speed prior (Schmidhuber, 2002),…
New method predicts wave height exceedance probabilities.
problem Forecasting significant wave height to prevent coastal disasters.
method Point forecasting approach using cumulative distribution function.
result Proposed method outperforms existing approaches.
Diffusion reach probability between two nodes on a network is defined as the probability of a cascade originating from one node reaching to another node. An infinite number of cascades would enable calculation of true diffusion reach probabilities between any two nodes. However, there exists only a finite number of cas…
While the accuracy of modern deep learning models has significantly improved in recent years, the ability of these models to generate uncertainty estimates has not progressed to the same degree. Uncertainty methods are designed to provide an estimate of class probabilities when predicting class assignment. While there …
Paper predicts embryo implantation probability from IVF time-lapse imaging.
problem Manual embryo selection in IVF has low success rates.
method Data-driven system trained on time-lapse imaging videos.
result Algorithm improves positive and negative predictive values.
This paper examines how different decoding algorithms for LLMs align with various goals.
problem Consistency of decoding algorithms with different goals in LLMs.
method Analysis of greedy, lookahead, random sampling, and temperature-scaled random sampling algorithms.
result Random sampling is consistent with the true probability distribution, but other goals require optimal algorithms for specific probability distributions.
Article addresses practical challenges in conformal prediction.
problem Challenges in determining, computing, and controlling conformal prediction regions.
method Proposes a quadratic-polynomial non-conformity measure.
result Allows circumventing three challenges in full conformal prediction framework.
VSPS creates flexible prediction regions for multi-target regression with guaranteed coverage.
problem Uncertainty quantification in multi-target regression with complex distributions.
method Conditional normalizing flows with conformal calibration to identify dense regions.
result VSPS produces smaller, more informative prediction regions with robust coverage guarantees.