Paper studies ensemble probabilistic regression trees for smooth approximations.
problem Smooth approximations of regression functions.
method Ensemble versions of probabilistic regression trees.
result Ensemble probabilistic regression trees are consistent and perform well.
NGBoost boosts multivariate probabilistic regression.
problem Joint probabilistic regression for multivariate targets.
method Natural Gradient Boosting for nonparametric modeling.
result Competitive performance in oceanographic velocity prediction.
VIR model improves regression accuracy and uncertainty estimation for imbalanced data.
problem Imbalanced regression datasets lead to poor model accuracy and uncertainty estimation.
method VIR model uses probabilistic smoothing and reweighting to estimate latent representations and uncertainty.
result VIR model outperforms state-of-the-art models in accuracy and uncertainty estimation.
Bayesian symbolic regression automates model discovery from data.
problem Learning closed-form mathematical models from data using heuristic methods.
method Probabilistic approach to symbolic regression, connecting to information theory and statistical physics.
result Probabilistic approach provides model plausibility and performance guarantees.
New method improves probabilistic electricity price predictions.
problem Improving point forecasts to probabilistic distributions for better decision-making.
method Isotonic Distributional Regression combined with other postprocessing methods.
result Isotonic Distributional Regression outperforms other methods in combining probabilistic distributions.
Enforces physical constraints in GP regression models.
problem Unbounded GP models can produce infeasible values.
method Enforces nonnegativity constraints probabilistically.
result Reduces model variance and enforces physical bounds.
Combines coarse learners for nonparametric probabilistic regression.
problem Predicting full probability density functions without strong assumptions.
method Combining gradient boosted forests trained on coarsened target values.
result Prediction intervals have high fidelity and provide valuable insights.
We present Natural Gradient Boosting (NGBoost), an algorithm for generic probabilistic prediction via gradient boosting. Typical regression models return a point estimate, conditional on covariates, but probabilistic regression models output a full probability distribution over the outcome space, conditional on the cov…
VaSST uses soft symbolic trees for probabilistic symbolic regression.
problem Efficiently recover symbolic expressions from noisy data.
method Variational inference with soft symbolic trees.
result Superior performance in structural recovery and predictive accuracy.
The paper presents a novel approach to multi-output regression using probabilistic circuits.
problem Capturing correlations between multiple output dimensions in large-scale regression problems.
method Employing a mixture of single-output Gaussian process experts encoded via a probabilistic circuit.
result The method can capture correlations between output dimensions and often outperforms other approaches.
Paper introduces methods to handle missing data in probabilistic regression trees.
problem Handling missing data in probabilistic regression trees.
method Three approaches: uniform probability, partial observation, and dimension-reduced smoothing.
result Preserves interpretability while extending applicability to incomplete datasets.
Collider regression improves predictive performance in regression tasks.
problem Discarding prior causal knowledge in regression tasks.
method Collider regression framework incorporating probabilistic causal knowledge from collider structures.
result Proves positive generalization benefit and provides closed-form estimators.
While deep learning-based classification is generally tackled using standardized approaches, a wide variety of techniques are employed for regression. In computer vision, one particularly popular such technique is that of confidence-based regression, which entails predicting a confidence value for each input-target pai…
New method improves calibration in multi-output probabilistic models.
problem Challenges in achieving multivariate calibration in multi-output regression.
method General regularization framework to enforce multivariate calibration during training for arbitrary pre-rank functions.
result Significant improvement in calibration across all pre-rank functions without sacrificing predictive accuracy.
SQR Averaging improves probabilistic electricity price forecasting.
problem Accurate short-term price forecasting in electricity markets.
method Smoothing Quantile Regression Averaging.
result SQR Averaging leads to profit increases of up to 3.5% in day-ahead power trading.
Paper proposes a unified framework for evaluating calibration of probabilistic models.
problem Evaluation of calibration for general probabilistic predictive models.
method Unified framework for calibration evaluation and tests for any probabilistic model.
result Generalization and reformulation of existing measures and tests.
Point forecasting of univariate time series is a challenging problem with extensive work having been conducted. However, nonparametric probabilistic forecasting of time series, such as in the form of quantiles or prediction intervals is an even more challenging problem. In an effort to expand the possible forecasting p…
We present a probabilistic viewpoint to multiple kernel learning unifying well-known regularised risk approaches and recent advances in approximate Bayesian inference relaxations. The framework proposes a general objective function suitable for regression, robust regression and classification that is lower bound of the…
Short-term load forecasting is a critical element of power systems energy management systems. In recent years, probabilistic load forecasting (PLF) has gained increased attention for its ability to provide uncertainty information that helps to improve the reliability and economics of system operation performances. This…
Multilabel classification is an important problem in a wide range of domains such as text categorization and music annotation. In this paper, we present a probabilistic model, Multilabel Logistic Regression with Hidden variables (MLRH), which extends the standard logistic regression by introducing hidden variables. Hid…
Parametric quantile regressions are a useful tool for creating probabilistic energy forecasts. Nonetheless, since classical quantile regressions are trained using a non-differentiable cost function, their creation using complex data mining techniques (e.g., artificial neural networks) may be complicated. This article p…
Adaptive probabilistic load forecasting improves performance in power systems.
problem Complexity of electricity load forecasting due to changing drivers and local generation.
method Adaptive probabilistic approach using Kalman filter and online gradient descent.
result Adaptive probabilistic forecasts improve performance in both point and probabilistic forecasting.
Unified probabilistic gradient boosting for entire conditional distribution modeling.
problem Creating accurate probabilistic forecasts from regression tasks.
method Unified probabilistic gradient boosting framework using XGBoost and LightGBM, modeling conditional moments or CDF via Normalizing Flows.
result Achieves state-of-the-art forecast accuracy.
New method calibrates probabilistic regression models without restrictive assumptions.
problem Ensuring predictive distributions accurately reflect true uncertainty.
method Nonparametric re-calibration algorithm based on conditional kernel mean embeddings.
result Consistently outperforms prior re-calibration approaches across various benchmarks.
The task of calibration is to retrospectively adjust the outputs from a machine learning model to provide better probability estimates on the target variable. While calibration has been investigated thoroughly in classification, it has not yet been well-established for regression tasks. This paper considers the problem…
New method provides calibrated feature importance explanations for regression models.
problem Lack of uncertainty quantification in existing local explanation methods.
method Extension of Calibrated Explanations method to support regression and probabilistic regression.
result Calibrated Explanations for regression provides quantified uncertainty and robust explanations.
Quantile regression improves urban water demand forecasting.
problem Improving probabilistic urban water demand forecasting.
method Comparing five quantile regression algorithms and their combinations for one-day ahead forecasting.
result Linear boosting algorithm performs best for probabilistic urban water demand forecasting.
Fenrir uses probabilistic numerics to simplify solving initial value problems.
problem Solving initial value problems in ordinary differential equations.
method Probabilistic numerics and Gauss--Markov regression.
result The method simplifies parameter estimation in ODEs, making it easier and more robust.
Enhances traffic forecasting with dynamic regression incorporating error modeling.
problem Improving accuracy of traffic forecasts using deep spatiotemporal models.
method Integrates matrix-variate autoregressive (AR) model into loss function for error series of base model.
result Improved traffic forecasting performance on SOTA models with interpretable AR coefficients.
A new beta-VAE based regression model accelerates oilfield optimization studies.
problem Computational expense of full-physics reservoir simulations.
method beta-VAE for interpretable latent space representation, probabilistic dense layers for uncertainty quantification.
result Interpretable latent representation and quantified uncertainty for optimization decisions.
Hybrid approach improves probabilistic forecasts for electricity trading.
problem Improving probabilistic forecasts for electricity trading markets.
method Combines QRA and factor-based averaging for probabilistic forecasting.
result The hybrid approach outperforms benchmarks in statistical measures and economic value.
New method turns any regression model into a calibrated probabilistic model.
problem Calibration and sharpness of uncertainty estimates in regression models.
method Modular Conformal Calibration (MCC) framework.
result MCC algorithms achieve near-perfect calibration and improved sharpness.
Bayesian method corrects for model selection multiplicity in regression.
problem Model selection multiplicity in regression analysis.
method Developed a Bayesian prior distribution based on Holm procedure analogy.
result Adequate multiplicity correction requires sparsity not provided by recommended priors.
FDN improves probabilistic regressors' adaptability to distribution shifts.
problem Overconfidence in modern probabilistic regressors under distribution shift.
method FDN uses input-conditioned distributions over network weights, trained with a Monte Carlo beta-ELBO objective.
result FDN produces predictive mixtures whose dispersion adapts to the input, providing shift-aware uncertainty.
Probabilistic grammars improve equation discovery from data.
problem Discovering scientific laws from data using equations.
method Proposed probabilistic context-free grammars to encode soft constraints and a Monte-Carlo algorithm.
result Probabilistic grammars lead to more efficient equation discovery.
Proposes a new algorithm for Sparse Bayesian Learning connected to Stepwise Regression.
problem Sparse Bayesian Learning for probabilistic models.
method Coordinate ascent algorithm (RMP) for SBL, showing connection to Stepwise Regression.
result RMP's noise variance parameter limit connects to Stepwise Regression, with derived guarantees.
Study compares deterministic and probabilistic ML for precise AM component dimensions.
problem Accurately estimate dimensions of additively manufactured parts with variability.
method Employed models integrating continuous and categorical factors, tested deterministic and probabilistic ML methods.
result Gaussian Process Regression and Bayesian Neural Networks provide strong predictive performance and uncertainty quantification.
A new GP method enforces physical constraints in probabilistic terms.
problem Unbounded model in GP regression leading to infeasible values.
method Introduces a new GP method using QHMC to enforce soft inequality and monotonicity constraints.
result Improves accuracy and reduces variance in GP model.
This paper improves SVM prediction uncertainty quantification methods.
problem Lack of comprehensive UQ methods for SVM predictions.
method Developed SSVQR model for sparse PI estimation and feature selection algorithm.
result Proposed SSVQR model achieves sparse solutions and improves PI quality.
Time series forecasting is one of the challenging problems for humankind. Traditional forecasting methods using mean regression models have severe shortcomings in reflecting real-world fluctuations. While new probabilistic methods rush to rescue, they fight with technical difficulties like quantile crossing or selectin…
Uncertainty analysis in the form of probabilistic forecasting can provide significant improvements in decision-making processes in the smart power grid for better integrating renewable energies such as wind. Whereas point forecasting provides a single expected value, probabilistic forecasts provide more information in …
Probabilistic programming languages can simplify the development of machine learning techniques, but only if inference is sufficiently scalable. Unfortunately, Bayesian parameter estimation for highly coupled models such as regressions and state-space models still scales poorly; each MCMC transition takes linear time i…
Novel probabilistic solver speeds up solving related linear systems.
problem Efficiently solving multiple related linear systems.
method Probabilistic linear solver over the parameter space, leveraging solved systems.
result Faster and more efficient solution of related linear systems.
A new method for Bayesian neural networks using probabilistic backpropagation.
problem Approximating posterior distributions in Bayesian neural networks.
method Variational Expectation Propagation (VEP) with probabilistic backpropagation.
result Efficient algorithm for approximate integration over posterior distributions.
We investigate whether the bid/ask queue imbalance in a limit order book (LOB) provides significant predictive power for the direction of the next mid-price movement. We consider this question both in the context of a simple binary classifier, which seeks to predict the direction of the next mid-price movement, and a p…
Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-…
We present a simple quantile regression-based forecasting method that was applied in a probabilistic load forecasting framework of the Global Energy Forecasting Competition 2017 (GEFCom2017). The hourly load data is log transformed and split into a long-term trend component and a remainder term. The key forecasting ele…
Probabilistic programming languages represent complex data with intermingled models in a few lines of code. Efficient inference algorithms in probabilistic programming languages make possible to build unified frameworks to compute interesting probabilities of various large, real-world problems. When the structure of mo…