We extend probabilistic programming to handle conditioning on marginal distributions.
problem Conditioning probabilistic programs on marginal distributions of observable variables.
method We define and implement stochastic conditioning, allowing inference in probabilistic programs conditioned on marginal distributions.
result We demonstrate the effectiveness of stochastic conditioning in various real-life scenarios.
This work combines recurrent models with diffusion for probabilistic time series forecasting.
problem Scalability and capturing high-dimensional distributions and cross-feature dependencies in time series forecasting.
method Combines recurrent neural networks' efficiency with diffusion models' probabilistic modeling, using stochastic interpolants and conditional generation.
result Offers scalable probabilistic time series forecasting methods.
Proposes a new framework for open set recognition using conditional probabilistic generative models.
problem Unknown samples can mislead traditional deep neural networks during testing.
method Conditional Probabilistic Generative Models (CPGM) that combine generative models with discriminative information.
result Significantly outperforms baselines on multiple benchmark datasets.
State-space systems generate probabilistic dependencies between inputs and outputs.
problem Understanding probabilistic dependencies in state-space systems.
method Introducing a probabilistic framework and proving sufficient conditions for output existence and uniqueness.
result State-space systems can generate probabilistic dependencies, even without functional relations.
Proposes a method to apply conformal prediction to probabilistic time series forecasting models.
problem Obtaining accurate prediction regions for multi-step time series forecasting with probabilistic models.
method Conformalises conditional normalising flows to generate potentially disjoint prediction regions.
result Improves predictive efficiency in time series forecasting with multimodal distributions.
CatBoostLSS predicts entire conditional distributions for probabilistic forecasting.
problem Limited to predicting only the conditional mean, traditional CatBoost is improved.
method Models all moments of a parametric distribution (mean, location, scale, shape).
result Enhanced flexibility in data analysis and probabilistic forecasting.
We present Natural Gradient Boosting (NGBoost), an algorithm for generic probabilistic prediction via gradient boosting. Typical regression models return a point estimate, conditional on covariates, but probabilistic regression models output a full probability distribution over the outcome space, conditional on the cov…
Deep neural operators learn complex probabilistic models efficiently.
problem Learning complex probabilistic models with global Lipschitz conditions.
method Deep neural-operator framework under global Lipschitz conditions.
result Explicit network-size bounds for universal approximation of probabilistic models.
ACE models allow flexible conditioning and prediction of latent variables.
problem Lack of flexibility in conditioning and prediction of latent variables in probabilistic models.
method Introduces Amortized Conditioning Engine (ACE) that explicitly represents latent variables and allows runtime conditioning and prediction.
result ACE models outperform existing methods in diverse tasks like image completion, classification, Bayesian optimization, and simulation-based inference.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
ICON learns differential equation operators from examples, revealing probabilistic inference.
problem Learning operators for differential equations from limited examples.
method Probabilistic operator learning using ICON architectures trained on diverse datasets.
result ICON implicitly performs Bayesian inference on solution operators.
Probabilistic graphical models are a central tool in AI; however, they are generally not as expressive as deep neural models, and inference is notoriously hard and slow. In contrast, deep probabilistic models such as sum-product networks (SPNs) capture joint distributions in a tractable fashion, but still lack the expr…
CW-Gen models improve probabilistic time series forecasting by incorporating prior information.
problem Challenges in probabilistic forecasting of multivariate time series due to non-stationarity, inter-variable dependencies, and distribution shifts.
method CW-Gen framework that incorporates prior information through conditional whitening. JMCE learns conditional mean and covariance, improving sample quality.
result CW-Gen consistently enhances predictive performance, capturing non-stationary dynamics and inter-variable correlations more effectively than prior-free approaches.
This paper introduces a neural operator for probabilistic conditioning.
problem Probabilistic conditioning of random variables X given Y. method Develops a single operator that maps any joint density to its conditional, approximated by neural operators.
result Neural operators can approximate the conditioning operator to arbitrary accuracy.
ProbRes calibrates probabilistic forecasts by learning volatility dynamics.
problem Quantifying risk and uncertainty in time series forecasting.
method ProbRes learns conditional mean and volatility separately, generating well-calibrated prediction intervals.
result ProbRes accurately captures predictive distributions and produces well-calibrated prediction intervals.
DIN framework directly models hydraulic conductivity and uncertainty.
problem Modeling hydraulic conductivity and uncertainty in groundwater flow.
method DIN utilizes DDPM as a prior learner, incorporating observational data through conditional injection mechanisms.
result DIN generates multiple constraint-satisfying realizations and accurate uncertainty quantification.
This paper presents foundational theoretical results on distributed parameter estimation for undirected probabilistic graphical models. It introduces a general condition on composite likelihood decompositions of these models which guarantees the global consistency of distributed estimators, provided the local estimator…
MD-CGAN models forecast time series with probabilistic posterior distributions.
problem Limited applications of GANs in time series forecasting, especially with probabilistic predictions.
method Mixture Density Conditional Generative Adversarial Model (MD-CGAN) using Gaussian mixture output.
result MD-CGAN outperforms benchmarks, especially in noisy time series.
A new probabilistic mixup framework improves deep learning generalization.
problem Improving deep learning generalization performance.
method Introduces a novel probabilistic mixup framework for conditional density estimation.
result Empirical results show benefits over existing mixup variants.
Develops effective adversarial attacks on probabilistic forecasting models.
problem Adversarial attacks on neural models outputting probability distributions.
method Effective generation of adversarial attacks through Monte-Carlo estimation and Bayesian conditioning.
result Demonstrates successful generation of attacks with small input perturbations.
Unified probabilistic gradient boosting for entire conditional distribution modeling.
problem Creating accurate probabilistic forecasts from regression tasks.
method Unified probabilistic gradient boosting framework using XGBoost and LightGBM, modeling conditional moments or CDF via Normalizing Flows.
result Achieves state-of-the-art forecast accuracy.
CSDI improves time series imputation by 40-65% over existing methods.
problem Imputing missing values in time series data.
method Conditional Score-based Diffusion models conditioned on observed data.
result CSDI improves by 40-65% over existing probabilistic imputation methods on popular metrics.
As inductive inference and machine learning methods in computer science see continued success, researchers are aiming to describe ever more complex probabilistic models and inference algorithms. It is natural to ask whether there is a universal computational procedure for probabilistic inference. We investigate the com…
Transformer model for probabilistic dynamical systems.
problem Modeling high-dimensional dynamical systems from noisy observations.
method Parallel between dynamical systems and language modeling; transformer-based model with geometrical properties; iterative training algorithm.
result Fine-grid approximation of conditional probabilities for high-dimensional systems.
Proposes a cGAN-based model for probabilistic predictions and uncertainty estimation.
problem Deterministic predictions in ANNs limit their use in fields requiring uncertainty.
method Reverses cGAN structure for probabilistic prediction, introduces entropy for uncertainty measurement.
result Superior estimation performance, especially on noisy data.
Proposes a new method for probabilistic forecasting using stochastic interpolants and Föllmer processes.
problem Probabilistic forecasting of dynamical systems.
method Generative modeling and stochastic interpolants to map current state to probabilistic ensemble of forecasts.
result The approach can be used to forecast complex, high-dimensional systems like Navier-Stokes and video sequences.
Bayesian approach for policy search in stochastic domains.
problem Policy search in stochastic domains.
method Nested probabilistic programs, Lightweight Metropolis-Hastings (LMH) adaptation.
result Similar quality policies learned with simpler algorithm.
New insights into risk aversion for complex decision models.
problem Understanding risk aversion in non-monotone decision models.
method Characterization of probabilistic risk aversion for generalized rank-dependent functions.
result Probabilistic risk aversion is determined by the distortion function, which is convex or scaled quantile-spread mixtures.
It is time-consuming and error-prone to implement inference procedures for each new probabilistic model. Probabilistic programming addresses this problem by allowing a user to specify the model and having a compiler automatically generate an inference procedure for it. For this approach to be practical, it is important…
ProFITi model forecasts irregular time series with missing values using conditional flows.
problem Probabilistic forecasting of irregularly sampled multivariate time series with missing values.
method ProFITi model uses conditional normalizing flows and invertible layers to learn joint distributions conditioned on past observations and queried channels and times.
result ProFITi model provides 4 times higher likelihood than the previous best model.
A new approach to continual learning using fully probabilistic models.
problem Handling data distribution changes in machine learning models.
method Gaussian Mixture Replay (GMR) based on fully probabilistic models.
result GMR achieves state-of-the-art performance on class-incremental learning problems.
Treeffuser predicts tabular data distributions using gradient-boosted trees.
problem Probabilistic prediction with flexible, non-parametric models.
method Gradient-boosted trees for score estimation in conditional diffusion model.
result Treeffuser outperforms existing methods in probabilistic prediction tasks.
Paper introduces PCP for efficient, reliable predictive inference.
problem Developing reliable predictive inference methods for target variables.
method Probabilistic conformal prediction using conditional random samples.
result PCP provides sharper predictive sets compared to existing methods.
Study builds ML models to predict fuel properties accurately.
problem Accurate prediction of liquid fuel properties over a wide range of conditions.
method Used Gaussian Processes and probabilistic conditional generative learning to train ML models on fuel density data.
result ML models can predict fuel properties accurately across various pressure and temperature conditions.
Paper introduces md-vtrees for efficient probabilistic and causal inference.
problem Efficient inference in complex probabilistic models.
method Introduces md-vtrees to generalize tractability conditions for advanced inference queries.
result Derives first polytime algorithms for causal inference queries.
We introduce a new category of multivariate conditional generative models and demonstrate its performance and versatility in probabilistic time series forecasting and simulation. Specifically, the output of quantile regression networks is expanded from a set of fixed quantiles to the whole Quantile Function by a univar…
HCNAF models complex conditional distributions for probabilistic occupancy forecasting.
problem Modeling complex conditional probability density functions for occupancy forecasting.
method Hyper-Conditioned Neural Autoregressive Flow (HCNAF) combining AF and hyper-network.
result HCNAF achieves state-of-the-art performance in self-driving datasets.
TSFlow uses Gaussian processes to match priors for better time series forecasting.
problem Difficulties in aligning generative models' priors with time series data.
method Conditional flow matching (CFM) with Gaussian processes, optimal transport, and data-dependent priors.
result TSFlow produces high-quality unconditional samples and competitive forecasting results.
In Bayesian classification, it is important to establish a probabilistic model for each class for likelihood estimation. Most of the previous methods modeled the probability distribution in the whole sample space. However, real-world problems are usually too complex to model in the whole sample space; some fundamental …
Efficient methods for answering complex probabilistic queries in sequential data.
problem Complex probabilistic queries in sequential data.
method Broad class of novel approximation techniques for marginalization in sequential models.
result Efficient techniques for answering long-range probabilistic queries.
Study efficient interactive learning for structured outputs with reliable computation.
problem Interactive learning with noisy labels and structured output spaces.
method Identify and utilize CRISPs (probabilistic models) that guarantee reliable and efficient computation of probabilistic quantities.
result CRISPs enable robust and efficient active and skeptical learning in large structured output spaces.
Kernel density matrices simplify probabilistic deep learning.
problem Representing joint probability distributions of continuous and discrete variables.
method Extending density matrices to a reproducing kernel Hilbert space.
result Versatile representation for marginal and joint probability distributions.
The paper investigates model collapse in language models from a probabilistic perspective.
problem Understanding and preventing model collapse in language model training.
method Investigates recursive parametric model training from a probabilistic standpoint, characterizing conditions for model collapse and proposing mitigation strategies.
result Progressively increasing sample size is necessary to prevent model collapse, with a superlinear growth rate required in the asymptotic regime.
In many supervised learning tasks, the entities to be labeled are related to each other in complex ways and their labels are not independent. For example, in hypertext classification, the labels of linked pages are highly correlated. A standard approach is to classify each entity independently, ignoring the correlation…
New model forecasts power consumption with high accuracy over months to years.
problem Probabilistic forecasting of power consumption in a middle-term horizon.
method Combines traditional time-series analysis with weather conditions using Gaussian Process.
result Promising results in Out-of-Sample density forecasts up to one year.
Generative adversarial network for probabilistic forecasting of random systems.
problem Forecasting random dynamical systems without distributional assumptions.
method Recurrent neural network and generative adversarial network (GAN) with regularization based on maximum mean discrepancy (MMD).
result The proposed model successfully forecasts complex stochastic processes with multiple-step predictions.
Hybrid model combines continuous and tractable probabilistic models.
problem Intractable probabilistic inference in continuous latent-space models.
method Continuous mixtures of tractable probabilistic models with finite integration points.
result Hybrid models achieve state-of-the-art performance in density estimation.
We propose a new framework of XGBoost that predicts the entire conditional distribution of a univariate response variable. In particular, XGBoostLSS models all moments of a parametric distribution (i.e., mean, location, scale and shape [LSS]) instead of the conditional mean only. Choosing from a wide range of continuou…