New grammar model learns sentence structure with latent variables.
problem Grammar induction for sentences with complex dependencies.
method Compound probabilistic context-free grammar with latent variables, variational inference.
result Effective unsupervised parsing compared to state-of-the-art methods.
SPECTRA improves probabilistic energy forecasting by separating trends and uncertainties.
problem Interacting uncertainties from renewable intermittency, demand flexibility, market volatility, and weather impact probabilistic forecasts.
method Adaptive state-space exogenous context and temporal-frequency resolution architecture.
result Achieved best CRPS in 14 out of 18 settings, reducing CRPS by 5.74% and upper-tail quantile risk by 7.27%.
Meta-RL algorithm improves sample efficiency and adaptability.
problem Challenges in meta-reinforcement learning, especially on-policy experience and task uncertainty.
method Develops an off-policy meta-RL algorithm that probabilistically infers task variables.
result Significantly outperforms prior algorithms in sample efficiency and asymptotic performance.
Optimally explores dynamical systems with varying properties using context inference.
problem Learning dynamics models for systems with varying properties.
method Formulates dynamics models as stochastic processes conditioned on a latent context variable inferred from system transitions. Uses probabilistic formulation to compute optimal action sequences for exploration.
result Demonstrates effectiveness of the method on non-linear toy-problems and reinforcement learning environments.
Arriving at the complete probabilistic knowledge of a domain, i.e., learning how all variables interact, is indeed a demanding task. In reality, settings often arise for which an individual merely possesses partial knowledge of the domain, and yet, is expected to give adequate answers to a variety of posed queries. Tha…
Datasets with hundreds of variables and many missing values are commonplace. In this setting, it is both statistically and computationally challenging to detect true predictive relationships between variables and also to suppress false positives. This paper proposes an approach that combines probabilistic programming, …
This book is a graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, but also introduces the techniques needed to design and build these systems. It is aimed at people who have an undergraduate-level understandi…
We extend probabilistic programming to handle conditioning on marginal distributions.
problem Conditioning probabilistic programs on marginal distributions of observable variables.
method We define and implement stochastic conditioning, allowing inference in probabilistic programs conditioned on marginal distributions.
result We demonstrate the effectiveness of stochastic conditioning in various real-life scenarios.
OCEAN infers online task identities from context variables.
problem Online task inference for compositional tasks with context adaptation.
method Variational inference framework OCEAN models global and local context variables in a joint latent space.
result OCEAN provides more effective task inference with sequential context adaptation.
Extends partitioned local depth concept with probabilistic considerations.
problem Uncertain, variable, and conflicting information in data.
method Partitioned local depth with probabilistic concepts of local relevance and support division.
result Extends original ideas to handle uncertain data.
MQTransformer improves forecast accuracy with context-aware attention.
problem Improving probabilistic demand prediction accuracy.
method Incorporates Transformer architectures for context alignment and feedback-aware attention.
result Significant improvements in forecast accuracy, reducing excess variability.
Paper justifies ideal point forecasts as measurable, clarifying conditions for their existence.
problem Justifying ideal point forecasts as measurable random variables.
method Clarifying and establishing measurability conditions for a wide class of functionals.
result Ideal point forecasts are shown to be measurable, providing theoretical justification.
Circular variables arise in a multitude of data-modelling contexts ranging from robotics to the social sciences, but they have been largely overlooked by the machine learning community. This paper partially redresses this imbalance by extending some standard probabilistic modelling tools to the circular domain. First w…
JoLT uses LLMs to make probabilistic predictions on tabular data.
problem Making probabilistic predictions on tabular data efficiently and without preprocessing.
method JoLT leverages LLMs' in-context learning to define joint distributions over tabular data.
result JoLT outperforms other methods on tabular classification and regression tasks.
A serious problem in learning probabilistic models is the presence of hidden variables. These variables are not observed, yet interact with several of the observed variables. Detecting hidden variables poses two problems: determining the relations to other variables in the model and determining the number of states of …
The paper examines how updates to probabilistic models influence behavior based on evidence.
problem Understanding how updates to probabilistic models influence behavior based on evidence.
method Study of KL-regularized soft updates as Bayesian posterior updates within a single probabilistic model.
result Posterior updates determine relative incentives but not absolute rewards, which are ambiguous up to context-specific baselines.
Paper improves Tm prediction of protein fragments using sparsity and probabilistic models.
problem Improving accuracy of melting temperature prediction for protein fragments.
method Promoting sparsity in pre-trained transformer models and adopting probabilistic frameworks.
result Mean absolute error of 0.23C for predicting melting temperature.
The paper models musical motif transformations in Beethoven's works.
problem Understanding how motifs transform in symbolic music.
method Developed a probabilistic framework using Conditional Random Fields.
result Identified patterns of motif transformations and their co-occurrences.
PNNs model aleatoric uncertainty in scientific machine learning with high accuracy.
problem Aleatoric uncertainty in scientific systems with unequal variance.
method Developed a probabilistic distance metric to optimize PNN architecture and used it in material science applications.
result PNNs yield remarkably accurate output mean estimates and high correlation in predicted intervals.
Connectivity estimation is challenging in the context of high-dimensional data. A useful preprocessing step is to group variables into clusters, however, it is not always clear how to do so from the perspective of connectivity estimation. Another practical challenge is that we may have data from multiple related classe…
A recurring problem when building probabilistic latent variable models is regularization and model selection, for instance, the choice of the dimensionality of the latent space. In the context of belief networks with latent variables, this problem has been adressed with Automatic Relevance Determination (ARD) employing…
Probabilistic grammars improve equation discovery from data.
problem Discovering scientific laws from data using equations.
method Proposed probabilistic context-free grammars to encode soft constraints and a Monte-Carlo algorithm.
result Probabilistic grammars lead to more efficient equation discovery.
New models explain image-based questions better with fewer examples.
problem Improving understanding and efficiency in visual question answering.
method Probabilistic neural-symbolic models with interpretable latent programs.
result Models generate more understandable programs with fewer examples and allow probing reasoning.
The paper tackles physical constraints in probabilistic machine learning for CG models of high-dimensional systems.
problem Introducing physical constraints in probabilistic machine learning objectives for coarse-graining dynamical systems.
method Formulating coarse-graining process using probabilistic state-space model and accounting for constraints as virtual observables.
result Probabilistic inference tools can identify coarse-grained variables without needing a fine-to-coarse projection or time-derivatives.
Proposes a method to reconcile count time series forecasts.
problem No formal framework for probabilistic reconciliation of count time series.
method Generalizes Bayes' rule for reconciling real-valued and count variables.
result Improves forecast accuracy for count variables compared to Gaussian reconciliation.
Improved inference in probabilistic programs using attention mechanisms.
problem Inference failure in existing IC network architectures due to long-range dependency issues.
method Inference compilation with attention mechanism to model latent variables.
result Attention mechanism enhances proposal distributions to better match true posterior.
Probabilistic learning for binary classification with categorical variables.
problem Binary classification with categorical covariates.
method Probabilistic analysis and two algorithms for learning boolean functions.
result Effective learning of boolean functions from binary data.
Paper introduces MLRH, a probabilistic model for multilabel classification.
problem Multilabel classification challenges in various domains.
method Introduces hidden variables to relax one-hot-encoding in logistic regression.
result Probabilistic model achieves competitive performance compared to other algorithms.
We present a class of models that, via a simple construction, enables exact, incremental, non-parametric, polynomial-time, Bayesian inference of conditional measures. The approach relies upon creating a sequence of covers on the conditioning variable and maintaining a different model for each set within a cover. Infere…
Paper evaluates CRPS for extreme event forecasts, finding it unsuitable.
problem Verifying probabilistic forecasts of extreme events is challenging.
method Formal framework using extreme value theory to assess CRPS as a random variable.
result CRPS is unsuitable for extreme event verification.
The paper develops a physics-aware method for modeling multiscale dynamics with reduced data.
problem Discovering effective, lower-dimensional models for high-dimensional dynamical systems.
method Probabilistic deep neural networks incorporating physical constraints.
result The method reduces the need for extensive multiscale simulations (Small Data regime).
We introduce an approximate search algorithm for fast maximum a posteriori probability estimation in probabilistic programs, which we call Bayesian ascent Monte Carlo (BaMC). Probabilistic programs represent probabilistic models with varying number of mutually dependent finite, countable, and continuous random variable…
A new method for efficient inference in probabilistic programs with mixed support.
problem Challenges in inference for programs with both continuous and discrete latent variables.
method Stochastic gradient Markov Chain Monte Carlo algorithms.
result Outperforms existing composing inference baselines and works almost as well as inference in marginalized versions.
Model detects patterns in noisy binary data, explaining neuron activity in terms of cell assemblies.
problem Detecting structure in noisy or approximate repeats of patterns in sparse binary data.
method Probabilistic binary latent variable model based on Noisy-OR model, inferring sparse activity in latent variables.
result Model successfully extracts and explains latent structure in spiking neural data.
Paper introduces MPPGA for integrating multiple PGA models on Riemannian manifolds.
problem Challenges in dimensionality reduction on Riemannian manifolds with multiple modalities.
method Develops a mixture probabilistic principal geodesic analysis (MPPGA) model.
result Demonstrates improved clustering and shape analysis using MPPGA.
This paper is concerned with a lesser-studied problem in the context of model-based, uncertainty quantification (UQ), that of optimization/design/control under uncertainty. The solution of such problems is hindered not only by the usual difficulties encountered in UQ tasks (e.g. the high computational cost of each forw…
Probabilistic numerics expands numerical tasks with black box methods.
problem Difficult conditioning of random variables in numerical tasks.
method Construct probabilistic numerical methods based on final outputs, extrapolating limiting quantities.
result Higher orders of convergence achieved in various numerical tasks.
GP+ is a Python library for Gaussian process learning.
problem Probabilistic learning and inference for complex models.
method Integration of nonlinear manifold learning with Gaussian processes.
result GP+ enables probabilistic data fusion and inverse parameter estimation.
In this paper we provide a conceptual overview of latent variable models within a probabilistic modeling framework, an overview that emphasizes the compositional nature and the interconnectedness of the seemingly disparate models commonly encountered in statistical practice.
Paper proposes semi-supervised learning with triplet Markov chains.
problem Lack of labels in training data.
method Variational Bayesian inference for semi-supervised learning.
result Derives semi-supervised algorithms for various sequential models.
Empirical study shows overparameterization benefits unsupervised learning of latent variable models.
problem Improving optimization landscape in unsupervised learning with overparameterization.
method Synthetic and semi-synthetic experiments with various models and training algorithms.
result Overparameterization significantly increases the number of ground truth latent variables recovered.
A new estimator improves training of probabilistic models with latent Gaussian variables.
problem Improving gradient estimation for models with latent Gaussian variables.
method Rao-Blackwellised Reparameterisation Gradients (R2-G2)
result R2-G2 consistently yields better performance in models with multiple applications of the reparameterisation trick.
Adaptive probabilistic PCA adapts complexity with varying subspaces.
problem Adaptive probabilistic PCA models varying complexity in data.
method Relaxed linear Gaussian model with discrete latent variables, Bayesian nonparametric approach.
result Proposes locally adaptive probabilistic PCA (A-PPCA) for varying subspaces.
Paper simplifies concentration inequalities for easier probabilistic analysis.
problem Complexity in probabilistic analysis of random variables.
method Compact notations for concentration inequalities.
result Simplified expressions for typical sizes and tails of random variables.
We consider the problem of training probabilistic conditional random fields (CRFs) in the context of a task where performance is measured using a specific loss function. While maximum likelihood is the most common approach to training CRFs, it ignores the inherent structure of the task's loss function. We describe alte…
ContextFlow++ improves generative models by conditioning on mixed-variable contexts.
problem Lack of effective methods for context conditioning in flow-based generative models.
method Proposes ContextFlow++ with additive conditioning and mixed-variable architecture.
result ContextFlow++ achieves higher performance metrics and faster training.
Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…
DDPMs can reproduce medical image context, showing interpolation between samples.
problem Understanding DDPMs' ability to learn spatial context in medical imaging.
method Used stochastic context models (SCMs) to produce training data and assess DDPMs' performance.
result DDPMs can generate contextually correct images, interpolating between samples.