NP-iMCMC algorithm for nonparametric models in universal PPLs.
problem Developing inference algorithms for arbitrary nonparametric models in universal PPLs.
method Unifying involutive MCMC framework with a general procedure for state movement.
result Proves the correctness of the NP-iMCMC sampler and shows significant performance improvements.
Probabilistic programming languages (PPLs) are powerful modelling tools which allow to formalise our knowledge about the world and reason about its inherent uncertainty. Inference methods used in PPL can be computationally costly due to significant time burden and/or storage requirements; or they can lack theoretical g…
NP-HMC extends HMC for nonparametric models in probabilistic programming.
problem Inference for nonparametric models in probabilistic programming.
method Introduces NP-HMC, a generalization of HMC for nonparametric models using tree representable functions.
result Empirically shows significant performance improvements over existing approaches.
PPL improves on Takacs-Fiksel estimation for Gibbs models.
problem Improving point process estimation methods.
method PPL uses cross-validation and a specific loss function to estimate parameters.
result PPL with specific loss functions and hyperparameters outperforms Takacs-Fiksel estimation in mean square error.
LIC compiles probabilistic models to generate efficient MCMC proposals.
problem Creating accurate Metropolis-Hastings proposals for Bayesian inference.
method Integrates probabilistic graphical models and neural networks in an open-source framework to optimize proposal distributions.
result LIC produces more efficient and robust MCMC proposals compared to existing methods.
DMVI uses diffusion models for efficient probabilistic inference in PPLs.
problem Efficient probabilistic inference in complex probabilistic programming languages.
method DMVI employs diffusion models as variational approximations to the posterior distribution, optimizing a bound on the marginal likelihood.
result DMVI produces more accurate posterior inferences than existing methods in PPLs with similar computational cost and less manual tuning.
Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the …
Optimizing an expensive-to-query function is a common task in science and engineering, where it is beneficial to keep the number of queries to a minimum. A popular strategy is Bayesian optimization (BO), which leverages probabilistic models for this task. Most BO today uses Gaussian processes (GPs), or a few other surr…
PClean automates Bayesian data cleaning for specific datasets.
problem Bayesian inference for diverse and complex data cleaning.
method Domain-specific probabilistic programming language with custom models and inference.
result PClean programs outperform general-purpose PPLs in accuracy and runtime.
Study uses PPLs to model and forecast COVID-19 spread and policy interventions.
problem Forecast and control the spread of COVID-19 using policy interventions.
method Compartmental models, probabilistic programming, inference, greedy algorithm.
result Optimal series of policy interventions can control the infected population.
Probabilistic programming languages (PPLs) are a powerful modeling tool, able to represent any computable probability distribution. Unfortunately, probabilistic program inference is often intractable, and existing PPLs mostly rely on expensive, approximate sampling-based methods. To alleviate this problem, one could tr…
Improves sampling from complex hierarchical models using HMC and automatic marginalization.
problem Sampling from complex hierarchical models is difficult for HMC.
method Proposes automatic marginalization as part of the sampling process using HMC in a graphical model extracted from a PPL.
result Significantly improves sampling from real-world hierarchical models.
Improves Bayesian optimisation for engineering design problems with many variables.
problem Efficiently searching for global minima in high-dimensional design spaces.
method Integrates input and output data to identify a reduced latent subspace using probabilistic partial least squares.
result Significant improvements in convergence to the global minimum compared to existing methods.
We consider probabilistic programming for birth-death models of evolution and introduce a new widely-applicable inference method that combines an extension of the alive particle filter (APF) with automatic Rao-Blackwellization via delayed sampling. Birth-death models of evolution are an important family of phylogenetic…
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs). However, the fact that HMC uses derivative information causes complications when the target distribution is non-differentiable with respect…
We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables. The key success of this language and its compilation scheme is in its ability to automatically distinguish parameters the density functio…
Type system captures CI relationships for probabilistic models.
problem Challenges in inference for models with mixed discrete and continuous parameters.
method Information flow type system for probabilistic programming.
result Well-typed programs guarantee certain CI relationships.
Deriving conditional and marginal distributions using conjugacy relationships can be time consuming and error prone. In this paper, we propose a strategy for automating such derivations. Unlike previous systems which focus on relationships between pairs of random variables, our system (which we call Autoconj) operates …
A new framework for PPLS combines noise estimation, optimization, and calibration.
problem Probabilistic PLS models need interpretable latent factors and calibrated uncertainty.
method End-to-end pipeline combining noise estimation, constrained optimization, and prediction calibration.
result Achieves near-nominal coverage and native calibrated uncertainty across benchmarks.
We present the Compressive Transformer, an attentive sequence model which compresses past memories for long-range sequence learning. We find the Compressive Transformer obtains state-of-the-art language modelling results in the WikiText-103 and Enwik8 benchmarks, achieving 17.1 ppl and 0.97 bpc respectively. We also fi…
This book is a graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, but also introduces the techniques needed to design and build these systems. It is aimed at people who have an undergraduate-level understandi…
New algorithm learns policies without uniform overlap assumption.
problem Learning optimal policies from non-uniformly collected data.
method Pessimistic Policy Learning (PPL) using lower confidence bounds.
result Efficient policy learning for adaptively collected data.
dynestyx: A library for probabilistic programming of dynamical systems
problem integrating state-space models into probabilistic programming languages
method a unified interface for specifying priors and performing inference
result principled uncertainty quantification for state and parameters
The front-end factor analysis (FEFA), an extension of principal component analysis (PPCA) tailored to be used with Gaussian mixture models (GMMs), is currently the prevalent approach to extract compact utterance-level features (i-vectors) for automatic speaker verification (ASV) systems. Little research has been conduc…
ProbDR framework interprets DR algorithms as probabilistic inference.
problem Efficiently compressing high-dimensional data into lower dimensions.
method ProbDR variational framework that treats DR as probabilistic inference.
result ProbDR unifies various DR algorithms and enables probabilistic reasoning.
Novel approach to universal online learning for bounded losses, closing open problems.
problem Characterizing processes for universal online learning under non-i.i.d. conditions.
method Characterization of processes admitting strong and weak universal learning, introduction of optimistically universal learning rule.
result Introduction of a novel 1NN algorithm that is optimistically universal for bounded losses.
Unified continuous diffusion model outperforms discrete alternatives in scalability and quality.
problem Continuous diffusion models were perceived as less scalable than discrete models.
method Reconstructed Plaid model and compared it with modern discrete DLMs, optimizing noise schedule and embeddings via likelihood.
result Unified continuous diffusion model (RePlaid) outperforms discrete models in compute efficiency and quality.
Universal Gaussian parity proven for 2D knots.
problem Proving universal Gaussian parity for 2D knots.
method Analyzing Gaussian parity on free 2D knots.
result Gaussian parity is universal for 2D knots.
GCNNs gain rotation invariance with more training augmentation, making SVD-Universal more effective.
problem Improving robustness of GCNNs to adversarial attacks.
method SVD-Universal technique applied to GCNNs trained with larger rotations.
result SVD-Universal becomes more effective as GCNNs gain rotation invariance.
The study of universal links in 3-manifolds and their properties.
problem Existence and characterization of universal links in 3-manifolds.
method Analyzing branched coverings and distinguishing between universal and complement universal links.
result Closed spherical 3-manifolds are the only ones admitting universal links.
Universal MLPs with a single hidden layer can learn any function.
problem Learning on various data structures like sequences, images, sets, and graphs.
method Using group theory, the paper proves the universality of a broad class of equivariant MLPs with a single hidden layer.
result Having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance).
Constructs universal link invariants from intersections in configuration spaces.
problem Globalise topologically all coloured Jones polynomials and ADO polynomials.
method Defines new link invariants from graded intersections in configuration spaces.
result Recover all coloured Jones polynomials and ADO polynomials for links.
We construct universal Lefschetz fibrations, defined in analogy with classical universal bundles. We also introduce the cobordism groups of Lefschetz fibrations, and we see how these groups are quotients of the singular bordism groups via the universal Lefschetz fibrations.
We present a universal knot polynomials for 2- and 3-strand torus knots in adjoint representation, by universalization of appropriate Rosso-Jones formula. According to universality, these polynomials coincide with adjoined colored HOMFLY and Kauffman polynomials at SL and SO/Sp lines on Vogel's plane, and give their ex…
Revives Vogel's diagrammatic technique for universal Lie algebra computations.
problem The universality of Lie algebra quantities remains open, despite many being described.
method Diagrammatic algebra based on Vogel's Λ-algebra.
result Diagrammatic technique enables truly universal computations in Lie theory.
We prove Runge-type theorems and universality results for locally univalent holomorphic and meromorphic functions. Refining a result of M. Heins, we also show that there is a universal bounded locally univalent function on the unit disk. These results are used to prove that on any hyperbolic simply connected plane doma…
Simple technique turns any adversarial attack into a universal one using few test examples.
problem Creating universal adversarial attacks with minimal data.
method Universalization technique using few adversarial test examples and spectral properties.
result Simple universalization technique achieves comparable fooling rates to state-of-the-art methods.
Universal inequalities found for Laplacian eigenvalues on convex domains.
problem Finding bounds for Laplacian eigenvalues on convex domains.
method Established two universal inequalities.
result Found new bounds for Laplacian eigenvalues.
Universal inequalities for Laplacian eigenvalues on discrete groups.
problem Proving inequalities for Laplacian eigenvalues on discrete groups.
method Analyzing Laplacian eigenvalues with Dirichlet boundary conditions on subsets of discrete groups.
result Yang-type universal inequalities for Cayley graphs of amenable groups and the d-regular tree.
New maps connect universal circles to ideal sphere for hyperbolic manifolds.
problem Understanding universal circles for Anosov foliations with branching.
method Introduced a new type of Cannon--Thurston map for leftmost universal circles.
result Fundamental group acts on leftmost universal circle with pseudo-Anosov dynamics.
This paper studies universal rates of ERM for binary classification under agnostic learning.
problem The challenge of achieving universal rates of ERM for binary classification under agnostic learning.
method The paper explores the agnostic universal rates of ERM for binary classification, revealing three possible rates: e−n, o(n−1/2), or arbitrarily slow. result The paper provides a complete characterization of which concept classes fall into each of the three categories of agnostic universal rates.
Develops a new approach to establish universality for any-dimensional machine learning models.
problem Understanding universality for models with inputs of varying sizes.
method Identifies any-dimensional functions with a unique function in an infinite-dimensional limit space, using symmetries and relations between inputs of different sizes.
result Establishes universality for several existing architectures and proposes modifications to restore it.
Softmax attention approximates complex functions and subsumes many known universal approximators.
problem Universal approximation of continuous sequence-to-sequence functions.
method Interpolation-based analysis of attention's internal mechanism, showing its ability to approximate ReLU functions.
result Softmax attention is a universal approximator for continuous sequence-to-sequence functions.
MLPs can approximate any function in context, challenging the importance of in-context universality.
problem Understanding why transformers are more effective than classical models.
method Proved MLPs with trainable activation functions are universal in context.
result Transformer success is likely due to factors other than in-context universality.
Solves open problem on universally consistent online learning with unbounded losses.
problem Open problem on universally consistent online learning with unbounded losses.
method Constructs random measurable partitions of the instance space.
result Simple memorization rule is optimistically universal for any unbounded loss.
Proves universal pairing result for 2-complexes, showing lack of positivity.
problem Detecting equivalence relations in 2-complexes.
method Analogous to Freedman et al. for manifolds, but for 2-complexes.
result Universal pairing does not detect simple homotopy equivalence vs 3-deformations for 2-complexes.
The paper explores universal circles for Anosov foliations and their uniqueness.
problem Exploring the uniqueness of universal circles for Anosov foliations.
method Using the flow space of an Anosov flow to parameterize the circle bundle at infinity of the foliations.
result Several constructions of a universal circle are typically distinct and not conjugate.
The paper tightens bounds on distances between Reeb graphs.
problem Certifying quasi-universality of distances between Reeb graphs.
method Establishes tight bi-Lipschitz bounds for various distances.
result Proves strict universality of the functional contortion distance for contour trees and coincides with interleaving distance for merge trees.