Automates infectious disease policy-making via inference in epidemiological models.
problem Improving policy-making for infectious diseases during pandemics.
method Performing inference in existing epidemiological models using a probabilistic programming language.
result Automated inference leads to better disease progression outcomes and policy prescriptions.
The automation of posterior inference in Bayesian data analysis has enabled experts and nonexperts alike to use more sophisticated models, engage in faster exploratory modeling and analysis, and ensure experimental reproducibility. However, standard automated posterior inference algorithms are not tractable at the scal…
The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming para…
EFI automates statistical inference for big data.
problem Statistical inference for model parameters based on observations.
method EFI uses stochastic gradient Markov chain Monte Carlo and sparse deep neural networks.
result EFI provides higher fidelity in parameter estimation and automates the inference process.
Automates model comparison in probabilistic programming.
problem Manual derivations for model comparison are error-prone and time-consuming.
method Message passing on a Forney-style factor graph with a custom mixture node.
result Automates Bayesian model averaging, selection, and combination.
New method speeds up inference for non-conjugate Gaussian processes.
problem Inference for non-conjugate Gaussian processes is slow and unreliable.
method Automated augmented conjugate inference method that constructs auxiliary variables to make the model conditionally conjugate.
result Our method is up to two orders of magnitude faster and more robust than existing methods.
AutoBayes automates Bayesian graph exploration for robust machine learning.
problem Learning representations invariant to nuisance variations in machine learning.
method Automated Bayesian inference framework exploring different graphical models.
result Significant performance improvement with nuisance-invariant machine learning pipelines.
PClean automates Bayesian data cleaning for specific datasets.
problem Bayesian inference for diverse and complex data cleaning.
method Domain-specific probabilistic programming language with custom models and inference.
result PClean programs outperform general-purpose PPLs in accuracy and runtime.
CausalPFN automates causal effect estimation from observational data.
problem Manual selection of causal effect estimators is time-consuming and requires domain expertise.
method CausalPFN is a transformer that learns to infer causal effects from raw observations without task-specific adjustments.
result CausalPFN achieves superior performance on various benchmarks and real-world tasks.
The proliferation of automated inference algorithms in Bayesian statistics has provided practitioners newfound access to fast, reproducible data analysis and powerful statistical models. Designing automated methods that are also both computationally scalable and theoretically sound, however, remains a significant chall…
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
Automated method bounds causal effects in discrete data.
problem Partial identification of causal effects in discrete settings.
method Polynomial programming and dual relaxation for automated bounds.
result Algorithm provides guaranteed non-sharp and ε-sharp bounds. Universal probabilistic programming systems (PPSs) provide a powerful framework for specifying rich probabilistic models. They further attempt to automate the process of drawing inferences from these models, but doing this successfully is severely hampered by the wide range of non--standard models they can express. As …
The roundworm C. elegans exhibits robust escape behavior in response to rapidly rising temperature. The behavior lasts for a few seconds, shows history dependence, involves both sensory and motor systems, and is too complicated to model mechanistically using currently available knowledge. Instead we model the process p…
Automates VI divergence selection for efficient few-shot learning.
problem Efficiently selecting divergence measures for VI to improve performance.
method Meta-learning algorithm to learn optimal divergence metric and variational parameter initialization.
result Meta-learning approach outperforms standard VI methods across various tasks.
Adversarial validation detects concept drift in user targeting systems.
problem Concept drift in user targeting automation systems deteriorates model performance over time.
method Adversarial validation approach to detect and adapt to concept drift.
result Adversarial validation effectively addresses concept drift in user targeting systems.
Improves reliability of BBVI optimization methods.
problem Reliability issues and expertise required for BBVI optimization.
method RABVI framework with automated learning rate adjustment and KL divergence estimation.
result RABVI detects inaccurate variational approximations and optimizes reliability.
We develop an automated variational inference method for Bayesian structured prediction problems with Gaussian process (GP) priors and linear-chain likelihoods. Our approach does not need to know the details of the structured likelihood model and can scale up to a large number of observations. Furthermore, we show that…
Optimal algorithm selects biological models without prior info.
problem Determining the correct biological model without prior knowledge.
method Systems biology models and likelihood-free inference.
result Improved model selection performance over conventional methods.
We present a new algorithm for approximate inference in probabilistic programs, based on a stochastic gradient for variational programs. This method is efficient without restrictions on the probabilistic program; it is particularly practical for distributions which are not analytically tractable, including highly struc…
Classical stochastic gradient methods for optimization rely on noisy gradient approximations that become progressively less accurate as iterates approach a solution. The large noise and small signal in the resulting gradients makes it difficult to use them for adaptive stepsize selection and automatic stopping. We prop…
We use SMC with twist functions to improve probabilistic inference in LLMs.
problem Improving probabilistic inference in large language models.
method We use Sequential Monte Carlo with learned twist functions to estimate expected future values and focus inference on promising sequences.
result Twisted SMC improves the accuracy of language model inference and evaluation.
Automates debiasing for large language model evaluations through Fisher random walk.
problem Rigorous and scalable evaluation of large language models.
method Semiparametric efficient estimator using Fisher random walk for weighted residual balancing.
result Efficient estimation of contextual preference scores for large language models.
This paper optimizes liquidity provision in automated market makers using auction theory.
problem Optimizing profit for a monopolist liquidity provider in automated market makers.
method Introduces a Bayesian-like belief inference framework to model AMMs, characterizes profit-maximizing strategies using Myerson's optimal auction theory.
result Characterizes the optimal demand curve and payments for an IC AMM, revealing a bid-ask spread caused by asymmetry and monopoly pricing.
Automates feature extraction from JSON data for machine learning.
problem Manual feature engineering for JSON data is laborious, lossy, and prone to bias.
method Automates feature extraction using Mill.jl and JsonGrinder.jl.
result Creates a differentiable machine learning model from raw JSON samples.
PySR method automates discovering equations from data in chaotic dynamics and epidemics.
problem Discovering equations from complex data in dynamical systems.
method Symbolic regression methods, focusing on PySR.
result PySR method efficiently infers equations from chaotic dynamics and epidemic models, matching original forms.
TripDecoder recovers metro routes and travel times from smart card data.
problem Recover unknown routes and travel times in metro systems.
method Decouples two inference tasks: travel time and route preference.
result TripDecoder improves accuracy and efficiency compared to competitors.
This work offers a broad perspective on probabilistic modeling and inference in light of recent advances in probabilistic programming, in which models are formally expressed in Turing-complete programming languages. We consider a typical workflow and how probabilistic programming languages can help to automate this wor…
Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …
The segmentation of the left ventricle (LV) from CINE MRI images is essential to infer important clinical parameters. Typically, machine learning algorithms for automated LV segmentation use annotated contours from only two cardiac phases, diastole, and systole. In this work, we present an analysis work-flow for fully-…
New method for finite approximations improves inference efficiency and applicability.
problem Challenges in Bayesian nonparametric inference due to infinite dimensionality.
method Automated Independent Finite Approximation (AIFA) for finite-dimensional approximations of CRMs and NCRMs.
result AIFA provides more efficient and straightforward derivations and parallel computing compared to Truncated Finite Approximations (TFAs).
Automated synthesis of inductive invariants is an important problem in software verification. Once all the invariants have been specified, software verification reduces to checking of verification conditions. Although static analyses to infer invariants have been studied for over forty years, recent years have seen a f…
DeepCap automates coronary artery segmentation from IVOCT images.
problem Automated segmentation of coronary arteries from IVOCT images is challenging.
method Developed a deep learning method based on capsules for robust, unbiased segmentation.
result DeepCap achieves segmentation quality comparable to state-of-the-art methods.
DMVI uses diffusion models for efficient probabilistic inference in PPLs.
problem Efficient probabilistic inference in complex probabilistic programming languages.
method DMVI employs diffusion models as variational approximations to the posterior distribution, optimizing a bound on the marginal likelihood.
result DMVI produces more accurate posterior inferences than existing methods in PPLs with similar computational cost and less manual tuning.
PGMax automates PGM inference on GPUs, improving quality and speed.
problem Efficient inference in complex discrete PGMs.
method Factor graph specification and loopy belief propagation in JAX.
result Higher-quality inference with up to 3x speedups.
Automates learning of multivariate diffusions for generative models.
problem Lack of automated methods for choosing and optimizing diffusion processes in generative models.
method Develops a recipe to maximize likelihood without model-specific analysis, parameterizes diffusion for target noise, and optimizes the inference diffusion process.
result Automatic search over all linear diffusions for generative models.
Lean Copilot uses LLMs to assist theorem proving in Lean, improving efficiency and automation.
problem Challenges in using existing neural theorem provers to prove novel theorems autonomously.
method Introduces Lean Copilot, a framework for integrating LLMs into Lean's theorem proving process.
result Lean Copilot automates 74.2% of proof steps on average, significantly improving over existing methods.
Automated tool reduces FPGA inference latency to 5 μs for deep neural networks.
problem Deploying ultra low-latency, low-power deep neural networks on FPGAs.
method Extending hls4ml library, using model compression techniques like pruning and quantization-aware training.
result Achieved inference latency of 5 μs with 97% resource reduction.
A theoretical framework that supports automated construction of dynamic prime models purely from experimental time series data has been invented and developed, which can automatically generate (construct) data-driven models of any time series data in seconds. This has resulted in the formulation and formalisation of ne…
With the advent of automated machine learning, automated hyperparameter optimization methods are by now routinely used in data mining. However, this progress is not yet matched by equal progress on automatic analyses that yield information beyond performance-optimizing hyperparameter settings. In this work, we aim to a…
Computer vision model automates residual plot assessment for diagnosing model assumptions.
problem Automating residual plot assessment for model diagnostics.
method Trains a computer vision model to predict disparity between residual distributions and reference distributions using Kullback-Leibler divergence.
result Computer vision model is less sensitive to non-linearity but more sensitive than human judgment and conventional tests.
Tree-AMP simplifies inference in complex tree-structured models.
problem Inference in high-dimensional tree-structured models.
method Approximate Message Passing algorithms for various machine learning tasks.
result Theoretical performance predictions and automated entropy estimation.
Probabilistic programming has emerged as a powerful paradigm in statistics, applied science, and machine learning: by decoupling modelling from inference, it promises to allow modellers to directly reason about the processes generating data. However, the performance of inference algorithms can be dramatically affected …
Two approaches scale up DNN optimization for diverse edge devices.
problem Optimizing DNNs for edge devices with varying performance requirements.
method Reuse performance predictors on proxy devices and build scalable predictors.
result Optimized DNN designs for many different edge devices without lengthy optimization.
We utilize copulas to constitute a unified framework for constructing and optimizing variational proposals in hierarchical Bayesian models. For models with continuous and non-Gaussian hidden variables, we propose a semiparametric and automated variational Gaussian copula approach, in which the parametric Gaussian copul…
New RL approach infers optimal policies via variational inference.
problem Manual design of reward functions for reinforcement learning.
method Variational inference for inferring policies achieving desired outcomes.
result Eliminates need for hand-crafted reward functions for diverse tasks.
Automated PDE discovery from multiple noisy experiments.
problem Inherent variability in experiments makes single experiment inference unreliable.
method Randomised adaptive group Lasso sparsity estimator in deep learning framework.
result More generalizable PDEs found from multiple datasets.
We propose to learn a kernel-based message operator which takes as input all expectation propagation (EP) incoming messages to a factor node and produces an outgoing message. In ordinary EP, computing an outgoing message involves estimating a multivariate integral which may not have an analytic expression. Learning suc…