Unified reinforcement learning methods using hybrid inference.
problem Combining model-based and model-free reinforcement learning approaches.
method Control as Hybrid Inference (CHI) framework.
result CHI algorithm balances model-based and model-free learning.
Kernel Bayesian inference is a principled approach to nonparametric inference in probabilistic graphical models, where probabilistic relationships between variables are learned from data in a nonparametric manner. Various algorithms of kernel Bayesian inference have been developed by combining kernelized basic probabil…
GIM learns by analogy to improve reinforcement learning efficiency.
problem Efficient reinforcement learning with limited data.
method GIM infers unknown dynamics from known dynamics based on spectral properties.
result GIM achieves lower sample complexity and computational efficiency.
EBMs improve sample efficiency and generalization in RL.
problem Improving sample efficiency and generalization in reinforcement learning.
method Developed an online algorithm to train EBMs for model-based planning, leveraging their ability to infer intermediate states.
result EBMs lead to significantly better online learning and state space planning compared to feed-forward networks.
Model-based methods and deep neural networks have both been tremendously successful paradigms in machine learning. In model-based methods, problem domain knowledge can be built into the constraints of the model, typically at the expense of difficulties during inference. In contrast, deterministic deep neural networks a…
A new Bayesian MBRL method improves performance in robotics tasks.
problem Enhancing model-based reinforcement learning with uncertainty.
method Introduces variational inference MPC and probabilistic action ensembles with trajectory sampling (PaETS).
result Consistently improves performance on challenging locomotion tasks.
VMBPO optimizes model and policy jointly using variational lower-bound.
problem Data efficiency in RL with biased simulated data.
method Formulate variational objective function, use EM, iteratively improve model and policy.
result VMBPO is more sample-efficient and robust than model-free algorithms.
Active inference implemented for high-dimensional tasks shows efficient exploration and improved sample efficiency.
problem Achieving efficient exploration and learning in complex, uncertain environments.
method Active inference framework applied to high-dimensional tasks, with Bayesian evidence maximization.
result Order of magnitude increase in sample efficiency over model-free baselines.
Two types of nonidentifiability in latent position graphs identified and characterized.
problem Identifying and characterizing nonidentifiability in latent position random graph models.
method Defined and examined subspace nonidentifiability and model-based nonidentifiability, providing examples and characterizing limits.
result Characterized the limits of model-based nonidentifiability and obtained additional limiting results for specific graph models.
Switching linear dynamics improves model-based reinforcement learning and system identification.
problem Complex and nonlinear systems can be approximated by linear dynamical systems.
method Bayesian inference, Variational Autoencoders, Concrete relaxations.
result Improved accuracy in learning dynamics from partial and high-dimensional observations.
This paper improves Bayesian inference for predictive models with limited data.
problem Effective uncertainty quantification for training predictive models with limited data.
method Entropy-regularized gradient estimators to approximate the Bayesian posterior.
result The method generates diverse samples from the posterior distribution efficiently.
Improved model-based estimation through tempered Bayes filter.
problem Improving predictive accuracy in partially-observable stochastic systems.
method Developed tempered Bayes filter combining likelihood and full posterior tempering.
result Tempered Bayes filter achieves improved predictive performance over the Bayes filter baseline.
This paper tackles distribution shift in model-based offline RL, proposing a shifts-aware reward method.
problem Distribution shift challenges model-based offline RL by distorting value estimation and policy optimization.
method The paper disentangles the problem into model bias and policy shift, proposing a shifts-aware reward through probabilistic inference.
result The proposed shifts-aware reward method effectively mitigates distribution shift and improves policy optimization.
SOLAR learns efficient representations for RL in complex image domains.
problem Efficient model-based reinforcement learning in domains with complex observations like images.
method Optimizes structured representations for inferring simple dynamics and cost models from data.
result Substantially better final performance than other model-based RL methods, more efficient than model-free RL.
Bayesian inference models failure distributions in autonomous systems.
problem Estimating the distribution of failures in complex systems.
method Bayesian inference using system dynamics rollouts and gradient computation.
result Improves sample efficiency and parameter space coverage in autonomous systems.
Synthetic data training improves neural networks for captcha breaking.
problem Training neural networks with synthetic data for improved performance.
method Connecting synthetic data training to model-based Bayesian inference.
result Demonstrated state-of-the-art performance and posterior uncertainty in captcha breaking.
Paper learns multimodal transition dynamics for RL using conditional VI.
problem Learning stochastic, multimodal transition dynamics in RL.
method Conditional variational inference (VI) for complex stochasticity.
result VI successfully predicts multimodal outcomes but ignores deterministic parts.
A model infers dynamic networks from partial observation of cascading processes.
problem Inferring evolving networks from partial observation of node and edge data.
method A novel framework based on a mixture of coupled hierarchical Dirichlet processes.
result Explicit predictive distribution over edges of the underlying network, including future edges.
NIPA aims to translate brain learning mechanisms into scalable Bayesian inference.
problem Scalable Bayesian inference for large-scale statistical machine learning problems.
method Neural-inspired algorithm combining model-based, model-free, and episodic-control modules.
result Advances Bayesian methods and facilitates their application to deep learning.
Learned factor graphs improve inference from time sequences using neural networks.
problem Inference from time sequences with limited labeled data.
method Combines model-based algorithms and data-driven ML tools for stationary time sequences.
result Learned factor graphs can accurately infer from small training sets.
Novel approach to learning models based on subjective timescales for better exploration and decision-making.
problem Learning models over multi-step timescales in environments with intermediate states.
method Developed a subjective-timescale model (STM) based on episodic memories, enabling systematic variation of temporal extent of predictions.
result STM produces more informative action-conditioned roll-outs, leading to better decision-making and exploration.
This paper builds a model to predict the long-term future in reinforcement learning.
problem Catastrophic failures due to flawed long-term predictions in reinforcement learning models.
method The authors develop a latent-variable autoregressive model using variational inference to incorporate future information.
result The model achieves higher rewards faster than baselines on various tasks and environments.
Infer-AVAE infers missing user attributes from incomplete data using a novel adversarial approach.
problem Incomplete user attributes in social networks.
method Infer-AVAE combines MLP and GNNs with adversarial training to infer missing attributes.
result Infer-AVAE outperforms baselines by 7.0% in accuracy on real-world datasets.
Bayesian model improves data-efficiency in reinforcement learning.
problem Data inefficiency in reinforcement learning.
method Bayesian approach with variational inference.
result Human-interpretable insight into reinforcement learning dynamics.
PIPPS solves deep learning's exploding gradient problem by reparameterization gradients.
problem Exploding gradients in deep learning and model-based RL.
method Develops PIPPS framework, a flexible policy search method robust to chaos-like gradients.
result PIPPS improves over reparameterization gradients by up to 10^6 times.
New method identifies latent variables in cognitive models using neural networks.
problem Inference of latent variables in complex cognitive models is limited.
method Recurrent neural networks and simulation-based inference for latent variable sequences.
result Extends neural Bayes estimation to broader classes of cognitive models.
Nonparametric mixture models based on the Dirichlet process are an elegant alternative to finite models when the number of underlying components is unknown, but inference in such models can be slow. Existing attempts to parallelize inference in such models have relied on introducing approximations, which can lead to in…
A new method for ABC reduces computational cost by intelligently choosing simulations.
problem High computational cost in ABC due to many simulations needed.
method Compute uncertainty in ABC posterior density and select next simulation point to minimise expected loss.
result The proposed method often produces more accurate approximations than common BO strategies.
Continuous-time MBRL framework tackles control systems with Bayesian ODEs.
problem Discretization of continuous-time systems in MBRL.
method Novel actor-critic method with Bayesian ODEs for state inference.
result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.
Neural methods improve likelihood-free inference with fewer simulations.
problem Inference when likelihood functions are unavailable.
method Comparing neural approaches for likelihood-free inference.
result Neural methods can achieve state-of-the-art results with fewer simulations.
Bayesian distance clustering improves robustness to kernel choice.
problem Kernel sensitivity in model-based clustering.
method Modeling pairwise distances instead of original data.
result Dramatic gains in cluster inference robustness.
Pen-and-paper exercises cover various machine learning topics.
problem None explicitly stated, focuses on learning through exercises.
method Pen-and-paper exercises on machine learning topics.
result Comprehensive coverage of machine learning concepts through exercises.
Bayesian methods improve decision-making in spatial environments.
problem Decision making in spatial environments.
method Model-based approaches using variational inference and neural networks.
result Performs comparably to specialized methods in simulated environments.
Enhances PlaNet for better planning in uncertain environments.
problem Improving deep planning networks for partially observable environments.
method Incorporates Bayesian inference to handle uncertainty in latent models and action candidates.
result Consistently improves asymptotic performance on continuous control tasks.
New methods improve inference for sparse, undirected models.
problem Inference for sparse, undirected models is challenging due to intractable partition functions.
method Persistent VI for variational inference and Fadeout for reparameterization under sparsity-inducing priors.
result Improved learning of sparse undirected graphical models in simulations and real-world problems.
The paper reclassifies RL algorithms using inference concepts.
problem To provide a unified perspective on RL algorithms.
method Using the control as inference framework, the paper classifies RL algorithms as amortised and iterative inference.
result A wide range of RL algorithms can be classified in this new manner.
Efficiently infers Gaussian process density models with Gibbs sampling and variational methods.
problem Density estimation for complex, nonparametric models.
method Augmented likelihood with latent variables, Gibbs sampling, and variational mean field approximations.
result Efficient inference for Gaussian process density models with up to thousands of data points.
We introduce stochastic variational inference for Gaussian process models. This enables the application of Gaussian process (GP) models to data sets containing millions of data points. We show how GPs can be vari- ationally decomposed to depend on a set of globally relevant inducing variables which factorize the model …
New methods improve likelihood-free inference from simulators.
problem Challenges in inverse problems due to intractable implicit densities.
method Extends traditional ABC methods and neural network surrogate models, using likelihood ratio and score.
result More sample efficient and higher-fidelity inference.
Single autoregressive model outperforms ensemble methods in offline reinforcement learning.
problem Offline reinforcement learning with limited data and model errors.
method Infer system dynamics from data and optimize policies on model rollouts, using a single autoregressive model.
result Single autoregressive model achieves better performance than ensembles on the D4RL benchmark.
AirRL uses RL to infer urban air quality from selected stations.
problem Inferring fine-grained urban air quality from limited monitoring stations.
method Reinforcement learning model with a dynamic station selector and air quality regressor.
result AirRL achieves highest performance in air quality inference experiments.
VLBM learns MDP transitions from limited data, improving OPE performance.
problem Limited coverage of state and action space in offline trajectories.
method VLBM uses variational inference with RSA and branching architecture.
result VLBM outperforms existing OPE methods on deep OPE benchmark.
Agents learn and control complex mechanical systems through shared memories.
problem Controlling multi-joint dynamical systems.
method Coupled autoregressive active inference agents using Bayesian filtering and minimizing expected free energy.
result Demonstrated learning and control of a double mass-spring-damper system.
PPI uses predictions and weighting to infer from partially labeled data.
problem Valid inference with partially labeled data.
method Combines model-based predictions with bias correction from labeled data, using Horvitz-Thompson and Hájek corrections.
result IPW-adjusted PPI with estimated propensities performs similarly to known-probability case.
OP3 models entities for better task generalization in reinforcement learning.
problem Generalizing to unseen physical tasks with combinatorial complexity.
method Object-centric perception, prediction, and planning (OP3) framework.
result OP3 outperforms oracle models and state-of-the-art video prediction models.
CYCLEGAN models are shown to be a special case of approximate Bayesian inference.
problem Learning correspondences between domains without paired data.
method Formalized as Bayesian inference in an LVM, developed a VI algorithm based on KL divergence minimization.
result CYCLEGAN models can be derived within the proposed VI framework.
Fast algorithm for analyzing huge social networks.
problem Analyzing dynamic social networks with large numbers of actors.
method Hierarchical strategy for latent space inference with spline processes and machine learning optimization.
result Can fit millions of nodes in a few minutes.
FBC clusters data fairly without needing cluster count.
problem Fairness in clustering groups of different sensitive groups.
method Developed a Bayesian model-based clustering method with a fair prior and efficient MCMC algorithm.
result Reasonably infers the number of clusters and achieves a fair utility trade-off.