EBU improves deep RL with episodic backward updates, achieving human-level performance with less data.
problem Efficiently learn from sparse and delayed rewards in reinforcement learning.
method Episodic Backward Update (EBU) algorithm that recursively propagates value updates.
result EBU achieves human-level performance in Atari games with significantly fewer samples.
Off-policy evaluation for MNAR rewards in MDPs
problem Off-policy evaluation in MDPs with MNAR rewards
method Formalizing a reward-dependent propensity model and using future states as shadow variables
result Proposed an Fitted-Q-Evaluation-style estimator that propagates recovered rewards while allowing target policies to depend on past missingness indicators
TD learning with neural networks can lead to worse solutions than Monte-Carlo methods, especially in discontinuous value functions.
problem TD learning with neural networks can propagate approximation errors, leading to worse solutions than Monte-Carlo methods.
method Investigated the issue of approximation errors in areas of sharp discontinuities of the value function being further propagated by bootstrap updates.
result Empirical and analytical evidence shows that leakage propagation occurs in TD learning with function approximation, especially in sharp discontinuities.
CRB tackles rising rewards in combinatorial online learning.
problem Rising rewards in combinatorial online learning.
method CRB framework and CRUCB algorithm.
result Empirical and theoretical validation of CRUCB's effectiveness.
A decentralized MARL algorithm for value propagation in non-linear settings.
problem Decentralized coordination in multi-agent systems with different local rewards.
method Decentralized optimization and value propagation algorithm.
result Non-asymptotic convergence rate of 1/T in nonlinear function approximation.
Proposes BSI for valid statistical inference on bandit algorithms.
problem Valid statistical inference on bandit algorithms' performance.
method Fits a simulator of the bandit environment from observed data and uses it to estimate mean reward under any policy.
result Proves asymptotically valid confidence intervals and maintains nominal coverage.
We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the trainin…
Develops neural network framework for risk-reward optimization problems.
problem Multi-period risk-reward optimization with constrained policies.
method Neural network framework with two coupled feedforward networks, parametrizing two-step policies.
result Empirical optimum converges to true optimal value as network capacity and training size increase.
Agent learns to solve unseen tasks with subtask dependencies.
problem Generalizing to unseen tasks with subtask dependencies.
method Neural subtask graph solver (NSGS) with graph reward propagation for pre-training and actor-critic finetuning.
result Agent can find near-optimal subtask execution and generalize to unseen subtask graphs.
Adversarial CBO optimizes under interventions by adversaries and non-stationarities.
problem Optimizing in the presence of adversaries and non-stationary factors.
method Formalizes CBO as ACBO, introduces CBO-MW algorithm combining online learning and causal modeling.
result First algorithm with bounded regret for ACBO, achieving superior performance in synthetic and real-world environments.
We present a new deep meta reinforcement learner, which we call Deep Episodic Value Iteration (DEVI). DEVI uses a deep neural network to learn a similarity metric for a non-parametric model-based reinforcement learning algorithm. Our model is trained end-to-end via back-propagation. Despite being trained using the mode…
IC3Net improves communication efficiency in multiagent tasks.
problem Efficient communication in multiagent tasks, especially in semi-cooperative and competitive settings.
method IC3Net uses a gating mechanism and individualized rewards to control continuous communication, improving scalability and performance.
result IC3Net yields improved performance and convergence rates compared to baselines as task scale increases.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
Proposes MCBO for causal Bayesian optimization with model learning and regret bounds.
problem Maximizing downstream variables in unknown structural models.
method Model-based causal Bayesian optimization (MCBO) that learns full system models and trades off exploration and exploitation.
result First non-asymptotic bounds for CBO and practical implementation showing superior performance.
This study improves lookahead Bayesian optimization using rollout approximation.
problem Error propagation in lookahead Bayesian optimization due to model mis-specification.
method Proves rollout's improving nature in lookahead BO and provides guidelines for choosing the rolling horizon.
result Empirical results show rollout improves over myopic and non-myopic BO algorithms.
New method trains GFlowNets from partial episodes to improve convergence and stability.
problem Improving convergence and stability of GFlowNets training.
method Introducing SubTB(λ) for GFlowNet training from partial action subsequences. result SubTB(λ) accelerates GFlowNet convergence and enables training in longer action sequences. Improves inference-time alignment for diffusion models without updating weights.
problem Aligning diffusion models without updating weights for high-reward outputs.
method Trust-Region Iterative Twisted Sequential Monte Carlo (TRI-TSMC) for variance reduction and efficiency.
result Improves primary alignment objectives on text generation tasks.
SDQL uses modular deep Q networks to efficiently learn multi-stage optimal control tasks.
problem Training complex deep reinforcement learning models for multi-stage control tasks is inefficient and unstable.
method Stacked Deep Q Learning (SDQL) with modular Q networks and backward training.
result SDQL efficiently learns optimal control policies for multi-stage tasks with high-dimensional state and action spaces.
Polynomial chaos surrogates quantify epistemic uncertainty in AI-driven scientific models.
problem Uncertainty in reward estimates hinders interpretability in sequential generative models.
method Fit polynomial chaos expansions to trained models to propagate epistemic uncertainty and quantify sensitivity.
result Interpretable decomposition of reward components driving generative decisions.
Paper closes neural-symbolic learning loop with grammar model and back-search algorithm.
problem Slow convergence in neural-symbolic learning due to error propagation issues.
method Introduces grammar model as symbolic prior and back-search algorithm for efficient error propagation.
result Significantly outperforms RL methods in performance, converging speed, and data efficiency.
Generalized belief propagation converges to optimal solutions on graphs with motifs.
problem Understanding belief propagation on loopy graphs.
method Study of generalized belief propagation on graphs with motifs.
result Generalized belief propagation converges to the global optimum of the Bethe free energy.
Belief propagation recovers backpropagation results.
problem Connection between backpropagation and belief propagation poorly understood.
method Converted backpropagation input to belief propagation input and showed results.
result Backpropagation is a special case of belief propagation.
A susceptibility propagation that is constructed by combining a belief propagation and a linear response method is used for approximate computation for Markov random fields. Herein, we formulate a new, improved susceptibility propagation by using the concept of a diagonal matching method that is based on mean-field app…
Propagates soft labels on hypergraphs using optimal transportation.
problem Semi-supervised learning on hypergraphs.
method Wasserstein barycenters and message-passing algorithm.
result Generalization error bounds for 2-Wasserstein distance.
Variational inference is a powerful concept that underlies many iterative approximation algorithms; expectation propagation, mean-field methods and belief propagations were all central themes at the school that can be perceived from this unifying framework. The lectures of Manfred Opper introduce the archetypal example…
This paper proposes an alternating back-propagation algorithm for learning the generator network model. The model is a non-linear generalization of factor analysis. In this model, the mapping from the continuous latent factors to the observed signal is parametrized by a convolutional neural network. The alternating bac…
Study reveals decurve flows in graph propagation models.
problem Limitations of traditional graph analysis and propagation mechanisms.
method Introduces Generalized Propagation Neural Networks (GPNNs) and Continuous Unified Ricci Curvature (CURC).
result Observation of decurve flow during training of graph neural networks, revealing propagation dynamics.
New back-propagation rules discovered through evolutionary methods.
problem Improving the back-propagation algorithm for faster training.
method Evolutionary approach to discover new update equations.
result Several new update equations that train faster and perform similarly at convergence.
Unified model combines feature and label propagation for semi-supervised classification.
problem Combining feature and label propagation for effective semi-supervised classification.
method Unified Message Passing Model (UniMP) using Graph Transformer and masked label prediction.
result Obtains new state-of-the-art results in Open Graph Benchmark (OGB).
Improved error correction using neural networks and belief propagation.
problem Inference in factor graphs with loops or poor approximations.
method Hybrid model combining FG-GNN and belief propagation.
result Hybrid model outperforms belief propagation in error correction tasks.
Proposes a method to propagate uncertainty in neural networks for sparse coding.
problem Uncertainty in neural networks for sparse coding.
method Representing the target vector as a spike and slab distribution at each layer, deriving gradients of normalisation constants, and using Bayesian inference.
result Designs a novel Bayesian neural network for sparse coding.
A new approach estimates propagators for trading risky assets.
problem Estimating price impact kernel from static data for optimal trading.
method Nonparametric estimation of propagator using offline reinforcement learning.
result Pessimistic trading strategy optimises execution costs under uncertainty.
This thesis investigates belief propagation's performance in graphical models with loops.
problem Belief propagation's performance and convergence guarantees in models with loops are uncertain.
method Investigates how model parameters affect belief propagation's performance, convergence, and approximation quality.
result Model parameters influence the number of fixed points, convergence properties, and approximation quality of belief propagation.
New algorithms for n-player games using a player-centered approach.
problem Learning near-perfect strategies in multiplayer games.
method Player-centered reinforcement learning with Final Adaptation RL.
result FARL is crucial for achieving near-perfect strategies in various games.
Paper proposes new neural network learning algorithms inspired by predictive coding.
problem Finding biologically plausible alternatives to back-propagation of errors.
method Error-driven Local Representation Alignment (LRA-E) and Difference Target Propagation.
result Both proposed algorithms yield stable performance and strong generalization in training deeper, highly nonlinear networks.
LNPE enhances local connections in embeddings using extended neighbor propagation.
problem Improving local connections and interactions in nonlinear dimensionality reduction.
method Inspired by GCN, LNPE extends 1-hop neighbors to n-hop neighbors in LLE.
result LNPE produces more faithful and robust embeddings with better topological and geometrical properties.
Decoupled GCN is shown to be equivalent to label propagation.
problem Improving semi-supervised node classification in graph learning.
method The paper proves the equivalence of decoupled GCN and label propagation, and proposes a new method named PTA.
result Decoupled GCN is equivalent to two-step label propagation and can automatically assign weights to pseudo-labels.
A new method for target propagation using iterative approximations converges fast and is more biologically plausible.
problem Improving target propagation methods for neural networks.
method Iterative approximate inverses and local auto-encoders.
result The method converges exponentially fast under certain conditions.
Global propagator for massless Dirac operator defined and analyzed.
problem Analyzing the massless Dirac operator on 3-manifolds.
method Constructing propagator as sum of oscillatory integrals, providing global definitions and small time expansions.
result Explicit calculation of propagators' symbols and coefficients in eigenvalue counting functions.
Belief Propagation algorithms are instruments used broadly to solve graphical model optimization and statistical inference problems. In the general case of a loopy Graphical Model, Belief Propagation is a heuristic which is quite successful in practice, even though its empirical success, typically, lacks theoretical gu…
Learn invariances in models using the marginal likelihood.
problem Generalizing well in supervised learning tasks.
method Learn invariances in model structure using the marginal likelihood.
result Demonstrated for Gaussian process models, reducing complexity of invariant models.
The article constructs Feynman propagators for normally hyperbolic operators on curved spacetimes.
problem Constructing Feynman propagators for non-scalar geometric operators on curved spacetimes.
method Global microlocalisation constructions for normally hyperbolic operators on globally hyperbolic spacetimes.
result Feynman propagators can be constructed to satisfy a positivity property for selfadjoint normally hyperbolic operators.
The paper proposes a scalable framework for uncertainty quantification and propagation in surrogate-based Bayesian inference.
problem Uncertainty in surrogate models and its impact on inference and decision-making.
method Bayesian inference methods for surrogate models with measurement data.
result Scalable framework for uncertainty quantification and propagation in surrogate models.
We introduce propagation kernels, a general graph-kernel framework for efficiently measuring the similarity of structured data. Propagation kernels are based on monitoring how information spreads through a set of given graphs. They leverage early-stage distributions from propagation schemes such as random walks to capt…
Improved BP algorithm outperforms loopy BP in MAP inference.
problem Limited understanding and poor performance of belief propagation in graphs with loops.
method Introduced α belief propagation, a minimization of localized α-divergence. result Significantly outperforms loopy BP in fully-connected graphs for MAP inference.
SOLBP extends efficient inference to uncertain Bayesian networks.
problem Inference in uncertain Bayesian networks with second-order probabilities.
method Extends Loopy Belief Propagation to second-order Bayesian networks.
result Generates inferences consistent with sum-product networks, more efficient and scalable.
We consider a dynamical model of distress propagation on complex networks, which we apply to the study of financial contagion in networks of banks connected to each other by direct exposures. The model that we consider is an extension of the DebtRank algorithm, recently introduced in the literature. The mechanics of di…
EKF improves DNN uncertainty quantification with less computation.
problem Uncertainty quantification in deep neural networks.
method Extended Kalman Filtering applied to DNNs.
result EKF reduces computational overhead for uncertainty quantification.