arXiv research

IDAC improves reinforcement learning efficiency by modeling implicit distributions.

Continuous semi-implicit models enable faster training and better performance in generative modeling.

Kernel semi-implicit variational inference improves variational inference without additional optimization.

Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.

Kernel SIVI improves variational inference by avoiding lower-level optimization.

Enhances graph modeling with hyperbolic geometry and variational inference.

We extend the existing framework of semi-implicit variational inference (SIVI) and introduce doubly semi-implicit variational inference (DSIVI), a way to perform variational inference and learning when both the approximate posterior and the prior distribution are semi-implicit. In other words, DSIVI performs inference …

Semi-implicit graph variational auto-encoder (SIG-VAE) is proposed to expand the flexibility of variational graph auto-encoders (VGAE) to model graph data. SIG-VAE employs a hierarchical variational framework to enable neighboring node sharing for better generative modeling of graph dependency structure, together with …

Stochastic recurrent neural networks with latent random variables of complex dependency structures have shown to be more successful in modeling sequential data than deterministic deep models. However, the majority of existing methods have limited expressive power due to the Gaussian assumption of latent variables. In t…

Unified theory for semi-implicit variational inference, bridging approximation and optimization.

Improved phylogenetic tree reconstruction using flexible branch length distributions.

To combine explicit and implicit generative models, we introduce semi-implicit generator (SIG) as a flexible hierarchical model that can be trained in the maximum likelihood framework. Both theoretically and experimentally, we demonstrate that SIG can generate high quality samples especially when dealing with multi-mod…

Neural network has attracted great attention for a long time and many researchers are devoted to improve the effectiveness of neural network training algorithms. Though stochastic gradient descent (SGD) and other explicit gradient-based methods are widely adopted, there are still many challenges such as gradient vanish…

PVI improves SIVI by directly optimizing ELBO without parametric assumptions.

Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution can assume any density function, explicit or not, as long as independent random samples can be generat…

Improved UIVI method shows better performance than state-of-the-art SIVI methods.

In this work, we propose learnable Bernoulli dropout (LBD), a new model-agnostic dropout scheme that considers the dropout rates as parameters jointly optimized with other model parameters. By probabilistic modeling of Bernoulli dropout, our method enables more robust prediction and uncertainty quantification in deep m…

Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by …

SGRNN models evolving graph data for better property prediction.

Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this pa…

SIFG uses noisy particles to efficiently sample from complex distributions.

The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.

Single-timescale actor-critic finds globally optimal policy.

New PAC-Bayesian approach stabilizes actor-critic learning.

Actor-critic methods can achieve incredible performance on difficult reinforcement learning problems, but they are also prone to instability. This is partly due to the interaction between the actor and critic during learning, e.g., an inaccurate step taken by one of them might adversely affect the other and destabilize…

FORK improves model-free reinforcement learning performance.

In science and especially in economics, agent-based modeling has become a widely used modeling approach. These models are often formulated as a large system of difference equations. In this study, we discuss two aspects, numerical modeling and the probabilistic description for two agent-based computational economic mar…

Smaller actor-critic models lead to performance degradation and overfitting, highlighting the critic's role in value underestimation.

New analysis shows actor-critic method converges efficiently in practical settings.

This work analyzes actor-critic methods for faster convergence.

Despite the empirical success of the actor-critic algorithm, its theoretical understanding lags behind. In a broader context, actor-critic can be viewed as an online alternating update algorithm for bilevel optimization, whose convergence is known to be fragile. To understand the instability of actor-critic, we focus o…

Actor-critic algorithms converge to an ODE as data samples change dynamically.

AGAC uses an adversary to enhance exploration in complex tasks.

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In th…

Neural networks trained with actor-critic algorithms converge to ODEs under weak convergence analysis.

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

Traffic actors' future motion predicted using a hybrid graph model.

In this paper we propose a hybrid architecture of actor-critic algorithms for reinforcement learning in parameterized action space, which consists of multiple parallel sub-actor networks to decompose the structured action space into simpler action spaces along with a critic network to guide the training of all sub-acto…

Proposes a new model for predicting future motion of road actors in autonomous vehicles.

Actor-Critic method achieves optimal regret for unichain MDPs.

TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution

Proposes a value-based method for continuous control without an actor.

Optimistic actor-critic tackles linear MDPs with parametric policies.

Paper introduces a meta-critic for accelerating off-policy actor-critic learning.

ESAC improves reinforcement learning by lookahead and intuition.

We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return. Our critic uses linear function approximation, and we extend the concept of compatible features to the variance-adjusted setting. We present an episodic actor-critic algorithm and show that it converges almost su…

We report a data mining pipeline and subsequent analysis to understand the core periphery power structure created in three national newspapers in Bangladesh, as depicted by statements made by people appearing in news. Statements made by one actor about another actor can be considered a form of public conversation. Name…

WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.