Bayesian model for multi-environment prediction with latent variable changes.
problem Prediction in environments with changing latent variable distributions.
method Bayesian model with empirical Bayes prior and amortized variational algorithm.
result Method outperforms previous approaches in new environments.
The paper explores how agents can generalize to new environments with unseen variables.
problem Generalizing to new environments with unseen variables.
method Investigates and proposes a method for efficient re-use of past marginal information to achieve out-of-variable generalization.
result The residual distribution after fitting a classifier reveals partial derivatives of the true generating function with respect to unobserved causal parents.
FPO optimizes policies for robust reinforcement learning by adjusting environment variables.
problem Slow learning or suboptimal policies due to unobservable environment variables.
method FPO uses Bayesian optimization to select optimal environment variable distributions.
result FPO efficiently learns robust policies for rare events.
Deep RL agents vary significantly in Atari environments.
problem Challenges in reproducibility due to stochasticity.
method Experiments with OpenAI Baselines agents.
result Variability in agent performance is significant and underreported.
Paper proposes EILLS for invariant linear regression across environments.
problem Estimating true parameter and important variable set in multi-environment settings.
method Environment invariant linear least squares (EILLS) objective function.
result EILLS estimator achieves variable selection consistency and efficient estimation.
Jigsaw-VAE tackles feature imbalance in VAE latent variables, improving generalization across environments.
problem Feature imbalance in VAE latent variables leads to poor generalization and biased sample generation.
method Proposes a regularization scheme to balance features in VAE latent variables and introduces a metric to measure balance.
result The regularization scheme substantially addresses feature imbalance, leading to improved generalization and diverse sample generation.
FAIR-NN finds invariant variables for causal inference across diverse environments.
problem Nonparametric invariance and causal learning in regression models with varying joint distributions.
method FAIR-NN framework using adversarial optimization and neural networks.
result FAIR-NN identifies invariant variables and quasi-causal variables under minimal conditions.
Meta-learners estimate CATE from multiple environments with partial identification.
problem Estimating CATE from observational data across multiple environments with violations of causal assumptions.
method Adapt IV literature for partial identification, propose model-agnostic meta-learners.
result Meta-learners effectively estimate CATE bounds across various experiments.
Tree-based regularization improves latent variable inference from related datasets.
problem Inferring latent variables from multiple related datasets in causal systems.
method Tree-Based Regularization (TBR) for sparse changes across environments.
result TBR identifies true latent variables up to simple transformations under sparse changes.
Adapts to new environments in robotics using latent variable models.
problem Learning dynamics in robotic environments with subtle variations.
method Variational inference for latent representation, online Bayesian inference, neural network ensemble.
result Positive transfer during training and online adaptation on HalfCheetah task.
Study identifies latent variables and causal relationships from multiple environments.
problem Identify latent variables and causal relationships from multiple environments.
method Proposes algorithm LiNGCReL for identifying causal graph up to surrounded-node ambiguity.
result Identifies latent variables up to surrounded-node ambiguity (SNA) in linear causal models.
New RL environments help AI learn causal relationships from visual data.
problem Learning causal relationships from visual data for AI agents.
method Designing benchmark RL environments and evaluating representation learning algorithms.
result Explicitly incorporating structure and modularity improves causal induction in model-based RL.
Paper proposes a deep RL method for hedging variable annuities, outperforming misspecified models.
problem Model miscalibration in variable annuity contracts with GMMB and GMDB riders.
method Two-phase deep reinforcement learning approach: training phase in a controlled environment, online learning phase in real market.
result Trained reinforcement learning agent hedges equally well as correct Delta in training phase and outperforms misspecified Deltas.
Study on estimating causal effects with limited data and multiple environments.
problem Estimating causal effects under hidden confounding with unpaired data and sparse effects.
method Instrumental variable (IV) regression with cross-fold sample splitting and ℓ1-regularized estimation. result Proposed GMM-type estimator is consistent as the number of environments grows.
New method identifies causal variables from multi-node interventions, expanding on previous single-node approaches.
problem Inferring high-level causal variables from low-level observations under multiple interventions.
method Exploits variance trace of ground truth causal variables and regularizes for sparsity.
result First identifiability result for causal representation learning with multiple node interventions.
We study causal inference in a multi-environment setting, in which the functional relations for producing the variables from their direct causes remain the same across environments, while the distribution of exogenous noises may vary. We introduce the idea of using the invariance of the functional relations of the vari…
New method recovers causal DAGs from general environments without strict assumptions.
problem Recovering causal DAGs from real-world data with varying distributions.
method Formalizes desiderata for causal representation learning in general environments, leveraging sufficient change conditions up to third-order derivatives.
result Fully recovers latent DAG and identifies latent variables up to minor indeterminacies under nonparametric mixing.
Method adapts policies for new environments efficiently.
problem Difficulties in transferring reinforcement learning policies to new environments.
method Variational Policy Embedding (VPE) learns latent variables and a master policy.
result Policies can quickly adapt to new environments in latent space.
Optimizes wireless systems using deep learning without supervision.
problem Optimizing resource allocation and transceivers in wireless networks.
method Introduces unsupervised and reinforced-unsupervised learning frameworks for variable and functional optimization problems.
result Demonstrates the effectiveness of the learning frameworks through a user association problem.
SADCBO optimizes contextual variables by balancing relevance and cost.
problem Optimizing contextual variables with varying costs and unknown relevance.
method Adaptive selection of relevant contextual variables using sensitivity analysis and early stopping.
result Consistent improvement in optimization across various examples.
New approach handles stochastic and partially-observable environments using discrete autoencoders and Monte Carlo tree search.
problem Challenges in planning for stochastic and partially-observable environments.
method Uses discrete autoencoders and a stochastic variant of Monte Carlo tree search.
result Significantly outperforms MuZero on stochastic chess and scales to DeepMind Lab.
Method identifies unknown intervention targets in structural causal models from diverse data.
problem Identifying unknown intervention targets in structural causal models from heterogeneous data.
method Two-phase approach: first recovers exogenous noises, second matches with endogenous variables.
result Proposed method uniquely identifies intervention targets under causal sufficiency assumption.
Bayesian optimisation has been successfully applied to a variety of reinforcement learning problems. However, the traditional approach for learning optimal policies in simulators does not utilise the opportunity to improve learning by adjusting certain environment variables: state features that are unobservable and ran…
Develops RL algorithm for lifelong non-stationary environments.
problem Challenges of reinforcement learning in environments with persistent change.
method Formalizes lifelong non-stationarity, uses latent variable models, and leverages online learning and probabilistic inference.
result Substantial improvement in performance over non-reasoning approaches in lifelong non-stationary environments.
The paper proposes a proactive method to improve model reliability by removing unstable relationships in the training data.
problem Improper generalization of predictive models due to dataset shift.
method Proactively removing unstable relationships using causal mechanisms and latent counterfactual variables.
result Models that remove vulnerable variables and use estimates of latent variables transfer better, often outperforming in the target domain.
Optimally explores dynamical systems with varying properties using context inference.
problem Learning dynamics models for systems with varying properties.
method Formulates dynamics models as stochastic processes conditioned on a latent context variable inferred from system transitions. Uses probabilistic formulation to compute optimal action sequences for exploration.
result Demonstrates effectiveness of the method on non-linear toy-problems and reinforcement learning environments.
Minimal learning agents can infer unobserved variables in complex environments.
problem How to infer unobserved variables in complex environments using minimal learning agents.
method Concrete operational definition of abstract concepts, minimal architecture supporting abstraction, reinforcement learning.
result Minimal learning agents can infer the existence of unobserved variables.
Fractal neural networks play SimCity and Conway's Game of Life on varying scales.
problem Generalizing agents' performance to larger gameboards than during training.
method Reinforcement learning in a custom environment, using fractal neural networks.
result Agents can generalize to larger gameboards, solving a minigame unsolvable with local strategies.
Paper establishes identifiability and achievability for causal representation learning.
problem Identifying and recovering latent causal models and variables from observational and interventional data.
method Establishes identifiability and achievability using uncoupled interventions and a recovery algorithm.
result Guaranteed perfect recovery of latent causal model and variables under uncoupled interventions.
Recurrent networks learn beliefs from history in partially observable environments.
problem Learning optimal policies in partially observable environments.
method Trained recurrent neural networks to approximate value functions, measuring mutual information between hidden states and beliefs.
result Recurrent networks' hidden states correlate with beliefs of relevant state variables, improving expected return.
Bayesian optimization adapted for experiments with changing environmental conditions.
problem Optimizing experiments influenced by uncontrollable environmental factors.
method Extends Bayesian optimization to handle both controllable and uncontrollable parameters, fitting a global surrogate model and optimizing only controllable parameters conditionally on measurements of uncontrollable variables.
result The proposed ENVBO algorithm finds solutions for the full domain of the environmental variable more efficiently and cost-effectively than traditional methods.
Paper proposes a semi-supervised method for detecting concept drift in streaming environments.
problem Detecting concept drift in streaming environments with limited labeled data.
method Utilizes density estimation of posterior probabilities in partially labeled streaming data.
result Demonstrates superior concept drift detection in streaming environments with limited labeled data.
Proposes method to learn state abstractions that generalize across environments.
problem Learning abstractions that generalize in block MDPs.
method Invariant causal prediction to learn model-irrelevant state abstractions (MISA).
result Proves high probability of outputting a state abstraction corresponding to causal feature set for return.
Bayesian Invariant Prediction models stable features from multi-environment data.
problem Analyzing stable features across multiple environments for better prediction and understanding.
method Developed Bayesian Invariant Prediction (BIP) model that encodes invariant feature indices as latent variables and infers them via posterior inference.
result BIP and its variational approximation (VI-BIP) outperform existing methods in accuracy and scalability for invariant prediction.
WILD-SCAV benchmarks AI in complex 3D FPS environments.
problem Lack of complexity and diversity in RL environments.
method Developed a 3D open-world FPS game environment.
result Demonstrates effectiveness in benchmarking RL algorithms.
PDD detects concept drift using explainable AI, improving model performance in dynamic environments.
problem Detecting and adapting to concept drift in predictive models.
method Profile Drift Detection (PDD) using Partial Dependence Profiles (PDPs).
result PDD outperforms existing methods in detecting concept drift and maintaining high predictive performance.
A game environment simulates competition among many agents for resources.
problem Understanding large-scale multiagent interactions and resource competition.
method Developed a persistent, massively multiplayer AI environment.
result Population size affects the development of skillful behaviors and niche differentiation.
New approach predicts under latent shifts using high-dimensional images.
problem Prediction under latent subgroup shifts with high-dimensional observations.
method Recognition-parametrised model (RPM) for identifying causal latent structure.
result Successfully adapts predictions for high-dimensional image data.
New benchmark tests machine learning's ability to learn causal overhypotheses.
problem Machine learning's difficulty in understanding causal overhypotheses.
method Adapted blicket detector environment for machine learning agents to test causal overhypotheses.
result Many state-of-the-art methods struggle with causal overhypotheses in the new benchmark.
Fitted Q iteration improves algorithmic trading by addressing dimensionality issues and data scarcity.
problem Dimensionality issues and data scarcity in algorithmic trading.
method Fitted Q iteration combined with model fitting and data simulation.
result The method performs well in both simulated and real-world environments.
New framework for designing cognitive experiments to infer latent cognitive mechanisms.
problem Designing cognitive experiments to infer latent cognitive mechanisms.
method Formulated as a Bayesian Experimental Design (BED) problem, treating the experimental environment as the design variable. Introduced an amortized Bayesian experimental design framework for efficient posterior inference and design evaluation.
result No single environment is uniformly optimal across cognitive inference objectives, revealing trade-offs between expected information gain, posterior recoverability, and information efficiency.
Paper proposes DEMER to reconstruct hidden confounders for better reinforcement learning in recommendation.
problem Reinforcement learning in real-world applications is costly due to exploration in the environment.
method DEMER uses a multi-agent generative adversarial imitation learning framework to learn the environment and hidden confounder.
result DEMER effectively reconstructs hidden confounders and improves recommendation policy performance.
New method for ancestral inference in branching processes with random environments.
problem Determining ancestor distribution parameters in branching processes with random environments.
method Generalized method of moments for ancestral inference.
result Limiting distribution of ancestor and offspring estimators decouple and converge to independent Gaussian variables under certain conditions.
New framework for designing cognitive experiments to infer latent cognitive mechanisms.
problem Designing optimal cognitive experiments for Bayesian inference of latent cognitive mechanisms.
method Formulated as a Bayesian Experimental Design (BED) problem, treating environments as design variables. Introduced an amortized Bayesian experimental design framework for efficient posterior inference and design evaluation.
result No single environment is uniformly optimal for all cognitive inference objectives, revealing trade-offs between expected information gain, posterior recoverability, and information efficiency.
We use copulas to improve SLAM in uncertain environments.
problem Uncertain data association and nonlinear transition models in SLAM.
method Integrate copulas into a Sequential Monte Carlo estimator for SLAM.
result Our method effectively handles SLAM in uncertain environments.
The paper tackles policy learning in dynamic environments using causal methods.
problem Existing reinforcement learning algorithms assume static mechanisms, but real-world systems often have changing mechanisms.
method The paper introduces multi-environment contextual bandits and policy invariance to handle environmental shifts.
result An optimal invariant policy is guaranteed to generalize across environments under suitable assumptions.
ICIL learns policies invariant to multiple environments, improving generalization.
problem Learning policies from multiple environments leads to spurious correlations.
method ICIL learns invariant feature representations and a matching imitation policy.
result ICIL policies generalize better to unseen environments.
A new method selects important variables for clustering from dependency networks.
problem Variable selection for clustering in high-cost data scenarios.
method Create dependency networks, rank variables by centrality, select top-n variables.
result Top-n variables improve clustering performance compared to existing methods.