Combines offline causal inference and online bandit learning for better decision-making.
problem Making adaptive decisions using both logged and streaming data to avoid user harm.
method Unified offline causal inference and online learning algorithms, deriving bounds on decision accuracy.
result First upper regret bound for forest-based online bandit algorithms.
FOCUS improves offline RL by incorporating causal structure into world-models.
problem Learning effective policies from historical data without interaction.
method FOCUS proposes a practical algorithm that learns and leverages causal structure in offline RL.
result FOCUS outperforms plain model-based offline RL algorithms and other causal model-based RL algorithms.
New method estimates and optimizes policy differences using orthogonal learning.
problem Offline reinforcement learning with safety concerns and cost limitations.
method Dynamic R-learner for estimating and optimizing Qπ(s,1)−Qπ(s,0), leveraging orthogonal estimation. result Consistent policy optimization with improved convergence rates.
CRL improves recommendation systems by reducing distribution shift.
problem Offline metrics fail to predict online performance due to distribution shift in recommender systems.
method Proposes an information-theoretic disentanglement criterion and a variational lower bound for better generalisation under distribution shift.
result CRL variants deliver substantial online gains in listener engagement compared to baseline models.
CausalCOMRL improves RL task representations by integrating causal relationships, enhancing generalizability.
problem Spurious correlations in context-based offline meta-reinforcement learning.
method CausalCOMRL integrates causal representation learning to uncover and incorporate causal relationships among task components.
result CausalCOMRL achieves better performance on meta-reinforcement learning benchmarks.
Enhances causal estimation using unlabeled offline ICU data.
problem Assessing unmeasured physiological variables in new ICU patients.
method Three-stage approach: non-causal and causal estimators, causal filter, and prediction for new patients.
result Enhanced causal estimation for new ICU patients using offline data.
Paper proposes methods to use observational data for reinforcement learning, addressing confounding issues.
problem Using observational data for reinforcement learning can lead to misleading outcomes due to unobserved confounders.
method The paper introduces two deconfounding methods in deep reinforcement learning to adjust for confounders.
result The proposed deconfounding methods improve the accuracy of reinforcement learning models using observational data.
The paper tackles bandit problems with biased offline data by using causal methods.
problem Improving bandit algorithms with biased offline data that includes confounding and selection biases.
method Formalizes the problem from a causal perspective, categorizes biases, and derives robust bounds for each arm.
result Causal bounds can guide the bandit agent to learn a nearly-optimal decision policy and consistently reduce asymptotic regret.
New method learns optimal policies in presence of unmeasured confounders.
problem Optimal policy learning with unobserved confounders.
method Causal-assisted policy learning methods using instrumental variables and negative controls.
result Policies are ildeO(n−1/2) quantile-optimal under mild coverage assumptions. This work explores the idea of a causal contextual multi-armed bandit approach to automated marketing, where we estimate and optimize the causal (incremental) effects. Focusing on causal effect leads to better return on investment (ROI) by targeting only the persuadable customers who wouldn't have taken the action orga…
Dynamical-VAE learns causal dynamics from POMDPs using future information.
problem Learning accurate state representations from partial observations in POMDPs.
method Dynamical Variational Auto-Encoder (DVAE) with hindsight framework.
result DVAE uncovers causal graph more effectively than history-based methods.
New method allows real-time audio synthesis using non-causal convolutions.
problem Real-time audio synthesis limitations due to offline model constraints.
method Post-training reconfiguration of non-causal models for real-time buffer-based processing.
result Non-causal streaming models can be transformed from offline-trained models without quality loss.
Causal Imitation Learning handles noisy measurements and distribution shifts.
problem Learning from noisy state observations and distributional shifts.
method Causal inference framework and adversarial RKHS learning.
result Improved robustness to distribution shifts compared to standard methods.
New algorithm tackles confounding in offline RL for partially observable MDPs.
problem Confounding in offline reinforcement learning for partially observable MDPs.
method P3O algorithm using proximal causal inference and pessimistic confidence regions.
result Achieves n−1/2-suboptimality for offline RL in POMDPs with confounded data. DOVI improves reinforcement learning with offline data, reducing trial-and-error in critical scenarios.
problem Lack of sample efficiency in deep reinforcement learning for critical applications.
method Proposes DOVI algorithm to incorporate confounded observational data provably efficiently.
result DOVI reduces regret by a multiplicative factor compared to pure online setting, especially when data are informative.
Study uses three sources to evaluate language models fairly.
problem Bias in offline model evaluation due to confounded model choice.
method Combines observational logs, randomized experiments, and simulators.
result Randomized experiment and simulator together recover causal model values.
GDT improves reinforcement learning by matching future state information efficiently.
problem Efficient learning of multi-task policies from trajectory data.
method Generalized Decision Transformer (GDT) for offline hindsight information matching.
result GDT enables effective offline multi-task state-marginal matching and imitation learning.
A new causal deepset framework improves off-policy evaluation under complex interference.
problem Handling spatio-temporal interference in off-policy evaluation.
method Permutation invariance assumption and novel algorithms incorporating it.
result Significantly more precise estimations than existing methods.
Paper tackles unobserved confounding in human-AI collaborations.
problem Unobserved confounding undermines human-AI collaboration effectiveness.
method Combines sensitivity analysis from causal inference with AI-driven statistical modeling.
result Enhances robustness and reliability of collaborative outcomes.
Normalizing flow regression approximates posterior distributions without additional sampling.
problem Bayesian inference with computationally expensive likelihood evaluations.
method Normalizing flow regression (NFR) for offline inference.
result NFR yields a tractable posterior approximation through regression on existing log-density evaluations.
Combining causality, control, and reinforcement learning for system control.
problem Learning to control dynamical systems using causal, control, and reinforcement learning approaches.
method Combining causal identification, control strategies, and reinforcement learning to control dynamical systems.
result Combining different learning paradigms for effective system control.
FOCAL tackles offline meta-reinforcement learning with efficient task inference and behavior regularization.
problem Efficiently adapt RL algorithms to unseen tasks without interactions, addressing bootstrapping errors and robust task inference.
method FOCAL combines behavior regularization, a deterministic context encoder, and a negative-power distance metric for efficient task inference.
result FOCAL outperforms prior algorithms on meta-RL benchmarks, demonstrating computational efficiency.
New method prevents invalid inference after causal discovery.
problem Invalid inference after causal discovery.
method Developed tools for valid post-causal-discovery inference.
result Our method provides reliable coverage while achieving more accurate causal discovery.
Probabilistic models can handle causal inference without special tools.
problem Confusion over necessary tools for causal inference.
method Demonstrated through concrete examples that causal questions can be answered using standard probabilistic models.
result Causal questions can be addressed using standard probabilistic modelling and inference.
This paper tackles sequential distribution shifts in representation learning.
problem Learning meaningful representations in a sequence of distribution shifts.
method Nonlinear Independent Component Analysis (ICA) framework for continual causal representation learning.
result The method achieves performance comparable to joint training on multiple offline distributions and shows no benefit from the incoming new distribution on all latent variables.
Deep learning aids causal inference in complex settings.
problem Estimating heterogeneous treatment effects in non-linear, time-varying, and encoded confounders.
method Intuitive introduction to deep learning and causal inference, focusing on observational data.
result Maximizes accessibility to causal inference through deep learning.
Book introduces ML and AI for causal inference.
problem Uncertainty in causal relationships.
method Structural equation models, DAGs, SCMs, and Double/Debiased Machine Learning.
result Improved inference in causal models using predictive tools.
Causal inference deals with identifying which random variables "cause" or control other random variables. Recent advances on the topic of causal inference based on tools from statistical estimation and machine learning have resulted in practical algorithms for causal inference. Causal inference has the potential to hav…
New offline RL study shows exponential sample requirement for accurate policy evaluation.
problem Understanding statistical limits of offline RL with linear function approximation.
method Analyzes necessary representational and distributional conditions for sample-efficient offline reinforcement learning.
result Even with realizability and good feature coverage, offline RL requires exponential samples for accurate policy evaluation.
PESCAL uses mediators to learn from confounded offline data.
problem Learning from confounded observational data in reinforcement learning.
method PESCAL uses mediator variables and the pessimistic principle to address confounding bias and distributional shift.
result It is sufficient to learn a lower bound of the mediator distribution function to mitigate distributional shift.
Two environments are enough to infer causal graphs and counterfactuals.
problem Inferring causal relations from multiple environments, especially for nonlinear mechanisms.
method Using structural causal models and the invariance principle, the study shows that only two auxiliary environments are sufficient for causal graph inference and counterfactual inference.
result Two auxiliary environments are sufficient for identifying causal graphs and counterfactuals.
ABCI infers causal models and queries simultaneously using Bayesian active learning.
problem Inference of causal models and effects in a two-stage process is inefficient and unnatural.
method Active Bayesian Causal Inference (ABCI) using Gaussian processes for sequentially designing experiments.
result ABCI is more data-efficient and accurate in learning causal queries from fewer samples.
Variational Causal Networks approximate Bayesian inference over causal structures.
problem Quantifying uncertainty in causal structure inference from finite data.
method Parametric variational family over DAGs, using Evidence Lower Bound (ELBO) for tractable learning.
result Approximation of the true posterior over DAGs is demonstrated to be good.
DECI combines causal discovery and inference in a single model for diverse data types.
problem Combining causal discovery and inference methods for diverse data types.
method Develops a single flow-based non-linear additive noise model (DECI) for causal discovery and inference.
result DECI can recover ground truth causal graphs and perform (C)ATE estimation.
BBCI uses meta prediction to estimate causal effects from datasets.
problem Estimating causal effects from observed data.
method Meta prediction to learn causal effect estimation.
result BBCI accurately estimates ATEs and CATEs across various causal inference problems.
Proposes DCNAR for dynamic causal inference from neural time series.
problem Uncertainty and evolution of causal structure in real-world domains.
method Two-stage neural causal modeling integrating discovery and inference.
result Dynamic causal inferences are more stable and meaningful than alternatives.
Causal inference is similar to prediction with treatment bias.
problem Generalizing from labeled to unlabeled data with treatment effects.
method Reframing causal inference as a prediction problem with explicit assumptions.
result Causal assumptions are not uniquely strong but more explicit.
New models infer causal effects from graph-based time-series data.
problem Inferring causal effects from graph-based relational time-series data.
method Proposes causal inference models leveraging graph topology and time-series data.
result Relational time-series causal inference models accurately estimate local causal effects of individual nodes.
PRL improves off-policy evaluation in partially observed MDPs.
problem Confounding and bias in offline reinforcement learning with unobserved state factors.
method Extends proximal causal inference to POMDPs, identifying and estimating target policy value.
result Semiparametrically efficient estimators for PRL in partially observed MDPs.
Deep learning method infers causal interactions from data.
problem Causal inference from observational data.
method Transform input vectors to NEPDFs, train CNN on NEPDFs.
result Improves upon prior methods for causal inference.
The paper proposes a policy learning framework for interpretable personalization.
problem Effective personalization of goods and services to improve revenues and maintain competitive edge.
method Policy learning with linear decision boundaries using causal inference and Bayesian optimization.
result The learned policy improves net sales revenue by 88.2% and provides insights into important features.
Blog post comparing neural network methods for causal inference.
problem Estimating heterogeneous treatment effects in causal inference.
method Developed and compared a fully connected neural network implementation of Bayesian Causal Forest.
result Improvements in performance in simulation settings.
RealCause provides a realistic benchmark for causal inference.
problem Lack of a reliable benchmark for comparing causal effect estimators.
method Flexible generative models to create a benchmark that is both ground-truth and realistic.
result Evaluation of over 1500 causal estimators provides evidence for choosing hyperparameters using predictive metrics.
Paper constructs unfaithful probability distributions in binary causal graphs.
problem Unfaithful probability distributions in binary causal graphs.
method Constructs unfaithful probability distributions in binary causal graphs.
result Examples of unfaithful probability distributions in binary causal graphs.
Bayesian causal inference method improves accuracy over traditional approaches.
problem Bayesian marginalisation over causal models is computationally infeasible.
method Decomposes structure marginalisation into causal orders and DAGs, using Gaussian processes for mechanisms and ARCO for orders.
result Method outperforms state-of-the-art in structure learning and inference.
ParKCa combines multiple causal inference methods to infer new causes from known and unknown factors.
problem Causal inference from observational data when randomized experiments are not feasible.
method ParKCa uses a stacking approach to combine results from multiple causal inference methods.
result ParKCa infers more causes than existing methods in real-world and simulated datasets.
New method infers causal effects without knowing control variables.
problem Inference errors when control variables are unknown.
method Proposes a method for inferring causal effects when control variables are unknown.
result Proves method yields asymptotically valid confidence intervals for average causal effects.
CInA method uses attention to improve causal inference.
problem Challenges in causal inference, especially in complex tasks.
method CInA method utilizes self-supervised causal learning with multiple unlabeled datasets and transformer-type architecture.
result CInA effectively generalizes to out-of-distribution datasets and various real-world datasets.