Paper investigates if RL can learn from observing others without explicit modeling.
problem Can RL learn from observing others without explicit modeling?
method Simple scenarios with RL agents observing and learning from another agent's actions.
result RL agents can learn from observing others, leveraging environmental effects.
CVRL tackles complex visual observations in reinforcement learning.
problem Complex visual observations in natural environments.
method Contrastive Variational Reinforcement Learning (CVRL) learns a contrastive variational model by maximizing mutual information between latent states and observations.
result CVRL achieves comparable performance with state-of-the-art model-based DRL methods and significantly outperforms them on tasks with complex observations.
Deep learning identifies space objects from uncorrelated observations.
problem Finding small groups of observations of the same space objects from a large set of uncorrelated data.
method Training a deep learning model on a large data set of uncorrelated observations to identify groups of observations likely of the same space objects.
result The model correctly identified 83.1% of observation pairs as belonging to the same space object.
New framework learns policies for partially observable systems.
problem Learning policies in partially observable dynamical systems.
method Partially Observable Bilinear Actor-Critic framework.
result Algorithm can learn against optimal policies in certain cases.
New algorithm improves reinforcement learning from partial observations.
problem Inferior performance of algorithms in real-world reinforcement learning due to partial observability.
method Representation-based approach to POMDPs, leading to a tractable algorithm.
result Empirically demonstrates superior performance with partial observations.
Improves deep RL for partially observable environments.
problem Handling partially observable environments in deep RL.
method Action-specific Deep Recurrent Q-Network (ADRQN) architecture.
result Demonstrates effectiveness in partially observable domains.
A new algorithm CAP learns optimal policies from observational data with confounding bias and missing observations.
problem Offline contextual bandit with confounding bias and missing observations.
method CAP policy learning, forming reward function as solution of integral equation system, building confidence set, and greedily taking action with pessimism.
result Developed an upper bound to the suboptimality of CAP for the offline contextual bandit problem.
Efficient RL in partially observable risk-sensitive environments with hindsight observations.
problem Risk-sensitive reinforcement learning in partially observable environments.
method Integrates hindsight observations into POMDP framework, develops novel RL algorithm.
result Achieves polynomial regret with provable efficiency, outperforming existing methods.
Optimal observation selection for complex hidden hypotheses.
problem Diagnosis of complex hidden hypotheses using limited observations.
method Active diagnosis through selection of most informative observations based on past results.
result An implication model that predicts future outcomes based on past observations, selecting the most informative next observation.
The paper proposes an algorithm to learn causal state representations for partially observable environments.
problem Learning task-agnostic state abstractions in partially observable environments.
method The approach involves learning approximate causal state representations from RNNs trained to predict observations given the history.
result The learned state representations are useful for efficient policy learning in reinforcement learning problems with rich observation spaces.
Algorithm improves imitation learning from visual data in partially observable environments.
problem Imitation learning from visual observations with missing expert actions and partial observability.
method Theoretical analysis and Latent Adversarial Imitation from Observations algorithm combining adversarial and latent representations.
result Latent Adversarial Imitation from Observations achieves state-of-the-art performance in high-dimensional robotic tasks.
Paper introduces a new approach to learn values from observations for faster RL.
problem Learning from expert demonstrations without action information.
method Directly learns values from observations, not rewards, for faster RL.
result Significantly speeds up RL by removing the need for bootstrapping action-values.
New framework for reinforcement learning with sporadic state observations.
problem Partial observability in reinforcement learning.
method Action-Triggered Sporadically Traceable Markov Decision Processes (ATST-MDPs).
result Optimistic algorithm achieving regret bound for episodic learning.
New algorithm learns policies from expert observations alone, efficiently.
problem Imitation Learning from expert observations in large-scale MDPs.
method Forward Adversarial Imitation Learning (FAIL) algorithm, minimizing IP metric between expert and learner observation distributions.
result First provably efficient algorithm in ILFO setting, learning near-optimal policies with polynomial sample complexity.
New algorithms learn POMDPs efficiently with hindsight observability.
problem Hardness of learning in POMDPs due to partial observability.
method Hindsight Observable Markov Decision Process (HOMDP) and new algorithms for tabular and function approximation settings.
result Sample-efficient learning in POMDPs with optimal dependence on latent state and observation cardinalities.
New partial models correct for confounding effects in reinforcement learning.
problem Confounding effects in partial models lead to incorrect planning.
method Introduces causally correct partial models for reinforcement learning.
result Causally correct partial models avoid confounding effects and improve planning accuracy.
Overfitting occurs when RL agents correlate rewards with spurious observation features.
problem Overfitting in reinforcement learning due to correlation with spurious observation features.
method Developed a framework to analyze and design synthetic benchmarks from modified observation spaces.
result Agents can overfit to different observation spaces even if the MDP dynamics are fixed.
New algorithm learns POMDPs without computational oracles.
problem Learning near-optimal policies in POMDPs with computationally hard oracles.
method Quasipolynomial-time algorithm using barycentric spanners for policy covers.
result First oracle-free learning algorithm for observable POMDPs.
PLOTS learns procedural actions from observed sequences, up to 100x faster.
problem Learning procedural actions from observed sequences efficiently.
method Exploits subtask structure to incrementally build action plans, optimistically explores actions.
result Explicit procedural learning is 100x faster than policy-gradient methods and model-based approaches.
New algorithm proves RL from partial obs is feasible.
problem Difficulty in learning from partial observability.
method Optimism combined with MLE for weakly revealing POMDPs.
result Simple algorithm guarantees polynomial sample efficiency.
The paper teaches robots to navigate by learning costs from expert demonstrations.
problem Teaching robots to navigate autonomously using only expert observations.
method Developed a map encoder and cost encoder to infer semantic class probabilities and a cost function from expert observations.
result Robots can learn to follow traffic rules in a simulator using only semantic observations.
New algorithms for efficient reinforcement learning with rich observations.
problem Efficient reinforcement learning with rich observations in environments with deterministic dynamics.
method Oracle-efficient algorithms using standard optimization primitives.
result Proved sample efficiency and presented examples of challenges.
Method learns dynamics from noisy partial observations.
problem Reconstructing stochastic dynamical systems from indirect noisy data.
method Amortized path generation method for nonlinear stochastic filtering.
result Learned conditional path generator quantifies uncertainty.
DPFRL uses particle filters for decision making with complex visual observations.
problem Decision making with partial complex visual observations.
method Discriminative Particle Filter Reinforcement Learning (DPFRL) with a differentiable particle filter in the neural network policy.
result DPFRL outperforms state-of-the-art POMDP RL models in complex visual observation tasks.
RL struggles with generalization due to implicit partial observability.
problem Generalization in RL is difficult due to implicit partial observability.
method Re-cast RL problem as solving epistemic POMDPs and propose ensemble-based techniques.
result Simple ensemble-based technique achieves significant generalization gains.
The paper develops algorithms for competitive RL in partially observable MGs.
problem Challenges in reinforcement learning with function approximation and partial observability.
method Proposes posterior sampling methods for self-play and adversarial learning in zero-sum MGs.
result Developed algorithms achieve low regret bounds scaling sublinearly with GEC and episode number.
Improves active learning efficiency by warping input space based on observed outputs.
problem Insensitivity of Gaussian process uncertainty to actual observations.
method Input warping with learned monotone reparameterization to adjust acquisition function behavior.
result Significantly improved sample efficiency across various benchmarks, especially in non-stationary conditions.
A new algorithm for distributed machine learning with observations and features.
problem Optimization in large-scale distributed machine learning.
method A stochastic algorithm for distributed observations and features with convergence analysis.
result The algorithm outperforms a benchmark in early iterations.
Blueprint for ESMs that learn from observations and high-res simulations.
problem Large uncertainties in climate projections due to parameterized processes.
method Integrates global observations and high-resolution simulations through machine learning and data assimilation.
result ESMs can learn from both global and high-resolution data, reducing uncertainties.
New approach uses observational data to learn policies with constraints.
problem Learning policies with application-specific constraints from observational data.
method Semiparametrically efficient estimation and algorithm for choosing whom to treat.
result Strong guarantees for the asymptotic utilitarian regret of the resulting policy.
New algorithm learns optimal decisions from imperfectly observed contexts.
problem Learning optimal decisions in bandits with unobserved contexts.
method Posterior sampling algorithm for imperfectly observed contexts.
result Efficient learning from noisy imperfect observations.
Study efficient reinforcement learning for partially observed systems with linear structure.
problem Efficient reinforcement learning for partially observed Markov decision processes with linear structure.
method Proposes OP-TENET algorithm using a Bellman operator with finite memory, adversarial integral equation, and optimistic exploration.
result Achieves ε-optimal policy within O(1/ε^2) episodes with polynomial sample complexity in intrinsic dimension.
Physics-constrained deep learning predicts geophysical dynamics with boundedness.
problem Forecasting geophysical systems with hidden variables and incomplete observations.
method Physics-constrained neural ordinary differential equation (NODE) representations with boundedness constraints.
result The approach generalizes learned dynamics to arbitrary initial conditions.
Controlled interventions provide the most direct source of information for learning causal effects. In particular, a dose-response curve can be learned by varying the treatment level and observing the corresponding outcomes. However, interventions can be expensive and time-consuming. Observational data, where the treat…
New method tackles RL with observational data, confounders.
problem Learning good policies from historical data with unobserved confounders.
method Extends Actor-Critic method to deconfounding variant.
result Proposed algorithms outperform traditional RL methods in confounded environments.
Paper tackles stochastic reinforcement learning with reduced observation costs.
problem Non-deterministic rewards and punishments with stochastic elements.
method Explicitly models stochastic elements and learning costs.
result Quantitative analysis of learning success criteria and observation cost probabilities.
Solves high-dimensional observation learning for control models.
problem Learning dynamics from high-dimensional images is challenging.
method Proposes a Beta DVBF approach to handle latent and observable space discrepancies.
result Demonstrates improved model learning from high-dimensional observations.
Combines observational and randomized data to estimate treatment effects.
problem Estimating heterogeneous treatment effects using only observational data is biased.
method Two-step framework: learn shared structure from observational data, then data-specific structures from randomized data.
result Combining observational and randomized data improves treatment effect estimation.
iTimER learns from reconstruction errors to represent irregularly sampled time series.
problem Learning from irregularly sampled time series with missing data.
method iTimER models reconstruction errors as a proxy for unobserved values, using a mixup strategy and a Wasserstein metric.
result iTimER outperforms state-of-the-art methods in classification, interpolation, and forecasting tasks.
A new approach to unsupervised learning using recognition-parametrised models.
problem Discovering meaningful latent structure in observational data.
method Recognition-Parametrised Model (RPM) combining parametric and non-parametric components.
result Effective learning of latent structure without explicit generative models.
Paper tackles reinforcement learning with complex observations and simple latent dynamics.
problem Understanding reinforcement learning with complex observations and simple latent dynamics.
method Statistical and algorithmic analysis of reinforcement learning under general latent dynamics.
result Identifies latent pushforward coverability as a condition for statistical tractability.
Improves LSTM performance by initializing states via manifold learning.
problem Improving LSTM performance through better initialization.
method Learning an intrinsic data manifold to initialize LSTM internal states.
result Improved LSTM performance through consistent initialization.
Method models other agents' behaviors without requiring direct observation.
problem Understanding and interacting effectively with other agents in reinforcement learning.
method Extracts representations from local observations of the controlled agent using encoder-decoder architectures.
result The method achieves higher returns than baseline methods in multi-agent environments.
Improved active learning for counterfactual learning from observational data.
problem Learning a classifier from observational data with selection bias.
method Active learning with a counterfactual risk minimizer, modifying both risk and active learning process.
result Statistically consistent and more label-efficient algorithm compared to prior work.
Method learns latent states from rich observations to improve RL exploration.
problem Improving RL performance with rich observations and latent states.
method Estimates latent states from observations through regression and clustering, providing finite-sample guarantees.
result Exponential improvement over Q-learning with naïve exploration. Spectral method learns hidden state mapping for RL in rich-observation MDPs.
problem Challenges in RL with large state spaces and hidden low-dimensional structure.
method Spectral decomposition method to learn hidden state to observation state mapping.
result Achieves low regret with weak dependence on observed space dimensionality.
Enhanced feedback model improves sample-efficiency in POMDPs.
problem Exponential hardness of learning in POMDPs.
method Multiple observations in hindsight feedback model.
result Sample-efficient learning possible for new subclasses of POMDPs.
The paper analyzes the tradeoff between bias and overfitting in reinforcement learning with partial observability.
problem Analyzing the tradeoff between asymptotic bias and overfitting in reinforcement learning with partial observability.
method Theoretical analysis and empirical illustration using truncated history of observations and function approximators.
result A smaller state representation decreases the risk of overfitting, but potentially increases asymptotic bias.