Meta-learning algorithms use past experience to learn to quickly solve new tasks. In the context of reinforcement learning, meta-learning algorithms acquire reinforcement learning procedures to solve new problems more efficiently by utilizing experience from prior tasks. The performance of meta-learning algorithms depe…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “unsupervised reinforcement learning”
New framework uses unsupervised learning for efficient exploration in RL.
problem Efficient exploration in reinforcement learning with rich observations.
method Combines unsupervised learning and no-regret RL algorithms.
result Proves sample complexity for finding near-optimal policies.
Optimizes wireless systems using deep learning without supervision.
problem Optimizing resource allocation and transceivers in wireless networks.
method Introduces unsupervised and reinforced-unsupervised learning frameworks for variable and functional optimization problems.
result Demonstrates the effectiveness of the learning frameworks through a user association problem.
Paper shows intrinsic motivation boosts exploration efficiency in HRL.
problem Efficient exploration and subgoal discovery in model-free HRL.
method Unsupervised learning over agent's experiences for subgoal discovery.
result Intrinsic motivation learning improves exploration efficiency.
CURL uses contrastive learning to improve reinforcement learning performance.
problem Improving reinforcement learning performance on complex tasks.
method Contrastive learning to extract high-level features from raw pixels, followed by off-policy control.
result CURL outperforms prior methods on DeepMind Control Suite and Atari Games.
Paper presents a world model that learns invariant causal features using contrastive unsupervised learning.
problem Learning invariant causal features in unsupervised settings.
method Contrastive unsupervised learning with intervention invariant auxiliary task.
result Significantly outperforms state-of-the-art methods on out-of-distribution point navigation tasks.
Paper proposes DRL for unsupervised IoT localization.
problem Challenges in unsupervised localization of IoT devices.
method Modeling localization as MDP, using DRL with reward-setting and RSS measurements.
result Demonstrated effectiveness of DRL in wireless localization.
New RL approach speeds up training across tasks.
problem Training reinforcement learning models quickly and efficiently.
method Combines planning quasi-metric and task-specific aimers.
result Achieves multiple-fold speed-up on bit-flip and robotic arm tasks.
Plan2Vec learns image representations without labels, improving control tasks.
problem Learning image representations without labeled data.
method Constructs a weighted graph using near-neighbor distances and extrapolates to global embedding.
result Plan2Vec achieves accurate long-term value estimates in control tasks with reduced computational and memory costs.
While reinforcement learning (RL) has the potential to enable robots to autonomously acquire a wide range of skills, in practice, RL usually requires manual, per-task engineering of reward functions, especially in real world settings where aspects of the environment needed to compute progress are not directly accessibl…
We assume that we are given a time series of data from a dynamical system and our task is to learn the flow map of the dynamical system. We present a collection of results on how to enforce constraints coming from the dynamical system in order to accelerate the training of deep neural networks to represent the flow map…
Tensor networks for unsupervised machine learningcond-mat.stat-mech
Tensor networks improve unsupervised learning performance.
problem Improving unsupervised machine learning models.
method Autoregressive Matrix Product States (AMPS) combining quantum and machine learning.
result AMPS significantly outperforms existing tensor network models and neural networks.
The paper proposes a framework to reason about object dynamics for faster reinforcement learning.
problem Current reinforcement learning approaches lack prior knowledge about the environment, limiting efficiency.
method Integrates object dynamics and behavior into reinforcement learning to improve efficiency.
result Demonstrates the need for reasoning about object behavior and dynamics, leading to faster learning.
CUDC collects diverse data for offline RL by predicting future states.
problem Challenges in collecting task-agnostic data for offline RL.
method Adaptive temporal distances for curiosity-driven data collection.
result CUDC outperforms existing unsupervised methods in offline RL tasks.
New unsupervised learning task improves RL performance.
problem Reward-driven feature learning limitations in RL from images.
method Introduce Augmented Temporal Contrast (ATC) for unsupervised learning of image representations.
result Training encoders using ATC matches or outperforms end-to-end RL in most environments.
Factor Representation and Decision Making in Stock Markets Using Deep Reinforcement Learningq-fin.ST
Paper uses deep reinforcement learning for optimal stock portfolio management.
problem Optimizing stock portfolio choices in complex market environments.
method Direct deep reinforcement learning to learn factor representations and make optimal decisions.
result Deep learning outperforms average market performance in portfolio allocation.
Paper proposes a new framework to improve policy optimization by aligning real and simulated data distributions.
problem Inaccurate model estimation leads to performance degradation in model-based reinforcement learning.
method Introduces unsupervised model adaptation to minimize the IPM between real and simulated data distributions.
result Achieves state-of-the-art performance in sample efficiency on various continuous control tasks.
New method learns diverse solutions in reinforcement learning without gradient bias.
problem Lack of diverse solutions in reinforcement learning tasks.
method Maximizes state-action-based mutual information directly, using variational lower bound.
result Successfully learns an infinite set of diverse solutions.
Deep reinforcement learning has made significant progress in the field of continuous control, such as physical control and autonomous driving. However, it is challenging for a reinforcement model to learn a policy for each task sequentially due to catastrophic forgetting. Specifically, the model would forget knowledge …
Interprets how intrinsic motivation shapes behavior in RL agents.
problem Understanding how intrinsic motivation influences behavior in reinforcement learning agents.
method Analyzed five RL agents in procedurally generated environments using various interpretability techniques.
result Curiosity-driven agents exhibit broader and more dynamic attention than extrinsically motivated agents.
Recent developments in deep reinforcement learning have enabled the creation of agents for solving a large variety of games given a visual input. These methods have been proven successful for 2D games, like the Atari games, or for simple tasks, like navigating in mazes. It is still an open question, how to address more…
Revisits VIC method to correct intrinsic reward bias in stochastic environments.
problem Intrinsic reward bias in VIC leading to suboptimal solutions.
method Proposes two methods based on transitional probability model and Gaussian mixture model to correct bias.
result Achieves maximal empowerment through corrected intrinsic reward.
Project explores reinforcement learning solutions for sparse reward environments.
problem Difficulty in navigating environments with infrequent rewards.
method Contrast and investigate existing reinforcement learning solutions in various video games.
result Introduces a novel reinforcement learning solution combining curiosity and auxiliary tasks.
Lecture note for data science students on machine learning basics and advanced topics.
problem No specific problem stated; preparing students for advanced machine learning.
method Introduction of basic machine learning concepts and advanced topics.
result Students are prepared to study advanced machine learning topics.
Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent s…
Disentangled representations have recently been shown to improve fairness, data efficiency and generalisation in simple supervised and reinforcement learning tasks. To extend the benefits of disentangled representations to more complex domains and practical applications, it is important to enable hyperparameter tuning …
Projective simulation converges to optimal behavior in Markov decision processes.
problem Optimizing reinforcement learning in Markov decision processes.
method Projective simulation framework applied to reinforcement learning.
result Projective simulation converges to optimal behavior in a large class of Markov decision processes.
We discuss deep reinforcement learning in an overview style. We draw a big picture, filled with details. We discuss six core elements, six important mechanisms, and twelve applications, focusing on contemporary work, and in historical contexts. We start with background of artificial intelligence, machine learning, deep…
CURL tackles unsupervised continual learning without task labels.
problem Learning from non-stationary distributions without task labels.
method Dynamic representation learning without task labels, incorporating rehearsal techniques.
result Strong performance in unsupervised and supervised settings.
Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record g…
A variety of representation learning approaches have been investigated for reinforcement learning; much less attention, however, has been given to investigating the utility of sparse coding. Outside of reinforcement learning, sparse coding representations have been widely used, with non-convex objectives that result in…
Providing a suitable reward function to reinforcement learning can be difficult in many real world applications. While inverse reinforcement learning (IRL) holds promise for automatically learning reward functions from demonstrations, several major challenges remain. First, existing IRL methods learn reward functions f…
Unified approach for learning state representations from streaming data.
problem Learning reusable state representations from high-dimensional, non-stationary data.
method Unified mathematical formulation for learning latent relations, enabling flexible and principled shaping of latent space.
result Improved understanding and evaluation of existing unsupervised learning approaches.
This paper proposes using variational autoencoders to model opponents in multi-agent systems.
problem Understanding and interacting with opponents in multi-agent systems.
method Variational autoencoders for opponent modeling, with a modification to use local information.
result Our opponent modeling methods achieve equal or greater episodic returns.
In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can…
A single pre-trained agent guides feature selection using knockoffs.
problem Feature selection challenges in AI-readiness of data.
method Generates knockoff features and uses reinforcement learning.
result Optimal feature subset identified with reduced dependency on target variable.
Jointly Pre-training with Supervised, Autoencoder, and Value Losses for Deep Reinforcement Learningcs.LG
Deep Reinforcement Learning (DRL) algorithms are known to be data inefficient. One reason is that a DRL agent learns both the feature and the policy tabula rasa. Integrating prior knowledge into DRL algorithms is one way to improve learning efficiency since it helps to build helpful representations. In this work, we co…
Modeling agent behavior is central to understanding the emergence of complex phenomena in multiagent systems. Prior work in agent modeling has largely been task-specific and driven by hand-engineering domain-specific prior knowledge. We propose a general learning framework for modeling agent behavior in any multiagent …
SAMPLR optimizes for ground truth in aleatoric parameters to avoid curriculum-induced covariate shift.
problem Curriculum learning shifts training distribution, leading to suboptimal policies in aleatoric settings.
method SAMPLR optimizes ground-truth utility function, avoiding curriculum-induced covariate shift.
result SAMPLR preserves optimality under ground-truth distribution, promoting robustness across various environments.
Unsupervised algorithm parses CSG images into CFG without pretraining.
problem Sparse reward problem in unsupervised program synthesis for images.
method Grammar-encoded tree LSTM, entropy regularization, sampling without replacement.
result Recover meaningful programs in large search spaces (up to ).
Stochastic computation graphs (SCGs) provide a formalism to represent structured optimization problems arising in artificial intelligence, including supervised, unsupervised, and reinforcement learning. Previous work has shown that an unbiased estimator of the gradient of the expected loss of SCGs can be derived from a…
Conventionally, model-based reinforcement learning (MBRL) aims to learn a global model for the dynamics of the environment. A good model can potentially enable planning algorithms to generate a large variety of behaviors and solve diverse tasks. However, learning an accurate model for complex dynamical systems is diffi…
Paper introduces TrufLL for language model training without labeled data.
problem Training language models from scratch with reinforcement learning.
method Dynamic vocabulary truncation in RL for language generation.
result TrufLL successfully learns language generation policies from scratch.
Behavior Transfer improves reinforcement learning by leveraging pre-trained policies.
problem Efficient transfer of knowledge in reinforcement learning.
method Behavior Transfer (BT) that uses pre-trained policies for exploration.
result BT combined with pre-training leads to better solutions than without pre-training.
SMiRL learns to minimize surprise in unstable environments, improving agent performance.
problem Learning useful behaviors in unpredictable, unstable environments.
method Alternates between learning a density model and improving policy to seek more predictable stimuli.
result SMiRL agents can play games, control robots, and navigate mazes without task-specific rewards.
Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Datacs.LG
Deep RL detects anomalies from few labeled examples and large unlabeled data.
problem Anomaly detection with limited labeled data and large unlabeled data.
method Deep reinforcement learning to optimize detection of labeled and unlabeled anomalies.
result Significantly outperforms state-of-the-art methods on 48 real-world datasets.
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations fro…
Regularity Normalization: Neuroscience-Inspired Unsupervised Attention across Neural Network Layerscs.LG
Inspired by the adaptation phenomenon of neuronal firing, we propose the regularity normalization (RN) as an unsupervised attention mechanism (UAM) which computes the statistical regularity in the implicit space of neural networks under the Minimum Description Length (MDL) principle. Treating the neural network optimiz…