GALA framework learns invariant graph representations via environment augmentation with minimal assumptions.
problem Learning invariant graph representations from different environments without additional assumptions.
method Developed GALA framework with minimal assumptions of variation sufficiency and consistency. Uses an assistant model to differentiate graph environment changes.
result Extracting maximally invariant subgraphs to proxy predictions identifies underlying invariant subgraphs for successful out-of-distribution generalization.
Self-play is an unsupervised training procedure which enables the reinforcement learning agents to explore the environment without requiring any external rewards. We augment the self-play setting by providing an external memory where the agent can store experience from the previous tasks. This enables the agent to come…
Enhances graph neural networks by creating virtual data examples.
problem Lack of examples to identify optimal graph rationales in graph applications.
method Introduces environment replacement to create virtual data examples and proposes a framework for rationale-environment separation and representation learning.
result Demonstrates the effectiveness and efficiency of the augmentation-based graph rationalization framework on molecular and polymer datasets.
AAMDRL uses DRL to manage assets in noisy, changing environments.
problem Learning in noisy, self-adapting environments with sequential data.
method Augmented state information, one-period lag, walk forward analysis.
result AAMDRL outperforms traditional methods in asset management.
CEA augments reinforcement learning by generating counterfactual experiences.
problem Challenges in reinforcement learning, especially out-of-distribution and inefficient exploration.
method CEA uses variational autoencoders to model state transitions and introduces randomness for non-stationarity. It expands learning data through counterfactual inference.
result CEA outperforms SOTA algorithms in diverse environments.
Synthetic augmentation improves financial machine learning performance in variance-dominant regimes.
problem Data scarcity in financial machine learning.
method Formalized synthetic augmentation, introduced size-matched null augmentation, and developed a non-parametric block permutation test.
result Synthetic augmentation is beneficial only in variance-dominant regimes, such as persistent volatility forecasting.
Vision and learning have made significant progress that could improve robotics policies for complex tasks and environments. Learning deep neural networks for image understanding, however, requires large amounts of domain-specific visual data. While collecting such data from real robots is possible, such an approach lim…
RIA method improves OoD generalization for covariate shift.
problem Improving out-of-distribution generalization under covariate shift.
method Adversarial label invariant graph data augmentations for OoD generalization.
result RIA method achieves high accuracy compared to OoD baselines.
This paper studies the problem of learning with augmented classes (LAC), where augmented classes unobserved in the training data might emerge in the testing phase. Previous studies generally attempt to discover augmented classes by exploiting geometric properties, achieving inspiring empirical performance yet lacking t…
Researchers improved Minecraft game performance using imitation learning.
problem Achieving state-of-the-art performance in immersive environments like Minecraft.
method Applied imitation learning to Minecraft, optimizing network architecture, loss function, and data augmentation.
result Reported stronger results than previous experiments, reaching second place in a competition.
New unsupervised learning task improves RL performance.
problem Reward-driven feature learning limitations in RL from images.
method Introduce Augmented Temporal Contrast (ATC) for unsupervised learning of image representations.
result Training encoders using ATC matches or outperforms end-to-end RL in most environments.
Augment small datasets with synthetic backgrounds to train lightweight CNNs for human pose estimation.
problem Training CNNs from limited real-world data for human pose estimation.
method Synthetic background substitution for data augmentation.
result Improves generalization to unseen environments.
DAL enhances black-box bandit algorithms for non-stationary environments.
problem Non-stationary environments in bandit problems.
method DAL combines any stationary bandit algorithm with a change detector.
result DAL consistently outperforms state-of-the-art methods in various non-stationary scenarios.
Self-Predictive Representations improves data-efficient reinforcement learning from limited interaction.
problem Efficient reinforcement learning from limited data.
method Train agents to predict future latent state representations using self-supervised objectives.
result Achieves a median human-normalized score of 0.415 on Atari with 100k steps of interaction, 55% improvement over previous state-of-the-art.
Enhances nighttime vehicle detection using style transfer and augmentation.
problem Nighttime object detection challenges due to lack of lighting and glare.
method Day-to-night style transfer and labeling-free augmentation with CARLA synthetic data.
result Significant improvements in nighttime vehicle detection with YOLO11 model.
Improves speech recognition in noisy environments using robust acoustic models.
problem Adverse environments with significant mismatch between training and test conditions.
method Theoretical analysis of data augmentation as vicinal risk minimization, using mixture of Gaussians to incorporate robust inductive bias.
result Waveform-based approach shows 150% relative improvement in out-of-distribution generalization.
Portfolio management (PM) is a fundamental financial planning task that aims to achieve investment goals such as maximal profits or minimal risks. Its decision process involves continuous derivation of valuable information from various data sources and sequential decision optimization, which is a prospective research d…
This study improves UAV identification using RF signals with one-shot generative methods.
problem Limited RF environments and signal variability make traditional RF identification ineffective.
method Introduces one-shot generative methods to augment RF signals for UAV identification.
result One-shot generative methods outperform traditional methods in low-data regimes.
We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a policy, I2As learn to inter…
This dissertation automates deep learning pipelines and uses meta-learning for better model selection and data augmentation.
problem Challenges in selecting and fine-tuning deep learning pipelines for new datasets.
method Meta-learning for DL pipeline selection and data augmentation, using synthetic data.
result Meta-learned approaches outperform traditional methods in automated DL pipeline selection and data augmentation.
This paper is concerned with multi-view reinforcement learning (MVRL), which allows for decision making when agents share common dynamics but adhere to different observation models. We define the MVRL framework by extending partially observable Markov decision processes (POMDPs) to support more than one observation mod…
In this paper, we investigate the problem of overfitting in deep reinforcement learning. Among the most common benchmarks in RL, it is customary to use the same environments for both training and testing. This practice offers relatively little insight into an agent's ability to generalize. We address this issue by usin…
Study adapts AI research methods to analyze image augmentation impacts on neural network operations.
problem Understanding how image augmentation affects neural network performance and sensitivity.
method Adapted treatment-control paradigm, uses variance decomposition, Sobol indices, and Shapley values for sensitivity analysis.
result Visualizes and quantifies sensitivity to different image augmentation parameters.
We propose an expert-augmented actor-critic algorithm, which we evaluate on two environments with sparse rewards: Montezumas Revenge and a demanding maze from the ViZDoom suite. In the case of Montezumas Revenge, an agent trained with our method achieves very good results consistently scoring above 27,000 points (in ma…
Robust Transformer-Based One-Step Stock Index Forecasting via Shifted Data Augmentation
problem Robust stock index forecasting
method Modified Transformer architecture with Shifted Data Augmentation
result Best performance on benchmark datasets
RAD enhances RL algorithms with data augmentations.
problem Challenges in RL learning from visual observations.
method RAD is a simple plug-and-play module for RL algorithms.
result RAD improves data-efficiency and final performance.
New algorithms tackle data challenges in physics model selection.
problem Lack of labeled data, high dimensionality, and inapplicability of data augmentation techniques to physics data.
method Two algorithms: feature selection and data augmentation combined with classifiers and stacking ensemble.
result Achieved 90% accuracy on nonlinear structural mechanics classification problem.
We propose a targeted communication architecture for multi-agent reinforcement learning, where agents learn both what messages to send and whom to address them to while performing cooperative tasks in partially-observable environments. This targeting behavior is learnt solely from downstream task-specific reward withou…
This paper introduces CENIE to quantify environment novelty for better UED.
problem Challenges in measuring environment novelty for effective UED.
method CENIE framework using state-action space coverage and Gaussian Mixture Models.
result CENIE improves UED performance across multiple benchmarks.
This paper augments the reward received by a reinforcement learning agent with potential functions in order to help the agent learn (possibly stochastic) optimal policies. We show that a potential-based reward shaping scheme is able to preserve optimality of stochastic policies, and demonstrate that the ability of an a…
A new method shapes reinforcement learning environments by abstracting large state spaces.
problem Learning in large, noisy environments with sparse feedback.
method Environment shaping using state abstraction.
result Agent's policy in shaped environment preserves near-optimal behavior in original environment.
Self-consistent models improve reinforcement learning by aligning predictions with future values.
problem Improving reinforcement learning by aligning model predictions with future values.
method Proposes multiple self-consistency updates to encourage a learned model and value function to be consistent with each other.
result Self-consistency helps both policy evaluation and control in both tabular and function approximation settings.
BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.
problem Exploration in visually complex environments.
method Optimizes a single prediction loss in latent space to learn world representation, dynamics, and exploration policy.
result Achieves superhuman performance on Atari games with simpler design.
Mixreg improves RL generalization by mixing diverse training environments.
problem Overfitting in RL agents trained in limited environments.
method Trains on a mixture of diverse observations and imposes linearity constraints.
result Mixreg outperforms baselines on unseen testing environments.
Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.
problem Memory models struggle in partially observable reinforcement learning environments.
method Introduces a novel memory model using the Hadamard product for efficient memory management and updates.
result Significantly outperforms state-of-the-art memory-based methods on challenging benchmarks.
Controller-Augmented Hidden Markov Models (CHMMs) are a framework for constrained sequential inference.
problem Hidden Markov models fail under pathwise constraints like precedence, visitation, or monotonic state progression.
method CHMMs compile constraints into finite-state controllers, then use standard forward-backward and Viterbi recursions to compute exact constrained posteriors and paths.
result CHMMs provide exact constrained inference, monotone ascent in constrained EM, and linear complexity in controller cardinality.
Early detection of breast cancer has a major contribution to curability, and using mammographic images, this can be achieved non-invasively. Supervised deep learning, the dominant CADe tool currently, has played a great role in object detection in computer vision, but it suffers from a limiting property: the need of a …
Online reinforcement learning agents are currently able to process an increasing amount of data by converting it into a higher order value functions. This expansion of the information collected from the environment increases the agent's state space enabling it to scale up to a more complex problems but also increases t…
Deep Reinforcement Learning has been shown to be very successful in complex games, e.g. Atari or Go. These games have clearly defined rules, and hence allow simulation. In many practical applications, however, interactions with the environment are costly and a good simulator of the environment is not available. Further…
A number of recent approaches to policy learning in 2D game domains have been successful going directly from raw input images to actions. However when employed in complex 3D environments, they typically suffer from challenges related to partial observability, combinatorial exploration spaces, path planning, and a scarc…
Improved reinforcement learning with deep learning.
problem Extending MultiGrid Reinforcement Learning to work with deep learning.
method Combining potential-based reward shaping with a learned potential function from interaction, and adapting it for deep learning algorithms.
result DQN augmented with the approach performs significantly better on continuous control tasks.
New algorithms detect changes in non-stationary MABs for better performance.
problem Non-stationary MAB environments where arm reward distributions change over time.
method Modular Detection Augmented Bandit (DAB) procedures with improved performance lower bounds.
result Modular DAB procedures achieve order-optimal regret bounds for various change detectors and bandit algorithms.
Semantic segmentation is a critical method in the field of autonomous driving. When performing semantic image segmentation, a wider field of view (FoV) helps to obtain more information about the surrounding environment, making automatic driving safer and more reliable, which could be offered by fisheye cameras. However…
The behavioral dynamics of multi-agent systems have a rich and orderly structure, which can be leveraged to understand these systems, and to improve how artificial agents learn to operate in them. Here we introduce Relational Forward Models (RFM) for multi-agent learning, networks that can learn to make accurate predic…
Deep reinforcement learning (RL) agents often fail to generalize to unseen environments (yet semantically similar to trained agents), particularly when they are trained on high-dimensional state spaces, such as images. In this paper, we propose a simple technique to improve a generalization ability of deep RL agents by…
Self-supervised reward prediction improves RL in sparse reward settings.
problem Data efficiency and sparse reward signals in reinforcement learning.
method Learning a state representation for reward prediction and using it to shape rewards.
result Self-supervised reward prediction enhances RL algorithms in single-goal environments.
A new algorithm adapts to changing user behaviors in finance.
problem Adapting to changing user behaviors in financial recommendations.
method History-Augmented Collaborative Filtering using a custom neural network.
result The algorithm provides dynamic financial recommendations.
Many decision-making problems naturally exhibit pronounced structures inherited from the characteristics of the underlying environment. In a Markov decision process model, for example, two distinct states can have inherently related semantics or encode resembling physical state configurations. This often implies locall…