Graph neural network executes value iteration for flexible environments.
problem Value iteration in flexible environments with direct supervision.
method Graph Neural Network (GNN) executing value iteration algorithm.
result GNN accurately models value iteration across diverse environments.
Learning in a non-stationary environment is an inevitable problem when applying machine learning algorithm to real world environment. Learning new tasks without forgetting the previous knowledge is a challenge issue in machine learning. We propose a Kalman Filter based modifier to maintain the performance of Neural Net…
GIB improves neural network generalization by dynamically selecting task-relevant features across different sequential environments.
problem Poor generalization of deep neural networks to unseen environments.
method Proposes a gated information bottleneck (GIB) approach that dynamically drops spurious correlations and selects task-relevant features.
result GIB outperforms other IB approaches in adversarial robustness and OOD detection.
In this paper, we train a recurrent neural network to learn dynamics of a chaotic road environment and to project the future of the environment on an image. Future projection can be used to anticipate an unseen environment for example, in autonomous driving. Road environment is highly dynamic and complex due to the int…
VASE uses Bayesian neural networks to improve exploration in sparse reward environments.
problem Exploration in environments with continuous control and sparse rewards.
method VASE uses a Bayesian neural network model of the environment dynamics and variational inference to alternately update the model's accuracy and policy.
result VASE outperforms other surprise-based exploration techniques in continuous control sparse reward environments.
PPOPT uses pretraining to speed up reinforcement learning in physics simulations.
problem High computational costs and inefficiency in reinforcement learning with small training samples.
method A novel policy neural network architecture that combines pretraining and fully-connected networks.
result PPOPT outperforms classic PPO on small training samples in terms of rewards and stability.
Recurrent networks learn beliefs from history in partially observable environments.
problem Learning optimal policies in partially observable environments.
method Trained recurrent neural networks to approximate value functions, measuring mutual information between hidden states and beliefs.
result Recurrent networks' hidden states correlate with beliefs of relevant state variables, improving expected return.
Deep Recurrent Q-Network improves autonomous driving in urban areas with pedestrians.
problem Challenges in urban autonomous driving due to complex road structures and unpredictable pedestrian behavior.
method Combines Deep Q-Network with LSTM for long-term memory, designed a 3-D state representation, and uses a reward function.
result The proposed DRQN-based approach outperforms rule-based methods in dense urban scenarios.
A new neural network model improves reinforcement learning efficiency.
problem Improving sample efficiency in reinforcement learning.
method Proposes a neural network architecture using VQ-VAE and convolutional LSTM for world modeling.
result Shows comparable performance to SimPLe with significantly smaller model.
This work develops agents to learn generalizable policies for dynamic network environments.
problem Real-world network topologies change due to attackers, defenders, or system failures, leading to failures in adaptive ACD systems.
method Developing agents to learn generalizable policies across dynamic network environments.
result Agents can learn robust policies for dynamic network topologies and diverse attackers.
A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatio-temporal representations. The world model's extracted features are fed into compact and simple policies trained by evolution, achieving state of the art resul…
In Business Intelligence, accurate predictive modeling is the key for providing adaptive decisions. We studied predictive modeling problems in this research which was motivated by real-world cases that Microsoft data scientists encountered while dealing with e-commerce transaction fraud control decisions using transact…
Reinforcement learning is concerned with identifying reward-maximizing behaviour policies in environments that are initially unknown. State-of-the-art reinforcement learning approaches, such as deep Q-networks, are model-free and learn to act effectively across a wide range of environments such as Atari games, but requ…
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…
Meta-learning improves drone trajectory design for dynamic wireless networks.
problem Designing optimal trajectories for energy-constrained drones in dynamic network environments.
method Proposes a meta-learning algorithm to adaptively tune a reinforcement learning solution for trajectory design.
result Meta-tuned RL yields faster convergence and improved communication performance compared to baseline algorithms.
Although various linear log-distance path loss models have been developed, advanced models are requiring to more accurately and flexibly represent the path loss for complex environments such as the urban area. This letter proposes an artificial neural network (ANN) based multi-dimensional regression framework for path …
Study improves systemic risk assessment by considering local network environments.
problem Identifying systemic financial institutions using network metrics.
method Two-step procedure: 1) recover network communities, 2) regress vulnerability on topological measures at global, local, and aggregated levels.
result Local network metrics predict distress better than global metrics during financial crises.
Predicting movement of objects while the action of learning agent interacts with the dynamics of the scene still remains a key challenge in robotics. We propose a multi-layer Long Short Term Memory (LSTM) autoendocer network that predicts future frames for a robot navigating in a dynamic environment with moving obstacl…
Bayesian segmentation and uncertainty estimation improve 3D model accuracy for factory planning.
problem Generating accurate 3D models from outdated and incomplete 2D data.
method Bayesian neural network for point cloud segmentation and entropy-based uncertainty estimation.
result Bayesian segmentation network significantly improves model accuracy and object identification.
Enhances graph neural networks by creating virtual data examples.
problem Lack of examples to identify optimal graph rationales in graph applications.
method Introduces environment replacement to create virtual data examples and proposes a framework for rationale-environment separation and representation learning.
result Demonstrates the effectiveness and efficiency of the augmentation-based graph rationalization framework on molecular and polymer datasets.
Paper tackles noise-robust domain adaptation in noisy environments.
problem Learning machines struggle with domain adaptation in noisy environments.
method The paper proposes offline curriculum learning and proxy distribution based margin discrepancy to mitigate label and feature noise.
result The proposed algorithm significantly outperforms state-of-the-art methods in noisy environments.
Framework tackles OOD challenges in molecule property prediction by modeling environments.
problem Challenges in modeling OOD samples for molecule property prediction.
method Soft causal learning framework incorporating chemistry theories and cross-attention mechanisms.
result Demonstrates well generalization ability on seven datasets.
Paper uses deep reinforcement learning for network slicing and traffic prediction.
problem Managing dynamic network slices while maintaining QoS in a 5G environment.
method Integrates LSTM for traffic prediction and DDRL for distributed decision-making.
result Significant improvements in network performance, reducing QoS violations.
FEPS models agents to learn and act in complex environments without deep neural networks.
problem Modeling complex adaptive systems and understanding self-organizing behavior.
method Introducing Free Energy Projective Simulation (FEPS) within the constraints of the free energy principle and active inference.
result FEPS agents resolve ambiguity and infer optimal policies in partially observable environments.
Deep reinforcement learning (RL) has been successfully applied to a variety of game-like environments. However, the application of deep RL to visual navigation with realistic environments is a challenging task. We propose a novel learning architecture capable of navigating an agent, e.g. a mobile robot, to a target giv…
This work investigates how a naive agent can acquire its own body image in a self-supervised way, based on the predictability of its sensorimotor experience. Our working hypothesis is that, due to its temporal stability, an agent's body produces more consistent sensory experiences than the environment, which exhibits a…
Assisted by neural networks, reinforcement learning agents have been able to solve increasingly complex tasks over the last years. The simulation environment in which the agents interact is an essential component in any reinforcement learning problem. The environment simulates the dynamics of the agents' world and henc…
FAVANO improves federated learning for resource-constrained environments.
problem Asynchronous communication in federated learning leads to bias and scalability issues.
method FAVANO is a novel asynchronous federated learning framework for resource-constrained environments.
result FAVANO outperforms existing methods on standard benchmarks.
Generative Flow Networks solve shortest path problems in graphs.
problem Finding shortest paths in graphs.
method Generative Flow Networks with flow regularization.
result Training a GFlowNet can solve pathfinding problems in arbitrary graphs.
XRM discovers environments without human annotations for OOD methods.
problem Costly and biased manual annotations limit OOD methods.
method XRM trains twin networks to mimic mistakes, eliminating hyper-parameters.
result XRM achieves oracle worst-group-accuracy for OOD methods.
LEADS improves model generalization across different environments.
problem Modeling dynamical systems from varied environments leads to biased or scarce solutions.
method LEADS learns a shared model capturing common dynamics and additional terms for environment-specific dynamics.
result LEADS improves model generalization for both known and novel environments.
Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions f…
AR-A3C improves A3C's robustness to noisy environments.
problem Neural networks are vulnerable to noise from unexpected sources in reinforcement learning.
method Introduced an adversarial agent to make the learning process more robust.
result AR-A3C outperforms A3C in both clean and noisy environments.
This research introduces an autonomous robot navigation method using reinforcement learning.
problem Improving robot navigation in complex environments.
method Deep Q Network (DQN) and Proximal Policy Optimization (PPO) models for path planning and decision-making.
result The models enhance robot navigation ability and adaptive learning in unknown environments.
Neural networks struggle with extrapolation, but a new framework allows them to learn counterfactual invariances.
problem Neural networks' inability to extrapolate beyond training data distribution.
method Introduces a learning framework that allows neural networks to extrapolate over group transformations based on counterfactual invariances.
result Neural networks can learn counterfactual invariances from a single environment, overcoming their limitations in extrapolation.
Dyna-style reinforcement learning is a powerful approach for problems where not much real data is available. The main idea is to supplement real trajectories, or sequences of sampled states over time, with simulated ones sampled from a learned model of the environment. However, in large state spaces, the problem of lea…
Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself a…
Autoencoder neural networks reconstruct missing indoor environment data.
problem Missing data in building operation data sets.
method Three different autoencoder neural networks trained to reconstruct missing data.
result Reconstructing variables with average RMSEs of 0.42 °C, 1.30 % and 78.41 ppm.
Equivariant CNNs improve RL performance in symmetric environments.
problem Learning equivariant representations for RL in symmetric environments.
method Proposed and studied equivariant CNNs for RL.
result Equivariant CNNs enhance RL performance and sample efficiency.
New algorithm guarantees domain generalization with few environments.
problem Performing well on unseen environments with limited training data.
method Iterative feature matching algorithm with theoretical guarantees.
result Guaranteed domain generalization with logarithmic environments.
NetHack Learning Environment (NLE) tests RL algorithms, offering scalable, complex, and challenging gameplay.
problem Challenging environments for testing RL algorithms.
method Procedurally generated, stochastic, rich, and complex NetHack environment.
result Demonstrates empirical success for early stages of NetHack using RL.
Fast and efficient motion planning algorithms are crucial for many state-of-the-art robotics applications such as self-driving cars. Existing motion planning methods become ineffective as their computational complexity increases exponentially with the dimensionality of the motion planning problem. To address this issue…
A new memory system handles non-stationary environments by self-sizing and retaining memories.
problem Non-stationary environments where memories arrive over time and must be distinguished from noise.
method A self-sizing continual associative memory that generalizes Hopfield's network, handling adaptation and novelty.
result The memory system grows to the intrinsic memory demand of the environment and retains memories without forgetting.
Fractal neural networks play SimCity and Conway's Game of Life on varying scales.
problem Generalizing agents' performance to larger gameboards than during training.
method Reinforcement learning in a custom environment, using fractal neural networks.
result Agents can generalize to larger gameboards, solving a minigame unsolvable with local strategies.
FAIR-NN finds invariant variables for causal inference across diverse environments.
problem Nonparametric invariance and causal learning in regression models with varying joint distributions.
method FAIR-NN framework using adversarial optimization and neural networks.
result FAIR-NN identifies invariant variables and quasi-causal variables under minimal conditions.
We propose Deep Q-Networks (DQN) with model-based exploration, an algorithm combining both model-free and model-based approaches that explores better and learns environments with sparse rewards more efficiently. DQN is a general-purpose, model-free algorithm and has been proven to perform well in a variety of tasks inc…
Reinforcement Learning (RL) is a research area that has blossomed tremendously in recent years and has shown remarkable potential for artificial intelligence based opponents in computer games. This success is primarily due to vast capabilities of Convolutional Neural Networks (ConvNet), enabling algorithms to extract u…
Paper proposes a hybrid model for financial time series prediction using sentiment analysis.
problem Challenges in forecasting in non-stationary, complex environments with heterogeneous data.
method Hybrid model combining GANs with NLP-based sentiment analysis.
result Hybrid model enhances robustness in non-stationary environments.