New method accelerates MBRL by 10x for complex quadrotor tasks.
problem Inefficient training of world models in MBRL for complex robotic tasks.
method Parallelized state-space models and privileged information for training.
result Significant speedup in world model and MBRL training times.
Improved tabular models learn better from real-world data.
problem Tabular models perform poorly on real-world datasets when trained only on synthetic data.
method Continued pre-training on a curated set of real-world datasets.
result Real-TabPFN achieves superior predictive accuracy on 29 datasets.
Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.
We explore building generative neural network models of popular reinforcement learning environments. Our world model can be trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation of the environment. By using features extracted from the world model as inputs to an agent, we c…
A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatio-temporal representations. The world model's extracted features are fed into compact and simple policies trained by evolution, achieving state of the art resul…
Dreamer 4 learns Minecraft tasks from videos alone.
problem Accurately predicting object interactions in complex environments.
method Reinforcement learning inside a fast, accurate world model.
result Dreamer 4 outperforms previous models in Minecraft, learning from only offline data.
The paper analyzes the training dynamics of neural networks using kernel methods.
problem Understanding the training dynamics of neural networks in high-dimensional settings.
method High-dimensional asymptotics and gradient flow on kernel least-squares objectives.
result The training dynamics of neural networks undergo three stages, characterized by behaviors in the Oracle and Empirical worlds.
Paper proposes hybrid modeling to improve surrogate accuracy using multiple data sources.
problem Improving surrogate model accuracy by integrating simulation and real-world data.
method Two novel probabilistic approaches: separate and combined surrogates with weighting strategy.
result Hybrid models improve predictive accuracy and coverage compared to single-source surrogates.
DreamerV2 learns Atari game behaviors from a world model, achieving human-level performance.
problem Learning complex behaviors in Atari games from limited data.
method DreamerV2 uses a world model with discrete representations to predict behaviors in a compact latent space.
result Achieves human-level performance on 55 Atari tasks.
The study examines how language models learn to represent the world, identifying conditions for ecological veridicality.
problem Understanding when language models learn to represent the world accurately and how this learning process can fail.
method Analyzes the Bayes-optimal next-token cross-entropy decomposition and the role of training ecology in shaping model representations.
result The minimum-complexity zero-excess solution is the quotient partition by training equivalence, and this solution is not preserved in in-context learning or per-task adaptation.
Deep Neural Networks (DNNs) are usually over-parameterized, causing excessive memory and interconnection cost on the hardware platform. Existing pruning approaches remove secondary parameters at the end of training to reduce the model size; but without exploiting the intrinsic network property, they still require the f…
GANs can learn hierarchical distributions in real-world images efficiently.
problem Understanding and efficiently learning complex, real-world distributions with GANs.
method Formally studying how GANs can learn hierarchically generated distributions close to real-life image distributions using SGDA.
result Training GANs via SGDA can efficiently learn distributions with a 'forward super-resolution' structure, both in sample and time complexities.
GPT learns a causal world model from token predictions, validated in game sequences.
problem Does GPT implicitly learn a causal world model from token predictions?
method Derived a causal interpretation of GPT's attention mechanism and proposed zero-shot causal structure learning.
result GPT can generate legal next moves with high confidence for sequences with encoded causal structures, but fails for illegal moves.
New algorithms improve federated learning accuracy and stability with real-world data.
problem Real-world data diversity and imbalance challenge federated learning.
method Developed new algorithms (FedVC, FedIR) to resample and reweight data.
result Significant improvements in accuracy and stability of federated learning.
Paper tackles AI driving competition challenges with mixed simulation and real-world data.
problem AI algorithms perform poorly in real-world environments compared to simulated ones and vice versa.
method Employed imitation learning on a mixed dataset to train algorithms equally well in all environments.
result Trained algorithms performed well in both simulated and real-world environments.
Self-consistency improves the accuracy of model comparison methods.
problem Improving the accuracy of model comparison methods when simulation models are misspecified.
method Supplement traditional simulation-based training with a self-consistency loss on unlabeled real data.
result Self-consistency training improves model comparison accuracy, especially in open-world scenarios.
Algorithm calibrates predictions for covariate shift using domain adaptation.
problem Uncertainty estimates overestimate certainty when real-world data differs from training data.
method Uses importance weighting and learns a feature map to equalize distributions.
result Outperforms existing approaches in calibrated prediction when covariate shift occurs.
MAT combines meta-learning and adversarial training to defend against universal patches.
problem Defending against universal patches that fool models in various contexts.
method Meta adversarial training (MAT) integrates meta-learning with adversarial training.
result MAT increases robustness against universal patch attacks on image classification and traffic-light detection.
Underspecified ML models can behave unpredictably in real-world use.
problem ML models can fail in real-world deployment due to ambiguous predictors.
method Identified underspecification as the cause, showing it affects various ML domains.
result Underspecified models can behave differently in deployment domains.
Robots hold promise in many scenarios involving outdoor use, such as search-and-rescue, wildlife management, and collecting data to improve environment, climate, and weather forecasting. However, autonomous navigation of outdoor trails remains a challenging problem. Recent work has sought to address this issue using de…
Modern computer vision algorithms typically require expensive data acquisition and accurate manual labeling. In this work, we instead leverage the recent progress in computer graphics to generate fully labeled, dynamic, and photo-realistic proxy virtual worlds. We propose an efficient real-to-virtual world cloning meth…
This work improves AI's ability to solve physical tasks by optimizing world models in abstracted spaces.
problem Developing AI agents capable of solving diverse physical tasks and generalizing to new environments.
method Investigates and optimizes a family of joint-embedding predictive world models (JEPA-WMs) for efficient planning in abstracted spaces.
result Proposes a model that outperforms two established baselines in both navigation and manipulation tasks.
Training large and highly accurate deep learning (DL) models is computationally costly. This cost is in great part due to the excessive number of trained parameters, which are well-known to be redundant and compressible for the execution phase. This paper proposes a novel transformation which changes the topology of th…
Improves reinforcement learning policies for robustness.
problem Lack of robustness in reinforcement learning policies.
method Risk-aware Distributional Reinforcement Learning (SDPG) with CVaR.
result Risk-averse policies achieve robustness against disturbances.
A large body of recent work has investigated the phenomenon of evasion attacks using adversarial examples for deep learning systems, where the addition of norm-bounded perturbations to the test inputs leads to incorrect output classification. Previous work has investigated this phenomenon in closed-world systems where …
Transformer learns long-term dependencies from real-world data.
problem Sample inefficiency in deep reinforcement learning.
method Transformer architecture applied to autoregressive real-world episodes.
result Transformer-based world model generates meaningful experience.
Agents compose pre-trained policies for complex tasks, improving zero-shot performance.
problem Challenges in long-horizon predictions and estimating visitation distributions induced by policy sequences.
method Learn predictive jumpy world models of multi-step dynamics, enhancing predictions with a consistency objective.
result Compositional planning with jumpy world models yields, on average, a 200% relative improvement over primitive actions on long-horizon tasks.
This work studies reinforcement learning in the Sim-to-Real setting, in which an agent is first trained on a number of simulators before being deployed in the real world, with the aim of decreasing the real-world sample complexity requirement. Using a dynamic model known as a rich observation Markov decision process (R…
WMPG reduces policy gradient variance using world models.
problem Reducing variance in policy gradient estimates.
method Trains a world model online to estimate policy gradients and uses imagined trajectories as a baseline.
result WMPG achieves better sample efficiency compared to AC and MAC.
Model-free deep reinforcement learning has been shown to exhibit good performance in domains ranging from video games to simulated robotic manipulation and locomotion. However, model-free methods are known to perform poorly when the interaction time with the environment is limited, as is the case for most real-world ro…
Multilingual end-to-end (E2E) models have shown great promise in expansion of automatic speech recognition (ASR) coverage of the world's languages. They have shown improvement over monolingual systems, and have simplified training and serving by eliminating language-specific acoustic, pronunciation, and language models…
We present the use of the fitted Q iteration in algorithmic trading. We show that the fitted Q iteration helps alleviate the dimension problem that the basic Q-learning algorithm faces in application to trading. Furthermore, we introduce a procedure including model fitting and data simulation to enrich training data as…
A new neural network model improves reinforcement learning efficiency.
problem Improving sample efficiency in reinforcement learning.
method Proposes a neural network architecture using VQ-VAE and convolutional LSTM for world modeling.
result Shows comparable performance to SimPLe with significantly smaller model.
Equivariant MuZero improves generalization in procedurally generated environments.
problem Leveraging world models for robust generalization in real-world scenarios.
method Incorporates symmetries of the environment into MuZero's world-model architecture, making MuZero equivariant to certain groups of transformations.
result Improves data efficiency and generalization capabilities, demonstrated on procedurally generated MiniPacman and Chaser.
Machine learning models trained on indirect data labels can fail on real-world examples.
problem Validity issues in machine learning when target labels are indirectly defined.
method Identification of problematic datasets and models using a general procedure.
result Machine learning models trained on indirect data labels will fail on real-world examples.
Panda predicts chaotic systems without retraining, showing emergent properties.
problem Predicting chaotic systems with small errors.
method Trained on a synthetic dataset of chaotic dynamical systems using evolutionary algorithms.
result Panda predicts unseen chaotic systems with zero-shot learning.
New model detects crying in real-world settings with improved accuracy.
problem Generalization of cry detection models to real-world environments.
method Evaluated machine learning approaches on a novel dataset of real-world infant crying.
result Improved F1 score of 0.613 for crying event recognition in real-world settings.
Deep neural networks are gaining increasing popularity for the classic text classification task, due to their strong expressive power and less requirement for feature engineering. Despite such attractiveness, neural text classification models suffer from the lack of training data in many real-world applications. Althou…
This paper describes our UNet based deep convolutional neural network approach on the Traffic4cast challenge 2019. Challenges task is to predict future traffic flow volume, heading and speed on high resolution whole city map. We used UNet based deep convolutional neural network to train predictive model for the short t…
MarS simulates financial markets using generative models.
problem Simulating realistic financial market effects.
method Order-level generative foundation model (LMM) for realistic, interactive, and controllable order generation.
result Strong scalability and robust realism in MarS.
BootsTAP uses real-world data to improve TAP tracking performance.
problem Lack of large-scale groundtruth training data for TAP.
method Bootstrapped training with a self-supervised student-teacher setup.
result State-of-the-art performance on TAP-Vid benchmarks.
UWM-JEPA predicts future scenarios in belief space, improving accuracy in partially observed environments.
problem Predicting future scenarios in partially observed environments with uncertainty.
method Introduces UWM-JEPA, a JEPA world model with a density-matrix latent and learned unitary predictor.
result UWM-JEPA achieves 0.77 accuracy on a hidden-velocity indicator task, outperforming LSTM-JEPA.
In recent years, deep neural networks (DNN) have become a highly active area of research, and shown remarkable achievements on a variety of computer vision tasks. DNNs, however, are known to often make overconfident yet incorrect predictions on out-of-distribution samples, which can be a major obstacle to real-world de…
Survey and framework for consistent uncertainty quantification in deep learning.
problem Partial uncertainty coverage and inconsistencies in deep learning uncertainty quantification.
method Bayes' theorem and conditional probability densities applied to all major sources of uncertainty.
result Improved robustness and reliability of neural network predictions in real-world scenarios.
In real-world recognition/classification tasks, limited by various objective factors, it is usually difficult to collect training samples to exhaust all classes when training a recognizer or classifier. A more realistic scenario is open set recognition (OSR), where incomplete knowledge of the world exists at training t…
Training machine learning models in parallel is an increasingly important workload. We accelerate distributed parallel training by designing a communication primitive that uses a programmable switch dataplane to execute a key step of the training process. Our approach, SwitchML, reduces the volume of exchanged data by …
Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time. Given that it is impractical …
Paper evaluates membership inference attacks on transfer learning models.
problem Evaluating membership inference attacks on transfer learning models.
method Shadow model training strategy to derive data for membership inference classifier.
result Membership inference attacks can achieve effective performance against transfer learning models.