Synthetic data training improves neural networks for captcha breaking.
problem Training neural networks with synthetic data for improved performance.
method Connecting synthetic data training to model-based Bayesian inference.
result Demonstrated state-of-the-art performance and posterior uncertainty in captcha breaking.
Paper proves efficiency of MARL with transformers, addressing agent complexity.
problem Theoretical understanding of MARL with many agents and limited relational reasoning.
method Set Transformer for relational reasoning, model-free and model-based MARL algorithms.
result Provable efficiency of MARL algorithms, suboptimality gaps independent of number of agents.
Disentangled representations improve abstract visual reasoning tasks.
problem The usefulness of disentangled representations for abstract visual reasoning.
method A large-scale study with 360 state-of-the-art unsupervised disentanglement models and 3600 abstract reasoning models.
result Disentangled representations lead to better down-stream performance in abstract reasoning tasks.
SAVE combines Q-learning and MCTS with amortized value estimates for improved performance.
problem Combining model-free Q-learning and model-based MCTS for efficient learning and planning.
method SAVE uses a learned prior to guide MCTS, which estimates improved state-action values. These estimates are used to update the prior, creating a cooperative relationship between learning and search.
result SAVE achieves higher rewards with fewer training steps and strong performance with small search budgets.
This paper tackles hard exploration in the game Pommerman, improving RL learning.
problem Hard exploration in sparse, delayed, and deceptive reward domains.
method Developed a model-based automatic reasoning module to prune unsafe actions.
result Model-based approach significantly improves RL learning in Pommerman.
The paper tackles model-based RL's inaccuracy issue by dynamically adjusting planning horizons.
problem Model-based RL's failure due to model inaccuracy over long planning horizons.
method State-dependent planning horizon, learning cumulative model errors with Temporal Difference methods.
result The proposed method successfully adapts planning horizons to state-dependent model accuracy, improving policy learning efficiency.
Survive method improves model-based RL by avoiding terminal states, reducing sample complexity.
problem High sample complexity in model-free RL methods limits real-world applications.
method Introduces 'survival' concept to model-based RL, focusing on avoiding terminal states instead of maximizing rewards.
result Survive method reduces training effort by focusing on terminal states, improving model-based RL performance.
Asynchronous method for hyperparameter and neural architecture search.
problem Efficiently searching for optimal hyperparameters and neural architectures.
method Model-based, asynchronous multi-fidelity method combining Hyperband and Gaussian process-based Bayesian optimization.
result Substantial speed-ups over current state-of-the-art methods on various benchmarks.
SeCTAR learns latent representations of trajectories for hierarchical reinforcement learning.
problem Learning lower layers in a hierarchy of reinforcement learning problems.
method SeCTAR uses variational autoencoders to learn latent representations of trajectories, combining policy and model consistency.
result SeCTAR effectively solves long-term and multi-stage problems with sparse rewards.
The purpose of this note is to reconcile two different results concerning the model-free upper bound on the price of an American option, given a set of European option prices. Neuberger (2007, `Bounds on the American option') and Hobson and Neuberger (2016, `On the value of being American') argue that the cost of the c…
This paper builds a model to predict the long-term future in reinforcement learning.
problem Catastrophic failures due to flawed long-term predictions in reinforcement learning models.
method The authors develop a latent-variable autoregressive model using variational inference to incorporate future information.
result The model achieves higher rewards faster than baselines on various tasks and environments.
Novel approach to learning models based on subjective timescales for better exploration and decision-making.
problem Learning models over multi-step timescales in environments with intermediate states.
method Developed a subjective-timescale model (STM) based on episodic memories, enabling systematic variation of temporal extent of predictions.
result STM produces more informative action-conditioned roll-outs, leading to better decision-making and exploration.
Paper cleans option price datasets by removing outliers.
problem Unusual option prices in datasets.
method Statistical techniques to identify and remove outliers.
result Removes option prices violating no arbitrage assumption.
FBC clusters data fairly without needing cluster count.
problem Fairness in clustering groups of different sensitive groups.
method Developed a Bayesian model-based clustering method with a fair prior and efficient MCMC algorithm.
result Reasonably infers the number of clusters and achieves a fair utility trade-off.
Derives RL framework for systems without velocity or acceleration measurements.
problem Learning control for systems with limited sensor data.
method Gaussian Process Regression with a novel derivative-free kernel.
result Improved estimation performance and data-efficiency compared to traditional methods.
Extended analysis of Q-learning's efficiency, matching optimal regret.
problem Theoretical guarantees of Q-learning's efficiency and optimal regret.
method Survey of related research, detailed proof reasoning.
result Q-learning with UCB exploration achieves sample efficiency matching optimal regret.
Paper develops a method for rapid online learning of deep models.
problem Rapid adaptation of deep models to changing tasks.
method Uses meta-learning to train a model that can adapt to new tasks and recall old ones.
result Meta-learning enables effective continuous adaptation in non-stationary task distributions.
This paper optimizes hyperparameters for random forest models.
problem Improving prediction performance of random forest models.
method Literature review followed by model-based optimization (MBO) using the tuneRanger R package.
result Tuning hyperparameters can significantly improve random forest model performance.
Modular RL modules solve complex 3D Sokoban tasks.
problem Solving complex, integrated tasks combining visual, physical, and abstract reasoning.
method Compose RL modules in a sense-plan-act hierarchy, using only model-free methods.
result Modular RL outperforms state-of-the-art monolithic RL on Mujoban.
New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
Hyperboost uses gradient boosting for hyperparameter optimization, outperforming state-of-the-art methods.
problem Hyperparameter tuning for machine learning algorithms
method Gradient boosting surrogate model with quantile regression and distance metric
result Hyperboost outperforms state-of-the-art techniques in empirical tests
The model predicts stock price trends and opening, minimum, and maximum prices with reasonable accuracy.
problem Forecasting stock prices and trends for investment decisions.
method Improvement of a model based on the association of three LSTM neural networks.
result The model predicts stock price trends and opening, minimum, and maximum prices with reasonable accuracy.
Empirical observations of CNN invertibility explained with a mathematical model.
problem Understanding why Convolutional Neural Networks (CNNs) are approximately invertible.
method Developed a mathematical model of sparse signal recovery consistent with random-weight CNNs, connecting to model-based compressive sensing.
result CNNs trained with random weights are consistent with the mathematical model and can be used for reasonable image reconstruction.
Study develops a cost model for field canals improvement projects in Egypt.
problem Accurately predict the preliminary costs of field canals improvement projects.
method Developed a parametric cost model using machine learning methods.
result Identified key cost drivers and developed a model for FCIPs.
Proposes a model to interpret complex ML algorithms.
problem Complex ML models are hard to interpret.
method Uses model-based regression trees and interpretable main-effects models.
result Surrogate model provides interpretable results with good predictive performance.
Novel technique detects adversarial samples in face recognition models.
problem Widespread adversarial sample attacks on DNN models.
method Bi-directional correspondence inference between attributes and internal neurons to identify critical neurons.
result 94% detection accuracy for 7 different kinds of attacks with 9.91% false positives.
Introduces LoCA regret to evaluate model-based RL methods.
problem Lack of consistent metrics to evaluate model-based RL methods.
method Inspired by neuroscience, introduces LoCA regret to measure model-based behavior.
result LoCA regret can identify model-based behavior and assess how close methods are to optimal model-based behavior.
Simple model-based reinforcement learning outperforms model-free methods in complex tasks.
problem Lagging performance of model-based reinforcement learning agents in non-trivial environments.
method Combining soft value estimates with stochastic value gradients.
result Simple model-based agents achieve state-of-the-art results in a high-dimensional humanoid control task.
PCI combines perception and control using Bayesian inference with object-based representations.
problem Separate perception and control in reinforcement learning.
method Joint Perception and Control as Inference (PCI) framework with Object-based Perception Control (OPC).
result OPC achieves good perceptual grouping quality and outperforms baselines in accumulated rewards.
POMBU improves model-based RL's asymptotic performance by estimating and using uncertainty.
problem Model-based reinforcement learning struggles with model errors, leading to suboptimal performance.
method POMBU uses estimated uncertainty to optimize policies conservatively, improving asymptotic performance.
result POMBU outperforms existing methods in sample efficiency and asymptotic performance.
M3PO improves model-based meta-RL with theoretical guarantees.
problem Improving sample efficiency in multi-task RL with theoretical guarantees.
method Extending Janner et al. (2019) theorems, proposing M3PO with performance guarantees.
result M3PO outperforms existing methods in continuous-control benchmarks.
Calibrates deep learning models to produce accurate uncertainty estimates.
problem Inaccurate uncertainty estimates in Bayesian and probabilistic models.
method Simple procedure inspired by Platt scaling to calibrate regression algorithms.
result Consistently produces well-calibrated credible intervals improving model performance.
A new perfect specialization model explains trade data better than imperfect models.
problem Improving the theoretical foundation of gravity equation in bilateral trade.
method Developed a perfect specialization model based on tradability.
result Tradability is the sole reason for deviations from basic models.
Develops a model to learn shared and idiosyncratic patterns in point processes.
problem Learning shared and unique patterns in point processes from diverse observations.
method Developed a parametric point process model with alternating optimization for learning shared structure and idiosyncratic effects.
result The method yields explainable point process models that perform well compared to existing methods.
Model-based neural networks generalize better than ReLU networks for sparse recovery.
problem Understanding and quantifying the superior generalization of model-based neural networks.
method Using complexity measures like global and local Rademacher complexities, the paper provides theoretical bounds on generalization and estimation errors.
result Model-based neural networks exhibit higher generalization capabilities for sparse recovery problems compared to ReLU networks.
Study batch reinforcement learning methods for personalized medical treatments.
problem Batch reinforcement learning for personalized medical treatments.
method Direct policy learning and model-based learning approaches.
result Model-based learning is impossible with finite model classes but feasible with relaxed conditions.
Model-based machine learning improves communication systems.
problem Improving symbol detection in communication receivers.
method Review and comparison of model-based and deep learning approaches, focusing on deep unfolding and DNN-aided hybrid algorithms.
result Different strategies of conventional deep architectures and hybrid algorithms show advantages and drawbacks.
Inference-Time Scaling can be extended to domains prone to systematic failure using intrinsic statistics.
problem Scaling inference time in domains prone to systematic failure
method Intrinsic Selection (iS), Intrinsic Particle Filtering (iPF), and Particle Distillation (dPF)
result Intrinsic Selection improves engineering design selection by 20% and pass@1 by 6.1 points on average.
Review of variable selection methods for model-based clustering.
problem Dealing with high-dimensional data in model-based clustering.
method Variable selection techniques to facilitate interpretation.
result Summary and illustration of existing methods.
A new method reduces compounding errors in model-based reinforcement learning.
problem Compounding errors in long horizon predictions from model-based reinforcement learning.
method Maximum Entropy Model Rollouts (MEMR) with non-uniform sampling and prioritized experience replay.
result Significantly reduces computation requirements compared to other model-based methods.
STOVE predicts convincing physical behavior in videos.
problem Learning models from videos with objects and interactions.
method Compositional state-space model combining image and dynamics models.
result Predicts videos with convincing physical behavior over hundreds of timesteps.
A framework schedules hyperparameters for model-based reinforcement learning, improving performance.
problem Inadequate scheduling of hyperparameters in model-based reinforcement learning.
method Theoretical analysis and AutoMBPO framework to automatically schedule real data ratio and other hyperparameters.
result Training with hyperparameters scheduled by AutoMBPO significantly improves performance.
A new multi-step model improves model-based reinforcement learning efficiency.
problem Expensive environmental interaction in reinforcement learning.
method Proposes a multi-step model for predicting action sequences with variable length.
result Multi-step model outperforms one-step model in preliminary tests.
DEVI uses neural nets to learn model-based meta-reinforcement learning.
problem Transfer learning for model-based reinforcement learning with high-dimensional states.
method End-to-end training of a deep neural network to learn a similarity metric for model-based Q-learning.
result DEVI achieves transfer learning even for high-dimensional state spaces.
Combines model-free Q-ensembles and model-based approaches for improved exploration.
problem Improving exploration strategies in reinforcement learning.
method Integrates model-free Q-ensembles and model-based trajectory memory approaches.
result Model-based trajectory memory combined with Q-ensembles outperforms Q-ensembles alone.
The paper shows how to learn the geometric structure of data manifolds using probabilistic methods.
problem Learning the differential geometric structure of data manifolds embedded in high-dimensional spaces.
method Probabilistic methods with proper regularizations to recover the geometric structure.
result Probabilistic methods can recover the differential geometric structure of data manifolds.
The study compares reinforcement learning models and finds model-based approaches superior for complex MDPs.
problem Complexity of optimal Q-functions and policies in MDPs exceeds dynamics, hindering model-free methods.
method Theoretical analysis and empirical testing of neural network expressivity for policies, Q-functions, and dynamics.
result Model-based planning yields better policies for complex MDPs, improving performance on MuJoCo tasks.
Greedy policies in model-based RL achieve tight regret bounds without full planning.
problem Achieving efficient RL algorithms in MDP settings.
method Using greedy policies for 1-step planning in model-based RL.
result Greedy policies achieve i l d e O ( H S A T ) ilde{\mathcal{O}}(\sqrt{HSAT}) i l d e O ( H S A T ) regret bounds.