An evolutionary algorithm (EA) is developed as an alternative to the EM algorithm for parameter estimation in model-based clustering. This EA facilitates a different search of the fitness landscape, i.e., the likelihood surface, utilizing both crossover and mutation. Furthermore, this EA represents an efficient approac…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Model-based Algorithms”
New hybrid RL algorithm outperforms model-free and model-based methods.
problem Improving reinforcement learning algorithms for MDPs.
method Combines model-free and model-based learning, with a PAC analysis.
result Outperforms both model-free and model-based methods in most cases.
Model-based machine learning improves communication systems.
problem Improving symbol detection in communication receivers.
method Review and comparison of model-based and deep learning approaches, focusing on deep unfolding and DNN-aided hybrid algorithms.
result Different strategies of conventional deep architectures and hybrid algorithms show advantages and drawbacks.
Adaptive discretization improves model-based RL in large spaces.
problem Efficient model-based reinforcement learning in large state-action spaces.
method Optimistic one-step value iteration with adaptive discretization.
result Adaptive discretization leads to better performance and lower memory usage.
Model-based reinforcement learning algorithms tend to achieve higher sample efficiency than model-free methods. However, due to the inevitable errors of learned models, model-based methods struggle to achieve the same asymptotic performance as model-free methods. In this paper, We propose a Policy Optimization method w…
A framework schedules hyperparameters for model-based reinforcement learning, improving performance.
problem Inadequate scheduling of hyperparameters in model-based reinforcement learning.
method Theoretical analysis and AutoMBPO framework to automatically schedule real data ratio and other hyperparameters.
result Training with hyperparameters scheduled by AutoMBPO significantly improves performance.
Designing effective model-based reinforcement learning algorithms is difficult because the ease of data generation must be weighed against the bias of model-generated data. In this paper, we study the role of model usage in policy optimization both theoretically and empirically. We first formulate and analyze a model-b…
Maximum Entropy Model Rollouts: Fast Model Based Policy Optimization without Compounding Errorscs.LG
A new method reduces compounding errors in model-based reinforcement learning.
problem Compounding errors in long horizon predictions from model-based reinforcement learning.
method Maximum Entropy Model Rollouts (MEMR) with non-uniform sampling and prioritized experience replay.
result Significantly reduces computation requirements compared to other model-based methods.
Unified reinforcement learning methods using hybrid inference.
problem Combining model-based and model-free reinforcement learning approaches.
method Control as Hybrid Inference (CHI) framework.
result CHI algorithm balances model-based and model-free learning.
VMBPO optimizes model and policy jointly using variational lower-bound.
problem Data efficiency in RL with biased simulated data.
method Formulate variational objective function, use EM, iteratively improve model and policy.
result VMBPO is more sample-efficient and robust than model-free algorithms.
Unified framework for policy improvement in RL with benefits in data efficiency and computation.
problem Improving data efficiency and computation in reinforcement learning for continuous control.
method Local, regularized policy improvement with tree search for continuous action spaces.
result Improves data efficiency and reduces wall-clock time in high-dimensional domains.
Unified framework for model-based RL with sample complexity guarantees.
problem Designing efficient posterior sampling methods for model-based RL.
method Optimistic posterior sampling, Hellinger distance reduction, data likelihood measurement.
result Unified algorithms with state-of-the-art sample complexity guarantees.
Proposes H-UCRL for efficient model-based RL with sublinear regret.
problem Greedy policy exploration in model-based RL ignores epistemic uncertainty.
method Reparameterizes plausible models, hallucinates control, augments input space, solves with greedy planners.
result H-UCRL achieves provably sublinear regret for Gaussian Process models.
Improved model-based reinforcement learning for multi-agent Markov games.
problem Suboptimal sample complexity for model-based algorithms in multi-agent reinforcement learning.
method Optimistic Nash Value Iteration (Nash-VI) for two-player zero-sum Markov games.
result First model-based algorithm matching information-theoretic lower bound with improved sample complexity.
Model-based learning algorithms have been shown to use experience efficiently when learning to solve Markov Decision Processes (MDPs) with finite state and action spaces. However, their high computational cost due to repeatedly solving an internal model inhibits their use in large-scale problems. We propose a method ba…
The design of symbol detectors in digital communication systems has traditionally relied on statistical channel models that describe the relation between the transmitted symbols and the observed signal at the receiver. Here we review a data-driven framework to symbol detection design which combines machine learning (ML…
MoMA improves model-based RL by using unrestricted policy classes.
problem Limited sample efficiency and generalizability in model-based offline RL.
method Model-based mirror ascent algorithm with general function approximations.
result Theoretical guarantees and practical implementation of MoMA.
The paper uses model-based trees to create interpretable surrogate models for complex machine learning models.
problem Interpreting complex machine learning models.
method Using model-based trees to partition feature space and create interpretable models.
result Model-based trees generate optimal surrogate models that balance interpretability and performance.
A new deep learning model improves phase retrieval performance.
problem Recovering signals from phaseless measurements.
method Hybrid model-based data-driven deep architecture (Unfolded Phase Retrieval, UPR).
result Significant improvement in phase retrieval performance.
FOCUS improves offline RL by incorporating causal structure into world-models.
problem Learning effective policies from historical data without interaction.
method FOCUS proposes a practical algorithm that learns and leverages causal structure in offline RL.
result FOCUS outperforms plain model-based offline RL algorithms and other causal model-based RL algorithms.
MOPO optimizes offline RL by penalizing dynamics uncertainty.
problem Learning policies from offline data with distributional shift.
method Modify model-based RL to avoid distributional shift.
result MOPO outperforms model-free and standard model-based RL.
MAGE optimizes policies using action gradients from model-based learning.
problem Lack of direct gradient information from critics in actor-critic methods.
method Model-based actor-critic algorithm that learns action-value gradient.
result MAGE outperforms model-free and model-based baselines on continuous control tasks.
Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL. However, the theoretical understanding of such methods has been rather limited. This paper introduces a novel algorithmic framework for designing and analyzing model-based RL algo…
Significant progress has been made in the area of model-based reinforcement learning. State-of-the-art algorithms are now able to match the asymptotic performance of model-free methods while being significantly more data efficient. However, this success has come at a price: state-of-the-art model-based methods require …
Machine Learning Risk Modelsq-fin.PM
We give an explicit algorithm and source code for constructing risk models based on machine learning techniques. The resultant covariance matrices are not factor models. Based on empirical backtests, we compare the performance of these machine learning risk models to other constructions, including statistical risk mode…
This study tackles adversarial corruption in model-based reinforcement learning.
problem Adversarial corruption in model-based reinforcement learning.
method Maximum likelihood estimation (MLE) approach for learning transition model in both online and offline settings.
result Proves a regret of for CR-OMLE and a suboptimality of for CR-PMLE.
Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.
problem Model-based RL for episodic MDPs with unknown transition probabilities.
method Upper confidence bound-based algorithm for exploration-exploitation balance.
result Achieves regret bound for multinomial logistic models.
New algorithms accelerate model-based optimization for stochastic problems.
problem Optimizing model-based stochastic optimization problems efficiently.
method Proposed new model-based algorithms with acceleration and minibatch techniques.
result Non-asymptotic convergence guarantees with linear speedup in minibatch size.
A simulation study of cluster search algorithms in data set generated by Gaussian mixture modelscs.LG
This study evaluates cluster search algorithms using Gaussian mixture models.
problem Determining the optimal number of clusters in data sets generated by Gaussian mixture models.
method Examined centroid- and model-based cluster search algorithms in various cases.
result Model-based algorithms are more robust to cluster overlap and covariance type than centroid-based methods.
Model-based clustering approaches concern the paradigm of exploratory data analysis relying on the finite mixture model to automatically find a latent structure governing observed data. They are one of the most popular and successful approaches in cluster analysis. The mixture density estimation is generally performed …
Improved model-based reinforcement learning for interactive dialogue tasks reduces sample needs and improves performance.
problem Limited data and high sample cost in interactive dialogue systems.
method Model-based actor-critic approach with an environment model and planner.
result 70 times fewer samples required compared to baseline model-free algorithm, with 2x better asymptotic performance.
We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, -functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal -func…
State-of-the-art efficient model-based Reinforcement Learning (RL) algorithms typically act by iteratively solving empirical models, i.e., by performing \emph{full-planning} on Markov Decision Processes (MDPs) built by the gathered experience. In this paper, we focus on model-based RL in the finite-state finite-horizon…
Model-free reinforcement learning methods such as the Proximal Policy Optimization algorithm (PPO) have successfully applied in complex decision-making problems such as Atari games. However, these methods suffer from high variances and high sample complexity. On the other hand, model-based reinforcement learning method…
FLEX optimizes exploration for nonlinear systems with minimal data.
problem Efficient exploration of unknown nonlinear systems with limited data.
method FLEX uses optimal experimental design to maximize information gain.
result FLEX outperforms other methods in nonlinear environments and control tasks.
PAC Bounds for Imitation and Model-based Batch Learning of Contextual Markov Decision Processescs.LG
Study batch reinforcement learning methods for personalized medical treatments.
problem Batch reinforcement learning for personalized medical treatments.
method Direct policy learning and model-based learning approaches.
result Model-based learning is impossible with finite model classes but feasible with relaxed conditions.
New model learning objective improves continuous control tasks.
problem Challenges in solving continuous control tasks using model-based reinforcement learning.
method Derived a novel value-aware model learning objective and identified and addressed stale value estimates issue.
result Value-aware objectives can be successfully deployed in solving continuous control tasks without tuning hyper-parameters.
The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms, which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. Among the few proposed approaches, the recent…
We revisit residual algorithms in both model-free and model-based reinforcement learning settings. We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in the DeepMind Control Suite benchmark. Moreover, we…
Optimistic RL algorithms are simplified for deep RL with competitive performance.
problem Achieving accurate optimism in model-based RL for large-scale problems.
method Interpreting scalable optimistic model-based algorithms as solving a tractable noise augmented MDP.
result Competitive regret bound of for Gaussian noise augmentation.
Proposes model-based robust deep learning to handle natural variation in data.
problem Deep learning's fragility to natural variation in data.
method Develops model-based robust training algorithms using deep generative models to learn natural variation.
result Deep neural networks trained with model-based algorithms outperform standard and norm-bounded robust algorithms.
A new reinforcement learning method uses model derivatives to improve policy optimization.
problem Improving sample efficiency and performance in model-based reinforcement learning.
method Constructs an actor-critic algorithm that uses the pathwise derivative of the learned model and policy.
result Consistently more sample efficient and matches model-free algorithms' asymptotic performance.
This work benchmarks diffusion model-based samplers for Bayesian inverse problems.
problem Optimizing diffusion models for uncertainty quantification in Bayesian inverse problems.
method Introduces three benchmark problems and a unified framework for diffusion model-based posterior sampling.
result Provides insights into strengths and limitations of diffusion model-based samplers.
The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms, which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. However, the current algorithms lack an effec…
UPR hybrid model improves phase retrieval performance.
problem Recovering signals from phase-less measurements.
method Model-based data-driven deep architecture (UPR).
result UPR shows potential in improving phase retrieval.
Paper proves efficiency of MARL with transformers, addressing agent complexity.
problem Theoretical understanding of MARL with many agents and limited relational reasoning.
method Set Transformer for relational reasoning, model-free and model-based MARL algorithms.
result Provable efficiency of MARL algorithms, suboptimality gaps independent of number of agents.
We present a new deep meta reinforcement learner, which we call Deep Episodic Value Iteration (DEVI). DEVI uses a deep neural network to learn a similarity metric for a non-parametric model-based reinforcement learning algorithm. Our model is trained end-to-end via back-propagation. Despite being trained using the mode…
GAMI-Tree uses model-based trees to fit low-order fANOVA models.
problem Fitting interpretable fANOVA models with low-order interactions.
method GAMI-Tree uses model-based trees and a new interaction filtering method.
result GAMI-Tree outperforms EBM and GAMI-Net in predictive performance and interpretability.