Monte Carlo Tree Search improves financial derivative hedging efficiency.
problem Optimizing pricing and hedging of derivative contracts in incomplete markets.
method Integrates tree search techniques with Reinforcement Learning for optimal control problems.
result Monte Carlo Tree Search outperforms Q-learning in sample efficiency and learning speed. TSN improves sparse signal recovery with less complexity.
problem Sparse regression problem of recovering sparse signals from measurements.
method Tree search algorithm driven by deep neural network with pruning.
result TSN outperforms conventional methods in various sensing matrices.
A new algorithm, Regular Tree Search, tackles non-convex simulation optimization problems.
problem Non-convex objective functions in simulation optimization.
method Integrates adaptive sampling with recursive partitioning of the search space.
result Proves global convergence and reliably identifies the global optimum.
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
Paper uses CMAB to improve NAS efficiency and accuracy.
problem Improving efficiency and accuracy of NAS for DNNs.
method Formulated NAS as CMAB, used Nested Monte-Carlo Search.
result Discovered cell structure achieves comparable accuracy to state-of-the-art, 20x faster.
Bayesian optimization improves Monte-Carlo tree search for better state value estimation.
problem Slow convergence in Monte-Carlo tree search due to averaging in backpropagation.
method Softmax MCTS and Monotone MCTS, using Bayesian optimization with Gaussian process prior.
result Our framework outperforms previous methods in computer Go.
This paper improves self-play learning in games by manipulating experience distributions.
problem Improving self-play learning in games through better experience sampling.
method Three approaches: weighted sampling, Prioritized Experience Replay, and diversifying trajectories.
result Major improvements in early training performance in some games, minor improvements overall.
Bayesian methods improve drug discovery experiment design.
problem Optimizing drug screening experiments in high-dimensional data.
method Bayesian inference and optimisation with upper confidence bound algorithms, Thompson sampling, and sparse tree search.
result Sparse tree search techniques outperform other methods in drug toxicity screening.
A new algorithm improves sample complexity for thresholding in Monte Carlo Tree Search.
problem Determining if the root node value of a tree is at least a given threshold.
method Developed a δ-correct sequential sampling algorithm based on the Track-and-Stop strategy.
result Ratio-based modification of D-Tracking strategy reduces sample complexity and computational cost.
DR-MCTS improves decision quality and sample efficiency in complex environments.
problem Improving decision quality and sample efficiency in complex environments.
method Integrates Doubly Robust off-policy estimation into Monte Carlo Tree Search (MCTS).
result DR-MCTS achieves superior performance in Tic-Tac-Toe and VirtualHome tasks.
Paper shows MCTS approximates policy optimization, proposing an improved variant.
problem Improving AI performance through better MCTS algorithms.
method Shows MCTS approximates policy optimization problem, proposes a new algorithm.
result Proposed algorithm reliably outperforms original AlphaZero in multiple domains.
Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success. Usually, the lookahead policies are implemented with specific planning methods such as Monte Carlo Tree Search (e.g. in AlphaZero). Referring to the planning problem as tree search, a reasonable…
Deep reinforcement learning has been successfully applied to several visual-input tasks using model-free methods. In this paper, we propose a model-based approach that combines learning a DNN-based transition model with Monte Carlo tree search to solve a block-placing task in Minecraft. Our learned transition model pre…
Parallelizes MCTS for continuous domains using leaf and root parallelization.
problem Solving challenging tasks in continuous domains using MCTS.
method Extends existing parallelization strategies to continuous domains, focusing on leaf and root parallelization.
result Proposes two final selection strategies for continuous states in root parallelization.
Tree-search algorithm optimizes noisy black-box functions with multi-fidelity queries.
problem Optimizing noisy functions with expensive evaluations and low-cost approximations.
method Tree-like hierarchical partitions and multi-fidelity bandit tree-search algorithm.
result Simple regret bounds for the proposed algorithm.
AutoML uses MCTS to optimize machine learning algorithms and hyperparameters.
problem Optimizing machine learning algorithms and hyperparameters efficiently.
method MCTS-based approach for hybrid optimization of machine learning portfolios.
result Mosaic outperforms Auto-Sklearn on OpenML 100 benchmark and Scikit-learn portfolio.
Combines MCTS and neural networks for efficient multi-period financial planning.
problem Solving multi-period financial planning models with high transaction costs and regime switching.
method Integrates Monte Carlo Tree Search with deep neural networks, using UTC and lookup search.
result Combined approach outperforms individual methods, solving complex models.
A hybrid model for Bayesian optimization handles mixed variables using MCTS for categorical and GP for continuous.
problem Optimizing functions with mixed variable types (continuous, integer, categorical).
method Merges MCTS for categorical and GP for continuous variables, integrates UCTS search strategy, and dynamically selects kernels.
result Hybrid models outperform traditional methods in Bayesian optimization.
Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
New algorithm identifies optimal subtrees in fixed-budget tree search.
problem Identifying optimal subtrees in fixed-budget Monte Carlo Tree Search.
method ε-agnostic algorithm for max-min action identification.
result Misidentification probability decays exponentially with sample size.
New approach handles stochastic and partially-observable environments using discrete autoencoders and Monte Carlo tree search.
problem Challenges in planning for stochastic and partially-observable environments.
method Uses discrete autoencoders and a stochastic variant of Monte Carlo tree search.
result Significantly outperforms MuZero on stochastic chess and scales to DeepMind Lab.
PCTS optimizes noisy, delayed, multi-fidelity feedbacks in black-box optimization.
problem Optimizing unknown functions with noisy, delayed, and multi-fidelity feedbacks.
method ProCrastinated Tree Search (PCTS) with DUCB1 and DUCBV algorithms.
result PCTS achieves better regret bounds for delayed, noisy, and multi-fidelity feedbacks.
Algorithm finds best mixture of training datasets for improved validation performance.
problem Learning from mixture distributions with covariate shift.
method Combines SGD with optimistic tree search and model re-use over mixture space.
result Proves simple regret guarantees for recovering optimal mixture.
We present an extension of Monte Carlo Tree Search (MCTS) that strongly increases its efficiency for trees with asymmetry and/or loops. Asymmetric termination of search trees introduces a type of uncertainty for which the standard upper confidence bound (UCB) formula does not account. Our first algorithm (MCTS-T), whic…
A new Chinese Checkers agent combines heuristics, MCTS, and deep RL.
problem Challenging state-space and unbounded depth in Chinese Checkers.
method Combines heuristics, Monte Carlo Tree Search, and deep reinforcement learning.
result Competent agent reaches human player level.
A reinforcement learning framework combining value function and tree search planner for strategic and tactical decisions.
problem Strategic and tactical decision-making in discrete environments.
method Combines value function and tree search planner, using uncertainty modeling and risk measurement.
result Improves performance and learning speed on hard exploration environments.
Deep learning synthesizes diverse solutions for NP-hard problems.
problem Finding optimal solutions for NP-hard problems.
method Graph convolutional network trained to estimate solution likelihood; guided tree search.
result Substantial performance improvement over recent deep learning work.
Active Reinforcement Learning (ARL) is a twist on RL where the agent observes reward information only if it pays a cost. This subtle change makes exploration substantially more challenging. Powerful principles in RL like optimism, Thompson sampling, and random exploration do not help with ARL. We relate ARL in tabular …
RiskMiner discovers formulaic alphas using MCTS for better performance.
problem Mining formulaic alphas without considering structural information and alpha correlations.
method Formulates alpha mining as an MDP and solves it with a risk-seeking MCTS.
result Our method outperforms state-of-the-art benchmarks and achieves the most profitable results.
While many recent advances in deep reinforcement learning (RL) rely on model-free methods, model-based approaches remain an alluring prospect for their potential to exploit unsupervised data to learn environment model. In this work, we provide an extensive study on the design of deep generative models for RL environmen…
WU-UCT parallelizes MCTS with linear speedup and limited performance loss.
problem Challenges in parallelizing Monte Carlo Tree Search (MCTS) due to its sequential nature.
method Introduces unobserved samples to track incomplete simulations and modify UCT tree policy.
result Achieves linear speedup and only limited performance loss with increasing parallel workers.
Recent advances in bandit tools and techniques for sequential learning are steadily enabling new applications and are promising the resolution of a range of challenging related problems. We study the game tree search problem, where the goal is to quickly identify the optimal move in a given game tree by sequentially sa…
New analysis improves Monte Carlo Tree Search for reinforcement learning.
problem Improving Monte Carlo Tree Search for reinforcement learning with finite simulations.
method Established polynomial concentration property of regret for non-stationary MABs, leading to a new UCB with polynomial bonus term.
result MCTS with polynomial bonus term requires nearly optimal sample size for learning value functions.
New algorithm reduces sample complexity for planning in MDPs.
problem Planning in MDPs with unknown transitions.
method MDP-GapE, a trajectory-based MCTS algorithm.
result Proves upper bound on sample complexity in terms of sub-optimality gaps.
Quantum algorithm speeds up MIP solving by a near-quadratic factor.
problem Solving Mixed Integer Programs (MIPs) efficiently.
method Incremental-Quantum-Branch-and-Bound algorithm combining quantum speedup with classical search heuristics.
result Universal near-quadratic speedup over classical Branch-and-Bound algorithms.
Monte-Carlo Tree Search (MCTS) methods are drawing great interest after yielding breakthrough results in computer Go. This paper proposes a Bayesian approach to MCTS that is inspired by distributionfree approaches such as UCT [13], yet significantly differs in important respects. The Bayesian framework allows potential…
Improved sample efficiency in reinforcement learning with action guidance.
problem Sample inefficiency in deep reinforcement learning with sparse, delayed, and deceptive rewards.
method Integrating a non-expert demonstrator (e.g., MCTS) into asynchronous distributed deep reinforcement learning.
result Our methods learn faster and converge to better policies on a game.
HAVER improves error bounds for estimating the largest mean in machine learning tasks.
problem Estimating the largest mean among multiple distributions.
method Proposes HAVER, a novel algorithm for maximum mean estimation.
result HAVER achieves better error bounds than the oracle in many cases.
CAPITAL algorithm identifies optimal patient subgroups for better treatment.
problem Identify maximum number of patients benefiting from better treatment.
method Constrained Policy Tree Search (CAPITAL) algorithm to find optimal subgroup selection rule (SSR).
result Maximizes the number of patients with enhanced treatment effects.
LA-MCTS learns search space partition for black-box optimization using Monte Carlo Tree Search.
problem High-dimensional black-box optimization challenges.
method LA-MCTS recursively splits search space into regions with high/low function values, learns nonlinear partition and local models online.
result LA-MCTS achieves strong performance in black-box optimization and reinforcement learning benchmarks, especially for high-dimensional problems.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
In the context of tree-search stochastic planning algorithms where a generative model is available, we consider on-line planning algorithms building trees in order to recommend an action. We investigate the question of avoiding re-planning in subsequent decision steps by directly using sub-trees as action recommender. …
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
KataGo accelerates Go self-play learning by 50x.
problem Efficiently learning in large state spaces like Go.
method Improved AlphaZero process and architecture.
result 50x reduction in computation time.
PGS uses neural networks to improve policies online without search trees.
problem Limited scalability of Monte Carlo Tree Search (MCTS) for high branching factor games.
method Adapts a neural network simulation policy via policy gradient updates, avoiding search trees.
result PGS achieves comparable performance to MCTS and defeats strong Hex agents.
Adaptive stress testing for autonomous vehicles identifies failure scenarios using reinforcement learning.
problem Identifying potential failure scenarios in autonomous vehicle decision-making systems.
method Formulated as a Markov decision process, used reinforcement learning (DRL) to find likely failure scenarios.
result Deep Reinforcement Learning (DRL) finds more likely failure scenarios with fewer simulator calls than Monte Carlo Tree Search (MCTS).
Planning problems are among the most important and well-studied problems in artificial intelligence. They are most typically solved by tree search algorithms that simulate ahead into the future, evaluate future states, and back-up those evaluations to the root of a search tree. Among these algorithms, Monte-Carlo tree …
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.