A new Chinese Checkers agent combines heuristics, MCTS, and deep RL.
problem Challenging state-space and unbounded depth in Chinese Checkers.
method Combines heuristics, Monte Carlo Tree Search, and deep reinforcement learning.
result Competent agent reaches human player level.
New algorithms identify optimal moves in game trees.
problem Identifying the best move in a game tree quickly.
method Best arm identification procedure applied to depth-one summaries.
result New algorithms outperform existing methods.
This paper improves self-play learning in games by manipulating experience distributions.
problem Improving self-play learning in games through better experience sampling.
method Three approaches: weighted sampling, Prioritized Experience Replay, and diversifying trajectories.
result Major improvements in early training performance in some games, minor improvements overall.
Monte Carlo Tree Search improves financial derivative hedging efficiency.
problem Optimizing pricing and hedging of derivative contracts in incomplete markets.
method Integrates tree search techniques with Reinforcement Learning for optimal control problems.
result Monte Carlo Tree Search outperforms Q-learning in sample efficiency and learning speed. The paper uses linear function approximators to bias MCTS for general games.
problem Improving MCTS playing strength for general games.
method Using linear function approximators with local features for self-play training.
result Significantly improved playing strength in multiple board games.
PGS uses neural networks to improve policies online without search trees.
problem Limited scalability of Monte Carlo Tree Search (MCTS) for high branching factor games.
method Adapts a neural network simulation policy via policy gradient updates, avoiding search trees.
result PGS achieves comparable performance to MCTS and defeats strong Hex agents.
Deep RL fails on deceptive games, revealing algorithm weaknesses.
problem Characterizing and understanding failures of deep reinforcement learning.
method Testing A2C on four deceptive games using a game framework.
result Deep RL fails in specific ways that differ from planning-based agents.
Improved MCTS for asymmetric trees and loops.
problem Efficiency issues in MCTS for asymmetric trees and loops.
method Extended MCTS-T algorithm with modifications for asymmetric termination and loops, using a modified UCB formula.
result Significantly improved efficiency in asymmetric domains and loops compared to standard MCTS.
This paper studies effective parallelization of MCTS for computer games.
problem Computational expense of MCTS due to many rollouts.
method Examined parallel MCTS performance, identified necessary conditions for desirable parallelization, and proposed BU-UCT algorithm.
result Theoretical foundation for effective parallel MCTS, with BU-UCT outperforming baselines.
Alpha Zero adapts to continuous action spaces for real-world tasks.
problem Real-world reinforcement learning domains often have continuous action spaces.
method Interleaves tree search and deep learning, extending Alpha Zero for continuous action spaces.
result Preliminary experiments on the Pendulum task show feasibility of the approach.
A RL approach finds Nash equilibrium for turn-based zero-sum games.
problem Finding Nash equilibrium in two-player turn-based zero-sum games.
method EIS method combining exploration, policy improvement, and supervised learning.
result EIS method finds an ε-approximate value function of Nash equilibrium in O(ε^(-(d+4))) steps.
The paper proposes a new policy training objective to reduce exploration in self-play.
problem Training policies to mimic MCTS search behavior can lead to excessive exploration.
method Derive a policy gradient expression using MCTS value estimates to reduce exploration.
result Empirically evaluated policies show reduced exploration compared to MCTS-based training.
Ranked Reward algorithm improves bin packing performance.
problem Improving reinforcement learning for combinatorial optimization.
method Ranking rewards from self-play to create a relative performance metric.
result Ranked Reward algorithm outperforms other methods on bin packing problems.
New method uses MCTS only at test time for faster game learning.
problem High computational demands in game learning.
method Combines MCTS with TD learning agents for faster, reproducible agents.
result First learning-from-scratch agent to beat Edax up to level 7.
Paper approximates deep neural network robustness using game theory.
problem Safety concerns due to adversarial examples in deep nets.
method Discretize input space, approximate problems as games, use Monte Carlo tree search and A* algorithms.
result Approximation has provable guarantees and competitive performance.
Deep RL algorithms tested on diverse video games.
problem Benchmarking AI on diverse video games.
method Interface GVGAI to OpenAI Gym, test deep RL algorithms.
result Characterized performance of deep RL algorithms on GVGAI games.
Deep neural network learns to play Big 2, a 4-player imperfect information game, outperforming amateurs.
problem Training a neural network to play a complex, imperfect information game with multiple players.
method Self-play reinforcement learning using Proximal Policy Optimization.
result Deep neural network trained via self-play reaches performance level surpassing amateur players.
MBExplainer provides explanations for models combining graph embeddings and tabular features.
problem Explaining models using a mix of graph embeddings and tabular features.
method Model-agnostic approach using Shapley values and Monte Carlo Tree Search.
result MBExplainer efficiently finds human-readable explanations for model predictions.
MuZero learns models to master complex games without domain knowledge.
problem Mastering complex, real-world domains with unknown dynamics.
method Combining tree-based search with learned models.
result Achieves superhuman performance in diverse domains.
We study an original problem of pure exploration in a strategic bandit model motivated by Monte Carlo Tree Search. It consists in identifying the best action in a game, when the player may sample random outcomes of sequentially chosen pairs of actions. We propose two strategies for the fixed-confidence setting: Maximin…
WU-UCT parallelizes MCTS with linear speedup and limited performance loss.
problem Challenges in parallelizing Monte Carlo Tree Search (MCTS) due to its sequential nature.
method Introduces unobserved samples to track incomplete simulations and modify UCT tree policy.
result Achieves linear speedup and only limited performance loss with increasing parallel workers.
Improved sample efficiency in reinforcement learning with action guidance.
problem Sample inefficiency in deep reinforcement learning with sparse, delayed, and deceptive rewards.
method Integrating a non-expert demonstrator (e.g., MCTS) into asynchronous distributed deep reinforcement learning.
result Our methods learn faster and converge to better policies on a game.
A game theory study on optimal hiding and searching strategies in discrete locations.
problem Optimal hiding and searching strategies in a two-person zero-sum game between a hider and a searcher.
method Proved the existence of optimal strategies, developed an algorithm to compute them, and compared with a simple strategy.
result Optimal hiding strategy involves hiding in each location with nonzero probability, and optimal searching strategy can be constructed with up to n simple sequences.
Paper examines financial engineering problems and introduces AlphaZero for better replication strategies.
problem Replication portfolio construction in incomplete markets with non-convex constraints.
method Introduces AlphaZero-based system to compare with deep hedging method.
result AlphaZero outperforms deep hedging in non-convex environments, finding near-optimal strategies.
A new algorithm, Regular Tree Search, tackles non-convex simulation optimization problems.
problem Non-convex objective functions in simulation optimization.
method Integrates adaptive sampling with recursive partitioning of the search space.
result Proves global convergence and reliably identifies the global optimum.
Paper develops a new method for game options in local volatility models.
problem Optimal stopping in one-dimensional diffusion with general coefficients.
method Skorokhod embedding to construct recombining tree approximations.
result Construct nearly optimal stopping times with convergence rates.
Paper uses CMAB to improve NAS efficiency and accuracy.
problem Improving efficiency and accuracy of NAS for DNNs.
method Formulated NAS as CMAB, used Nested Monte-Carlo Search.
result Discovered cell structure achieves comparable accuracy to state-of-the-art, 20x faster.
Optimizes tree models for better beam search performance.
problem Beam search causes retrieval performance deterioration in tree models.
method Develops Bayes optimality and calibration under beam search, proposes a novel algorithm for optimal tree model learning.
result Eliminates the training-testing discrepancy in tree models.
Enhances tree search methods in reinforcement learning for better convergence.
problem Non-contractive nature of standard tree search methods in reinforcement learning.
method Proposes a new method to back up values at the root using the optimal tree path return.
result Establishes a γh-contracting procedure leading to better convergence rates. A neural network approach to Monte-Carlo tree search.
problem Improving tree search algorithms for planning problems.
method Learning neural network architecture to control search parameters.
result The learned search algorithm outperformed traditional MCTS.
TSN improves sparse signal recovery with less complexity.
problem Sparse regression problem of recovering sparse signals from measurements.
method Tree search algorithm driven by deep neural network with pruning.
result TSN outperforms conventional methods in various sensing matrices.
Automates optimizer design for diverse tasks efficiently.
problem Scalability, generability, and sample efficiency in optimizer search.
method Re-arranged optimizer space into a super-tree, applying tree traversal methods.
result Discover optimizers with only 128 evaluations, surpassing human-designed and prior methods.
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.
New algorithm avoids re-planning in tree-search algorithms, reducing suboptimal actions.
problem Avoiding re-planning in tree-search algorithms to reduce suboptimal actions.
method A new algorithm that decides at each step whether to re-plan or use a sub-tree, based on sub-tree statistics.
result The probability of selecting a suboptimal action converges to zero and decays logarithmically.
Optimal decision trees learned via dynamic programming and search.
problem Finding the best classification trees that are both accurate and compact.
method Dynamic programming and search techniques.
result Our algorithm learns optimal classification trees significantly faster and scales better than existing methods.
Generative adversarial tree search fails to outperform DQN in Atari environments.
problem Improving deep reinforcement learning algorithms for Atari environments.
method Proposes generative adversarial tree search (GATS) that learns the environment model and implements Monte Carlo tree search (MCTS) on the learned model.
result GATS fails to outperform DQN, despite theoretical analysis showing potential benefits.
Cover trees speed up MRI fingerprint recovery by reducing computation.
problem Efficiently reconstructing MRI fingerprint signals from compressed sensing data.
method Use cover trees for fast approximate nearest neighbor searches in IPG algorithm.
result Achieves 2-3 orders of magnitude reduction in computations.
Bayesian optimization improves Monte-Carlo tree search for better state value estimation.
problem Slow convergence in Monte-Carlo tree search due to averaging in backpropagation.
method Softmax MCTS and Monotone MCTS, using Bayesian optimization with Gaussian process prior.
result Our framework outperforms previous methods in computer Go.
SMAC method optimizes tree-boosting hyperparameters best.
problem Optimizing hyperparameters for tree-boosting to improve model accuracy.
method Compared and evaluated various hyperparameter optimization methods.
result SMAC method outperforms other methods for hyperparameter tuning.
A hybrid model for Bayesian optimization handles mixed variables using MCTS for categorical and GP for continuous.
problem Optimizing functions with mixed variable types (continuous, integer, categorical).
method Merges MCTS for categorical and GP for continuous variables, integrates UCTS search strategy, and dynamically selects kernels.
result Hybrid models outperform traditional methods in Bayesian optimization.
Transformer learns to search through reinforcement learning, mimicking DFS.
problem Understanding how transformers learn search capabilities in RL.
method Two-head transformer, depth-wise curriculum, discounted returns.
result Transformer policy generalizes depth and prioritizes high-probability branches.
A new algorithm improves sample complexity for thresholding in Monte Carlo Tree Search.
problem Determining if the root node value of a tree is at least a given threshold.
method Developed a δ-correct sequential sampling algorithm based on the Track-and-Stop strategy.
result Ratio-based modification of D-Tracking strategy reduces sample complexity and computational cost.
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
Automatically tunes hyperparameters for faster approximate nearest neighbor search.
problem Tuning hyperparameters for efficient approximate nearest neighbor search is slow and impractical.
method Proposes an algorithm using randomized space-partitioning trees to automatically tune hyperparameters.
result Significantly faster than existing approaches and competitive in query time.
Proposes a method to interpret linguistic data models using parse trees and least-squares scores.
problem Interpreting trained classification models in linguistic data sets.
method Assigns least-squares based importance scores to words in a sentence using syntactic constituency structure and relates them to the Banzhaf value in coalitional game theory.
result Demonstrates the effectiveness of the proposed method in aiding interpretability and diagnostics for language models.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
Combines MCTS and neural networks for efficient multi-period financial planning.
problem Solving multi-period financial planning models with high transaction costs and regime switching.
method Integrates Monte Carlo Tree Search with deep neural networks, using UTC and lookup search.
result Combined approach outperforms individual methods, solving complex models.
Paper shows MCTS approximates policy optimization, proposing an improved variant.
problem Improving AI performance through better MCTS algorithms.
method Shows MCTS approximates policy optimization problem, proposes a new algorithm.
result Proposed algorithm reliably outperforms original AlphaZero in multiple domains.