Improves RL planning by proposing sub-goals hierarchically.
problem Sequential planning assumption in RL.
method Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS).
result Improves navigation and control tasks.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “sequential planning”
When is Particle Filtering Efficient for Planning in Partially Observed Linear Dynamical Systems?cs.LG
This paper studies when particle filtering is efficient for planning in partially observed systems.
problem The efficiency of particle filtering for planning in partially observed linear dynamical systems.
method Coupling of ideal and approximate sequences to bound particle complexity.
result Polynomially many particles suffice for stable systems to approximate optimal planning.
CriticSMC improves planning efficiency in constrained environments.
problem Planning with hard constraints in dynamic environments.
method Sequential Monte Carlo with learned heuristic factors.
result CriticSMC reduces collision rates with low computational cost.
Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the "Imagination-based Planner", the first m…
New method optimizes experiments under constraints.
problem Adapting BED to dynamic constraints in real-world tasks.
method Offline pre-training of an amortized policy and posterior network with online multi-step lookahead planning.
result Significantly more informative design sequences than existing methods.
This paper presents a unifying framework for reinforcement learning and planning.
problem Sequential decision making in AI, formalized as MDP optimization.
method A unifying algorithmic framework (FRAP) for reinforcement learning and planning.
result Identifies common dimensions in MDP planning and learning algorithms.
We present a representation learning algorithm that learns a low-dimensional latent dynamical system from high-dimensional \textit{sequential} raw data, e.g., video. The framework builds upon recent advances in amortized inference methods that use both an inference network and a refinement procedure to output samples f…
Survey of integrating planning and learning in model-based reinforcement learning.
problem Sequential decision making in AI, formalized as MDP optimization.
method Systematic coverage of dynamics model learning and planning-learning integration.
result Broad conceptual overview of model-based reinforcement learning.
Machine teaching addresses the problem of finding the best training data that can guide a learning algorithm to a target model with minimal effort. In conventional settings, a teacher provides data that are consistent with the true data distribution. However, for sequential learners which actively choose their queries,…
In recent years, deep generative models have been shown to 'imagine' convincing high-dimensional observations such as images, audio, and even video, learning directly from raw data. In this work, we ask how to imagine goal-directed visual plans -- a plausible sequence of observations that transition a dynamical system …
GO-CBED optimizes experiments for specific causal queries, improving efficiency.
problem Efficiently infer causal relationships with limited resources.
method Goal-oriented Bayesian framework that maximizes expected information gain on user-specified causal quantities.
result GO-CBED outperforms existing methods in various causal tasks, especially with limited budgets.
In the wake of the highly electrified future ahead of us, the role of energy storage is crucial wherever distributed generation is abundant, such as in microgrid settings. Given the variety of storage options that are becoming more and more economical, determining which type of storage technology to invest in, along wi…
A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty. We cast POMDP filtering and planning problems as two closely related Sequential Monte Carlo (SMC) processes, one over the real…
Planning methods can solve temporally extended sequential decision making problems by composing simple behaviors. However, planning requires suitable abstractions for the states and transitions, which typically need to be designed by hand. In contrast, model-free reinforcement learning (RL) can acquire behaviors from l…
New model for personalized online advertising with multi-user interaction.
problem Realistic online advertising scenarios with multiple users interacting simultaneously.
method Introduces Multi-User Contextual Cascading Bandit (MCCB) model and proposes UCBBP and AUCBBP algorithms.
result Proves UCBBP and AUCBBP achieve optimal regret bounds for multi-user context.
New method reduces memory usage for Bayesian inverse problems on large grids.
problem Solving large-scale linear inverse problems with Gaussian process priors.
method Implicit representation of posterior covariance matrices, sequential disintegrations of Gaussian measures.
result Significant reduction in uncertainty for high-density regions estimation.
A moment constraint that limits the number of dividends in the optimal dividend problem is suggested. This leads to a new type of time-inconsistent stochastic impulse control problem. First, the optimal solution in the precommitment sense is derived. Second, the problem is formulated as an intrapersonal sequential dyna…
New AI model optimizes personalized care for elderly residents.
problem Limited care workforce impacts health outcomes and quality of life.
method Bandit algorithms for personalized care planning.
result Improves care quality and health outcomes through personalized care planning.
In this paper, we solve the arms exponential exploding issue in multivariate Multi-Armed Bandit (Multivariate-MAB) problem when the arm dimension hierarchy is considered. We propose a framework called path planning (TS-PP) which utilizes decision graph/trees to model arm reward success rate with m-way dimension interac…
Equity-Transformer solves NP-hard min-max routing problems efficiently.
problem Min-max routing problems with multiple agents and large-scale applications.
method Sequential planning approach with Transformer and equitable workload distribution inductive biases.
result Significant runtime and cost reductions in min-max mTSP and min-max mPDP tasks.
Generative ML learns optimal pursuit trajectories in pursuit-evasion games.
problem Optimizing Blue's pursuit trajectory to intercept Red in a game of pursuit-evasion.
method Applying generative machine learning to learn optimal action policies for Blue.
result Generative ML models can learn relevant representations for pursuit-evasion dynamics.
Risk-Control Strategiesq-fin.RM
In this paper, we consider the pricing of derivative products that involve dynamic hedging strategies and payments within the planning horizon. Equity-indexed annuities (EIAs), Guaranteed investment certificate (GIC), American and Barrier options are typical examples of these products. Our exploration involves evaluati…
Paper tackles continual reinforcement learning by forgetting, proposing a planning method with online world models.
problem Catastrophic forgetting in reinforcement learning when learning new tasks.
method Planning with an online world model using model predictive control.
result The proposed FTL Online Agent (OA) learns new tasks without forgetting old skills.
This paper presents a novel nonmyopic adaptive Gaussian process planning (GPP) framework endowed with a general class of Lipschitz continuous reward functions that can unify some active learning/sensing and Bayesian optimization criteria and offer practitioners some flexibility to specify their desired choices for defi…
We study the problem of learning sequential decision-making policies in settings with multiple state-action representations. Such settings naturally arise in many domains, such as planning (e.g., multiple integer programming formulations) and various combinatorial optimization problems (e.g., those with both integer pr…
Pursuit-evasion is a multi-agent sequential decision problem wherein a group of agents known as pursuers coordinate their traversal of a spatial domain to locate an agent trying to evade them. Pursuit evasion problems arise in a number of import application domains including defense and route planning. Learning to opti…
Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics. This paper offers a consolidated presentation of a theory of bounded rationality based on i…
Model-based reinforcement learning is an appealing framework for creating agents that learn, plan, and act in sequential environments. Model-based algorithms typically involve learning a transition model that takes a state and an action and outputs the next state---a one-step model. This model can be composed with itse…
Learning the minimum/maximum mean among a finite set of distributions is a fundamental sub-task in planning, game tree search and reinforcement learning. We formalize this learning task as the problem of sequentially testing how the minimum mean among a finite set of distributions compares to a given threshold. We deve…
Increasing availability of vehicle GPS data has created potentially transformative opportunities for traffic management, route planning and other location-based services. Critical to the utility of the data is their accuracy. Map-matching is the process of improving the accuracy by aligning GPS data with the road netwo…
STOIC improves energy demand forecasting with reliable uncertainty estimates.
problem Accurate point forecasts alone are insufficient for energy systems; reliable uncertainty estimates are needed.
method Integrates graph-based forecasting with tabular foundation models for zero-shot calibration of spatial-temporal residuals.
result STOIC delivers more reliable and robust uncertainty estimates for complex graph-structured energy time series.
Deep neural network learns discrete state abstractions for efficient planning.
problem Efficient sequential decision making in large state spaces.
method Information bottleneck method for learning approximate bisimulations using deep neural encoders and action-conditioned HMM.
result Trained method efficiently plans for unseen goals in multi-goal reinforcement learning.
Model-based approaches bear great promise for decision making of agents interacting with the physical world. In the context of spatial environments, different types of problems such as localisation, mapping, navigation or autonomous exploration are typically adressed with specialised methods, often relying on detailed …
This paper explores the possibility of near-optimally solving multi-agent, multi-task NP-hard planning problems with time-dependent rewards using a learning-based algorithm. In particular, we consider a class of robot/machine scheduling problems called the multi-robot reward collection problem (MRRC). Such MRRC problem…
Bridges uplift modeling and sequential decision-making with online budget allocation.
problem Treatment allocation under budget constraints in digital advertising.
method Budget-Constrained Causal Bandits (BCCB) integrates learning, exploration, and budget pacing.
result Data-efficiency crossover: BCCB operates effectively from the first user, 3-5x lower performance variance.
New risk metric for RL in finance considers time splits of returns.
problem Optimizing financial decisions with a balance between return and risk.
method Developed a new risk metric for reinforcement learning that allows for flexible target levels of rewards over time.
result Proposed risk metric optimizes for arbitrary time splits of returns, improving upon classical risk measures.
New algorithm reduces reinforcement learning complexity, approaching contextual bandits.
problem Episodic reinforcement learning's difficulty compared to contextual bandits.
method Proposes MVP algorithm with a new Bernstein-type bonus for episodic reinforcement learning.
result Achieves near-optimal regret bound of $O\left(\left(\sqrt{SAK} + S^2A
ight) \poly\log \left(SAHK
ight)
ight)$, improving state-of-the-art results.
New method designs fairer transport plans with uncertainty.
problem Designing fair and balanced mass transport plans.
method Hierarchical fully probabilistic design (HFPD) for transport plans.
result Optimal hyperprior for transport plans with uncertain marginals.
Learning Sequential Decisions from Multiple Sources via Group-Robust Markov Decision Processesstat.ME
Paper tackles robust decision-making from multiple sites with shared structure.
problem Learning robust sequential decisions from heterogeneous multi-site datasets.
method Group-Robust MDPs with d-rectangular uncertainty sets, feature-wise worst-case aggregation, and cluster-level pooling.
result Proves suboptimality bound for robust planning policy under robust partial coverage assumption.
A new algorithm learns optimal personalized treatment plans online with low regret.
problem Learning optimal dynamic treatment regimes in an online setting.
method Developed a novel algorithm balancing exploration and exploitation for rate-optimal regret.
result Guaranteed rate-optimal regret for linear transition and reward models.
We aim to reduce the burden of programming and deploying autonomous systems to work in concert with people in time-critical domains, such as military field operations and disaster response. Deployment plans for these operations are frequently negotiated on-the-fly by teams of human planners. A human operator then trans…
Study integrates reliability constraints into generation planning models.
problem Challenges in integrating reliability constraints with generation planning models.
method Leverages a weighted oblique decision tree (WODT) technique to embed reliability verification constraints.
result Demonstrates effectiveness in achieving reliable and optimal planning solutions.
In the modern healthcare system, rapidly expanding costs/complexity, the growing myriad of treatment options, and exploding information streams that often do not effectively reach the front lines hinder the ability to choose optimal treatment decisions over time. The goal in this paper is to develop a general purpose (…
New approach improves black-box planning efficiency by discovering focused macros.
problem Difficulty of deterministic planning increases exponentially with depth.
method Discovering macro-actions with focused effects to improve goal-count heuristics.
result Focused macros dramatically improve black-box planning efficiency.
We propose a framework for general probabilistic multi-step time series regression. Specifically, we exploit the expressiveness and temporal nature of Sequence-to-Sequence Neural Networks (e.g. recurrent and convolutional structures), the nonparametric nature of Quantile Regression and the efficiency of Direct Multi-Ho…
Study motion planning for points avoiding obstacles in a plane.
problem Avoiding collisions for multiple points in a plane with unknown obstacles.
method Algebraic and topological tools for motion planning.
result New topological complexity for planar motion planning.
Selective planning with imperfect models reduces harmful effects of model inadequacy.
problem Harmful effects of using an imperfect model in reinforcement learning.
method Selective planning with heteroscedastic regression to estimate predictive uncertainty from model inadequacy.
result Effective selective planning requires considering both parameter uncertainty and model inadequacy.
CoMPNetX uses neural networks to efficiently solve constrained motion planning problems.
problem Finding collision-free paths on constraint manifolds efficiently.
method Neural generator and discriminator with neural gradients-based projection operator.
result CoMPNetX finds path solutions with high success rates and lower computation times.