Efficient algorithms identify true hypothesis from many options with minimal actions.
problem Identifying true hypothesis from a large set of options with minimal actions.
method Greedy approximation algorithms for active sequential hypothesis testing.
result First approximation guarantees for ASHT, independent of the number of hypotheses.
Diffusion models mimic human actions in sequential tasks.
problem Cloning human behavior in dynamic environments is challenging.
method Adapting diffusion models to handle stochastic, multimodal, and correlated actions.
result Diffusion models closely replicate human behavior in robotic and gaming tasks.
The study examines robust decision-making in volatile financial markets, finding action robustness is more impactful than uncertainty tolerance.
problem Sequential decision making in high-frequency markets under evolving uncertainty.
method Analyzes two dimensions of robustness: uncertainty tolerance and action robustness, using simulations and empirical evidence.
result Action robustness has a larger impact on profitability than uncertainty tolerance, and excessive robustness can reduce profitability in illiquid markets.
Develops methods for finding counterfactual explanations in sequential decision making.
problem Finding counterfactual explanations for sequential decision making processes.
method Formal characterization of sequential actions and states using Markov decision processes and Gumbel-Max structural causal model. Introduces a polynomial time algorithm based on dynamic programming.
result Algorithm finds optimal counterfactual explanations for sequential decision making.
SLHF uses sequential game theory to optimize preferences from human feedback.
problem Optimizing preferences from human feedback in sequential settings.
method SLHF frames the problem as a sequential-move game between Leader and Follower, decomposing the optimization into refinement and adversarial optimization.
result SLHF achieves strong alignment across diverse preference datasets and scales to large models.
Investigates sequential problems on graph structures and large action spaces.
problem Sequential decision-making on graph structures and large action spaces.
method Spectral bandits, side observations, influence maximization, kernel bandits, polymatroid bandits, function optimization, infinitely many-arms bandits.
result Contributions to graph and structured bandits.
AR CI framework handles complex confounders and sequential actions.
problem Low-dimensional confounders and singleton actions in causal inference.
method Sequencification to transform data into sequences, enabling CI with complex confounders and sequential actions.
result AR model can estimate multiple causal quantities using a single model, simplifying inference and improving outcome prediction.
Algorithm maximizes rewards with a budget and giving up option.
problem Sequential decision-making with stochastic rewards and resource consumption.
method Upper Confidence Bound (UCB) algorithm for maximizing cumulative reward.
result Logarithmic regret bound with improved dependence on problem parameters.
Study personalizes user experience to maximize rewards with patience budget.
problem Maximizing rewards for a platform while respecting user patience.
method Proposes bandit algorithms for sequential choice with feedback models.
result Upper and lower bounds on regret of order O(N2/3) and Ω(N2/3). Study batch learning in linear bandits with context, achieving near-optimal performance.
problem Sequential batch learning in linear contextual bandits with finite actions.
method Established regret bounds and provided algorithms for two settings: arbitrary contexts and i.i.d. contexts.
result Regret upper bound nearly matches lower bound, showing polynomial and logarithmic batch requirements.
The paper extends a learning heuristic to high-dimensional contexts, reducing the risk of unusual actions.
problem Sequential learning problems in high dimensions, especially in dynamic pricing and auctions.
method Introducing a conservative εt-greedy rule that limits the adoption of new actions to a focused set of promising actions. result Reasonable bounds for cumulative regret and improved regret bound for conservative version compared to non-conservative.
New algorithms handle unpredictable actions in sequential learning.
problem Learning with unreliable composite actions in online optimization.
method Follow-The-Perturbed-Leader method with Counting Asleep Times loss estimation.
result Significant improvement in performance guarantees for sleeping bandit problem.
E-valuator converts verifier scores into reliable decision rules.
problem Ensuring the correctness of agent trajectories based on heuristic scores.
method Sequential hypothesis testing framework for online monitoring of agent trajectories.
result E-valuator provides better false alarm rate control and statistical power than other strategies.
Recently, it has been shown how sampling actions from the predictive distribution over the optimal action-sometimes called Thompson sampling-can be applied to solve sequential adaptive control problems, when the optimal policy is known for each possible environment. The predictive distribution can then be constructed b…
Paper proposes a new HMM approach for better action recognition.
problem Capturing complex temporal dependency patterns in skeleton-based actions.
method Introduces a hierarchical HMM with a latent variable layer for dynamic inference.
result Proposed approach effectively models complex sequential data and handles missing values.
A new MDP with Bandits approach for sequential decision making in linear-flow scenarios.
problem Sequential decision making with limited feedback in a linear-flow context.
method Formulated as an MDP with Bandits, using Thompson sampling for action selection and exact dynamic programming for allocation.
result The proposed MDP with Bandits algorithm outperforms other methods in sequential decision making.
Algorithm learns to play against unknown opponents in sequential games.
problem Designing strategies for a learner to interact with an unknown opponent in repeated sequential games.
method Kernel-based regularity assumptions and a novel algorithm combining bilevel optimization and online learning.
result Algorithm achieves sublinear regret guarantees and is effective in specific game settings.
Interactive machine learning improves learning efficiency with user input.
problem Expensive, time-consuming, or risky acquisition of labeled data and decision-making.
method Develops new algorithms for active learning, sequential decision making, and model selection under partial feedback.
result First efficient algorithms achieving exponential label savings and independent of action space size.
IDS algorithm optimizes sequential decisions in various monitoring settings.
problem Optimizing sequential decisions in complex monitoring scenarios.
method Information-directed sampling (IDS) algorithm for linear partial monitoring.
result IDS achieves nearly worst-case rate optimality in finite-action games.
Diffusion approximations optimize sequential experimentation for uncertain parameters.
problem Maximizing reward from unknown parameter Θ with delayed action.
method Bayesian sequential experimentation framework, dynamic programming, diffusion asymptotics.
result Derives diffusion approximation for optimal experimentation strategy.
We extend Bayesian multi-armed bandit (MAB) algorithms beyond their original setting by making use of sequential Monte Carlo (SMC) methods. A MAB is a sequential decision making problem where the goal is to learn a policy that maximizes long term payoff, where only the reward of the executed action is observed. In the …
Recommender system, as an essential part of modern e-commerce, consists of two fundamental modules, namely Click-Through Rate (CTR) and Conversion Rate (CVR) prediction. While CVR has a direct impact on the purchasing volume, its prediction is well-known challenging due to the Sample Selection Bias (SSB) and Data Spars…
AI system narrows human decision options for better outcomes.
problem Improving human decision-making in sequential tasks.
method Developed a decision support system using a pre-trained AI agent to limit human action choices.
result Participants outperformed AI and solo play in a wildfire mitigation game.
Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the "Imagination-based Planner", the first m…
This paper solves the normalizability crisis in sequential inference by introducing bounded information geometry.
problem Structural failure in standard sequential inference architectures when dealing with extreme outliers.
method Non-parametric field actions and bounded information geometry to truncate infinite tails of spatial distributions.
result Empirical benchmarks across three domains show robust estimation without infinite-tailed distributional assumptions.
Bayesian framework for learning optimal action-value function in MDPs.
problem Uncertainty quantification in MDPs for optimal decision-making strategies.
method Full Bayesian framework including modelling, inference, and decision-making.
result Demonstrates exploration benefits of posterior sampling in MDPs.
New method optimizes portfolios by dynamically integrating ESG constraints.
problem Static ESG scores mismatch sequential portfolio decisions.
method MACF-X, a family of adapters that learns ESG costs from multimodal evidence.
result Reduces tail ESG budget pressure while maintaining financial performance.
We study an original problem of pure exploration in a strategic bandit model motivated by Monte Carlo Tree Search. It consists in identifying the best action in a game, when the player may sample random outcomes of sequentially chosen pairs of actions. We propose two strategies for the fixed-confidence setting: Maximin…
Study on special warped product manifolds with Einstein metrics.
problem Characterizing Einstein metrics in specific warped product manifolds.
method Derive general formulas and prove existence of solutions for invariant families.
result Existence of families of solutions invariant under transformations for positive constant Ricci curvature.
The paper tackles fair sequential decision making with biased linear bandit feedback.
problem Fair sequential decision making with biased linear bandit feedback.
method Phased elimination algorithm to correct unfair evaluations, establishing upper bounds on regret.
result The worst-case regret is smaller than O(κ∗1/3log(T)1/3T2/3). New reinforcement learning framework for adapting to new actions.
problem Making reinforcement learning agents adaptable to new actions without retraining.
method Two-stage framework: infer action representations first, then train a flexible policy.
result Agents can make decisions from new action sets without retraining.
In many real-world sequential decision making problems, the number of available actions (decisions) can vary over time. While problems like catastrophic forgetting, changing transition dynamics, changing rewards functions, etc. have been well-studied in the lifelong learning literature, the setting where the action set…
Proposes using frequent sequences to improve sequential recommendation models.
problem Combining user history and recent actions for personalized recommendations.
method Uses frequent sequences to identify relevant parts of user history, embedding items based on preferences and dynamics in a unified metric model.
result Outperforms state-of-the-art methods, especially on sparse datasets.
Deep RL learns to construct objects from 2D images by avoiding brick overlaps.
problem Learning to construct objects from partial 2D images with constraints.
method Combination of reinforcement learning and action validity prediction.
result Successfully constructs unseen objects from 2D images.
Reinforcement learning aids decision-making in economics and finance.
problem Optimal decision-making in dynamic, uncertain environments.
method Reinforcement learning algorithms to learn optimal policies.
result Deep learning enhances solving complex behavioral problems.
Novel ML approach solves complex warehouse routing problem.
problem Efficiently routing pickers in mixed-shelves warehouses.
method Hierarchical and parallel multi-agent reinforcement learning.
result State-of-the-art performance in solution quality and inference speed.
A textbook on machine learning explaining patterns, predictions, and actions.
problem Understanding patterns in data for predictions and actions.
method Explains foundations of decision making, representation, optimization, and generalization.
result Equips readers with tools to reason about actions and their consequences.
Improved Bayesian regret bound for linear Thompson sampling with general distributions.
problem Proving an improved Bayesian regret bound for linear Thompson sampling with general distributions.
method Generalized elliptical potential lemma for non-Gaussian noise and prior distributions.
result Minimax optimal regret bound for changing action sets with general prior and noise distributions.
The paper tackles finding optimal treatment sequences in continuous state spaces.
problem Finding counterfactually optimal action sequences in continuous state spaces.
method Formalizes the problem using finite horizon Markov decision processes and structural causal models. Develops a search method based on the A* algorithm.
result The method can find optimal action sequences in polynomial time under certain conditions.
The Markov decision process (MDP) formulation used to model many real-world sequential decision making problems does not efficiently capture the setting where the set of available decisions (actions) at each time step is stochastic. Recently, the stochastic action set Markov decision process (SAS-MDP) formulation has b…
PopArt efficiently solves sparse linear bandits with tighter recovery guarantees.
problem Sparse linear bandits where rewards depend on a few covariates.
method PopArt: a simple, computationally efficient sparse linear estimation method.
result Improved regret bounds compared to state-of-the-art algorithms.
Paper proposes DAC-ML, a cognitive architecture that learns quickly from few episodes.
problem Sample inefficiency in AI learning action policies.
method Incorporates hippocampus-inspired sequential memory system into DAC theory of mind.
result DAC-ML rapidly converges to effective action policies maximizing reward.
Algorithm reduces regret in safe Bayesian optimization with monotonicity constraints.
problem Sequentially maximize unknown function with safety constraints.
method Sequential algorithms using Gaussian processes with safety constraints modeled as monotonicity.
result Sublinear regret achieved for expanding safe region and finding optimal s. Safe actions learned in finite trials, without infinite exploration.
problem Learning safe actions in unknown environments efficiently.
method Defining a handicap metric and using sequential probability ratio test for discarding unsafe actions.
result Achieves constant handicap, discarding unsafe machines with probability one in finite rounds.
Adaptive sequential decision making is one of the central challenges in machine learning and artificial intelligence. In such problems, the goal is to design an interactive policy that plans for an action to take, from a finite set of n actions, given some partial observations. It has been shown that in many applicat…
Bayesian RL tackles uncertainty with deep generative models and sequential samplers.
problem Optimal decision-making in uncertain environments with limited data.
method Bayesian approach using deep generative models and prequential scoring rule for posterior inference. Policy learning via expected Thompson sampling.
result Improves policy learning in high-dimensional parameter spaces and continuous action spaces.
Practical algorithm for contextual bandits with large action spaces.
problem Efficient algorithms for decision making in large, continuous action spaces.
method Uses computational oracles for supervised learning and optimization over the action space.
result Achieves sample complexity, runtime, and memory independent of the size of the action space.
We propose the factorized action variational autoencoder (FAVAE), a state-of-the-art generative model for learning disentangled and interpretable representations from sequential data via the information bottleneck without supervision. The purpose of disentangled representation learning is to obtain interpretable and tr…