Efficient algorithms identify true hypothesis from many options with minimal actions.
problem Identifying true hypothesis from a large set of options with minimal actions.
method Greedy approximation algorithms for active sequential hypothesis testing.
result First approximation guarantees for ASHT, independent of the number of hypotheses.
Diffusion models mimic human actions in sequential tasks.
problem Cloning human behavior in dynamic environments is challenging.
method Adapting diffusion models to handle stochastic, multimodal, and correlated actions.
result Diffusion models closely replicate human behavior in robotic and gaming tasks.
New algorithm adapts to changing action sets in sequential decision making.
problem Adapting to a changing action set in sequential decision making problems.
method Breaks the problem into inferring action structure and optimizing policy.
result Demonstrates efficiency on real-world lifelong learning problems.
The study examines robust decision-making in volatile financial markets, finding action robustness is more impactful than uncertainty tolerance.
problem Sequential decision making in high-frequency markets under evolving uncertainty.
method Analyzes two dimensions of robustness: uncertainty tolerance and action robustness, using simulations and empirical evidence.
result Action robustness has a larger impact on profitability than uncertainty tolerance, and excessive robustness can reduce profitability in illiquid markets.
Develops methods for finding counterfactual explanations in sequential decision making.
problem Finding counterfactual explanations for sequential decision making processes.
method Formal characterization of sequential actions and states using Markov decision processes and Gumbel-Max structural causal model. Introduces a polynomial time algorithm based on dynamic programming.
result Algorithm finds optimal counterfactual explanations for sequential decision making.
SLHF uses sequential game theory to optimize preferences from human feedback.
problem Optimizing preferences from human feedback in sequential settings.
method SLHF frames the problem as a sequential-move game between Leader and Follower, decomposing the optimization into refinement and adversarial optimization.
result SLHF achieves strong alignment across diverse preference datasets and scales to large models.
Investigates sequential problems on graph structures and large action spaces.
problem Sequential decision-making on graph structures and large action spaces.
method Spectral bandits, side observations, influence maximization, kernel bandits, polymatroid bandits, function optimization, infinitely many-arms bandits.
result Contributions to graph and structured bandits.
AR CI framework handles complex confounders and sequential actions.
problem Low-dimensional confounders and singleton actions in causal inference.
method Sequencification to transform data into sequences, enabling CI with complex confounders and sequential actions.
result AR model can estimate multiple causal quantities using a single model, simplifying inference and improving outcome prediction.
Algorithm maximizes rewards with a budget and giving up option.
problem Sequential decision-making with stochastic rewards and resource consumption.
method Upper Confidence Bound (UCB) algorithm for maximizing cumulative reward.
result Logarithmic regret bound with improved dependence on problem parameters.
Study personalizes user experience to maximize rewards with patience budget.
problem Maximizing rewards for a platform while respecting user patience.
method Proposes bandit algorithms for sequential choice with feedback models.
result Upper and lower bounds on regret of order O(N2/3) and Ω(N2/3). Study batch learning in linear bandits with context, achieving near-optimal performance.
problem Sequential batch learning in linear contextual bandits with finite actions.
method Established regret bounds and provided algorithms for two settings: arbitrary contexts and i.i.d. contexts.
result Regret upper bound nearly matches lower bound, showing polynomial and logarithmic batch requirements.
The paper extends a learning heuristic to high-dimensional contexts, reducing the risk of unusual actions.
problem Sequential learning problems in high dimensions, especially in dynamic pricing and auctions.
method Introducing a conservative εt-greedy rule that limits the adoption of new actions to a focused set of promising actions. result Reasonable bounds for cumulative regret and improved regret bound for conservative version compared to non-conservative.
Paper improves action selection for accurate parameter estimation in linear bandits.
problem Best action identification in stochastic linear bandits with fixed confidence constraints.
method Designs a sequential adaptive policy to estimate underlying parameter efficiently.
result The designed policy achieves the same estimation error scaling as a lower bound.
New algorithms handle unpredictable actions in sequential learning.
problem Learning with unreliable composite actions in online optimization.
method Follow-The-Perturbed-Leader method with Counting Asleep Times loss estimation.
result Significant improvement in performance guarantees for sleeping bandit problem.
E-valuator converts verifier scores into reliable decision rules.
problem Ensuring the correctness of agent trajectories based on heuristic scores.
method Sequential hypothesis testing framework for online monitoring of agent trajectories.
result E-valuator provides better false alarm rate control and statistical power than other strategies.
New method uses SMC for Bayesian MABs with time-varying, nonlinear rewards.
problem Learning optimal actions in dynamic, non-stationary environments.
method Sequential Monte Carlo (SMC) for estimating statistics of unknown reward distributions.
result SMC-based Bayesian MAB policies achieve good regret performance in non-stationary, nonlinear reward scenarios.
Recently, it has been shown how sampling actions from the predictive distribution over the optimal action-sometimes called Thompson sampling-can be applied to solve sequential adaptive control problems, when the optimal policy is known for each possible environment. The predictive distribution can then be constructed b…
Paper proposes a new HMM approach for better action recognition.
problem Capturing complex temporal dependency patterns in skeleton-based actions.
method Introduces a hierarchical HMM with a latent variable layer for dynamic inference.
result Proposed approach effectively models complex sequential data and handles missing values.
Paper tackles adaptive submodularity in sequential decision making.
problem Maximizing adaptive submodular functions with limited adaptive rounds.
method Proposes an efficient semi-adaptive policy with logarithmic adaptive rounds.
result Achieves an almost tight 1−1/e−ε approximation guarantee. A new MDP with Bandits approach for sequential decision making in linear-flow scenarios.
problem Sequential decision making with limited feedback in a linear-flow context.
method Formulated as an MDP with Bandits, using Thompson sampling for action selection and exact dynamic programming for allocation.
result The proposed MDP with Bandits algorithm outperforms other methods in sequential decision making.
Interactive machine learning improves learning efficiency with user input.
problem Expensive, time-consuming, or risky acquisition of labeled data and decision-making.
method Develops new algorithms for active learning, sequential decision making, and model selection under partial feedback.
result First efficient algorithms achieving exponential label savings and independent of action space size.
Algorithm learns to play against unknown opponents in sequential games.
problem Designing strategies for a learner to interact with an unknown opponent in repeated sequential games.
method Kernel-based regularity assumptions and a novel algorithm combining bilevel optimization and online learning.
result Algorithm achieves sublinear regret guarantees and is effective in specific game settings.
Diffusion approximations optimize sequential experimentation for uncertain parameters.
problem Maximizing reward from unknown parameter Θ with delayed action.
method Bayesian sequential experimentation framework, dynamic programming, diffusion asymptotics.
result Derives diffusion approximation for optimal experimentation strategy.
IDS algorithm optimizes sequential decisions in various monitoring settings.
problem Optimizing sequential decisions in complex monitoring scenarios.
method Information-directed sampling (IDS) algorithm for linear partial monitoring.
result IDS achieves nearly worst-case rate optimality in finite-action games.
AI system narrows human decision options for better outcomes.
problem Improving human decision-making in sequential tasks.
method Developed a decision support system using a pre-trained AI agent to limit human action choices.
result Participants outperformed AI and solo play in a wildfire mitigation game.
A new model predicts conversion rates by analyzing post-click actions.
problem Challenges in predicting conversion rates due to sample selection bias and data sparsity.
method Post-click behavior decomposition and multi-task learning.
result The model effectively addresses sample selection bias and data sparsity issues.
Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the "Imagination-based Planner", the first m…
This paper solves the normalizability crisis in sequential inference by introducing bounded information geometry.
problem Structural failure in standard sequential inference architectures when dealing with extreme outliers.
method Non-parametric field actions and bounded information geometry to truncate infinite tails of spatial distributions.
result Empirical benchmarks across three domains show robust estimation without infinite-tailed distributional assumptions.
Bayesian framework for learning optimal action-value function in MDPs.
problem Uncertainty quantification in MDPs for optimal decision-making strategies.
method Full Bayesian framework including modelling, inference, and decision-making.
result Demonstrates exploration benefits of posterior sampling in MDPs.
New method optimizes portfolios by dynamically integrating ESG constraints.
problem Static ESG scores mismatch sequential portfolio decisions.
method MACF-X, a family of adapters that learns ESG costs from multimodal evidence.
result Reduces tail ESG budget pressure while maintaining financial performance.
We study an original problem of pure exploration in a strategic bandit model motivated by Monte Carlo Tree Search. It consists in identifying the best action in a game, when the player may sample random outcomes of sequentially chosen pairs of actions. We propose two strategies for the fixed-confidence setting: Maximin…
Study on special warped product manifolds with Einstein metrics.
problem Characterizing Einstein metrics in specific warped product manifolds.
method Derive general formulas and prove existence of solutions for invariant families.
result Existence of families of solutions invariant under transformations for positive constant Ricci curvature.
Deep RL model learns 2.5D fighting games with height ambiguity.
problem Ambiguity in character height/depth and sequential action orders in 2.5D fighting games.
method Modified A3C network with Recurrent Info network for combo skill observation.
result Successfully learned and played Little Fighter 2 (LF2) 2.5D fighting game.
The paper tackles fair sequential decision making with biased linear bandit feedback.
problem Fair sequential decision making with biased linear bandit feedback.
method Phased elimination algorithm to correct unfair evaluations, establishing upper bounds on regret.
result The worst-case regret is smaller than O(κ∗1/3log(T)1/3T2/3). New reinforcement learning framework for adapting to new actions.
problem Making reinforcement learning agents adaptable to new actions without retraining.
method Two-stage framework: infer action representations first, then train a flexible policy.
result Agents can make decisions from new action sets without retraining.
Proposes using frequent sequences to improve sequential recommendation models.
problem Combining user history and recent actions for personalized recommendations.
method Uses frequent sequences to identify relevant parts of user history, embedding items based on preferences and dynamics in a unified metric model.
result Outperforms state-of-the-art methods, especially on sparse datasets.
Budgeted hyper-parameter tuning algorithm improves performance.
problem Optimizing hyper-parameters with resource constraints.
method Sequential decision making, Bayesian model, action-value function.
result Superior performance across various budgets.
Deep RL learns to construct objects from 2D images by avoiding brick overlaps.
problem Learning to construct objects from partial 2D images with constraints.
method Combination of reinforcement learning and action validity prediction.
result Successfully constructs unseen objects from 2D images.
FAVAE learns disentangled representations from sequential data.
problem Learning disentangled and interpretable representations from sequential data.
method FAVAE uses the information bottleneck principle without supervision.
result FAVAE can disentangle multiple dynamic factors.
Reinforcement learning aids decision-making in economics and finance.
problem Optimal decision-making in dynamic, uncertain environments.
method Reinforcement learning algorithms to learn optimal policies.
result Deep learning enhances solving complex behavioral problems.
Novel ML approach solves complex warehouse routing problem.
problem Efficiently routing pickers in mixed-shelves warehouses.
method Hierarchical and parallel multi-agent reinforcement learning.
result State-of-the-art performance in solution quality and inference speed.
The paper improves recommendation models by considering user interactions with recommended items.
problem Improving next item prediction in recommendation systems.
method Extending RNN framework with a recommendation action module and state-action fusion module.
result Improved performance on next item prediction compared to baselines.
Improved Bayesian regret bound for linear Thompson sampling with general distributions.
problem Proving an improved Bayesian regret bound for linear Thompson sampling with general distributions.
method Generalized elliptical potential lemma for non-Gaussian noise and prior distributions.
result Minimax optimal regret bound for changing action sets with general prior and noise distributions.
A textbook on machine learning explaining patterns, predictions, and actions.
problem Understanding patterns in data for predictions and actions.
method Explains foundations of decision making, representation, optimization, and generalization.
result Equips readers with tools to reason about actions and their consequences.
The paper tackles finding optimal treatment sequences in continuous state spaces.
problem Finding counterfactually optimal action sequences in continuous state spaces.
method Formalizes the problem using finite horizon Markov decision processes and structural causal models. Develops a search method based on the A* algorithm.
result The method can find optimal action sequences in polynomial time under certain conditions.
New RL algorithms handle stochastic action sets, addressing divergence issues.
problem Handling stochastic action sets in reinforcement learning.
method Developed new policy gradient algorithms with variance reduction techniques.
result Proved conditions for convergence of new algorithms.
PopArt efficiently solves sparse linear bandits with tighter recovery guarantees.
problem Sparse linear bandits where rewards depend on a few covariates.
method PopArt: a simple, computationally efficient sparse linear estimation method.
result Improved regret bounds compared to state-of-the-art algorithms.
Co-training improves sequential decision-making policies from multiple views.
problem Learning policies in settings with multiple state-action representations.
method Inspired by co-training for classification, we present a co-training framework for sequential decision making.
result Our framework improves upon learning from a single view alone.