Unified framework for lower bounds in interactive decision making.
problem Challenges in interactive decision making, especially bandits and reinforcement learning.
method Interactive Fano method and Fractional Covering Number.
result Unified characterization of learnability for stochastic bandit problems and tight lower bounds for interactive decision making.
Study on information evolution in interactive decision making using multi-armed bandits.
problem Understanding information dynamics in interactive decision making.
method Stochastic multi-armed bandit problem, focusing on optimal arm with a fixed margin.
result Distinct growth phases in mutual information, showing decoupling between success probability and information gain.
New complexity measure for interactive learning reduces regret to near-optimal levels.
problem Challenges in sample-efficient, adaptive learning algorithms for interactive decision making.
method Introduces the Decision-Estimation Coefficient and the Estimation-to-Decisions (E2D) principle.
result Unified algorithm design principle E2D achieves optimal sample-efficient learning.
Improved model-free reinforcement learning with decision-estimation coefficient.
problem Interactive decision making, including structured bandits and reinforcement learning.
method Combining Estimation-to-Decisions with optimistic estimation to achieve better regret bounds.
result Regret bounds for model-free reinforcement learning with value function approximation.
Interactive machine learning improves learning efficiency with user input.
problem Expensive, time-consuming, or risky acquisition of labeled data and decision-making.
method Develops new algorithms for active learning, sequential decision making, and model selection under partial feedback.
result First efficient algorithms achieving exponential label savings and independent of action space size.
In order to drive safely and efficiently under merging scenarios, autonomous vehicles should be aware of their surroundings and make decisions by interacting with other road participants. Moreover, different strategies should be made when the autonomous vehicle is interacting with drivers having different level of coop…
Lecture notes on reinforcement learning using statistical methods.
problem Addressing the exploration-exploitation dilemma in decision making.
method Frequentist and Bayesian approaches, function approximation, neural networks.
result Unified framework for decision making and estimation.
Study on multi-agent decision making complexity, showing sample efficiency gaps.
problem Understanding sample efficiency in multi-agent decision making.
method General framework for interactive decision making, focusing on equilibrium computation.
result No 'reasonable' complexity measure can close gaps between single and multiple agents.
New AI assistant for power grid operators simplifies complex decision-making.
problem Complexity and uncertainty in power grid operations.
method Unified human-machine interface and AI integration.
result Development of a new assistant framework for power grid operators.
The influence of additional information on the decision making of agents, who are interacting members of a society, is analyzed within the mathematical framework based on the use of quantum probabilities. The introduction of social interactions, which influence the decisions of individual agents, leads to a generalizat…
Framework for robust decision making in changing environments with privacy constraints.
problem Interactive decision making in changing environments with constraints.
method Hybrid Decision Making with Structured Observations (hybrid DMSO) framework, local differentially private decision making, query-based learning, robust and smooth decision making.
result Strong connections and bounds derived for DEC, SQ dimension, local minimax complexity, learnability, and joint differential privacy.
DECE visualizes machine learning decisions with counterfactual explanations.
problem Making machine learning models transparent and explainable.
method Interactive visualization system supporting counterfactual explanations at instance- and subgroup-levels.
result DECE enables users to explore and understand machine learning model decisions.
The paper tackles AI advice giving by considering adherence levels and defer options.
problem Inadequate consideration of human adherence to AI recommendations.
method Sequential decision-making model that considers adherence levels and incorporates a defer option.
result Specialized learning algorithms provide better convergence and empirical performance.
Platform combines RL and language models to study narrative influence on AI decisions.
problem Understanding how narrative elements shape AI decision-making.
method Dual-system architecture with reinforcement learning and language model integration.
result Initial experiments show narrative frameworks can influence AI decision-making.
In important applications involving multi-task networks with multiple objectives, agents in the network need to decide between these multiple objectives and reach an agreement about which single objective to follow for the network. In this work we propose a distributed decision-making algorithm. The agents are assumed …
Investigates safe decision-making in interactive environments.
problem Learning the best safe decision in real-time systems.
method Reduces to a constrained linear bandits problem, proposes adaptive experimental design-based algorithm.
result First results on best-arm identification in linear bandits with safety constraints.
In many machine learning applications, there are multiple decision-makers involved, both automated and human. The interaction between these agents often goes unaddressed in algorithmic development. In this work, we explore a simple version of this interaction with a two-stage framework containing an automated model and…
Efficient algorithm for learning from indirect feedback in complex decision-making scenarios.
problem Learning from indirect feedback in realistic scenarios with personalized mechanisms.
method IGW algorithm for policy optimization, extending reward-estimator construction from single-step to multi-step.
result Achieves sublinear regret guarantee for contextual episodic MDPs with personalized feedback.
In Interactive Machine Learning (IML), we iteratively make decisions and obtain noisy observations of an unknown function. While IML methods, e.g., Bayesian optimization and active learning, have been successful in applications, on real-world systems they must provably avoid unsafe decisions. To this end, safe IML algo…
New DEC variant improves sample complexity bounds in decision making.
problem Understanding sample-efficient learning guarantees in decision making.
method Introducing a new Constrained Decision-Estimation Coefficient (DEC) and using it to derive improved lower bounds.
result New lower bounds improve upon prior work in three aspects: expectation, global applicability, and improper reference models.
Interactive IL beats BC by state-wise annotation cost.
problem Behavior Cloning struggles with annotation cost in sequential decision making.
method Proved Stagger and Warm Stagger algorithms to outperform BC.
result Interactive and hybrid IL methods outperform BC with state-wise annotation.
Framework for responsible LLM deployment with human involvement and decentralized technologies.
problem Challenges in deploying LLMs for high-stakes decisions, including data security and accountability.
method Interactive human involvement through multiple iterations, decentralized technologies, and automated auditing.
result Enhanced security and accountability in LLM deployment for financial decisions.
New algorithm achieves instance-optimality in decision making.
problem Develop adaptive algorithms for interactive decision making.
method Introduce Allocation-Estimation Coefficient (AEC) and develop AE2 algorithm. result First non-asymptotic instance-optimal performance guarantees.
New framework converts offline to online estimation using black-box offline estimators.
problem Convert offline estimation algorithms to online estimation algorithms.
method Oracle-Efficient Online Estimation (OEOE) framework.
result Achieves near-optimal online estimation error via black-box offline estimators.
Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.
problem Balancing exploration and exploitation in online decision making.
method Introducing a batch Thompson Sampling framework for stochastic multi-arm bandit and linear contextual bandit problems.
result Achieves asymptotic regret bound with O(logT) batch queries, significantly reducing interactions. RLMM extends psychometric models to larger tasks.
problem Sequential process data from interactive assessments are not well handled by conventional models.
method RLMM decouples person-level choice sensitivity from task-level value representation through a shared parametric action-value function.
result RLMM achieves higher estimation accuracy and lower runtime than MDP-MM in peg-solitaire simulations and AQUALAB gameplay logs.
ACS is an interactive framework for model-free selection with guaranteed error control.
problem Model-free selection with rigorous error control.
method Adaptive conformal selection with human-in-the-loop data exploration and new information incorporation.
result ACS provides concrete selection algorithms for various goals, including model update/selection, diversified selection, and incorporating new data.
We propose and investigate a class of new algorithms for sequential decision making that interacts with \textit{a batch of users} simultaneously instead of \textit{a user} at each decision epoch. This type of batch models is motivated by interactive marketing and clinical trial, where a group of people are treated simu…
Bayesian framework for SSP problem learns optimal strategy through interactions.
problem Sequential decision-making in stochastic shortest path problems.
method Develops a Bayesian framework to learn optimal action-value function Q∗ through interactions, avoiding unrealistic assumptions. result Demonstrates data efficiency and uncertainty quantification compared to other methods.
Time series data that are not measured at regular intervals are commonly discretized as a preprocessing step. For example, data about customer arrival times might be simplified by summing the number of arrivals within hourly intervals, which produces a discrete-time time series that is easier to model. In this abstract…
MIDAS learns to adaptively control other cars in urban driving scenarios.
problem Autonomous vehicles need to interact with other agents on the road.
method Reinforcement learning with attention mechanism to handle multiple agents.
result MIDAS policies are adaptive and robust to external changes.
Variable selection for optimal treatment regime in a clinical trial or an observational study is getting more attention. Most existing variable selection techniques focused on selecting variables that are important for prediction, therefore some variables that are poor in prediction but are critical for decision-making…
The paper tackles decision making problems with funnel structure in email marketing campaigns.
problem Decision making challenges in systems with funnel structure, where fewer observations are received from deeper layers.
method Formulated as a contextual bandit with funnel structure and developed a multi-task learning algorithm.
result Our algorithms offer significant improvement over previous methods in email marketing campaigns.
Bayesian principles improve agentic AI decision-making.
problem Decision-making under uncertainty in agentic AI systems.
method Bayesian decision theory applied to the orchestration layer of agentic AI.
result Bayesian principles enhance agentic AI's ability to make decisions under uncertainty.
A new MDP with Bandits approach for sequential decision making in linear-flow scenarios.
problem Sequential decision making with limited feedback in a linear-flow context.
method Formulated as an MDP with Bandits, using Thompson sampling for action selection and exact dynamic programming for allocation.
result The proposed MDP with Bandits algorithm outperforms other methods in sequential decision making.
Unified framework for efficient RL in MDP, POMDP, and PSR.
problem Sample efficient reinforcement learning under various decision-making frameworks.
method Proposed GEC complexity measure and a posterior sampling algorithm.
result Sample efficient learning achieved with sublinear regret bound in terms of GEC.
The paper addresses private and Byzantine-proof cooperative decision-making in multi-agent systems.
problem Designing algorithms for multi-agent decision-making that are private and resilient to faulty agents.
method Upper-confidence bound algorithms for stochastic bandit problems under privacy and Byzantine conditions.
result Optimal regret achieved in both private and Byzantine-tolerant settings.
Explainable AI improves human decision accuracy but does not enhance it significantly.
problem Improving human decision-making through explainable AI.
method Comparing human decision accuracy with and without AI predictions, including or excluding explanations.
result Providing AI predictions improves human decision accuracy, but explanations do not significantly enhance it.
Interactive EMA combines multiple explainability methods to improve model understanding.
problem Isolated explanations of machine learning models lead to misunderstandings and wrong reasoning.
method Interactive EMA (IEMA) combines multiple explainability methods sequentially.
result Interactive EMA increases the performance and confidence of human decision making.
When an AI system interacts with multiple users, it frequently needs to make allocation decisions. For instance, a virtual agent decides whom to pay attention to in a group setting, or a factory robot selects a worker to deliver a part. Demonstrating fairness in decision making is essential for such systems to be broad…
Bayesian method helps decision-makers find preferred solutions in multi-objective optimization.
problem Identifying preferred solutions from the Pareto set in multi-objective optimization problems.
method Bayesian model to estimate decision-maker's utility function based on pairwise comparisons, guided by a principled elicitation strategy.
result Superior performance in finding high-utility solutions with a small number of queries.
Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics. This paper offers a consolidated presentation of a theory of bounded rationality based on i…
This review examines bandit problems in AI using statistical methods.
problem Sequential decision-making under uncertainty in AI environments.
method Foundational models, concentration inequalities, minimax regret bounds, frequentist and Bayesian algorithms, K-armed contextual bandits, SCAB, functional data analysis.
result Exploration-exploitation trade-offs and regret analyses in various bandit problems.
We study dynamic allocation problems for discrete time multi-armed bandits under uncertainty, based on the the theory of nonlinear expectations. We show that, under strong independence of the bandits and with some relaxation in the definition of optimality, a Gittins allocation index gives optimal choices. This involve…
KGRL uses reinforcement learning with knowledge graphs for better interactive recommendation.
problem Achieving responsiveness and accuracy in dynamic user-item interactions.
method KGRL combines reinforcement learning and knowledge graphs, using a local knowledge network and attention mechanism.
result KGRL outperforms state-of-the-art methods in simulated and real-world environments.
This paper proposes a web-based visual graph analytics platform for interactive graph mining, visualization, and real-time exploration of networks. GraphVis is fast, intuitive, and flexible, combining interactive visualizations with analytic techniques to reveal important patterns and insights for sense making, reasoni…
Proposes a new model for context-dependent decision-making.
problem Constant preference parameters in decision models are too rigid.
method Introduces Context-aware Bayesian mixed multinomial logit model using neural networks.
result Models context-dependent intra-respondent heterogeneity effectively.
The common pipeline in autonomous driving systems is highly modular and includes a perception component which extracts lists of surrounding objects and passes these lists to a high-level decision component. In this case, leveraging the benefits of deep reinforcement learning for high-level decision making requires spec…