The paper proposes a method to learn and leverage contextual preference distributions for better decision-making.
problem Heterogeneous and context-dependent human preferences in decision-making problems.
method A sequential learning-and-optimization pipeline using a bounded-variance score function gradient estimator to train a predictive model mapping contextual features to preference distributions.
result The approach reduces average post-decision surprise by up to 25 times compared to risk-averse baselines in a ridesharing environment.
Study minimax-optimal rates for offline decision-making with function approximation.
problem Statistical complexity of offline decision-making with function approximation.
method Near minimax-optimal rates for stochastic contextual bandits and Markov decision processes, using pseudo-dimension and behavior policy.
result Established performance limits and new characterization of behavior policy.
Novel algorithm reduces feature inclusion in online decision-making.
problem Optimizing decision-making for personalized user experiences with fairness.
method Online Batched Sequential Inclusion (OBSI) algorithm for sequential feature inclusion.
result OBSI outperforms other algorithms in terms of regret, relevance of features, and compute.
The paper addresses statistical inference for online decision-making in a contextual bandit setting.
problem Understanding the performance of reward models in online decision-making with contextual information.
method The paper uses the contextual bandit framework with a linear reward model and the ε \varepsilon ε -greedy policy to address the exploration-exploitation dilemma. It employs the martingale central limit theorem and inverse propensity score weighting to establish asymptotic normality of parameter estimators. result The online ordinary least squares estimator and the online weighted least squares estimator are asymptotically normal, providing insights into the performance of the reward model.
Universal algorithm learns unknown distribution for various decision-making problems.
problem Various statistical measures in contextual sequential decision-making.
method Infinite-dimensional functional regression oracle for cumulative distribution functions.
result Utility regret rate bounded by polynomial decay of eigenvalue sequence.
New algorithm for partially observable contexts in finance.
problem Decision making based on partially observable, correlated market information.
method EMKF-Bandit algorithm integrating system identification, filtering, and bandit algorithms.
result Sub-linear regret under conditions on filtering.
Unified framework for human-like decision making in various sequential tasks.
problem Real-life decision-making involves diverse strategies leading to similar outcomes.
method Two-stream reward processing mechanism for flexible and unified models.
result Framework unified MAB, CB, and RL with comparable performance.
New algorithm estimates treatment effects for more efficient contextual bandits.
problem Contextual bandits struggle with action-independent reward redundancies.
method Reduces contextual bandits to heterogeneous treatment effect estimation.
result Heterogeneous treatment effect estimation leads to more efficient model estimation.
CB-RL solves complex decision-making problems with contextual information and exogenous events.
problem Optimal policy in strategic decision-making problems that depend on environmental configuration and exogenous events.
method Contextual Bilevel Reinforcement Learning (CB-RL) with a stochastic Hyper Policy Gradient Descent (HPGD) algorithm.
result Demonstrated convergence and performance of the HPGD algorithm for reward shaping and tax design.
Aims to improve personalized treatment decisions through Bayesian experimental design.
problem Evaluating and improving personalized treatment decisions in contexts like customer service.
method Model-agnostic Bayesian Experimental Design to efficiently gather data and avoid highly sub-optimal treatments.
result Our method achieves superior performance in evaluating and improving treatment decisions compared to traditional approaches.
SPARKLE handles high-dimensional covariates for online decision-making.
problem Complex reward-covariate relationships in high-dimensional settings.
method SPARKLE uses a sparse additive reward model with doubly penalized estimator and adaptive screening.
result SPARKLE achieves sublinear regret bound logarithmic in covariate dimensionality.
Fairness in AI decisions for users with varying performance.
problem Ensuring fairness in AI decisions for users with different performance levels.
method Contextual Multi-Armed Bandit algorithm with fairness constraints.
result Accounting for user contexts improves fairness in AI decisions.
Study wSAA for contextual decisions, improving uncertainty quantification under computational constraints.
problem Uncertainty quantification limitations in wSAA for contextual stochastic optimization.
method Establish central limit theorems and asymptotic-normality-based confidence intervals for optimal costs.
result Over-optimizing can mitigate misspecification and preserve asymptotic normality, albeit at a slower convergence rate.
Unified framework for sequential decision making using meta-learning surrogate models.
problem Sequential decision making problems in various domains.
method Probabilistic model-based approach with meta-learning for data-efficient adaptation.
result Efficient and general black-box learning approach across different problem domains.
Algorithm improves decision-making with partially observed contexts using pretrained models.
problem Improving decision-making with partially observed contexts in online linear contextual bandits.
method PULSE-UCB algorithm that uses pretrained models trained on auxiliary data to impute missing features.
result Achieves near-optimal performance in i.i.d. context case with Hölder-smooth missing features.
New method for online statistical inference in contextual bandits using SGD.
problem Online decision-making in contextual bandits with statistical inference.
method Weighted stochastic gradient descent for adaptive data collection.
result Asymptotic normality of the parameter estimator with improved efficiency.
Practical algorithm for contextual bandits with large action spaces.
problem Efficient algorithms for decision making in large, continuous action spaces.
method Uses computational oracles for supervised learning and optimization over the action space.
result Achieves sample complexity, runtime, and memory independent of the size of the action space.
Paper tackles infinite action linear bandits with tight regret bounds.
problem Linear contextual bandit with infinite action sets.
method Proves a regret upper bound of O ( d 2 T log T ) i m e s e x t p o l y ( log log T ) O(\sqrt{d^2T\log T}) imes ext{poly}(\log\log T) O ( d 2 T log T ) im ese x t p o l y ( log log T ) . result Upper bound matches previous lower bound of Ω ( d 2 T log T ) Ω(\sqrt{d^2 T\log T}) Ω ( d 2 T log T ) up to iterated logarithmic terms. Human irrationality can improve AI design, study shows.
problem Improving AI by learning from human decision-making biases.
method Developed a novel POMDP model to simulate human decision-making in contextual choice tasks.
result Reinforcement learners can exploit human irrationalities to make better decisions.
Parallelized bandit algorithms speed up decision-making.
problem Sequential exploration limits decision-making speed.
method Developed parallel contextual bandit algorithms.
result Regret nearly identical to sequential algorithms.
Efficiently handles contextual bandits with diffusion models.
problem Challenges in online decision-making with contextual bandits.
method Leverage pre-trained diffusion models as priors to capture action dependencies.
result Developed an algorithm for efficient posterior approximation.
A smart method predicts and optimizes decisions online with resource constraints.
problem Online decision-making with resource constraints.
method Combines prediction and optimization with dual update using mirror descent.
result Regret bounds and convergence rates for general convex feasible regions.
Framework reduces contextual bandit learning to offline regression with near-optimal regret.
problem Efficient learning with large action spaces and complex reward functions.
method Offline Estimation to Decisions (OE2D) algorithm that minimizes regret with near-optimal oracle calls.
result Near-optimal regret for contextual bandits with large action spaces and O ( l o g ( T ) ) O(log(T)) O ( l o g ( T )) offline oracle calls. OE2D framework reduces contextual bandits to offline regression for near-optimal regret.
problem Efficiently learning contextual bandits with large action spaces and complex reward functions.
method Offline Estimation to Decisions (OE2D) algorithm that reduces contextual bandits to offline regression.
result Near-optimal regret for contextual bandits with large action spaces and O ( log T ) O(\log T) O ( log T ) calls to an offline regression oracle. The paper tackles decision making problems with funnel structure in email marketing campaigns.
problem Decision making challenges in systems with funnel structure, where fewer observations are received from deeper layers.
method Formulated as a contextual bandit with funnel structure and developed a multi-task learning algorithm.
result Our algorithms offer significant improvement over previous methods in email marketing campaigns.
New methods improve robust decision-making under uncertainty in off-policy evaluation.
problem Statistical uncertainty and causal considerations in off-policy evaluation.
method Marginal Ratio (MR) estimator, Conformal Off-Policy Prediction (COPP), causal bounds.
result Improved robustness and uncertainty quantification in off-policy decision-making.
A new algorithm balances exploration and exploitation in online decision-making.
problem Balancing exploration and exploitation in online decision-making.
method Proposed C 4 C^4 C 4 -UCB algorithm incorporating conservative mechanism. result Proved n-step upper regret bound for two situations.
Paper proposes a risk-aware decision-making framework for real-world sequential decisions.
problem Real-world sequential decision-making problems often have critical constraints that learning solutions often neglect.
method Actor multi-critic architecture with risk characterization.
result Our approach consistently satisfies system constraints with minimal performance toll.
Study benchmarks contextual bandit algorithms for precision oncology using in vitro data.
problem Designing effective protocols for individual treatment assignment in precision oncology.
method Proposed a benchmark dataset of in vitro drug responses to evaluate contextual bandit algorithms.
result Bayesian bandit algorithms performed better than a rule-based baseline in minimizing regret.
Optimizes decision-making in dueling bandits with contextual features.
problem Identifying the best arm in dueling bandits with contextual features.
method Develops algorithms for minimizing regret in stochastic contextual dueling bandits.
result Proves optimal regret bounds for contextual dueling bandits.
Simple method improves exploration in various decision problems.
problem Improving exploration in sequential decision problems.
method Parameterized Exploration (PE) method that considers time horizon and state of knowledge.
result PE outperforms un-tuned methods in various bandit and decision problem settings.
OTSS learns personalized decision weights from logged decisions and outputs.
problem Learning context-specific decision weights from logged decisions and outputs.
method Output-targeted soft-segmentation model that deploys personalized decision-ready weight vectors.
result OTSS achieves the lowest mean regret in benchmark settings.
A new DRL model for intraday trading incorporating positional context.
problem Neglecting positional context in existing DRL intraday trading strategies.
method Introducing positional features into the state space of a DRL model.
result Significant improvement in profitability and risk-adjusted metrics.
Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.
problem Improving decision-making in contexts with uncertain cost coefficients using partial feedback.
method Unified class of offline learning algorithms for CLO with different types of feedback, using IERM framework.
result Fast-rate regret bound for IERM with partial feedback and misspecified model classes.
Kernel method improves cooperative decision-making among agents.
problem Cooperative multi-agent decision making with contextual information.
method Proposed extsc{Coop-KernelUCB} algorithm for near-optimal per-agent regret.
result Near-optimal bounds on per-agent regret with efficient computation and communication.
The paper addresses contextual optimization problems with feedback, aiming to minimize regret.
problem Contextual optimization with feedback information.
method Characterizing the optimal minimax policy in offline setting and leveraging geometric characterization in online setting to optimize cumulative regret.
result Developed an algorithm yielding logarithmic regret bound in the online setting.
Framework for robust decision making in changing environments with privacy constraints.
problem Interactive decision making in changing environments with constraints.
method Hybrid Decision Making with Structured Observations (hybrid DMSO) framework, local differentially private decision making, query-based learning, robust and smooth decision making.
result Strong connections and bounds derived for DEC, SQ dimension, local minimax complexity, learnability, and joint differential privacy.
New algorithm matches best regret bound for tabular Contextual Bandit problems.
problem Learning from observations in uncertain environments.
method A minor variant of a reinforcement learning algorithm for MDPs.
result The algorithm matches the best possible regret bound i l d e O ( S A T ) ilde O (\sqrt{SAT}) i l d e O ( S A T ) for tabular Contextual Bandit problems. Thompson Sampling improves decision-making in partially observed contexts.
problem Balancing exploration and exploitation in partially observed contextual bandits.
method Thompson Sampling policy for learning optimal arms from noisy linear functions of unobserved context vectors.
result Thompson Sampling achieves poly-logarithmic regret and square-root consistency of parameter estimation.
Paper tackles binary feedbacks in contextual search learning.
problem Learning underlying mean value function in context with binary feedbacks.
method Tri-section search combined with margin-based active learning.
result Algorithm achieves O ( 1 / ε 2 ) O(1/\varepsilon^2) O ( 1/ ε 2 ) queries for ε ε ε -estimation accuracy. New online method for statistical inference with matrix context in decision-making.
problem Statistical inference in decision-making with matrix context.
method Proposes a fully online procedure to conduct statistical inference with adaptive data collection, handling low-rank structure.
result Establishes asymptotic normality of debiased estimators and proves validity of confidence intervals.
Bayesian algorithms perform well even with misspecified priors, especially in meta-learning.
problem Performance degradation of Bayesian algorithms with misspecified priors.
method Thompson sampling and meta-learning analysis with misspecified priors.
result Thompson sampling's performance degrades gracefully with misspecification, with a bound of i l d e O ( H 2 ε ) ilde{\mathcal{O}}(H^2 ε) i l d e O ( H 2 ε ) . A contextual care protocol is used by a medical practitioner for patient healthcare, given the context or situation that the specified patient is in. This paper proposes a method to build an automated self-adapting protocol which can help make relevant, early decisions for effective healthcare delivery. The hybrid mode…
Contextual linear optimization shows naive plug-in methods can outperform direct optimization.
problem Optimizing decisions with side observations to reduce uncertainty.
method Using off-the-shelf machine learning methods to learn a predictive model and plug it in for optimization.
result The naive plug-in approach achieves faster regret convergence rates than direct optimization methods.
A new Thompson Sampling framework handles uncertainty by imputing missing data.
problem Handling uncertainty in contextual bandit problems.
method Generative model to impute missing outcomes, fit policy, and select actions.
result Established a state-of-the-art regret bound that depends on generative model quality.
Lecture notes on reinforcement learning using statistical methods.
problem Addressing the exploration-exploitation dilemma in decision making.
method Frequentist and Bayesian approaches, function approximation, neural networks.
result Unified framework for decision making and estimation.
Study batch learning in linear bandits with context, achieving near-optimal performance.
problem Sequential batch learning in linear contextual bandits with finite actions.
method Established regret bounds and provided algorithms for two settings: arbitrary contexts and i.i.d. contexts.
result Regret upper bound nearly matches lower bound, showing polynomial and logarithmic batch requirements.
Paper tackles domain adaptation for contextual bandits with sub-linear regret.
problem Adapting contextual bandit algorithms across domains with distribution shift.
method Learn a bandit model for the target domain using feedback from the source domain.
result Sub-linear regret bound maintained across domains.