New algorithm for contextual bandits with linear constraints using regression.
problem Contextual bandits with packing and covering constraints.
method Modular Lagrangian approach via regression.
result First vanishing-regret guarantees for CBwLC (or CBwK).
Proposes a Thompson sampling algorithm for multi-objective contextual bandit problems with auxiliary constraints.
problem Real-world applications with multiple competing objectives and auxiliary constraints.
method Thompson sampling algorithm for multi-outcome contextual bandit problems with auxiliary constraints.
result Empirically evaluated and applied to a real-world video transcoding problem.
New loss function handles uncertain constraints in CSLO problems.
problem Handling uncertain inequality constraints in CSLO with machine learning predictions.
method Introduces SPO-RC loss and SPO-RC+ surrogate, trains on truncated datasets, corrects bias.
result SPO-RC+ effectively manages constraint uncertainty and improves performance.
Study contextual bandits with stage-wise constraints, proving regret bounds and extending results.
problem Contextual bandits with stage-wise constraints in high probability and expectation settings.
method Upper-confidence bound algorithms for linear and non-linear reward/cost functions, extending to multiple constraints.
result Regret bounds for various settings, including non-linear reward/cost functions.
Integrates contextual constraints into embedding models for better recommendation quality.
problem Contextual constraints lead to incomplete or low-quality recommendations when applied independently.
method Merges constraint application and retrieval into one operation in the embedding space.
result Significant improvements in predictive performance compared to context-aware and standard models.
Study on adaptivity constraints in linear contextual bandits with optimal design.
problem Impact of adaptivity constraints on linear contextual bandits.
method Two models of limited adaptivity: batch learning and rare policy switches. Proposed distributional optimal design.
result Achieves minimax-optimal regret with optimal number of policy switches and batches.
Algorithm tackles clustered contextual bandits with resource constraints.
problem Maximizing reward while respecting resource limits in unknown cluster contexts.
method Combines econometrics and bandit constraints techniques for sublinear regret.
result Achieves sublinear regret without needing all arm information.
In this work we consider adversarial contextual bandits with risk constraints. At each round, nature prepares a context, a cost for each arm, and additionally a risk for each arm. The learner leverages the context to pull an arm and then receives the corresponding cost and risk associated with the pulled arm. In additi…
Study wSAA for contextual decisions, improving uncertainty quantification under computational constraints.
problem Uncertainty quantification limitations in wSAA for contextual stochastic optimization.
method Establish central limit theorems and asymptotic-normality-based confidence intervals for optimal costs.
result Over-optimizing can mitigate misspecification and preserve asymptotic normality, albeit at a slower convergence rate.
HATCH learns optimal recommendations with resource constraints.
problem Resource-constrained recommendation systems.
method Hierarchical adaptive contextual bandits with adaptive resource allocation.
result HATCH achieves a regret bound of O ( T ) O(\sqrt{T}) O ( T ) . Paper proposes a risk-aware decision-making framework for real-world sequential decisions.
problem Real-world sequential decision-making problems often have critical constraints that learning solutions often neglect.
method Actor multi-critic architecture with risk characterization.
result Our approach consistently satisfies system constraints with minimal performance toll.
A novel approach learns constraints and maximizes rewards for autonomous agents.
problem Ensuring autonomous agents align with societal norms and values.
method Inverse reinforcement learning for constraints, contextual bandit orchestrator for policy mixing.
result Agent learns to act optimally within constraints and maximize rewards.
New method reduces total cost constraints in CBwK to sqrt(T) with fairness application.
problem Maximize rewards while adhering to total cost constraints in CBwK.
method Dual strategy based on projected-gradient-descent updates.
result Total cost constraints reduced to sqrt(T) with poly-logarithmic terms.
Paper develops a fair pricing algorithm for dynamic settings with uncertain demand.
problem Fair pricing in dynamic, uncertain demand scenarios.
method Contextual bandit algorithm with dynamic pricing and demand learning.
result Achieves optimal regret bound with fairness constraints.
Study dynamic batch learning in high-dimensional sparse linear bandits.
problem Dynamic batch learning in high-dimensional sparse linear contextual bandits under batch constraints.
method Characterized fundamental learning limits via regret lower bound and provided matching upper bound.
result Prescribed an optimal scheme for dynamic batch learning in high-dimensional sparse linear contextual bandits.
Study nonparametric contextual bandits with batched updates, achieving optimal regret.
problem Optimal regret in nonparametric contextual bandits with batch constraints.
method Dynamic binning of covariate space, optimal regret achieved.
result Achieves optimal regret (up to logarithmic factors) for nonparametric contextual bandits.
Improved regret bounds for contextual combinatorial semi-bandits with linear payoffs.
problem Maximizing rewards in decision-making problems with feature vectors and constraints.
method Proposed C^2UCB algorithm and modified reward estimates for general constraints.
result Optimal regret bounds of C^2UCB algorithm and modified algorithm for various constraints.
We study contextual bandits with budget and time constraints, referred to as constrained contextual bandits.The time and budget constraints significantly complicate the exploration and exploitation tradeoff because they introduce complex coupling among contexts over time.Such coupling effects make it difficult to obtai…
An algorithm for maximizing rewards under linear cost constraints.
problem Maximizing rewards while adhering to cost constraints in a linear bandit problem.
method Proposes an upper-confidence bound algorithm called optimistic pessimistic linear bandit (OPLB) for constrained contextual linear bandits.
result Proves an O ~ ( d T τ − c 0 ) \widetilde{\mathcal{O}}(\frac{d\sqrt{T}}{τ-c_0}) O ( τ − c 0 d T ) bound on regret for the proposed algorithm. A smart method predicts and optimizes decisions online with resource constraints.
problem Online decision-making with resource constraints.
method Combines prediction and optimization with dual update using mirror descent.
result Regret bounds and convergence rates for general convex feasible regions.
Bayesian classifier improves robustness with optimistic score ratio.
problem Limited information on class-conditional distribution.
method Optimistic score ratio for robust binary classification.
result Bayesian classifier using optimistic score ratio is robust and computationally tractable.
This paper achieves optimal regret bounds for locally private linear contextual bandit.
problem Designing locally private linear contextual bandit algorithms with optimal regret bounds.
method New algorithmic and analytical ideas, including mean absolute deviation analysis and layered principal component regression.
result Achieves an i l d e O ( T ) ilde O(\sqrt{T}) i l d e O ( T ) regret upper bound for locally private linear contextual bandit. A new algorithm balances global reward and group constraints in federated multi-armed bandits.
problem Maximizing global reward while protecting client privacy in federated learning.
method Combinatorial contextual bandit with group constraints, using a two-output Gaussian process.
result TCGP-UCB incurs low regret, balancing super arm reward and group reward constraints.
Develops algorithms for CCBs with non-linear costs, improving safety and performance.
problem Safety constraints in sequential decision making with non-linear arm costs.
method Innovative algorithms using Inverse Gap Weighting (IGW) and online regression oracle.
result Sub-linear regret bounds for C-SquareCB and first-order regret for C-FastCB.
This paper tackles fair online decision-making in contextual bandits, achieving optimal performance and fairness.
problem Fairness in online decision-making systems under strategic manipulation.
method Develops algorithms for linear and smooth reward functions, maintaining fairness and optimal regret.
result Achieves nearly minimax-optimal regret with strong fairness guarantees, even in the presence of attacks.
Modified CTGAN-Plus-Features method optimizes asset allocation with CVaR constraint.
problem Optimizing portfolio weights in asset allocation problems.
method Combines synthetic data generation with CVaR-constraint optimization.
result Synthetic data captures key characteristics of original data and outperforms conventional strategies.
New approach tackles resource constraints in bandit problems with weakly adaptive algorithms.
problem Maximizing rewards while adhering to general long-term constraints.
method Weakly adaptive primal and dual regret minimizers.
result Achieves sublinear constraints violations and competitive ratios in both stochastic and adversarial settings.
New approach optimizes sales process for B2B businesses.
problem Optimizing the sales process for B2B businesses.
method Causal Predictive Optimization and Generation with three layers: prediction, optimization, and serving.
result Significant wins over legacy systems in LinkedIn implementation.
Optimal algorithm for maximizing rewards in contextual bandits with resource constraints.
problem Maximizing rewards in contextual bandits with resource constraints.
method Proposed a universal and optimal algorithmic framework for CBwK by reducing it to online regression.
result Established the optimality of the proposed algorithm for various function classes.
New algorithm improves online learning under performance constraints.
problem Improving performance of existing systems in various fields.
method Conservative Constrained LinUCB (CLUCB2) algorithm for contextual linear bandits.
result Empirically outperforms existing conservative bandit algorithms.
Paper proposes a network framework for prosumers to manage peak loads in Iran.
problem Balancing renewable prosumers' self-sufficiency with grid integration under uncertainty.
method Distributed contextual stochastic optimization (DCSO) framework with consensus-based sharing.
result Integration of prediction and optimization reduces peak loads and costs.
Safety is a desirable property that can immensely increase the applicability of learning algorithms in real-world decision-making problems. It is much easier for a company to deploy an algorithm that is safe, i.e., guaranteed to perform at least as well as a baseline. In this paper, we study the issue of safety in cont…
Develops locally private methods for nonparametric contextual bandits.
problem Privacy concerns in sequential decision-making on sensitive data.
method Uniform-confidence-bound-type estimator and jump-start scheme.
result Minimax optimality of proposed methods supported by lower bounds.
New insights into multi-armed bandits with budget constraints.
problem Multi-armed bandits with supply/budget constraints.
method Characterization of logarithmic regret rates, simple regret, and reduction to other bandit problems.
result Full characterization of logarithmic, instance-dependent regret rates for BwK.
We consider a contextual version of multi-armed bandit problem with global knapsack constraints. In each round, the outcome of pulling an arm is a scalar reward and a resource consumption vector, both dependent on the context, and the global knapsack constraints require the total consumption for each resource to be bel…
New method for private linear regression under privacy constraints, achieving optimal rates.
problem Statistical complexity of private linear regression under unknown, ill-conditioned covariates.
method Information-Weighted Regression method
result Optimal convergence rates for both central and local privacy models.
Sharp policy value estimation for contextual bandits with unobserved confounders.
problem Estimating policy value under unobserved confounders with sensitivity analysis.
method Kernel method to approximate conditional moment constraints, leveraging f-divergence.
result Sharp lower bound of policy value, avoiding coarse relaxation of uncertainty set.
Combines machine learning and optimization for real-time decision-making.
problem Optimizing decisions in contextually constrained problems.
method Generative model combining interior point methods and adversarial learning.
result Generative model produces optimal decisions with in-sample and out-of-sample guarantees.
A new algorithm for differential privacy in kernelized contextual bandits reduces error rate.
problem Joint differential privacy in kernelized contextual bandits.
method Proposes a novel algorithm with a specific error rate and privacy parameter dependence.
result Achieves an error rate of $\mathcal{O}\left(\sqrt{\frac{γ_T}{T}} + \frac{γ_T}{T \varepsilon}
ight)$ after T T T queries. Paper optimizes UAV navigation for IoT data freshness and energy efficiency.
problem Improving data freshness and connectivity for IoT devices with UAVs.
method Deep reinforcement learning model with experience replay for energy-efficient UAV trajectory optimization.
result The proposed approach is 3.6% and 3.13% more energy efficient than greedy and baseline methods.
PS framework selects best policy from library for CSO problems.
problem Policy selection in CSO with heterogeneous performance across covariate space.
method PS framework constructs library of candidate policies and learns a meta-policy to select the best one.
result PS consistently outperforms best single policy in heterogeneous CSO problems.
C3T-Budget optimizes drug efficacy in dose-finding trials with budget and safety constraints.
problem Heterogeneous patient populations and budget constraints make dose-finding clinical trials challenging.
method Contextual constrained clinical trial algorithm that maximizes drug efficacy while learning subgroup responses.
result Demonstrates efficient budget usage and balanced learning-treatment trade-off in simulated trials.
This paper optimizes product assortment decisions with changing contextual information.
problem Optimizing product assortment decisions in a dynamic context.
method Developed an upper confidence bound (UCB) policy to learn and make decisions under a changing contextual MNL model.
result Established a regret bound of O ~ ( d T ) \widetilde O(d\sqrt{T}) O ( d T ) and a lower bound of Ω ( d T / K ) Ω(d\sqrt{T}/K) Ω ( d T / K ) for dynamic assortment optimization. Two algorithms address limited adaptivity in generalized linear contextual bandits.
problem Limited adaptivity in generalized linear contextual bandits.
method Two algorithms, B-GLinCB and RS-GLinCB, designed for two settings of limited adaptivity.
result Achieved i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret in both settings. New algorithms robust to adversarial data achieve optimal performance.
problem Adversarial robustness in high-dimensional online learning problems.
method Alternating minimization scheme combining least-squares and convex reweighting.
result Achieves optimal robustness guarantees without distributional assumptions.
Fairness in AI decisions for users with varying performance.
problem Ensuring fairness in AI decisions for users with different performance levels.
method Contextual Multi-Armed Bandit algorithm with fairness constraints.
result Accounting for user contexts improves fairness in AI decisions.
An algorithm for efficient experimentation in a dynamic environment with personalized preferences and context drifts.
problem Efficiently recommending decisions to users with personalized preferences in a context where the environment is changing over time.
method Dri-MED, inspired from the linear version of the MED strategy, adapted to handle non-stationary heteroskedastic noise.
result The instance-dependent regret scales as $ ilde{\mathcal O}\left(\fracκ{ ildeΔ}d^2(\log(T)
ight)$ , with i l d e Δ ildeΔ i l d e Δ being the constraint-aware sub-optimality gap. A new model optimizes portfolios by learning stock return distributions conditioned on factors.
problem Optimizing portfolios with high-dimensional asset-specific factors.
method Conditional Diffusion Transformer architecture linking each asset's return to its factor vector.
result The model outperforms benchmarks in mean-variance and mean-CVaR optimization.