AdaLinUCB optimizes exploration-exploitation for contextually varying costs.
problem Optimizing decision-making in environments with varying exploration costs.
method Adaptive Upper-Confidence-Bound (AdaLinUCB) algorithm for opportunistic learning.
result AdaLinUCB achieves O((log T)^2) regret bound, significantly outperforming other algorithms.
New algorithm balances exploration and exploitation in opportunistic bandits.
problem Regret of pulling suboptimal arms varies with environmental conditions.
method Proposes AdaUCB algorithm to adaptively balance exploration and exploitation.
result AdaUCB achieves O ( log T ) O(\log T) O ( log T ) regret with a smaller coefficient than traditional UCB. This paper uses bandit algorithms to reduce the cost of user interface experimentation in online retail.
problem Reducing the cost of user interface experimentation in online retail.
method Modeling user interface experimentation as an opportunistic bandit problem, reducing the cost of exploration.
result Significant regret reduction and improved contextual information for testing.
We consider the task of opportunistic channel access in a primary system composed of independent Gilbert-Elliot channels where the secondary (or opportunistic) user does not dispose of a priori information regarding the statistical characteristics of the system. It is shown that this problem may be cast into the framew…
A novel bandit problem with context-dependent rewards and blocking.
problem Contextual bandit problem with blocking.
method Online bipartite matching, UCB algorithm, delayed exploitation, opportunistic subsampling.
result Guaranteed O ( log T ) \mathcal{O}(\log T) O ( log T ) -regret in bandit setting. This paper examines unfair trading practices in NFT markets.
problem Sophisticated actors exploit market inefficiencies for unfair profits.
method Analyzes three types of opportunistic trading strategies.
result Identifies and categorizes unfair trading practices in NFT markets.
Opportunistic spectrum access is one of the emerging techniques for maximizing throughput in congested bands and is enabled by predicting idle slots in spectrum. We propose a kernel-based reinforcement learning approach coupled with a novel budget-constrained sparsification technique that efficiently captures the envir…
The paper explores cost-aware spectrum access strategies in cognitive radio systems.
problem Optimizing spectrum usage in cognitive radio systems with uncertain channel states and costs.
method Discrete time model with sensing and transmission phases, considering random costs and rewards.
result The optimal policy for spectrum access has a recursive double threshold structure, and online algorithms achieve near-optimal performance.
Deep learning optimizes vehicular communication zones for efficient data dissemination.
problem Overdimensioning and inefficient communication in vehicular floating content.
method Deep learning is used to select optimal broadcasting areas (Anchor Zones) for efficient message dissemination.
result The proposed method achieves an accuracy of 89.7% in predicting optimal Anchor Zones, saving up to 27% of resources.
Empirical study shows carriers ignore past shippers' behavior, focusing only on current actions.
problem Opportunistic behavior by shippers and carriers in dynamic freight markets.
method Empirical analysis of carrier reciprocity in US truckload transportation sector.
result Carriers do not remember shippers' past behaviors but respond to current actions.
Develops methods for dynamic pricing in incomplete data settings.
problem Incomplete historical data makes optimal pricing difficult.
method Nonparametric partial identification framework for offline dynamic pricing.
result Pessimistic and opportunistic policies with regret bounds.
Proposes a multi-stage algorithm for efficient spectrum access in CR networks.
problem High demand for wireless spectrum and need for high throughput and energy efficiency in SUs.
method Centralized multi-stage algorithm with non-parametric learning and adaptive collision avoidance.
result Ensures minimum interference to licensed users while providing high throughput and energy efficiency.
This paper improves fairness in recommendation systems by learning individual preferences across multiple dimensions.
problem Fairness in recommender systems, especially in areas with social impact.
method Opportunistic multi-aspect re-ranking approach that learns individual preferences and enhances provider fairness.
result Achieves a better trade-off between accuracy and fairness across multiple fairness dimensions.
We propose a design for schedule-based execution trading strategies based on uncertainty bands. This formulation: 1) simplifies strategy specification and implementation; 2) provides for flexible allocation among passive, opportunistic, aggressive, and dark pool crossing execution tactics; 3) allows for rapid enhanceme…
A new approach uses deep learning to manage vehicular content efficiently.
problem Managing content replication and caching in vehicular networks efficiently.
method Data-driven, centralized approach using a Convolutional Neural Network (CNN).
result Effective strategies derived to modulate FC operation in space and adapt to mobility changes.
RL-Exec uses reinforcement learning to optimize BTC-USD liquidation, outperforming traditional methods.
problem Optimizing liquidation strategies on BTC-USD limit-order books with transient impact and latency.
method PPO agent trained on historical BTC-USD limit-order book replays, incorporating impact resilience and fees.
result RL-Exec significantly outperforms TWAP and a VWAP-like baseline on BTC-USD liquidation, with performance improving with longer execution horizons.
OverQ increases model accuracy by handling outliers in neural networks with minimal hardware changes.
problem Handling outliers in neural network weights and activations for low-precision quantization.
method Overwrite quantization (OverQ) that opportunistically increases bitwidth for activation outliers.
result OverQ can handle over 90% of outliers and achieve +5% ImageNet Top-1 accuracy on a quantized ResNet-50 at 4 bits.
In this paper, we address the general case of a coordinated secondary network willing to exploit communication opportunities left vacant by a licensed primary network. Since secondary users (SU) usually have no prior knowledge on the environment, they need to learn the availability of each channel through sensing techn…
AnyThreat detects insider threats with minimal false positives.
problem High false positives in detecting insider threats.
method Opportunistic knowledge discovery system with four components: feature engineering, oversampling, class decomposition, and classification.
result Detects 87.5% of malicious insider threats with minimal false positives.
Traditional activity recognition systems work on the basis of training, taking a fixed set of sensors into account. In this article, we focus on the question how pattern recognition can leverage new information sources without any, or with minimal user input. Thus, we present an approach for opportunistic activity reco…
Paper proposes a new neural machine translation method for wave data.
problem Limited real-world sensor data for continuous signal waves.
method Introduces window-based representation and iterative back-translation for wave data.
result Significant performance improvements in wave translation tasks.
New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Unified formulation bridges adversarial and nonstationary bandits.
problem Handling time-varying reward distributions in multi-armed bandit problems.
method Unified oracle that switches between adversarial and nonstationary bandit oracles based on window size.
result Optimal regret achieved with matching lower bound.
Paper studies attacks on bandit algorithms and shows how attackers can manipulate data to hijack behavior.
problem Potential attacks on bandit algorithms can cause catastrophic loss in real-world applications.
method Proposes a framework of offline and online attacks on bandit algorithms using convex optimization and adaptive strategies.
result Attackers can force bandit algorithms to pull target arms with high probability by manipulating data.
Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
Algorithm identifies best arm in linked bandits with reduced feedback.
problem Best arm identification in linked bandits with reduced feedback.
method Combines uniform sampling with regular bandit algorithm.
result Almost matching upper and lower bounds on sample complexity.
Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.
problem Modeling sequential decision-making problems with evolving arm rewards.
method Graph-Triggered Bandits (GTBs) framework that generalizes rested and restless bandits using a graph.
result Rested and restless bandits are special cases of GTBs for suitable graphs.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.
Unified approach for non-stationary and clustered bandits.
problem Solving non-stationary and clustered bandits with overlapping solutions.
method Test of homogeneity for seamless integration of non-stationary and clustered bandits.
result Unified solution framework for change detection and cluster identification.
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
New algorithm learns optimal exploration parameters for contextual bandits.
problem Learning optimal exploration in contextual bandits.
method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.
result Demonstrates improved performance in learning optimal exploration compared to traditional methods.
A new method for dueling bandits improves performance.
problem Improving decision-making in dueling bandits.
method Sup-KLUCB method for K-armed dueling bandits, converting Copeland dueling bandits into standard MAB problems.
result Sup-KLUCB outperforms state-of-the-art methods in Copeland dueling bandits.
Study on indexability of restless multi-armed bandits and rollout policy performance.
problem Maximizing discounted rewards in finite state restless multi-armed bandit problems.
method Decouple the problem into single-armed restless bandits, analyze using value iteration, and compare with Whittle index policy.
result Demonstrates conditions for indexability and compares performance of index policy and rollout policy.
Investigates sequential problems on graph structures and large action spaces.
problem Sequential decision-making on graph structures and large action spaces.
method Spectral bandits, side observations, influence maximization, kernel bandits, polymatroid bandits, function optimization, infinitely many-arms bandits.
result Contributions to graph and structured bandits.
New insights into multi-armed bandits with budget constraints.
problem Multi-armed bandits with supply/budget constraints.
method Characterization of logarithmic regret rates, simple regret, and reduction to other bandit problems.
result Full characterization of logarithmic, instance-dependent regret rates for BwK.
Optimal algorithm for identifying best arm in stochastic linear bandits with fixed confidence.
problem Identifying the best arm in stochastic linear bandits with fixed confidence.
method Extending an algorithm designed for Best Arm Identification to the ε ε ε -Thresholding Bandit Problem (TBP). result Asymptotically optimal algorithm for TBP.
A meta-UCB method combines stochastic bandit algorithms.
problem Combining multiple stochastic bandit algorithms efficiently.
method Meta-UCB procedure solving an N-armed bandit problem.
result Final regret depends only on the best base algorithm's regret.
New method for contextual bandits with corrupted context.
problem Contextual bandits with corrupted context in online settings.
method Combining contextual bandit and multi-armed bandit approaches.
result Improved learning from all iterations, including corrupted ones.
A new algorithm for top-k combinatorial bandits with full-bandit feedback.
problem Optimizing decisions among subsets of arms with only aggregate rewards observed.
method CSAR algorithm using Hadamard matrices for estimating individual arm rewards.
result Proved a tight lower bound on sample complexity for k = O ( 1 ) k=O(1) k = O ( 1 ) . A new framework for structured bandits using influence diagrams and variational Thompson sampling.
problem Complex statistical dependencies in structured bandit problems.
method Influence diagram framework, variational Thompson sampling, tracking structured posterior distribution.
result Empirically evaluated algorithms perform as well as or better than existing baselines.
Smooth Contextual Bandits bridge two previously studied extremes of non-differentiable and parametric-response bandits.
problem Nonparametric contextual bandits with Hölder smoothness.
method Developed a novel algorithm that optimally balances between non-differentiable and parametric-response bandits.
result Proved the algorithm achieves rate-optimal regret for all smoothness settings.
Factored bandits model learns with limited feedback using decomposable actions.
problem Limited feedback learning with decomposable actions.
method Introduces factored bandits model, provides anytime algorithm, and matching upper and lower bounds.
result Improves regret bounds for utility-based dueling bandits.
New algorithm for nonstationary multi-armed bandits with optimal performance.
problem Nonstationary multi-armed bandits with changing model parameters over time.
method Adaptive Resetting Bandit (ADR-bandit) algorithm using adaptive windowing techniques.
result ADR-bandit achieves nearly optimal performance in both abrupt and gradual changes.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
Unified framework for high-dimensional bandit problems with low-dimensional structures.
problem Stochastic high-dimensional bandit problems with low-dimensional structures.
method Proposed a simple unified algorithm and a general analysis framework for the regret upper bound.
result Unified algorithm achieves comparable regret bounds in various high-dimensional bandit problems.
Adapts multi-armed bandits to contextual bandits using logistic regression.
problem Online decision-making with contextual information and binary rewards.
method Adapts multi-armed bandits policies to contextual bandits using logistic regression and bootstrapping.
result Adaptive-Greedy algorithm shows better performance than upper confidence bound and Thompson sampling.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.