We consider interactive learning and covering problems, in a setting where actions may incur different costs, depending on the response to the action. We propose a natural greedy algorithm for response-dependent costs. We bound the approximation factor of this greedy algorithm in active learning settings as well as in …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “greedy actions”
Simpler ε-greedy with longer action durations improves exploration.
problem Limited exploration capability of ε-greedy in complex domains.
method Temporally extended ε-greedy with repeated actions for random durations.
result Temporally extended ε-greedy outperforms sophisticated methods on various domains.
Efficient algorithms identify true hypothesis from many options with minimal actions.
problem Identifying true hypothesis from a large set of options with minimal actions.
method Greedy approximation algorithms for active sequential hypothesis testing.
result First approximation guarantees for ASHT, independent of the number of hypotheses.
Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policiescs.LG
A new RL approach learns near-equivalent actions for healthcare decisions.
problem Finding optimal actions in healthcare settings where actions may be near-equivalent.
method Temporal difference learning with a near-greedy heuristic for action selection.
result The proposed algorithm discovers meaningful near-equivalent actions and converges well.
Greedy algorithm nearly outperforms exploration in contextual bandits.
problem Balancing exploration and exploitation in online learning.
method Smoothed analysis of the greedy algorithm in linear contextual bandits.
result Greedy algorithm nearly matches Bayesian regret rate under diversity conditions, with regret at most .
Study on rapid policy changes in reinforcement learning.
problem Rapid change of greedy policy in reinforcement learning.
method Empirical study and ablation analysis.
result Policy churn is a beneficial form of implicit exploration.
GUM tackles MARL by avoiding overestimation through state-marginal restriction.
problem Overestimation of values in large joint state-action spaces.
method Greedy UnMixing through state-marginal restriction and unmixing.
result Superior performance compared to existing Q-learning and general MARL algorithms.
Greedy policies perform poorly in imperfectly observed contextual bandits.
problem Performance of Greedy policies in bandits with partially observed contexts.
method Analysis of Greedy reinforcement learning policies under imperfectly observed contextual bandits.
result Worst-case regret grows poly-logarithmically with the time horizon and the failure probability.
We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…
We consider a class of discrete optimization problems that aim to maximize a submodular objective function subject to a distributed partition matroid constraint. More precisely, we consider a networked scenario in which multiple agents choose actions from local strategy sets with the goal of maximizing a submodular obj…
In this paper, we propose an active perception method for recognizing object categories based on the multimodal hierarchical Dirichlet process (MHDP). The MHDP enables a robot to form object categories using multimodal information, e.g., visual, auditory, and haptic information, which can be observed by performing acti…
Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In th…
The paper extends a learning heuristic to high-dimensional contexts, reducing the risk of unusual actions.
problem Sequential learning problems in high dimensions, especially in dynamic pricing and auctions.
method Introducing a conservative -greedy rule that limits the adoption of new actions to a focused set of promising actions.
result Reasonable bounds for cumulative regret and improved regret bound for conservative version compared to non-conservative.
The question of how to explore, i.e., take actions with uncertain outcomes to learn about possible future rewards, is a key question in reinforcement learning (RL). Here, we show a surprising result: We show that Q-learning with nonlinear Q-function and no explicit exploration (i.e., a purely greedy policy) can learn s…
Enhances PMD with lookahead to improve RL performance.
problem Improving RL performance with greedy policies over 1-step.
method Integrates multi-step greedy policies into PMD with lookahead.
result Shows faster convergence rate for -PMD.
New RL theory predicts deep RL success based on greedy actions under random policies.
problem Current RL theory struggles to predict practical performance.
method Introduced BRIDGE dataset and effective horizon measure.
result Effective horizon correlates better with deep RL success than prior bounds.
DE is a new exploration method that limits resource usage based on expected improvement and surprise.
problem Limited exploration in large action spaces when resources are scarce.
method Delight-gated exploration (DE) that limits exploration actions based on a gate price set by the product of expected improvement and surprise.
result DE outperforms -greedy and Thompson Sampling in terms of regret across various bandit and MDP settings.
Proposes a new algorithm for accurate tree-based models with guaranteed recourse actions.
problem Ensuring recourse actions for models optimized for predictive performance.
method Formulates and solves an optimization task to ensure reasonable actions for as many instances as possible, using adversarial training techniques.
result Successfully provided reasonable actions to more instances than baselines without significantly degrading accuracy and computational efficiency.
Regularization-induced exploration improves contextual bandit performance.
problem Complex reward models in real-world contextual bandits are hard to explore effectively.
method Regularization-induced exploration using stochasticity in cross-validation.
result Regularization-induced exploration leads to reliable exploration in large-scale business environments.
New algorithm for combinatorial bandit problems reduces regret.
problem Optimal selection of sets of arms in bandit problems.
method SGB algorithm with optimized exploration of unselected arms.
result Achieves -regret bound of .
We study adversarial attacks that manipulate the reward signals to control the actions chosen by a stochastic multi-armed bandit algorithm. We propose the first attack against two popular bandit algorithms: -greedy and UCB, \emph{without} knowledge of the mean rewards. The attacker is able to spend only logarithmic …
Multi-step greedy policies have been extensively used in model-based reinforcement learning (RL), both when a model of the environment is available (e.g.,~in the game of Go) and when it is learned. In this paper, we explore their benefits in model-free RL, when employed using multi-step dynamic programming algorithms: …
We propose a policy improvement algorithm for Reinforcement Learning (RL) which is called Rerouted Behavior Improvement (RBI). RBI is designed to take into account the evaluation errors of the Q-function. Such errors are common in RL when learning the -value from finite past experience data. Greedy policies or even …
Simple greedy algorithms can excel in multi-objective bandits with multiple good arms.
problem Optimizing multiple objectives in bandits is traditionally harder.
method Introduced greedy algorithms that exploit multiple good arms for multiple objectives.
result Simple greedy algorithms achieve strong performance in multi-objective bandits.
Efficient algorithms for planning in cooperative multi-agent reinforcement learning with combinatorial action spaces.
problem Planning in cooperative multi-agent reinforcement learning with a combinatorial action space.
method Efficient algorithms using local access to a simulator and linear function approximation, with improvements for additive feature decomposition and kernelized settings.
result Polynomial compute and query complexity in relevant problem parameters.
Paper introduces a new analysis framework for stochastic linear bandits.
problem Optimizing decision-making in online experiments with noisy rewards.
method Develops a general analysis framework and algorithms for stochastic linear bandits.
result Introduces new algorithms like SG that improve performance and provide new regret bounds.
Proposes H-UCRL for efficient model-based RL with sublinear regret.
problem Greedy policy exploration in model-based RL ignores epistemic uncertainty.
method Reparameterizes plausible models, hallucinates control, augments input space, solves with greedy planners.
result H-UCRL achieves provably sublinear regret for Gaussian Process models.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves cumulative expected regret.
Greedy algorithm optimizes consumption habits with power utility.
problem Optimizing lifetime consumption with habit formation under power utility.
method Developed a greedy algorithm using Monte Carlo simulation.
result Greedy solution is a good approximation to the optimal solution.
This paper investigates trajectory tracking problem for a class of underactuated autonomous underwater vehicles (AUVs) with unknown dynamics and constrained inputs. Different from existing policy gradient methods which employ single actor-critic but cannot realize satisfactory tracking control accuracy and stable learn…
Exploration is a difficult challenge in reinforcement learning and even recent state-of-the art curiosity-based methods rely on the simple epsilon-greedy strategy to generate novelty. We argue that pure random walks do not succeed to properly expand the exploration area in most environments and propose to replace singl…
Paper uses RL for better credit scoring and underwriting.
problem Traditional underwriting methods are ungeneralizable in complex scenarios.
method Adapts RL principles for credit scoring, incorporating action space renewal and multi-choice actions.
result RL-based algorithms outperform traditional methods in aligned data scenarios.
Exploration is a difficult challenge in reinforcement learning and is of prime importance in sparse reward environments. However, many of the state of the art deep reinforcement learning algorithms, that rely on epsilon-greedy, fail on these environments. In such cases, empowerment can serve as an intrinsic reward sign…
We present an information-theoretic framework for sequential adaptive compressed sensing, Info-Greedy Sensing, where measurements are chosen to maximize the extracted information conditioned on the previous measurements. We show that the widely used bisection approach is Info-Greedy for a family of -sparse signals b…
New Classes of the Greedy-Applicable Arm Feature Distributions in the Sparse Linear Bandit Problemcs.LG
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.
Model-based reinforcement learning algorithms make decisions by building and utilizing a model of the environment. However, none of the existing algorithms attempts to infer the dynamics of any state-action pair from known state-action pairs before meeting it for sufficient times. We propose a new model-based method ca…
This paper is a follow up to the previous author's paper on convex optimization. In that paper we began the process of adjusting greedy-type algorithms from nonlinear approximation for finding sparse solutions of convex optimization problems. We modified there three the most popular in nonlinear approximation in Banach…
Introduces greedy feature selection for classifier-dependent feature ranking.
problem Feature selection for classification tasks.
method Greedy feature selection, identifying the most important feature at each step based on the selected classifier.
result Theoretical and numerical benefits of greedy feature selection.
An automatic program that generates constant profit from the financial market is lucrative for every market practitioner. Recent advance in deep reinforcement learning provides a framework toward end-to-end training of such trading agent. In this paper, we propose an Markov Decision Process (MDP) model suitable for the…
In statistical dialogue management, the dialogue manager learns a policy that maps a belief state to an action for the system to perform. Efficient exploration is key to successful policy optimisation. Current deep reinforcement learning methods are very promising but rely on epsilon-greedy exploration, thus subjecting…
Greedy policy maximizes information in unknown linear systems.
problem Exploration in unknown linear dynamical systems.
method Online greedy policy maximizing information.
result Competitive performance compared to gradient-based methods.
Greedy selection works well in a toy model of independent increments.
problem Iterative selection of maximum-value processes from i.i.d. stochastic processes.
method Fixed greedy selection at each stage.
result Optimal strategy is greedy selection under independent increments.
This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima. When greediness is set at maximum, KES co…
Reinforcement learning addresses the dilemma between exploration to find profitable actions and exploitation to act according to the best observations already made. Bandit problems are one such class of problems in stateless environments that represent this explore/exploit situation. We propose a learning algorithm for…
Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control. In a recent work \cite{efroni2018beyond}, multiple-step greedy policies and their use in vanilla Policy Iteration algorithms were proposed and analy…
Paper analyzes Greedy-GQ for reinforcement learning with Markovian noise.
problem Analyzing Greedy-GQ for reinforcement learning with Markovian noise.
method Develops finite-sample analysis for Greedy-GQ with linear function approximation under Markovian noise.
result Provides theoretical justification for choosing stepsizes for faster convergence.
Off-policy reinforcement learning with eligibility traces is challenging because of the discrepancy between target policy and behavior policy. One common approach is to measure the difference between two policies in a probabilistic way, such as importance sampling and tree-backup. However, existing off-policy learning …
We consider stochastic multi-armed bandit problems with graph feedback, where the decision maker is allowed to observe the neighboring actions of the chosen action. We allow the graph structure to vary with time and consider both deterministic and Erdős-Rényi random graph models. For such a graph feedback model, we fir…