We introduce Parameterized Exploration (PE), a simple family of methods for model-based tuning of the exploration schedule in sequential decision problems. Unlike common heuristics for exploration, our method accounts for the time horizon of the decision problem as well as the agent's current state of knowledge of the …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “decision exploration”
Study explores loss design for decision trees to improve robustness against noisy labels.
problem Improving decision tree robustness to noisy labels.
method Investigated loss correction and symmetric losses, found ineffective.
result Other loss design directions need exploration for robust decision trees.
DECE visualizes machine learning decisions with counterfactual explanations.
problem Making machine learning models transparent and explainable.
method Interactive visualization system supporting counterfactual explanations at instance- and subgroup-levels.
result DECE enables users to explore and understand machine learning model decisions.
Exploration Unboundcs.LG
An agent explores indefinitely in an environment with unlimited rewards.
problem Balancing exploration and exploitation in environments with unlimited rewards.
method Simple example of an environment with unbounded rewards and optimal agent behavior.
result An optimal agent always explores to maximize rewards, regardless of accumulated knowledge.
In Interactive Machine Learning (IML), we iteratively make decisions and obtain noisy observations of an unknown function. While IML methods, e.g., Bayesian optimization and active learning, have been successful in applications, on real-world systems they must provably avoid unsafe decisions. To this end, safe IML algo…
Paper presents an algorithm for optimal regret in communicating Markov decision processes.
problem Achieving optimal regret in Markov decision processes with a communicating assumption.
method The algorithm explicitly tracks the constant K(M) to learn optimally, balancing exploration, co-exploration, and exploitation.
result The algorithm achieves asymptotically optimal regret for communicating Markov decision processes.
Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.
problem Balancing exploration and exploitation in online decision making.
method Introducing a batch Thompson Sampling framework for stochastic multi-arm bandit and linear contextual bandit problems.
result Achieves asymptotic regret bound with batch queries, significantly reducing interactions.
A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space …
The paper explores the geometry and topology of DNN decision boundaries.
problem Understanding the geometric and topological properties of DNN decision boundaries.
method Differential geometry and the Gauss-Bonnet-Chern theorem.
result Computed the Euler characteristics of compact decision boundaries.
Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.
problem Exploration-exploitation in communicating Markov Decision Processes.
method Analysis of UCRL2 with Empirical Bernstein inequalities (UCRL2B).
result Regret bound of for UCRL2B.
Proposes EVE for efficient exploration in reinforcement learning.
problem Efficient exploration in reinforcement learning.
method EVE: a recipe for posterior over parameters, facilitating efficient exploration.
result Competitive performance on benchmarks, efficient exploration confirmed.
This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables …
Study minimax optimal RL in factored MDPs with bonus exploration.
problem Optimal reinforcement learning in episodic factored MDPs.
method Proposes two model-based algorithms with bonus exploration for minimax optimal regret.
result Achieves minimax optimal regret guarantees for rich factored structures.
Parallelized bandit algorithms speed up decision-making.
problem Sequential exploration limits decision-making speed.
method Developed parallel contextual bandit algorithms.
result Regret nearly identical to sequential algorithms.
Lecture notes on reinforcement learning using statistical methods.
problem Addressing the exploration-exploitation dilemma in decision making.
method Frequentist and Bayesian approaches, function approximation, neural networks.
result Unified framework for decision making and estimation.
This review examines bandit problems in AI using statistical methods.
problem Sequential decision-making under uncertainty in AI environments.
method Foundational models, concentration inequalities, minimax regret bounds, frequentist and Bayesian algorithms, K-armed contextual bandits, SCAB, functional data analysis.
result Exploration-exploitation trade-offs and regret analyses in various bandit problems.
Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learningcs.LG
Rectified decision trees improve machine learning interpretability and effectiveness.
problem Combining interpretability and effectiveness in machine learning models.
method Knowledge distillation and modified decision tree splitting criteria.
result Soft labels improve model performance and reduce model size.
Decision Machines embeds decision trees into vector spaces for improved optimization.
problem Overfitting and difficulty in finding optimal decision tree structure.
method Embedding Boolean tests into a binary vector space and representing tree structure as matrices.
result Optimized decision trees with enhanced predictive power.
In many machine learning applications, there are multiple decision-makers involved, both automated and human. The interaction between these agents often goes unaddressed in algorithmic development. In this work, we explore a simple version of this interaction with a two-stage framework containing an automated model and…
VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Treescs.LG
VisRuler simplifies decision extraction from bagged and boosted trees.
problem Complexity and lack of interpretability in ensemble models.
method Visual analytics tool for selecting robust models, important features, and essential decisions.
result Users successfully extracted and explained decisions from ensemble models.
We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as possible. Similarly to active exploration in multi-armed bandit (MAB), states may have…
New framework uses unsupervised learning for efficient exploration in RL.
problem Efficient exploration in reinforcement learning with rich observations.
method Combines unsupervised learning and no-regret RL algorithms.
result Proves sample complexity for finding near-optimal policies.
Proposes a method to learn from historical data for personalized decision-making.
problem Sample hunger in sequential decision-making algorithms for personalized medicine.
method Identifiable latent bandit framework using nonlinear independent component analysis.
result Optimal decision-making with shorter exploration time than classical bandits.
We study the Combinatorial Pure Exploration problem with Continuous and Separable reward functions (CPE-CS) in the stochastic multi-armed bandit setting. In a CPE-CS instance, we are given several stochastic arms with unknown distributions, as well as a collection of possible decisions. Each decision has a reward accor…
In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring fin…
The paper tackles efficient exploration in MDPs to learn accurate models.
problem Efficient exploration in MDPs to learn accurate models.
method Formalizes the problem, introduces an algorithm for -accurate model estimation, and proposes a heuristic-based algorithm.
result Heuristic-based algorithm outperforms original algorithm in small sample regime.
This paper improves Bayesian decision tree learning using HMC.
problem Bayesian decision tree learning is challenging due to a large parameter space.
method Develops and compares HMC-based algorithms for exploring Bayesian decision tree posteriors.
result HMC-based methods outperform existing methods in predictive accuracy and tree complexity.
Active inference enhances RL by balancing exploration and exploitation.
problem Traditional RL's balance between exploration and exploitation is often suboptimal.
method Developed a new decision-making objective based on active inference.
result The new algorithm successfully balances exploration and exploitation on various RL benchmarks.
New algorithm identifies best policy in MDPs faster.
problem Identifying the best policy in Markov Decision Processes.
method Problem-dependent lower bound and first algorithm with instance-specific sample complexity.
result First algorithm with reduced exploration rate for faster convergence.
New approach uses autoregressive models to explore and quantify uncertainty in decision-making.
problem Quantifying and exploring uncertainty in online decision-making.
method Reformulates uncertainty as missing future outcomes, training autoregressive models for next-outcome prediction.
result Establishes a reduction from online learning to offline next-outcome prediction, controlling Bayesian regret by sequence prediction loss.
New strategies improve multi-agent decision-making on irregular networks.
problem Maximizing group reward in multi-agent settings with heterogeneous strategies.
method Design and analysis of heterogeneous explore-exploit strategies for multi-star networks.
result Group performance improves under heterogeneous strategies compared to homogeneous strategies.
In many healthcare settings, intuitive decision rules for risk stratification can help effective hospital resource allocation. This paper introduces a novel variant of decision tree algorithms that produces a chain of decisions, not a general tree. Our algorithm, -Carving Decision Chain (ACDC), sequentially carves o…
Improved ExO method achieves near-optimal bounds in both stochastic and adversarial settings.
problem Finding optimal exploration strategies in online decision-making with limited feedback.
method Exploration by Optimization with hybrid regularizers for locally observable games.
result Achieved nearly optimal bounds of in stochastic and adversarial environments.
Platform combines RL and language models to study narrative influence on AI decisions.
problem Understanding how narrative elements shape AI decision-making.
method Dual-system architecture with reinforcement learning and language model integration.
result Initial experiments show narrative frameworks can influence AI decision-making.
The paper extends physics-based information maximization to complex bandit problems.
problem Designing efficient decision-making policies for complex bandit problems.
method Information and free-energy maximization principles adapted to three distinct bandit types.
result Information maximization leads to strong performance in complex bandit problems.
Methodology explores fairness limits in decision tree classifiers.
problem Understanding statistical limits of bias mitigation in machine learning.
method Multi-objective framework optimizing accuracy and fairness.
result Decision tree models can be optimized for fairness with minimal accuracy loss.
New rule reduces exploration regret to logarithmic, improving bad episode handling.
problem Improving exploration regret in average reward MDPs.
method Replacing Doubling Trick with Vanishing Multiplicative rule in EVI-based algorithms.
result Regret is logarithmic under the new rule, significantly better than linear.
Study task-guided exploration in linear dynamical systems, improving sample complexity.
problem Efficiently learning about an environment to complete a specific task.
method Proposed a computationally efficient experiment-design based exploration algorithm.
result Optimally explores the environment, collecting precise information needed to complete the task.
ERTS uses Thompson sampling for Gaussian entropic risk bandits, achieving regret bounds.
problem Risk in decision making complicates reward maximization in MAB problems.
method ERTS (Entropic Risk Thompson Sampling) using Thompson sampling with an entropic risk measure.
result Regret bounds for ERTS under entropic risk measure provided.
TS-Insight visualizes Thompson Sampling for better debugging and trust.
problem Thompson Sampling's black box nature hinders debugging and trust.
method TS-Insight is a visual analytics tool that traces evolving posteriors and evidence counts.
result Visualizations help in verifying, diagnosing, and explaining Thompson Sampling dynamics.
Maximize to Explore integrates RL components for efficient policy discovery.
problem Balancing exploration and exploitation in online RL with general function approximators.
method Integrates estimation, planning, and exploration into a single objective function.
result Achieves sublinear regret for MDPs and MGs with general function approximations.
We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses. SCAL is a variant of SCAL (Fruit et al., 2018) that performs efficient exploration-exploitation in any unknown weakly-communicating MDP for which an upper bo…
The paper analyzes frameworks for integrating sustainability into investment decisions.
problem Understanding how ESG factors influence investment choices.
method Examined and analyzed various theoretical frameworks including Behavioral Finance, Modern Portfolio, and Risk Management.
result Investors increasingly integrate ESG factors to optimize financial outcomes and societal goals.
We propose and investigate a class of new algorithms for sequential decision making that interacts with \textit{a batch of users} simultaneously instead of \textit{a user} at each decision epoch. This type of batch models is motivated by interactive marketing and clinical trial, where a group of people are treated simu…
Trieste optimizes black-box functions using TensorFlow for efficient decision-making.
problem Optimizing complex, black-box functions efficiently.
method Bayesian optimization and active learning with TensorFlow integration.
result Trieste enables efficient exploration of black-box functions.
Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either regret or -consistent inference, with trade-off between exploration and exploitation.
Deep imagination optimizes decision-making in large trees with limited resources.
problem Optimal planning in large decision trees with limited resources and time.
method Analytical solutions and numerical analysis of sampling capacity allocation.
result Optimal policy is to allocate few samples per level for deep exploration, favoring depth over breadth.
This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. B…