A new method improves reinforcement learning by directing exploration towards new knowledge.
problem Efficient exploration in reinforcement learning, especially in complex environments.
method Proposed E E E -values, a generalization of visit-counters, for directed exploration in model-free reinforcement learning. result Improves learning and performance in continuous Markov Decision Processes (MDPs) compared to traditional methods.
Directed exploration improves reinforcement learning efficiency and robustness.
problem Achieving good sample efficiency in reinforcement learning with efficient exploration.
method Directed exploration through goal-conditioned policies that are independent of uncertainty.
result Directed exploration is more efficient and robust to uncertainty than reward bonuses.
A new exploration method for RL using parameter space noise.
problem Improving exploration in deep reinforcement learning.
method Switching isotropic and directional exploration in parameter space with parameter space noise.
result The proposed method achieves competitive results and better performance in sparse reward environments.
Agent learns directed exploration policies to improve performance in hard games.
problem Improving exploration in complex games.
method Episodic memory-based intrinsic reward, self-supervised inverse dynamics, UVFA framework.
result Doubles performance in hard exploration games, achieves non-zero rewards in Pitfall!.
Unified algorithm for efficient pure exploration using dual variables.
problem Efficiently achieving a specific goal through adaptive experimentation.
method Introducing dual variables to derive optimal allocation conditions, leading to Information-Directed Selection.
result Top-two Thompson sampling attains asymptotic optimality for Gaussian best-arm identification.
IDS improves exploration in deep reinforcement learning.
problem Efficient exploration in reinforcement learning, especially with heteroscedastic returns.
method Information-Directed Sampling (IDS) for deep Q-learning.
result Significant improvement in Atari game performance over alternative approaches.
A new metric based on hitting probabilities for directed graphs and Markov chains.
problem Lack of metrics specifically adapted to asymmetric structure of directed graphs and Markov chains.
method Metric based on hitting probabilities, insensitive to shortest and average walk distances.
result New structural theory of directed graphs and utility for various applications.
This work shows how to use simulators to learn efficient exploration in real-world RL.
problem Sample complexity of real-world reinforcement learning.
method Coupling exploratory policies learned in simulators with practical approaches.
result Polynomial sample complexity in real world, exponential improvement over direct sim2real transfer.
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.
New algorithm balances exploration and exploitation in multi-armed bandits with structured priors.
problem Balancing exploration and exploitation in multi-armed bandits with structured priors.
method Value-function-driven online planning techniques with n-step lookahead.
result Sub-linear performance guarantee and strong practical performance in structured priors.
We begin an exploration of parametric Backlund transformations for hyperbolic Monge-Ampere systems. We compute invariants for such transformations and explore the behavior of four examples regarding their invariants, symmetries, and conservation laws. We prove some preliminary results and indicate directions for furthe…
A new algorithm STE for model-based RL improves learning rates.
problem Sparse rewards and computational intractability of estimating information gain.
method Developed a novel algorithm based on Stein Information Directed Exploration (STE)E.
result Achieves sublinear Bayesian regret, outperforming prior approaches.
Optimistic Actor-Critic improves exploration efficiency in reinforcement learning.
problem Poor sample efficiency in existing actor-critic methods.
method Introduces Optimistic Actor-Critic, approximating upper and lower bounds on state-action value function.
result Achieves state-of-the-art sample efficiency in challenging continuous control tasks.
Hypermodels improve exploration efficiency and accuracy.
problem Efficiently approximating Thompson sampling with large ensembles.
method Introducing hypermodels as a generalization of ensembles, including linear and neural network hypermodels.
result Hypermodels enable more accurate exploration and performance gains over Thompson sampling.
New RL method finds near-optimal policies efficiently.
problem Efficient exploration in model-free RL.
method Modifies Delayed Q-learning with long-term exploration bonus.
result Polynomial time PAC-MDP solution for near-optimal policy finding.
The study analyzes optimization trajectories in neural networks to reveal redundancy and redundancy-reducing strategies.
problem Understanding the directional structure and redundancy in neural network optimization.
method Introducing natural notions of complexity for optimization trajectories and analyzing their directional nature.
result Training only scalar batchnorm parameters can match the performance of training the entire network, indicating potential for hybrid optimization schemes.
In this paper, we explore and detail our experiments in a high-dimensionality, multi-class image classification problem often found in the automatic recognition of Sign Languages. Here, our efforts are directed towards comparing the characteristics, advantages and drawbacks of creating and training Support Vector Machi…
PC-PG balances exploration and exploitation in reinforcement learning.
problem Local policy gradient methods struggle with exploration.
method PC-PG uses an ensemble of learned policies (policy cover) to balance exploration and exploitation.
result PC-PG provides strong theoretical guarantees and empirical validation.
Noise can affect the overparametrization of QNNs, enabling new directions but also suppressing sensitivity.
problem The overparametrization of QNNs in the presence of noise.
method Analyzing the Quantum Fisher Information Matrix (QFIM) to understand how noise affects the rank of QFIM.
result Noise can turn previously-zero eigenvalues of the QFIM to non-zero, enabling exploration of new directions.
R3L uses planning algorithms to efficiently explore sparse reward environments.
problem Balancing exploration and exploitation in sparse reward reinforcement learning.
method Formulate exploration as a search problem using RRT, leverage demonstrations from initial solutions to refine RL policy.
result R3L outperforms classic and intrinsic exploration techniques, requiring fewer samples and achieving better asymptotic performance.
Proposes goal-based exploration for better learning efficiency.
problem Difficult exploration in reinforcement learning.
method Random goal selection followed by several steps in their direction.
result Generates longer and safer trajectories than individual random actions.
MAX uses ensemble models to efficiently explore novel situations in reinforcement learning.
problem Efficient exploration in reinforcement learning.
method Model-based active exploration using ensemble forward models and Bayesian disagreement.
result MAX is at least an order of magnitude more efficient than baselines in semi-random environments.
ManifoldMind uses adaptive-curvature probabilistic spheres for trustworthy recommendations in semantic hierarchies.
problem Sparse and abstract recommendation domains where users explore diverse conceptual paths.
method Adaptive-curvature probabilistic spheres, soft multi-hop inference, and curvature-aware semantic kernel.
result Superior NDCG, calibration, and diversity compared to baselines on public benchmarks.
Study uses deep learning to predict asset prices, finds complex target processes lead to meaningless predictions.
problem Complexity of successful price prediction models hinders understanding.
method Deep learning models for high-frequency price prediction, focusing on volatility and directional prediction.
result Inadequately defined target price process renders predictions meaningless.
We explore the geometrical interpretation of the PCA based clustering algorithm Principal Direction Divisive Partitioning (PDDP). We give several examples where this algorithm breaks down, and suggest a new method, gap partitioning, which takes into account natural gaps in the data between clusters. Geometric features …
The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.
problem Exploration in policy-gradient algorithms and its impact on policy optimization.
method Numerical optimization criteria and stochastic gradient analysis.
result Exploration techniques improve policy optimization by smoothing the learning objective and modifying gradient estimates.
The paper tackles exploration in reinforcement learning by accounting for two types of uncertainty.
problem Directed exploration for reinforcement learning agents with two sources of uncertainty.
method The approach involves learning parametric and return uncertainty with deep neural networks and estimating them in a Double Uncertain Value Network.
result The policy is derived from learned distributions based on Thompson sampling, showing improvement in domains with strong exploration challenges.
Machine learning models predict EUR/USD currency direction with 58.52% accuracy.
problem Predicting the directional movement of EUR/USD in the Foreign Exchange market.
method Comparative analysis of machine learning models, including decorrelated and non-decorrelated feature sets, and meta-estimators.
result 58.52% accuracy for one-day ahead forecasts.
Study explores loss design for decision trees to improve robustness against noisy labels.
problem Improving decision tree robustness to noisy labels.
method Investigated loss correction and symmetric losses, found ineffective.
result Other loss design directions need exploration for robust decision trees.
New discrete Ricci curvature for directed networks developed.
problem Directed networks require a new curvature measure.
method Extended Forman-Ricci curvature for directed networks, incorporating vertex and edge weights, and edge direction.
result New curvature measure captures higher-order correlations in directed networks.
Paper proposes DAG-DB for learning discrete DAGs via backpropagation.
problem Learning Directed Acyclic Graphs (DAGs) from data.
method DAG-DB uses Discrete Backpropagation with I-MLE and Straight-Through Estimation.
result DAG-DB learns DAGs effectively using probabilistic sampling and backpropagation.
New algorithm optimizes exploration in unknown MDPs.
problem Optimal exploration in unknown Markov decision processes.
method Trajectory-optimization-based approximate solution using Bayesian experimental design.
result Empirically superior to intrinsic-motivation-based algorithms in convergence and model fidelity.
A new distance measure balances projection exploration and informativeness.
problem Inefficient and incomplete projection sampling in existing sliced-Wasserstein distances.
method Proposes Distributional Sliced-Wasserstein (DSW) that optimally balances projection exploration and informativeness.
result DSW generalizes Max-SW and can be computed efficiently.
ASD algorithm maximizes model estimates by adaptively labeling points.
problem Maximizing model estimates through adaptive labeling of points in a sequential decision-making problem.
method Formulated a general information-directed sampling (IDS) algorithm with theoretical guarantees for linear, graph, and low-rank models.
result IDS algorithm outperforms in both simulation and real-data experiments for discovering chemical reaction conditions.
NEXT learns efficient paths in high dimensions using neural exploration-exploitation trees.
problem Learning efficient path planning in high-dimensional spaces.
method Neural Exploration-Exploitation Trees (NEXT) integrating neural architecture and UCB algorithm.
result NEXT achieves better sample efficiency and outperforms state-of-the-art methods.
The paper interprets policy-gradient algorithms using continuation theory.
problem Optimizing nonconvex functions in reinforcement learning.
method Formulates policy optimization as optimization by continuation, interprets policy-gradient algorithms as implicitly optimizing deterministic policies.
result Exploration in policy-gradient algorithms is seen as computing a continuation of the return of the policy.
IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
Improves neural network search in combinatorial spaces of mathematical symbols.
problem Early commitment and initialization bias limit exploration in neural network search.
method Entropy regularization and distribution initialization methods.
result Improves performance, increases sample efficiency, lowers solution complexity.
Go-Explore improves performance on hard-exploration problems in Atari games.
problem Challenges in reinforcement learning, especially with sparse or deceptive rewards.
method Exploits principles of remembering states, returning to promising states, and solving simulated environments.
result Scores significantly higher than previous state-of-the-art on Montezuma's Revenge and Pitfall.
The mixed scalar curvature of a foliated Riemannian manifold, i.e., an averaged mixed sectional curvature, has been considered by several geometers. We explore the Yamabe type problem: to prescribe the constant mixed scalar curvature for a foliation by a conformal change of the metric in normal directions only. For a h…
Robot learns from multiple teachers to efficiently achieve various motor skill outcomes.
problem Efficiently learning motor skills from multiple teachers and strategies.
method Hierarchical active decisions based on empirical evaluation of learning progress.
result Significantly more efficient learning and coherent strategy selection.
OPPO optimizes policy learning with provable efficiency.
problem Designing efficient policy optimization algorithms that incorporate exploration.
method Proposes an Optimistic Proximal Policy Optimization (OPPO) algorithm.
result Achieves i l d e O ( d 2 H 3 T ) ilde{O}(\sqrt{d^2 H^3 T} ) i l d e O ( d 2 H 3 T ) regret in episodic MDPs. New method learns DAGs from data without acyclicity constraint.
problem Learning DAGs from data without imposing acyclicity.
method Sparse matrix factorization and ℓ 1 \ell_1 ℓ 1 -penalized optimization. result Empirical success in recovering true graphs and almost-DAG graphs.
DLM for BNNs fails to improve over ELBO optimization.
problem Performance of DLM for Bayesian Neural Networks (BNNs).
method Direct Loss Minimization (DLM) compared to ELBO optimization.
result DLM does not significantly improve over ELBO optimization for BNNs.
New RL agent learns sparse rewards efficiently.
problem Sparse-reward environments and computational expense.
method Active inference with novel free energy minimization.
result High sample efficiency and online operation.
The paper explores updating multiple goals in reinforcement learning with neural networks.
problem Training agents to achieve mastery in complex, non-tabular domains.
method Three extensions of Kaelbling's all-goals updating approach using deep neural networks.
result Many-goals updating can be used to pre-train networks and improve learning on a single task.
Reward-free RL in linear MDPs is as hard as reward-aware RL.
problem Reward-free RL in linear MDPs without access to the reward function during exploration.
method Developed a computationally efficient algorithm with sample complexity O ~ ( d 2 H 5 / ε 2 ) \widetilde{\mathcal{O}}(d^2 H^5/ε^2) O ( d 2 H 5 / ε 2 ) . result Achieved optimal d d d dependence in linear MDPs for reward-free RL, matching the reward-aware RL setting. The paper explores Wiener-Granger causality and its computational enhancements.
problem Analyzing causal relationships between time series data.
method Detailed overview of Granger causality, historical development, and computational advancements.
result Enhanced application of Granger causality in various fields.