Active search is a learning paradigm for actively identifying as many members of a given class as possible. A critical target scenario is high-throughput screening for scientific discovery, such as drug or materials discovery. In this paper, we approach this problem in Bayesian decision framework. We first derive the B…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “policy discovery”
PODNet discovers plannable options from unstructured demonstrations.
problem Learning from unstructured, multi-objective demonstrations.
method Custom categorical variational autoencoder, recurrent option inference network, option-conditioned policy network, and option dynamics model.
result PODNet enables learning from demonstration for multiple tasks and planning.
ALIAS uses RL to learn DAGs without acyclicity constraints.
problem Efficiently learning DAGs from observational data without acyclicity constraints.
method ALIAS employs RL to generate DAGs in a single step with optimal complexity, bypassing acyclicity constraints.
result ALIAS outperforms state-of-the-art methods in causal discovery.
Option Encoder compresses reinforcement learning options into a policy basis.
problem Redundant options in reinforcement learning frameworks.
method Auto-encoder framework with constrained weights to discover a policy basis.
result Option Encoder reduces the number of options while maintaining performance.
SAC-NF improves RL exploration by using normalizing flows to discover policies faster.
problem Discovering optimal policies in sparse reward domains.
method Extending SAC with normalizing flows to improve exploration efficiency.
result SAC-NF accelerates policy discovery with smaller parameter usage.
Paper proposes NAC for efficient network discovery in incomplete networks.
problem Efficiently discover vertices with specific attributes in incomplete networks.
method Formulates network discovery as a reinforcement learning problem, uses deep reinforcement learning with task-specific network embeddings.
result Offline planning leads to significantly improved performance compared to online discovery algorithms.
AI-Interpret transforms opaque policies into simple, interpretable decision rules.
problem Designing effective decision aids for professionals to mitigate decision-making biases.
method Combining imitation learning, program induction, and clustering to transform learned policies into interpretable descriptions.
result Providing interpretable decision rules as flowcharts significantly improves people's planning strategies and decisions.
The paper improves experimental design by weighting diversity metrics with quality, leading to more diverse and effective discoveries.
problem Existing experimental design techniques favor exploitation over exploration, leading to local optima and insufficient diversity.
method The paper extends Vendi scores to account for quality and applies them to various experimental design problems.
result Quality-weighted Vendi scores allow for better balance between quality and diversity, resulting in 70%-170% more effective discoveries.
Bayesian model for cost-effectiveness analysis with subgroup discovery.
problem Statistical challenges in cost-effectiveness analysis, especially with non-random treatment assignment and censored data.
method Developed a nonparametric Bayesian model using Dirichlet and Gamma processes to estimate cost-survival distributions and identify cost-effectiveness subgroups.
result Identified and estimated policy-relevant causal CEA estimands using a Bayesian nonparametric g-computation procedure.
New method learns decisions from collective preferences without individual covariates.
problem Making decisions online without individual covariates.
method Collaborative filtering, matrix completion bandit, ε-greedy policy, online gradient descent, inverse propensity weighting.
result Method outperforms benchmarks and reveals new discoveries.
Automated process discovery is a class of process mining methods that allow analysts to extract business process models from event logs. Traditional process discovery methods extract process models from a snapshot of an event log stored in its entirety. In some scenarios, however, events keep coming with a high arrival…
Finite resources limit false discovery rate control in structured hypothesis spaces.
problem Controlling false discovery rate in hypothesis testing with finite data and structured hypothesis spaces.
method Framework for exact FDR control and adaptive power maximization.
result Exact FDR control and adaptive power maximization.
DiffATD efficiently discovers targets in partially observable environments using diffusion dynamics.
problem Efficiently discovering targets in partially observable environments with limited sampling.
method DiffATD uses diffusion dynamics to maintain a belief distribution over unobserved states, balancing exploration and exploitation.
result DiffATD outperforms baselines and supervised methods in diverse domains.
A new method optimizes material discovery by balancing exploration and exploitation.
problem Substantial experimental costs and lengthy development periods in material discovery.
method Threshold-Driven UCB-EI Bayesian Optimization (TDUE-BO) method.
result TDUE-BO significantly outperforms traditional BO methods in material discovery.
Optimizes COVID-19 testing policy using a Multi-Armed Bandit approach.
problem Balancing discovery of positive cases with population surveillance.
method Risk scoring and random sampling based on Multi-Armed Bandit theory.
result Effective prioritization captures 65-92% of positive cases with varying testing capacity.
DECI combines causal discovery and inference in a single model for diverse data types.
problem Combining causal discovery and inference methods for diverse data types.
method Develops a single flow-based non-linear additive noise model (DECI) for causal discovery and inference.
result DECI can recover ground truth causal graphs and perform (C)ATE estimation.
New framework for dynamic causal graph modeling and effect estimation.
problem Dynamic changes in causal relationships over time.
method Score-based causal discovery with autoregressive model structure.
result Dynamic causal graph with time-varying causal relations.
New method recovers causal order from dependent data.
problem Causal discovery methods fail with shared volatility or common scale effects.
method Linear Mean-Independent Acyclic Model (LiMIAM) with mean-independence restrictions.
result Compatible causal order can be recovered from dependent disturbances.
AutoGrow automatically discovers optimal depth in DNNs.
problem Designing optimal depth in deep neural networks is difficult and time-consuming.
method AutoGrow grows new layers in a seed architecture if it improves accuracy; stops if no improvement. Robust policies generalize to different architectures and datasets.
result AutoGrow discovers near-optimal depth on various datasets, improving accuracy-computation trade-off in ResNets.
NeuPL learns diverse policies in strategy games efficiently.
problem Iterative training of policies in strategy games leads to under-trained good-responses and wasteful repetition.
method NeuPL uses a single conditional model to represent a population of policies, offering convergence guarantees and transfer learning.
result NeuPL achieves better performance and efficiency across various domains, enabling access to novel strategies.
Bayesian nonparametrics enables dynamic skill discovery from expert data.
problem Fixed K for offline skill discovery in reinforcement learning.
method Variational inference, continuous relaxations, Bayesian nonparametrics.
result Nonparametric model with dynamically-changing number of options.
GeneDisco benchmarks experimental design for drug discovery.
problem Vast experimental design space in drug discovery.
method Machine learning for optimal experimental design.
result Standardised benchmark suite for active learning.
Gradient-based algorithm improves model performance with triage.
problem Improving model accuracy with human expert involvement.
method Formal characterization of triage, optimal triage policy as deterministic threshold rule, gradient-based algorithm.
result Gradient-based algorithm finds triage policies and models with increasing performance.
Adaptive teacher improves sample efficiency and mode coverage in sampling tasks.
problem Efficient exploration and mode coverage in sampling tasks with amortized inference.
method An adaptive training distribution (teacher) guides the training of the primary sampler (student).
result Improves sample efficiency and mode coverage across various tasks.
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.
This paper detects Markov violations in RL with noise, improving policy development.
problem Partial observability and sensor/actuator noise invalidate Markovian assumptions in RL.
method Combines PCMCI causal discovery with Markov Violation score (MVS).
result Even substantial noise doesn't always disrupt multi-step dependencies.
New method shows data-driven causal studies can be misleading.
problem Misattribution of causality in data-driven earth science studies.
method Subsample-based ensemble approach for robust causality analysis.
result Transfer entropy-based causal graphs can be spurious.
DG separates successes and failures by gating updates with advantage and surprisal.
problem Negative learning from surprising data in distributed reinforcement learning.
method DG gates each update with the product of advantage and surprisal, suppressing failures and preserving successes.
result DG outperforms other methods in various challenging reinforcement learning tasks.
AGFN improves causal discovery by integrating expert feedback and handling latent confounding.
problem Inaccurate causal discovery due to unreliable expert knowledge and latent confounding.
method Ancestral GFlowNet (AGFN) is a reinforcement learning algorithm that iteratively refines a policy based on noisy expert feedback to infer ancestral graphs.
result AGFN converges to the true ancestral graph given accurate expert responses and outperforms baselines in structural Hamming distance and Bayesian Information Criterion.
MO2 learns useful behaviours from past experience for new tasks.
problem Discovering useful behaviours from past experience and transferring them to new tasks.
method Model-Based Offline Options (MO2) framework supporting sample-efficient bottleneck option discovery over continuous state-action spaces.
result MO2 outperforms recent option learning methods on complex long-horizon continuous control tasks.
Local discovery method uncovers direct unfairness in complex systems.
problem Identifying causal pathways of unfairness in complex domains.
method Local discovery for direct discrimination (LD3) method.
result LD3 returns a valid adjustment set (VAS) for assessing unfairness.
POETS optimizes LLMs by combining policy ensembles and KL regularization.
problem Balancing exploration and exploitation in decision-making and optimization.
method POETS uses policy ensembles and KL regularization to optimize LLMs efficiently.
result POETS achieves state-of-the-art sample efficiency across various domains.
Successor Options discovers reusable skills using landmark states.
problem Discovering reusable skills in reinforcement learning.
method Leverages Successor Representations to build a state space model and learns intra-option policies using a novel pseudo-reward.
result Demonstrates the approach's efficacy on grid-worlds and high-dimensional robotic control environments.
The paper learns robot skills from demonstrations without supervision.
problem Discovering robotic options from unlabelled demonstrations.
method Temporal variational inference for latent variable learning.
result The framework can learn options across multiple datasets.
On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluationcs.LG
Study on future-dependent value functions for off-policy evaluation in complex environments.
problem Exponential dependence on horizon in off-policy evaluation for complex observations.
method Developed novel coverage assumptions for POMDPs to achieve polynomial bounds.
result Achieved polynomial bounds on previously exponential quantities, improving off-policy evaluation.
New methods minimize GFlowNet training divergences for better sampling.
problem Training GFlowNets with KL divergence leads to biased and high-variance estimators.
method Design and implement efficient estimators for four divergence measures.
result Properly minimizing these divergences yields a provably correct and effective training scheme.
Algorithm finds causal effects from observational data using auxiliary variables.
problem Estimating causal effects from observational data with confounders.
method Gradient-based optimization using auxiliary variables.
result Algorithm outperforms alternatives in estimating true causal effect.
Logit-GFN accelerates GFlowNets training by scaling logits based on temperature.
problem Training temperature-conditional GFlowNets is numerically challenging.
method Logit-GFN uses a learned function of temperature to scale policy logits.
result Logit-GFN greatly accelerates GFlowNets training and improves generalization and mode discovery.
RODE learns roles to simplify multi-agent tasks.
problem Efficiently discovering roles for complex multi-agent tasks.
method Clustering actions based on effects, bi-level learning hierarchy, integrating action effects into role policies.
result RODE outperforms state-of-the-art MARL algorithms on StarCraft II benchmarks.
New method learns temporal abstractions by defining interest functions.
problem Learning temporal abstractions with limited, variable durations.
method Introduced interest functions to define initiation sets, enabling gradient-based learning.
result Demonstrated effectiveness in discrete and continuous environments.
Uses RL to discover causal DAGs from data.
problem Discovering causal structure among variables.
method Reinforcement Learning to search for DAGs.
result Improved search ability and flexible scoring.
Active-GRPO improves molecular optimization by actively deciding when to imitate or self-improve.
problem Training robust and efficient molecular optimization models with large language models.
method Active-GRPO combines imitation and reinforcement learning, upgrading references and policies dynamically.
result Improves molecular optimization performance, achieving statistically significant gains.
We consider the problem of learning the functions computing children from parents in a Structural Causal Model once the underlying causal graph has been identified. This is in some sense the second step after causal discovery. Taking a probabilistic approach to estimating these functions, we derive a natural myopic act…
Proposes a method to improve few-shot transfer in off-dynamics RL.
problem Traditional RL struggles with transferring policies between environments with different dynamics.
method Introduces a penalty to regulate source-trained policies in target environments with limited data.
result Improves performance in various off-dynamics RL scenarios compared to existing methods.
Novel online algorithm for hierarchical imitation learning.
problem Scalability issue in reinforcement learning and options discovery.
method Online Baum-Welch algorithm for hierarchical imitation learning.
result The online algorithm outperforms the batch version in both discrete and continuous environments.
Hierarchical MARL learns complementary skills for team coordination.
problem Creating intelligent agents that coordinate like human sports teams.
method Two-level hierarchical MARL with unsupervised skill discovery.
result Emergence of useful and complementary skills in team games.
MolecularRNN generates realistic molecules with desired properties.
problem Designing new molecules with specific properties.
method Graph recurrent generative model with likelihood pretraining and policy gradient tuning.
result Significant distribution shift to desired ranges for lipophilicity, drug-likeness, and melting point.
HyPE improves sample efficiency in DRL by discovering objects and hierarchies of skills.
problem Poor sample efficiency in DRL methods, especially in complex tasks.
method HyPE algorithm that discovers objects and generates hypotheses about their controllability, learning a hierarchy of skills.
result HyPE learns high-scoring policies an order of magnitude faster than state-of-the-art methods.