To solve a text-based game, an agent needs to formulate valid text commands for a given context and find the ones that lead to success. Recent attempts at solving text-based games with deep reinforcement learning have focused on the latter, i.e., learning to act optimally when valid actions are known in advance. In thi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Action Validity”
Recourse explanations can become invalid if collective actions change statistical data.
problem Recourse explanations may become invalid due to collective behavior changing data statistics.
method Formal characterization of conditions under which recourse explanations remain valid under performativity.
result Recourse actions may become invalid if they are influenced by or intervene on non-causal variables.
Introduces Conditional Action Trees to simplify RL action spaces.
problem Challenges in RL with large, complex action spaces.
method Structures action spaces and reduces complexity through Conditional Action Trees.
result Demonstrates effectiveness in reducing action space and improving decision making.
A new model validation framework for agentic AI systems based on POMDPs.
problem Model validation of agentic AI systems.
method A POMDP-based framework for belief-state, forecast, and policy validation.
result The framework decomposes autonomous decision making into information, beliefs, forecasts, actions, and utility.
Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we consider the scenario with discrete-continuous hybrid action space. To handle hybrid action space, previous works either approximate the hyb…
From Reviews to Actionable Insights: An LLM-Based Approach for Attribute and Feature Extractionstat.ML
LLM extracts actionable insights from customer reviews.
problem Extracting actionable insights from customer reviews.
method Large language model approach distinguishing perceptual attributes from actionable features.
result High consistency and predictive validity of LLM insights compared to human coders.
Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actionscs.LG
EbC learns equivariant embeddings from unlabeled group actions.
problem Learning equivariant embeddings from unlabeled group actions.
method Equivariance by Contrast (EbC) method to learn equivariant embeddings from observation pairs .
result High-fidelity equivariance in latent space for diverse groups.
Improving cancer treatment decisions requires considering causal effects, not just model accuracy.
problem Cancer outcome prediction models may cause harm when used for treatment decisions.
method Explains the importance of considering causal effects in model validation and provides guidelines.
result Building and validating models that are useful for decision making requires considering causal effects.
The standard P. A. Smith theory of p-group actions on spheres, disks, and euclidean spaces is extended to the case of p-group actions on tori (i.e., products of circles) and coupled with topological surgery theory to give a complete topological classification, valid in all dimensions, of the locally linear, orientation…
New AR framework handles missing values for better recourse actions.
problem Existing AR methods fail with missing values, leading to invalid or costly actions.
method Formulates task using multiple imputation and mixed-integer linear optimization.
result Efficacy of new method demonstrated in experiments.
The paper investigates the effects of invalid action masking in policy gradient algorithms.
problem Invalid actions in policy gradient algorithms can lead to suboptimal performance.
method The paper provides theoretical justification and empirical demonstrations of the importance of invalid action masking.
result Invalid action masking is crucial as the number of invalid actions increases.
Deep RL learns to construct objects from 2D images by avoiding brick overlaps.
problem Learning to construct objects from partial 2D images with constraints.
method Combination of reinforcement learning and action validity prediction.
result Successfully constructs unseen objects from 2D images.
Causal Bayesian networks interpret actions as interventions to connect models to real-world outcomes.
problem Connecting causal model predictions to real-world outcomes.
method Formal framework to interpret actions as interventions and prove impossibility results.
result No non-circular interpretation exists that satisfies natural desiderata without violating some.
New method reduces bias and variance in OPE for large action spaces.
problem High bias and variance in OPE for large, combinatorial action spaces.
method Factored action spaces and decomposed importance sampling.
result Decomposed IS estimators have less variance than non-decomposed versions.
We introduce a framework for model learning and planning in stochastic domains with continuous state and action spaces and non-Gaussian transition models. It is efficient because (1) local models are estimated only when the planner requires them; (2) the planner focuses on the most relevant states to the current planni…
New systolic inequality for 3D contact forms on Seifert bundles.
problem Bounding the shortest Reeb orbit period in terms of contact volume.
method Proved a general systolic inequality for S1-invariant contact forms on Seifert bundles.
result Validated systolic inequality on Seifert bundles with non-zero Euler number.
Paper tackles action selection in deep RL, proposing a data-driven approach.
problem High-dimensional action selection in deep RL environments.
method Data-driven approach with knockoff sampling for minimal sufficient actions.
result Method surpasses alternative techniques in performance and rewards.
New insights into symplectic loops and their flux groups.
problem Understanding symplectic loops and their properties.
method Analyzing symplectic diffeomorphisms and their orbits.
result Flux of symplectic loops vanishes for contractible orbits.
Geometry of Gauged Skyrmionsmath.DG
Generalizes skyrmion theory to gauged maps with -action.
problem Finding non-trivial minimizers of gauged skyrmion energy.
method Introduces a gauged energy functional and studies BPS equations.
result Classifies solutions for and .
Stochastic Q-learning tackles large action spaces with reduced computation.
problem Effective decision-making in complex environments with large discrete action spaces.
method Stochastic value-based RL approaches that consider a sublinear number of actions in each iteration.
result Stochastic Q-learning achieves near-optimal returns with significantly reduced computation time.
Develops optimal decision-making framework for uncertain counterfactuals.
problem Ensuring reliability of predictions in high-stakes decisions.
method Policy-Coupled Risk-Averse Conformal Prediction (PC-RACP).
result Optimal prediction sets for counterfactual decisions with valid coverage.
Portfolio traders strive to identify dynamic portfolio allocation schemes so that their total budgets are efficiently allocated through the investment horizon. This study proposes a novel portfolio trading strategy in which an intelligent agent is trained to identify an optimal trading action by using deep Q-learning. …
Action chunking and data exploration improve behavior cloning in robotics.
problem Exponential errors in learning from demonstrations for continuous control tasks.
method Action chunking and exploratory data collection.
result Control-theoretic stability is key to improving imitation learning.
Policy certifies inventory levels meeting service requirements.
problem Maintaining stock levels meeting service requirements despite unknown demand.
method Data-driven order policy using online learning and integral action.
result Valid inference method for finite samples.
Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.
problem Exploration-exploitation dilemma in finite-horizon reinforcement learning with metric state-action spaces.
method Kernel-UCBVI, leveraging smoothness and kernel estimators of rewards and transitions.
result First regret bound for kernel-based RL using smoothing kernels, .
Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.
problem Optimizing robot navigation in continuous-valued spaces with uncertainty.
method Autoregressive active inference agent using message passing on a factor graph.
result Agent modulates action based on predictive uncertainty, leading to better model of dynamics.
Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep -learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the …
We propose a new low-cost machine-learning-based methodology which assists designers in reducing the gap between the problem and the solution in the design process. Our work applies reinforcement learning (RL) to find the optimal task-oriented design solution through the construction of the design action for each task.…
Identification of clusters of investors from their real trading activity in a financial marketq-fin.TR
We use statistically validated networks, a recently introduced method to validate links in a bipartite system, to identify clusters of investors trading in a financial market. Specifically, we investigate a special database allowing to track the trading activity of individual investors of the stock Nokia. We find that …
DRL agents perform poorly at high decision frequencies, but a new algorithm improves performance.
problem DRL agents struggle at high decision frequencies, leading to poor performance.
method Proved that DRL agents' action-conditioned return distributions collapse to their policy's return distribution as decision frequency increases. Defined superiority as a probabilistic generalization of advantage for high-frequency value-based RL.
result Proper modeling of superiority distribution improves performance of controllers at high decision frequencies.
A new method, Count-MORL, improves offline reinforcement learning by using state-action frequency.
problem Improving offline reinforcement learning performance.
method Integrates count-based conservatism into model-based offline reinforcement learning.
result The learned policy is near-optimal and outperforms existing methods.
MACC learns communication protocols by adapting counterfactual reasoning.
problem Credit assignment and non-stationarity in communication environments.
method Adapts counterfactual reasoning to overcome credit assignment and uses action policy and Q-function of other agents to handle non-stationarity.
result MACC outperforms state-of-the-art baselines in four scenarios.
The paper proposes a principle for dynamically adjusting the granularity of reinforcement learning abstractions.
problem Lack of general principles for dynamically adjusting the granularity of reinforcement learning abstractions.
method The paper proposes a principle based on rate-distortion theory, formalized through a performance certificate decomposing value error into learning and abstraction error bounds.
result Soft state-action abstractions can achieve near-optimal performance under substantial lossy compression of state and action information.
Predicts next actions in soccer possessions using path signatures.
problem Predicting next actions in soccer possessions with high accuracy.
method Leveraging path signatures to encode spatio-temporal structure of recent possessions, avoiding manual feature engineering.
result Our approach outperforms transformer-based benchmarks across various loss metrics and reduces computational cost.
The use of Reinforcement Learning (RL) is still restricted to simulation or to enhance human-operated systems through recommendations. Real-world environments (e.g. industrial robots or power grids) are generally designed with safety constraints in mind implemented in the shape of valid actions masks or contingency con…
Proposes meTS for efficient exploration in correlated bandits.
problem Efficiently explore in bandits with correlated actions.
method Mixed-effect model with Thompson Sampling.
result Bound on Bayes regret with two terms.
For a smooth (locally trivial) principal bundle in Ehresmann's sense, the relation between the commuting vertical and horizontal actions of the structural Lie group and the structural Lie groupoid (isomorphisms between vertical fibers) is regarded as a special case of a symmetrical concept of conjugation between "princ…
New algorithm balances personalization and statistical validity in MRTs.
problem Optimizing decisions in nonstationary settings with habituation and recovery.
method ROGUE-TS Thompson Sampling with probability clipping.
result Achieves lower regret and maintains high statistical power.
Graph neural Thompson Sampling improves online decision-making for graph data.
problem Online decision-making with graph-structured rewards.
method GNN-TS algorithm using GNN for mean reward estimation and graph neural tangent features for uncertainty.
result GNN-TS achieves a state-of-the-art regret bound of .
KeRNS tackles non-stationary reinforcement learning in metric spaces.
problem Non-stationary reinforcement learning in metric spaces.
method KeRNS uses time-dependent kernels to model non-stationary Markov Decision Processes (MDPs).
result KeRNS achieves a regret bound that scales with the covering dimension and total variation of the MDP.
We study Lagrangian submanifolds foliated by (n-1)-spheres in R^2n for n>2. We give a parametrization valid for such submanifolds, and refine that description when the submanifold is special Lagrangian, self-similar or Hamiltonian stationary. In all these cases, the submanifold is centered, i.e. invariant under the act…
Causal inference analysis is the estimation of the effects of actions on outcomes. In the context of healthcare data this means estimating the outcome of counter-factual treatments (i.e. including treatments that were not observed) on a patient's outcome. Compared to classic machine learning methods, evaluation and val…
Develops methods for finding counterfactual explanations in sequential decision making.
problem Finding counterfactual explanations for sequential decision making processes.
method Formal characterization of sequential actions and states using Markov decision processes and Gumbel-Max structural causal model. Introduces a polynomial time algorithm based on dynamic programming.
result Algorithm finds optimal counterfactual explanations for sequential decision making.
Configuration spaces for computer systems can be challenging for traditional and automatic tuning strategies. Injecting task-specific knowledge into the tuner for a task may allow for more efficient exploration of candidate configurations. We apply this idea to the task of index set selection to accelerate database wor…
Realtime price impact detectionq-fin.TR
Detecting real-time price impact in algo trading
problem Identifying the impact of traders' actions on market prices
method Measuring timing synchronicity between trader actions and adverse market events
result Detecting price impact on a per-action basis
Paper proposes a hybrid AI method to optimize pandemic actions.
problem Optimizing government actions to balance public health and economy.
method Combines Deep Q-Learning and Genetic Algorithms for optimal sequences of actions.
result Deep Q-Learning outperforms Genetic Algorithms in optimizing action sequences.
It is shown that on a compact spin symmetric space with a Kähler or Quaternion-Kähler structure, the first eigenvalue of the Dirac operator is linked to a ''{lowest}'' action of the holonomy, given by the fiberwise action on spinors of the canonical forms characterized by this holonomy. The result is also verified for …
Bayes-CPACE: PAC Optimal Exploration in Continuous Space Bayes-Adaptive Markov Decision Processescs.LG
We present the first PAC optimal algorithm for Bayes-Adaptive Markov Decision Processes (BAMDPs) in continuous state and action spaces, to the best of our knowledge. The BAMDP framework elegantly addresses model uncertainty by incorporating Bayesian belief updates into long-term expected return. However, computing an e…