Active Reinforcement Learning (ARL) is a twist on RL where the agent observes reward information only if it pays a cost. This subtle change makes exploration substantially more challenging. Powerful principles in RL like optimism, Thompson sampling, and random exploration do not help with ARL. We relate ARL in tabular …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
20 results for “ARL”
ARL uses queries to learn rewards, focusing on cost vs. reward value.
problem How to efficiently use queries to learn rewards in reinforcement learning.
method Proposed and evaluated heuristic approaches for ARL in multi-armed bandits and MDPs.
result Challenging aspects of ARL highlighted, including intractability of value computation.
ARL makes market makers resilient to adversarial conditions.
problem Creating resilient market makers against adversarial attacks.
method Transformed single-agent RL into a zero-sum game between market maker and adversary.
result ARL leads to risk-averse behavior and significant performance improvements.
ARL and Hawkes processes improve market-making strategies with variable volatility.
problem Enhancing market-making strategies to adapt to varying volatility levels and self-exciting behaviors.
method Integrates ARL, Hawkes processes, and variable volatility levels; shifts from Poisson to Hawkes process.
result 4-action MM trained in low-volatility environment adapts to high-volatility conditions, providing stable performance.
Non-parametric estimators improve quickest changepoint detection under irregular sequence lengths.
problem Limited and irregular sequence lengths hinder application of ARL and ADD in QCD.
method Analogies with survival analysis to model detection probabilities under truncation.
result KM-ARL and KM-ADD non-parametric estimators are asymptotically unbiased.
ARL-GEN adapts to the smallest model class in nested families for RL with improved regret.
problem Model selection for Reinforcement Learning with nested model families.
method Adaptive Reinforcement Learning (ARL-GEN) with value targeted regression and model selection module.
result ARL-GEN achieves a matching regret to an oracle with knowledge of the true model class.
ARL improves fairness without protected features, showing AUC improvements for worst-case groups.
problem Training fairness in ML without known protected features.
method Adversarially Reweighted Learning (ARL) using non-protected features and task labels.
result ARL improves Rawlsian Max-Min fairness with notable AUC improvements for worst-case groups.
KQT-EWMA monitors multivariate data streams online with flexible and practical change detection.
problem Online monitoring of multivariate data streams for detecting changes.
method Combines Kernel-QuantTree histogram and EWMA statistic for non-parametric monitoring.
result Controls Average Run Length (ARL0) while achieving comparable detection delays.
Optimizes quickest change detection with bounded means under ARL constraint.
problem Quickest detection of changepoints with bounded means under ARL constraint.
method Derives universal lower and upper bounds for detection delay.
result Achieves universal lower bound in the bounded mean detection setting.
A Markov Decision Process (MDP) is a popular model for reinforcement learning. However, its commonly used assumption of stationary dynamics and rewards is too stringent and fails to hold in adversarial, nonstationary, or multi-agent problems. We study an episodic setting where the parameters of an MDP can differ across…
Efficient online kernel CUSUM detects changes quickly and accurately.
problem Detecting changes in online data streams efficiently.
method Online kernel CUSUM using maximum kernel statistics.
result Increased sensitivity to small changes compared to existing methods.
Anticipatory Reinforcement Learning: From Generative Path-Laws to Distributional Value Functionscs.LG
ARL bridges non-Markovian decision processes with reinforcement learning, improving foresight and stability.
problem Inaccurate foresight in non-Markovian environments due to state-based methods' limitations.
method Lifted state space into a signature-augmented manifold, using a self-consistent field approach to anticipate future path-law.
result ARL achieves deterministic evaluation of expected returns with reduced computational complexity and variance.
We present new algorithms for detecting the emergence of a community in large networks from sequential observations. The networks are modeled using Erdos-Renyi random graphs with edges forming between nodes in the community with higher probability. Based on statistical changepoint detection methodology, we develop thre…
We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a "white" identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sampl…
Research uses deep learning and copulas to predict multivariate survival data.
problem Handling right-censored and correlated multivariate survival data.
method Integrates deep learning, copula functions, and survival analysis. Uses copula-based activation functions to model nonlinear dependencies.
result Enhanced prediction accuracy for multivariate survival responses.
We study sequential change-point detection procedures based on linear sketches of high-dimensional signal vectors using generalized likelihood ratio (GLR) statistics. The GLR statistics allow for an unknown post-change mean that represents an anomaly or novelty. We consider both fixed and time-varying projections, deri…
We develop a mixture procedure for multi-sensor systems to monitor data streams for a change-point that causes a gradual degradation to a subset of the streams. Observations are assumed to be initially normal random variables with known constant means and variances. After the change-point, observations in the subset wi…
Adversarial representation learning is a promising paradigm for obtaining data representations that are invariant to certain sensitive attributes while retaining the information necessary for predicting target attributes. Existing approaches solve this problem through iterative adversarial minimax optimization and lack…
NN-CUSUM detects changes in high-dimensional data using neural networks.
problem Detecting abrupt changes in high-dimensional data.
method Neural network-based CUSUM for online change-point detection.
result NN-CUSUM performs well in detecting changes in high-dimensional data.
New method detects changes in high-dimensional data from small samples.
problem Detecting changes in high-dimensional data with limited samples.
method Angular kernel scan framework for detecting marginal distributional shifts.
result Exact population mean factorization and asymptotically distribution-free test.