A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The paper introduces a new intrinsic reward method for exploration in reinforcement learning.
New RL approach uses future state and action visitation measures for better exploration.
The paper analyzes how SGD visits different regions of a non-convex problem's state space.
The paper introduces return parity for fairness in MDPs, addressing delayed and adverse effects.
RECODE uses clustering and embedding to track state visitation counts in RL.
BRPO optimizes batch RL policies to better exploit state-action differences.
Study how untrained policies explore in RL environments.
Unsupervised model detects healthcare fraud from patient visit data.
MedGraph learns patient visit embeddings from EMRs, capturing both attributes and temporal sequences.
New algorithm stabilizes RL policy learning through divergence regularization.
Hidden tree Markov models allow learning distributions for tree structured data while being interpretable as nondeterministic automata. We provide a concise summary of the main approaches in literature, focusing in particular on the causality assumptions introduced by the choice of a specific tree visit direction. We w…
BeBold improves exploration in sparse-reward tasks by regulating visitation counts.
HDT improves MCMC on graphs with history-dependent sampling.
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approaches have generally ignored the mismatch between the distribution of states visited under the behavior policy used to collect data, and what …
Computational models that forecast the progression of Alzheimer's disease at the patient level are extremely useful tools for identifying high risk cohorts for early intervention and treatment planning. The state-of-the-art work in this area proposes models that forecast by using latent representations extracted from t…
VisitHGNN predicts visit probabilities between neighborhoods and POIs using graph neural networks.
New method reduces state distribution mismatch in off-policy RL.
The paper develops personalized DAG models for web user behavior.
Framework identifies comorbidities for frequent ED and inpatient visits.
Estimating statistical uncertainties allows autonomous agents to communicate their confidence during task execution and is important for applications in safety-critical domains such as autonomous driving. In this work, we present the uncertainty-aware imitation learning (UAIL) algorithm for improving end-to-end control…
Study online RL with mismatched dynamics, achieving sublinear regret.
Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.
Algorithm optimizes constrained reinforcement learning with dual variables.
ConCare personalizes healthcare predictions by capturing EMR features.
Pediatric asthma is the most prevalent chronic childhood illness, afflicting about 6.2 million children in the United States. However, asthma could be better managed by identifying and avoiding triggers, educating about medications and proper disease management strategies. This research utilizes deep learning methodolo…
Conventional Monte Carlo simulations are stochastic in the sense that the acceptance of a trial move is decided by comparing a computed acceptance probability with a random number, uniformly distributed between 0 and 1. Here we consider the case that the weight determining the acceptance probability itself is fluctuati…
Is it true that patients with similar conditions get similar diagnoses? In this paper we show NLP methods and a unique corpus of documents to validate this claim. We (1) introduce a method for representation of medical visits based on free-text descriptions recorded by doctors, (2) introduce a new method for clustering…
Effective modeling of electronic health records presents many challenges as they contain large amounts of irregularity most of which are due to the varying procedures and diagnosis a patient may have. Despite the recent progress in machine learning, unsupervised learning remains largely at open, especially in the healt…
Study automates detection of visitation disruptions in ICU patients.
Paper tackles efficient exploration of unseen graph-structured environments.
We consider the off-policy estimation problem of estimating the expected reward of a target policy using samples collected by a different behavior policy. Importance sampling (IS) has been a key technique to derive (nearly) unbiased estimators, but is known to suffer from an excessively high variance in long-horizon pr…
Study best arm identification in restless bandits with unknown TPMs.
The paper introduces FMCI and hybrid decoding for hidden Markov models.
Clinical data for ambulatory care, which accounts for 90% of the nations healthcare spending, is characterized by relatively small sample sizes of longitudinal data, unequal spacing between visits for each patient, with unequal numbers of data points collected across patients. While deep learning has become state-of-th…
A web browser should not be only for browsing web pages but also help users to find out their target websites and recommend similar type websites based on their behavior. Throughout this paper, we propose two methods to make a web browser more intelligent about link prediction which works during typing on address-bar a…
Self-guided ALPs improve MDP policies without domain knowledge.
Standard reinforcement learning methods aim to master one way of solving a task whereas there may exist multiple near-optimal policies. Being able to identify this collection of near-optimal policies can allow a domain expert to efficiently explore the space of reasonable solutions. Unfortunately, existing approaches t…
Optimizes RTB campaigns by selecting user profiles and website configurations.
What is a good exploration strategy for an agent that interacts with an environment in the absence of external rewards? Ideally, we would like to get a policy driving towards a uniform state-action visitation (highly exploring) in a minimum number of steps (fast mixing), in order to ease efficient learning of any goal-…
Suppose an agent is in a (possibly unknown) Markov Decision Process in the absence of a reward signal, what might we hope that an agent can efficiently learn to do? This work studies a broad class of objectives that are defined solely as functions of the state-visitation frequencies that are induced by how the agent be…
New results on inferring hidden states in trackable weak models.
Paper proposes machine learning model for early Alzheimer's diagnosis.
Bootstrap method for Markov chains in reinforcement learning.
Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurre…
This work bridges offline RL and DRL to address distributional shift.
Dyna is an architecture for model-based reinforcement learning (RL), where simulated experience from a model is used to update policies or value functions. A key component of Dyna is search-control, the mechanism to generate the state and action from which the agent queries the model, which remains largely unexplored. …
In this paper, we apply neural networks into digital marketing world for the purpose of better targeting the potential customers. To do so, we model the customer online behaviours using dedicated neural network architectures. Starting from user searched keywords in a search engine to the landing page and different foll…
When using reinforcement learning (RL) algorithms to evaluate a policy it is common, given a large state space, to introduce some form of approximation architecture for the value function (VF). The exact form of this architecture can have a significant effect on the accuracy of the VF estimate, however, and determining…