Study online RL with mismatched dynamics, achieving sublinear regret.
problem Exploration challenges in online RL with mismatched training and deployment dynamics.
method Introduce supremal visitation ratio, propose efficient algorithm with f-divergence. result Achieves sublinear regret in online RMDPs with optimal dependence on supremal visitation ratio and interaction episodes.
What happens when the Supreme Court of the United States decides a case impacting one or more publicly-traded firms? While many have observed anecdotal evidence linking decisions or oral arguments to abnormal stock returns, few have rigorously or systematically investigated the behavior of equities around Supreme Court…
The Brazilian court system is currently the most clogged up judiciary system in the world. Thousands of lawsuit cases reach the supreme court every day. These cases need to be analyzed in order to be associated to relevant tags and allocated to the right team. Most of the cases reach the court as raster scanned documen…
Two trees in the boundary of outer space are said to be \emph{primitive-equivalent} whenever their translation length functions are equal in restriction to the set of primitive elements of FN. We give an explicit description of this equivalence relation, showing in particular that it is nontrivial. This question is …
New algorithm guarantees optimal convergence rate for stochastic optimization.
problem Optimal convergence rate for stochastic optimization algorithms.
method Regularized versions of Minimization by Incremental Surrogate Optimization (MISO) with arbitrary recurrent data sampling.
result Expected optimality gap converges at O(n−1/2) under general recurrent sampling schemes. Robust learning method combines kernel smoothing and robust optimization.
problem Certifying robustness against distribution shifts in machine learning models.
method Adapting integral operator using supremal convolution for robustness, leveraging optimal transport.
result The method provides theoretical guarantees for certified robustness and competitive performance.
MedGraph learns patient visit embeddings from EMRs, capturing both attributes and temporal sequences.
problem Limited EMR embedding methods fail to capture patient demographics, utilisation, and code descriptions.
method MedGraph constructs an attributed bipartite graph and uses a point process to model temporal sequences.
result MedGraph outperforms state-of-the-art methods in medical risk prediction tasks.
Framework identifies comorbidities for frequent ED and inpatient visits.
problem Reducing resource usage and costs in frequent patients.
method Developed MSAR algorithm to identify comorbidities.
result MSAR identifies conditions most associated with reoccurring ED and inpatient visits.
Is it true that patients with similar conditions get similar diagnoses? In this paper we show NLP methods and a unique corpus of documents to validate this claim. We (1) introduce a method for representation of medical visits based on free-text descriptions recorded by doctors, (2) introduce a new method for clustering…
Study automates detection of visitation disruptions in ICU patients.
problem Difficulty in detecting frequent visitation disruptions in ICU patients.
method Used DensePose R-CNN model to count people in video frames, analyzed disruptions and patient outcomes.
result Automated method detects visitation disruptions, impacts on pain and length of stay examined.
RECODE uses clustering and embedding to track state visitation counts in RL.
problem Efficient novelty-based exploration in nonstationary RL environments.
method Non-parametric clustering, online density estimation, inverse dynamics loss.
result RECODE achieves state-of-the-art performance in challenging RL tasks.
Computational models that forecast the progression of Alzheimer's disease at the patient level are extremely useful tools for identifying high risk cohorts for early intervention and treatment planning. The state-of-the-art work in this area proposes models that forecast by using latent representations extracted from t…
The paper introduces a new intrinsic reward method for exploration in reinforcement learning.
problem Improving exploration in reinforcement learning agents.
method Intrinsic rewards proportional to the entropy of future state-action features.
result The new objective leads to improved visitation of features within individual trajectories.
Successful attempts to predict judges' votes shed light into how legal decisions are made and, ultimately, into the behavior and evolution of the judiciary. Here, we investigate to what extent it is possible to make predictions of a justice's vote based on the other justices' votes in the same case. For our predictions…
The paper analyzes how over-parameterization affects reinforcement learning performance.
problem Understanding the impact of over-parameterization in reinforcement learning.
method Theoretical analysis of Least-Square Temporal Difference (LSTD) algorithm with random features and asymptotic regime.
result Identification of a double descent phenomenon in reinforcement learning performance.
BeBold improves exploration in sparse-reward tasks by regulating visitation counts.
problem Efficient exploration in deep reinforcement learning under sparse rewards.
method Regulated difference of inverse visitation counts.
result BeBold solves 12 challenging tasks in MiniGrid with fewer steps than previous state-of-the-art.
A web browser should not be only for browsing web pages but also help users to find out their target websites and recommend similar type websites based on their behavior. Throughout this paper, we propose two methods to make a web browser more intelligent about link prediction which works during typing on address-bar a…
Unsupervised model detects healthcare fraud from patient visit data.
problem Detecting fraudulent healthcare bills from patient visit data.
method Uses LSTM and seq2seq models for anomaly detection, normalizes scores with EDF.
result Improves anomaly detection for high class imbalance problems.
The paper analyzes how SGD visits different regions of a non-convex problem's state space.
problem Understanding the long-run distribution of stochastic gradient descent in non-convex problems.
method Large deviations theory and randomly perturbed dynamical systems.
result The long-run distribution of SGD resembles the Boltzmann-Gibbs distribution with temperature equal to the step-size.
VisitHGNN predicts visit probabilities between neighborhoods and POIs using graph neural networks.
problem Estimating visit probabilities between neighborhoods and POIs for urban planning.
method Heterogeneous, relation-specific graph neural network (VisitHGNN) trained on mobility data.
result Strong predictive performance with high fidelity to observed travel behavior.
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approaches have generally ignored the mismatch between the distribution of states visited under the behavior policy used to collect data, and what …
Optimizes RTB campaigns by selecting user profiles and website configurations.
problem Maximizing impressions and profitability in RTB campaigns.
method Optimizes user profiles and website configurations, combines with other strategies.
result As the required number of visits increases, average profitability decreases.
New RL approach uses future state and action visitation measures for better exploration.
problem Improving exploration in reinforcement learning.
method Intrinsic reward based on future state and action visitation measures, using contraction operators.
result Policies achieve good state-action space coverage and high performance.
New method stabilizes FQE by reweighting Bellman targets.
problem Stability guarantees for FQE often rely on Bellman completeness, which can fail with function approximation.
method Proposes stationary-weighted FQE, reweighting Bellman targets by stationary target-to-behavior density ratio.
result Proves finite-sample linear convergence to stationary projected Bellman fixed point without Bellman completeness.
Surface parameterizations and registrations are important in computer graphics and imaging, where 1-1 correspondences between meshes are computed. In practice, surface maps are usually represented and stored as 3D coordinates each vertex is mapped to, which often requires lots of storage memory. This causes inconvenien…
Paper proposes machine learning model for early Alzheimer's diagnosis.
problem Early and accurate diagnosis of Alzheimer's Disease.
method Machine learning models, demographic, biomarker, and cognitive test data.
result 90% accuracy and 87% accuracy in predicting Alzheimer's development.
The paper develops personalized DAG models for web user behavior.
problem Understanding user behavior transitions between websites with user heterogeneity and network dependency.
method Personalized Binomial DAG models with network-structured covariates, embedding network structure into a dimension-reduced covariate, learning node neighborhoods, and exploring variance-mean relation.
result The proposed algorithm outperforms state-of-the-art competitors in heterogeneous data.
Given a positive function u∈W1,n, we define its John-Nirenberg radius at point x to be the supreme of the radius such that ∫Bt∣∇logu∣n<ε0n when n>2, and ∫Bt∣∇u∣2<ε02 when n=2. We will show that for a collapsing sequence in a fixed conformal class under some curvature c…
In this paper, we apply neural networks into digital marketing world for the purpose of better targeting the potential customers. To do so, we model the customer online behaviours using dedicated neural network architectures. Starting from user searched keywords in a search engine to the landing page and different foll…
Algorithm beats sports betting markets, showing inefficiencies.
problem Inefficiencies in sports betting markets.
method Created a betting algorithm using a novel dataset and win probability model.
result Above market returns for various sports betting markets.
We consider the off-policy estimation problem of estimating the expected reward of a target policy using samples collected by a different behavior policy. Importance sampling (IS) has been a key technique to derive (nearly) unbiased estimators, but is known to suffer from an excessively high variance in long-horizon pr…
This article was withdrawn by the arXiv.org administrators since it plagiarizes math.GT/0011056.
Unsupervised learning summarizes EHR data into a patient status vector.
problem Challenges in modeling electronic health records due to irregularities and varying procedures/diagnoses.
method Two-step unsupervised representation learning scheme using auto-encoders and forecasting tasks.
result Improved generalization performance on mortality and readmission tasks.
New batch RL method avoids overly optimistic policies.
problem Challenges in applying RL to large domains with limited data.
method Modified Bellman optimality and evaluation back-up for conservative updates.
result Can find approximately best policies within explored state-action space.
Enhances early risk assessments for pediatric outcomes using contrastive learning.
problem Improving risk assessments in early stages of pediatric development.
method Contrastive multi-modal framework that treats each time window as a distinct modality, training on all available data.
result Consistent improvements in early-stage risk assessments validated on real-world tasks.
We show how to learn low-dimensional representations (embeddings) of patient visits from the corresponding electronic health record (EHR) where International Classification of Diseases (ICD) diagnosis codes are removed. We expect that these embeddings will be useful for the construction of predictive statistical models…
Modeling infection hotspots to quantify effects of contact tracing and testing.
problem Capturing the role of infection hotspots in disease transmission.
method Temporal point process modeling framework to represent visits and disease transmission.
result Estimation of transmission rates at sites and households using Bayesian optimization.
Pediatric asthma is the most prevalent chronic childhood illness, afflicting about 6.2 million children in the United States. However, asthma could be better managed by identifying and avoiding triggers, educating about medications and proper disease management strategies. This research utilizes deep learning methodolo…
Improved off-policy evaluation for MDPs with weak distributional overlap.
problem Evaluation of policies when target and data-collection distributions are not strongly overlapping.
method Truncated Doubly Robust (TDR) estimators for off-policy evaluation in MDPs under weak distributional overlap.
result TDR estimators can recover large-sample behavior and are consistent even when distribution ratios are not square-integrable.
ConCare personalizes healthcare predictions by capturing EMR features.
problem Predicting patient outcomes from EMR data with personalization.
method Captures personal characteristics and time-aware distribution in EMR data.
result Improves healthcare prediction accuracy through personalized health context.
Unified treatment of reinforcement learning via convex duality.
problem Applying Fenchel-Rockafellar duality to reinforcement learning.
method Unified derivation of various RL settings using convex duality.
result Ability to perform policy evaluation and on-policy policy gradient with offline data.
Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.
problem Training RL policies to maximize sparse outcomes.
method Incentivizing policy matching state-action visitations of successful episodes.
result Significantly faster RL finetuning performance.
The paper introduces return parity for fairness in MDPs, addressing delayed and adverse effects.
problem Fairness in MDPs for dynamic domains with delayed and adverse effects.
method Proposes return parity, decomposes return disparity, and develops algorithms for state visitation distributional alignment.
result The proposed algorithms can successfully close the disparity gap while maintaining policy performance.
Algorithm optimizes constrained reinforcement learning with dual variables.
problem Minimizing convex functional subject to convex constraint in large state spaces.
method VPDPO algorithm using Lagrangian and Fenchel duality.
result Achieves sublinear regret and constraint violation, globally optimal policy.
We present the Bayesian Echo Chamber, a new Bayesian generative model for social interaction data. By modeling the evolution of people's language usage over time, this model discovers latent influence relationships between them. Unlike previous work on inferring influence, which has primarily focused on simple temporal…
Study how untrained policies explore in RL environments.
problem Challenges in reinforcement learning, especially sparse or adversarial reward structures.
method Theoretical and empirical analysis of untrained deep neural policies in a toy model.
result Untrained policies generate correlated actions and non-trivial state-visitation distributions.
A new approach for deep exploration in sparse reward reinforcement learning.
problem Slow or no learning in reinforcement learning with rare rewards.
method Long-term visitation count planning and decoupling exploration and exploitation.
result Significantly outperforms existing methods in sparse reward environments.
Point-of-Interest (POI) recommender systems play a vital role in people's lives by recommending unexplored POIs to users and have drawn extensive attention from both academia and industry. Despite their value, however, they still suffer from the challenges of capturing complicated user preferences and fine-grained user…