Due to the recent advancements in wearables and sensing technology, health scientists are increasingly developing mobile health (mHealth) interventions. In mHealth interventions, mobile devices are used to deliver treatment to individuals as they go about their daily lives. These treatments are generally designed to im…
Proposes pT-Learning for optimal dynamic treatment regimes in mHealth.
problem Challenges in learning optimal dynamic treatment regimes with large intervention options and infinite time horizon.
method Proximal Temporal consistency Learning (pT-Learning) framework for adaptively adjusting between deterministic and stochastic policies.
result Minimax estimator avoids double sampling issue and can incorporate off-policy data.
New method improves mHealth user engagement using Thompson sampling for count data.
problem Optimizing mHealth interventions for distal outcomes through proximal context.
method Combines count data models with Thompson sampling for contextual bandits.
result Improves user engagement in mHealth trials compared to existing methods.
Proposes an efficient algorithm for mHealth that makes real-time physical activity suggestions.
problem Training complex models efficiently in real-time mHealth applications.
method Streamlined empirical Bayes procedure for fitting linear mixed effects models.
result Improves accuracy and speed over state-of-the-art approaches in mHealth applications.
An app-based mHealth intervention uses reinforcement learning to send effective reminder notifications.
problem Designing an effective reinforcement learning algorithm for app-based mHealth interventions.
method Developed a reinforcement learning algorithm to send reminder notifications based on participant likelihood of app engagement.
result The algorithm improved app-based mHealth interventions by reducing participant burden and promoting behavior change.
Meta-learning method for estimating time-varying mHealth intervention effects.
problem Complex mHealth data and uncertain randomization probabilities.
method DR-WCLS meta-learning procedure for causal excursion effects.
result More efficient and consistent estimates of causal excursion effects.
IntelligentPooling improves treatment decisions in mHealth.
problem Optimizing treatment decisions in mobile health with limited data and non-stationary responses.
method Generalized Thompson-Sampling bandit algorithms to IntelligentPooling, addressing differential response, limited data, and non-stationary responses.
result IntelligentPooling achieves 26% lower regret compared to state-of-the-art methods.
In mobile health (mHealth), reinforcement learning algorithms that adapt to one's context without learning personalized policies might fail to distinguish between the needs of individuals. Yet the high amount of noise due to the in situ delivery of mHealth interventions can cripple the ability of an algorithm to learn …
This paper considers the actor-critic contextual bandit for the mobile health (mHealth) intervention. The state-of-the-art decision-making methods in mHealth generally assume that the noise in the dynamic system follows the Gaussian distribution. Those methods use the least-square-based algorithm to estimate the expect…
Bayesian ATM improves stability and efficiency in mobile health interventions.
problem Balancing intervention efficacy with user burden in mobile health interventions.
method Bayesian extension to ATM using Kalman filter-style updates.
result Bayesian ATM achieves comparable or improved scalarized returns with lower variance and more stable policy behavior.
Generative model predicts menstrual cycle lengths accounting for self-tracking artifacts.
problem Uncertainty in self-tracked health data due to user adherence.
method Hierarchical, generative model using machine learning.
result Model yields state-of-the-art performance in predicting menstrual cycle lengths.
Efficient algorithm for mobile health provides timely physical activity suggestions.
problem Inefficient reinforcement learning for mobile health settings.
method Contextual bandit algorithm with linear mixed effects model and hyper-parameter updating.
result Improves speed and accuracy by up to 99% and 56%.
New method for efficient personalized learning in mobile health.
problem Efficient and personalized learning in mobile health.
method Proposes a novel generative process on kernel composition for online Gaussian Process regression.
result Trajectories of kernel evolutions can be transferred between users to improve learning and kernels are meaningful for mHealth prediction.
Develops a real-time exercise recommendation system using deep learning.
problem Improving accuracy in exercise recommendation systems without user feedback.
method Deep recurrent neural network with attention mechanisms, real-time expert feedback.
result Improved accuracy in exercise recommendation system after real-time active learning.
We introduce Parameterized Exploration (PE), a simple family of methods for model-based tuning of the exploration schedule in sequential decision problems. Unlike common heuristics for exploration, our method accounts for the time horizon of the decision problem as well as the agent's current state of knowledge of the …
We introduce Contrastive Multivariate Singular Spectrum Analysis, a novel unsupervised method for dimensionality reduction and signal decomposition of time series data. By utilizing an appropriate background dataset, the method transforms a target time series dataset in a way that evinces the sub-signals that are enhan…
A significant challenge for a supervised learning approach to inertial human activity recognition is the heterogeneity of data between individual users, resulting in very poor performance of impersonal algorithms for some subjects. We present an approach to personalized activity recognition based on deep embeddings der…
Method constructs prediction intervals for time-varying individual treatment effects.
problem Accurately quantify uncertainty of individual treatment effects across multiple decision points.
method Conformal inference techniques for time-varying ITEs with weaker assumptions.
result Guaranteed lower bound for coverage dependent on data non-exchangeability.
Activity and motion analysis has the potential to be used as a diagnostic tool for mental disorders. However, to-date, little work has been performed in turning stratification measures of activity into useful symptom markers. The research presented in this thesis has focused on the identification of objective activity …
The paper evaluates index-based allocation policies using data from randomized control trials.
problem Evaluating index-based allocation policies in resource-scarce scenarios.
method Using data from randomized control trials, the paper introduces an efficient estimator and methods for computing asymptotically correct confidence intervals.
result Valid statistical conclusions can be drawn for index-based allocation policies.
New method preserves privacy while improving machine learning accuracy.
problem Privacy-preserving machine learning for daily data.
method Compressive Privacy and multi-kernel method.
result Improved utility classification accuracy with privacy preservation.
New setting combines state evolution and corrupted context for better decision-making.
problem Decision-making in a changing state with unreliable context.
method Proposes a new algorithm using a referee to dynamically combine contextual bandit and multi-armed bandit policies.
result Improved empirical performance compared to existing algorithms.