Method learns optimal treatment sequences from observational data.
problem Optimal dynamic treatment regimes for public policies and medical interventions.
method Doubly robust classification-based approach via backward induction.
result Achieves optimal convergence rate of n^(-1/2) for welfare regret.
In this paper, we consider multi-stage stochastic optimization problems with convex objectives and conic constraints at each stage. We present a new stochastic first-order method, namely the dynamic stochastic approximation (DSA) algorithm, for solving these types of stochastic optimization problems. We show that DSA c…
We consider dynamics of financial markets as dynamics of expectations and discuss such a dynamics from the point of view of phenomenological thermodynamics. We describe a financial Carnot cycle and the financial analogue of a heat machine. We see, that while in physics a perpetuum mobile is absolutely impossible, in ec…
Online health communities such as the online breast cancer forum enable patients (i.e., users) to interact and help each other within various subforums, which are subsections of the main forum devoted to specific health topics. The changing nature of the users' activities in different subforums can be strong indicators…
TK-GCN forecasts spatiotemporal dynamics using Koopman-enhanced graph convolutional networks.
problem Forecasting complex spatiotemporal dynamics over irregular domains.
method Two-stage framework: Koopman-enhanced Graph Convolutional Network (K-GCN) for spatial encoding and Transformer for temporal modeling.
result TK-GCN outperforms state-of-the-art methods in spatiotemporal cardiac dynamics forecasting.
Attention-only transformers learn from context via two stages of inference.
problem Learning from corrupted token sequences in minimal transformers.
method Two-stage empirical Bayes interpretation: kernel-weighted posterior mean and particle dynamics.
result Effective denoising without explicit noise schedules, showing posterior-mean recovery under asymptotic conditions.
Proposes a new method for dynamic treatment regimes that improves sample efficiency and stability.
problem Challenges in estimating optimal treatments for individuals with dynamic decision-making stages.
method Focuses on prioritizing alignment between observed and optimal treatment trajectories across decision stages.
result Improves sample efficiency and stability of IPWE-based methods by relaxing the alignment requirement.
The paper develops methods to estimate optimal treatment sequences under policy constraints.
problem Estimating the best sequence of treatments over multiple stages for individuals.
method Empirical welfare maximization approach, solving treatment assignment sequentially or simultaneously.
result Established convergence rates and upper bounds for estimation methods.
Algorithm achieves optimal pricing with minimal exploration for dynamic markets.
problem Optimal pricing in dynamic markets with contextual information.
method Localized exploration-then-commit (LetC) algorithm with pure exploration, refinement, and exploitation stages.
result Achieves minimax optimal, dimension-free regret bound.
A new method reduces complexity in estimating dynamic choice models.
problem Estimating structural parameters in dynamic discrete choice models using behavioral data.
method Two-stage approach: inverse reinforcement learning for Q-function estimation, state selection via clustering, and maximum likelihood estimation with nested fixed-point algorithm.
result The method mitigates the curse of dimensionality and provides finite-sample bounds on estimation error.
Proposes ML methods for robust price-sensitivity estimation in dynamic pricing.
problem Estimating price elasticities robustly in the presence of feature-dependent sensitivity.
method Poisson semi-parametric model with two-stage estimation: first-stage ML for observed purchases, second-stage Bayesian GLM for price-sensitivity.
result Reduces estimation error in price-sensitivity parameters from 25% to 4%.
A method for dynamic portfolio choice with uncertain parameters using Pontryagin projection.
problem Continuous-time CRRA portfolio choice in markets with estimated and uncertain coefficients.
method Simulation-based two-stage solver (DPO + Pontryagin projection) to maximize ex-ante objective.
result Projection stabilizes learning and accurately recovers analytic decisions, improving over model-free PPO.
This paper studies a composite problem involving the decision making of the optimal entry time and dynamic consumption afterwards. In stage-1, the investor has access to full market information subjecting to some information costs and needs to choose an optimal stopping time to initiate stage-2; in stage-2, the investo…
Transformers learn to integrate information from past positions incrementally, specializing heads in distinct patterns.
problem How transformers learn to integrate information from multiple past positions with varying statistical significance.
method High-order Markov chain task, incremental learning, sparse attention patterns, simplified differential equations, stage-wise convergence, early stopping as regularizer.
result Transformers learn to specialize heads in distinct patterns, shifting from competitive to cooperative learning dynamics.
The paper analyzes the training dynamics of neural networks using kernel methods.
problem Understanding the training dynamics of neural networks in high-dimensional settings.
method High-dimensional asymptotics and gradient flow on kernel least-squares objectives.
result The training dynamics of neural networks undergo three stages, characterized by behaviors in the Oracle and Empirical worlds.
To integrate strategic, tactical and operational decisions, the two-stage optimization has been widely used to guide dynamic decision making. In this paper, we study the two-stage stochastic programming for complex systems with unknown response estimated by simulation. We introduce the global-local metamodel assisted t…
Proposes DCNAR for dynamic causal inference from neural time series.
problem Uncertainty and evolution of causal structure in real-world domains.
method Two-stage neural causal modeling integrating discovery and inference.
result Dynamic causal inferences are more stable and meaningful than alternatives.
Estimates and infers multi-stage stationary treatment policies with variable selection.
problem Valid inference for multi-stage stationary treatment policies with high-dimensional feature variables.
method Estimate the value function using augmented inverse probability weighted estimator, apply penalty for variable selection, construct one-step improvements for valid inference.
result Improved estimators are asymptotically normal, valid inference for policy parameters demonstrated.
Improved latent dynamics identification framework reduces training time and improves accuracy.
problem Accurate numerical solutions of partial differential equations require computationally expensive solvers.
method Sequential decoder training (mLaSDI) to correct residual errors from previous stages.
result mLaSDI consistently outperforms standard LaSDI, achieving lower prediction errors and reduced training time.
Industrial recommender systems usually consist of the matching stage and the ranking stage, in order to handle the billion-scale of users and items. The matching stage retrieves candidate items relevant to user interests, while the ranking stage sorts candidate items by user interests. Thus, the most critical ability i…
With the increasing complexity of modern power systems, conventional dynamic load modeling with ZIP and induction motors (ZIP + IM) is no longer adequate to address the current load characteristic transitions. In recent years, the WECC composite load model (WECC CLM) has shown to effectively capture the dynamic load re…
CycleFQI tackles offline reinforcement learning for cyclic MDPs, mitigating state distribution mismatch.
problem Offline reinforcement learning for cyclic MDPs with heterogeneous dynamics and discount factors.
method CycleFQI decomposes the cyclic process into stage-wise sub-problems, using vector of stage-specific Q-functions.
result CycleFQI mitigates the curse of dimensionality and provides finite-sample suboptimality error bounds.
Dynamic treatment strategies on networks amplify policy impact through spillovers.
problem Effective dynamic treatment allocation in network settings.
method Q-Ising, a three-stage pipeline integrating Bayesian dynamic Ising model, treatment adoption histories, and offline reinforcement learning.
result Adaptive targeting outperforms static centrality benchmarks in Indian village microfinance networks and synthetic data.
Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Principle (F-Principle): a DNN tends to learn a target function from low to high frequencies during the training. The F-Principle has been very use…
SGD converges to critical points of normalized margin in late-stage training for homogeneous neural networks.
problem Analyzing the implicit bias of SGD on homogeneous neural networks.
method Interpreting SGD dynamics as an Euler-like discretization of a conservative field flow associated with the normalized classification margin.
result Normalized SGD iterates converge to the set of critical points of the normalized margin at late-stage training.
Study uses deep neural networks for causal inference tasks, especially in high-dimensional settings.
problem Challenges in direct estimation for complex causal inference tasks.
method Sequential multi-stage learning with doubly robust deep neural networks.
result Theoretical guarantees for DNNs' effectiveness in high-dimensional causal inference.
Electroencephalographic (EEG) monitoring of neural activity is widely used for sleep disorder diagnostics and research. The standard of care is to manually classify 30-second epochs of EEG time-domain traces into 5 discrete sleep stages. Unfortunately, this scoring process is subjective and time-consuming, and the defi…
RAHMC improves sampling from multimodal distributions using dissipative dynamics.
problem Sampling from multimodal distributions efficiently.
method RAHMC uses mode-repelling and mode-attracting stages with a single tuning parameter.
result RAHMC generates proposals that cross low-probability barriers efficiently.
We develop a tractable and flexible approach for incorporating side information into dynamic optimization under uncertainty. The proposed framework uses predictive machine learning methods (such as k-nearest neighbors, kernel regression, and random forests) to weight the relative importance of various data-driven unc…
Dynamic Influence Tracker measures changing sample importance during model training.
problem Static influence measurements during training overlook how sample importance varies over time.
method Dynamic Influence Tracker (DIT) captures time-varying sample influence across arbitrary time windows.
result DIT reveals distinct learning phases with shifting priorities and detects corrupted samples more efficiently.
This work extends reduction processes for nonholonomic discrete mechanical systems.
problem Nonholonomic discrete mechanical systems and their reductions.
method Introduces a category LDPd of discrete-time dynamical systems and a two-stage reduction process. result Two-stage reduction process produces systems isomorphic to one-stage reduction.
New model explains volatility after extreme stock market events.
problem Understanding volatility dynamics after extreme stock market events.
method Proposed a new dynamical model using high frequency minute data.
result Volatility after extreme events follows a stretched exponential decay initially and a power law decay later.
Q-learning adapted to find near-equivalent treatment strategies.
problem Finding optimal treatment sequences in dynamic treatment regimes.
method Introducing a worst-value tolerance criterion to find sets of near-equivalent policies.
result Constructs families of near-equivalent treatment strategies.
In this paper, we analyze dynamic programming as a novel approach to solve the problem of maximizing the profits of a bank. The mathematical model of the problem and the description of a bank's work is described in this paper. The problem is then approached using the method of dynamic programming. Dynamic programming m…
Improved forecasting of investment dynamics across heterogeneous panels using a two-stage model.
problem Forecasting investment dynamics in heterogeneous panels with varying dynamics.
method Two-stage architecture: global pooled AR(1) for shared persistence, local models for residual dynamics.
result Significant improvement in out-of-sample R2 from 0.630 to 0.677, with a gain of 0.047. Sleep plays a vital role in human health, both mental and physical. Sleep disorders like sleep apnea are increasing in prevalence, with the rapid increase in factors like obesity. Sleep apnea is most commonly treated with Continuous Positive Air Pressure (CPAP) therapy. Presently, however, there is no mechanism to moni…
ABS dynamically adjusts batch size based on policy stability, improving RL performance.
problem Diminishing returns with large batch sizes in RL due to non-stationary data.
method Adaptive Batch Scaling (ABS) with Behavioral Divergence metric.
result Larger batch sizes can improve RL performance, contrary to conventional wisdom.
We introduce the Pricing Engine package to enable the use of Double ML estimation techniques in general panel data settings. Customization allows the user to specify first-stage models, first-stage featurization, second stage treatment selection and second stage causal-modeling. We also introduce a DynamicDML class tha…
Time-continuous emotion prediction has become an increasingly compelling task in machine learning. Considerable efforts have been made to advance the performance of these systems. Nonetheless, the main focus has been the development of more sophisticated models and the incorporation of different expressive modalities (…
We present an algorithm producing a dynamic non-self-financing hedging strategy in an incomplete market corresponding to investor-relevant risk criterion. The optimization is a two stage process that first determines admissible model parameters that correspond to the market price of the option being hedged. The second …
Proposes a method for forecasting time series with multiple seasonality.
problem Forecasting time series with both short-term and long-term seasonality is challenging.
method Two-stage method: first generalizes ARMA model for multiple seasonality, second selects lag order.
result Method outperforms `Facebook Prophet` model in predictive performance.
Market-GAN adds context control to financial market data generation.
problem Lack of context labels and precision in generating context-aligned financial data.
method Proposes Contextual Market Dataset and Market-GAN architecture integrating GAN, autoencoder, and supervisors.
result Market-GAN outperforms state-of-the-art models in Dow Jones data generation.
MAntRA combines machine learning and Bayesian methods for time-dependent reliability analysis of unknown systems.
problem Time-dependent reliability analysis of systems with unknown governing physics.
method Combines machine learning, Bayesian statistics, and stochastic integration to discover and analyze SDEs from data.
result Demonstrates the effectiveness of MAntRA on three numerical examples, indicating its potential for in-situ and heritage structure analysis.
TNDE quantifies dynamic gene drivers from single-cell snapshots.
problem Reconstructing time-resolved regulatory effects in biological processes.
method Time-varying Network Driver Estimation (TNDE) using shared graph attention encoder and partial optimal transport.
result TNDE identifies stage-specific driver genes in mouse erythropoiesis.
The paper analyzes the training dynamics of a transformer for next-token prediction.
problem Understanding the non-asymptotic performance of transformers in next-token prediction.
method Characterizes training dataset properties, designs a two-stage training algorithm, and analyzes attention gradient properties.
result Trained transformers converge sub-linearly to max-margin solutions and exhibit linear convergence in cross-entropy loss.
Ancillaries have become a major source of revenue and profitability in the travel industry. Yet, conventional pricing strategies are based on business rules that are poorly optimized and do not respond to changing market conditions. This paper describes the dynamic pricing model developed by Deepair solutions, an AI te…
A new algorithm learns optimal personalized treatment plans online with low regret.
problem Learning optimal dynamic treatment regimes in an online setting.
method Developed a novel algorithm balancing exploration and exploitation for rate-optimal regret.
result Guaranteed rate-optimal regret for linear transition and reward models.
Motivated by the need for accurate frequency information, a novel algorithm for estimating the fundamental frequency and its rate of change in three-phase power systems is developed. This is achieved through two stages of Kalman filtering. In the first stage a quaternion extended Kalman filter, which provides a unified…