Predicting delayed outcomes in recommender systems using proxies.
problem Predicting delayed outcomes in recommender systems.
method Formalized as adversarial, delayed online learning problem; proposed Factored Forecaster (FF) and Residual Factored Forecaster (RFF) neural network architectures.
result Residual Factored Forecaster (RFF) outperforms direct forecaster and Factored Forecaster (FF) in predicting human behavior.
Firms delay write-downs for adverse macroeconomic and industry outcomes but not for firm-specific issues.
problem Timeliness of write-downs for adverse macroeconomic and industry outcomes versus firm-specific issues.
method Comparative analysis of write-downs driven by macroeconomic and industry outcomes versus firm-specific outcomes.
result Firms delay write-downs for adverse macroeconomic and industry outcomes but not for firm-specific issues.
Optimal policy for sequential treatment allocation minimizing risk.
problem Minimizing risk in sequential treatment allocation.
method Proposes a minimax optimal regret policy for unknown number of treatments.
result Regret grows slowly with the number of treatments.
Quantum machine learning without measurements using time-delayed equations.
problem Efficiently solving problems encoded in quantum controlled unitary operations.
method Iteration of a time-delayed equation for feedback in dynamics, eliminating measurements.
result Performance comparison with classical machine learning methods shows enhanced efficiency.
New algorithm optimizes long-term user satisfaction in recommendation systems.
problem Optimizing long-term user satisfaction in recommendation systems with delayed rewards.
method Developed a predictive model of delayed rewards and a bandit algorithm that balances exploration and exploitation.
result Our approach results in substantially better performance compared to short-term or delayed optimization.
This study aims to predict vessel stay and delay times at ports to optimize logistics.
problem Uncertainties in maritime logistics, including weather, cargo diversity, and port dynamics, lead to increased costs and inefficiencies.
method Developed predictive analytics to address shortcomings in previous works, using feature analysis and SHAP explanations.
result Predictive analytics can assist in efficient planning and scheduling of port operations, reducing costs and improving logistics.
New algorithm optimizes for long-term user satisfaction in delayed reward settings.
problem Optimizing for long-term user satisfaction in delayed reward settings.
method Developed a predictive model of delayed rewards and a bandit algorithm that combines rewards and surrogate outcomes.
result Our algorithm significantly outperforms methods that optimize for short-term proxies or rely solely on delayed rewards.
Study predicts hearing recovery in MD patients using TEOAE signals.
problem Predicting hearing recovery in MD patients during acute episodes.
method Applied machine learning to TEOAE signals from MD patients, using SVM for classification.
result Baseline TEOAE parameters can predict hearing recovery in MD patients.
Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.
problem Learning from sparse and delayed rewards in reinforcement learning.
method Randomized Return Decomposition (RRD) algorithm to redistribute rewards.
result Substantial improvement over baseline algorithms in experiments.
The paper tackles long-term treatment effects with persistent confounders using sequential short-term outcomes.
problem Estimating long-term treatment effects with persistent unmeasured confounders.
method Exploiting the sequential structure of short-term outcomes, the paper develops three novel identification strategies and corresponding estimators.
result The proposed methods outperform existing approaches in handling persistent confounders.
Agent decides when to measure latent states in RL to improve efficiency.
problem Costly state measurement in RL negatively affects future outcomes.
method Introduces AOMDP with measurement action, uses online RL and sequential Monte Carlo.
result Reduced uncertainty improves sample efficiency and policy value.
The paper targets optimal interventions for long-term outcomes using imputed data and policy learning.
problem Maximizing long-term outcomes observed only in the future.
method Imputing missing long-term outcomes and using a doubly-robust approach for policy evaluation and optimization.
result The approach outperforms simple short-term proxies and achieves significant revenue impact over three years.
New algorithm optimizes multi-objective outcomes in uncertain environments.
problem Optimizing global concave rewards in online Markov decision processes with multiple actions.
method No-regret algorithm based on online convex optimization and UCRL2, with a gradient threshold procedure.
result Non-stationary policy diversifies outcomes to optimize the global concave reward.
The paper introduces a framework for prescriptive process monitoring that generates alarms to prevent or mitigate undesired outcomes.
problem Existing predictive process monitoring techniques do not prescribe when and how to intervene to decrease undesired outcomes.
method The paper proposes a framework that extends predictive monitoring with the ability to generate alarms, incorporating a cost model to assess the trade-off between generating alarms and the cost of undesired outcomes.
result The net cost of undesired outcomes can be minimized by optimizing the generation of alarms based on the progress of the process instance and introducing delays for triggering alarms.
A Q-learning approach optimizes RTB ad campaigns for mobile app installs.
problem Optimizing RTB ad campaigns for mobile app installs with delayed rewards.
method State space based policy trained via Q-learning algorithm to handle delayed install notifications.
result Significant increase in profit and number of efficient campaigns.
Study finds non-adherence to schizophrenia meds leads to earlier adverse events.
problem Impact of medication non-adherence on adverse outcomes in schizophrenia patients.
method Survival analysis, causal inference methods (T-learner, S-learner, nearest neighbor matching), different amounts of longitudinal information.
result Non-adherence to schizophrenia meds advances adverse events by 1 to 4 months.
A new RL framework evaluates dynamic mediation effects over time.
problem Dynamic mediation effects in sequentially assigned treatments.
method Reinforcement Learning framework for decomposition and estimation of causal effects.
result Superior performance demonstrated through numerical studies and real data analysis.
CausalLongPFN predicts counterfactual outcomes from time-series data.
problem Predicting future outcomes under varying treatments in time-series data with confounding and heterogeneity.
method Prior-fitted network pretrained on synthetic episodes of temporal structural causal models.
result CausalLongPFN outperforms domain-trained models on factual and counterfactual prediction tasks.
Develops a stochastic approach to financial market delays.
problem Modeling delays in financial markets with multiple assets.
method Introduces a general stochastic framework for information and order execution delays.
result Delayed markets maintain fundamental asset pricing theorems and no asymptotic free lunch condition.
Paper tackles action delays in reinforcement learning, proposing a delay-aware framework.
problem Action delays degrade reinforcement learning performance in real-world systems.
method Formal definition of delay-aware MDP, transformation into standard MDP with augmented states, delay-aware model-based reinforcement learning framework.
result Proposed framework is more efficient in training and transferable between systems with various delay durations.
We show that time-dependent fluctuations { Δ x } \{Δx\} { Δ x } in foreign exchange rates are accurately described by a random walk in a complex plane that is demarcated into the gain (+) and loss (-) sectors. { Δ x } \{Δx\} { Δ x } is the outcome of N N N random steps from the origin and ∣ Δ x ∣ |Δx| ∣Δ x ∣ is the square of the Euclidean distance of the final …
New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.
problem Delayed feedback in Lipschitz bandits.
method Design of algorithms for bounded and unbounded stochastic delays.
result Sublinear regret guarantees for both bounded and unbounded delays.
New algorithms ensure fair selection in combinatorial semi-bandit with unrestricted delays.
problem Fair selection in stochastic combinatorial semi-bandit with delayed feedback.
method Introduced merit-based fairness constraints and new bandit algorithms for reward and fairness.
result Achieved sublinear expected reward and fairness regrets with dependence on delay distribution quantiles.
Banker-OMD improves online learning with delayed feedback.
problem Handling delayed feedback in online learning.
method Generalized Online Mirror Descent (OMD) framework.
result Achieves nearly-optimal performance in three bandit scenarios.
New algorithm handles delayed feedback robustly, reducing regret without knowing delay bounds.
problem Bandits with variably delayed feedback, especially excessive delays.
method Implicit exploration scheme, adaptive skipping, drifted regret control.
result Can tolerate arbitrary excessive delays up to order T, reducing regret.
Proposes a nonparametric model for predicting conversion rates with delayed feedback.
problem Predicting conversion rates with time delays and unknown distribution.
method Nonparametric delayed feedback model without assuming a specific distribution.
result The proposed model outperforms existing methods in conversion rate prediction.
Gradient descent with delayed updates converges faster with noise, even when delays are significant.
problem Analyzing convergence of gradient descent with delayed gradients and stochastic noise.
method Novel technique using generating functions for convergence analysis.
result Convergence bounds show that stochastic noise mitigates the negative effects of delays, improving performance.
New algorithm for multiarmed bandits with variable, unbounded delays achieves similar regret bounds.
problem Variable, unbounded delays in multiarmed bandits.
method Introduces a new algorithm that skips rounds with excessively large delays and uses a doubling scheme.
result Achieves the same regret bound as Exp3 with variable, unbounded delays.
Paper tackles delays in multi-agent reinforcement learning, improving performance.
problem Challenges in reinforcement learning due to delays in real-world systems.
method Proposes a novel framework for multi-agent reinforcement learning with delays, using Delay-Aware Markov Games and centralized-decentralized training.
result Demonstrates significant improvement in performance with delay-aware multi-agent reinforcement learning.
Derives a Feynman-Kac formula for a fixed delay CIR model.
problem Modeling financial processes with fixed delay.
method Proves existence and uniqueness of a strong solution for a specific SDDE.
result Derives a Feynman-Kac type formula leading to an affine bond pricing formula.
New bandit problem with delayed, aggregated feedback analyzed.
problem Stochastic K K K -armed bandit problem with delayed, aggregated anonymous feedback. method Developed algorithm matching worst case regret of non-anonymous problem.
result Regret increase can be maintained in the harder delayed, aggregated anonymous feedback setting.
Study on synchronization in financial markets with time delays.
problem Understanding market dynamics and synchronization in financial systems with time delays.
method Examined a system of coupled non-linear delay-differential equations, linearized for small delays, and analyzed collective dynamics using bifurcation diagrams and numerical solutions.
result Demonstrated that limit cycles can be maintained in coupled N-asset models with appropriate parameterization, leading to market synchronization.
BayTiDe discovers time-delayed differential equations from noisy data.
problem Discovering time-delayed differential equations from data with large delays and noise.
method Bayesian inference with a sparsity-promoting prior.
result BayTiDe accurately identifies time-delayed differential equations with accuracy proportional to data resolution.
New algorithm tackles stochastic bandits with varying arm-dependent delays.
problem Applying existing algorithms to stochastic delayed bandit settings is restricted by strong assumptions on delay distributions.
method Proposes a simple UCB-based algorithm called PatientBandits that weakens assumptions on delay distributions.
result Provides bounds on regret and performance lower bounds for the PatientBandits algorithm.
Model analyzes how delayed information impacts option pricing.
problem Effects of delayed information on option pricing.
method Binomial model, closed form formula for convex contingent claims, convergence analysis.
result Delayed information exaggerates the volatility smile.
TSMB handles time delays in multivariate time series data.
problem Varying time delays in multivariate time series data complicate predictions.
method Time Series Model Bootstrap (TSMB) framework for nonparametric time delay estimation.
result TSMB improves model performance in dynamic data environments.
Model shows bailout stigma affects firm funding and market performance.
problem Bailout stigma impacts firm funding and market stability.
method Developed a model to analyze the effects of bailout stigma on firms and markets.
result Firms avoid stigma by withdrawing or refusing bailouts, leading to market freeze or revival.
Delayed-RNN approximates stacked and bidirectional RNNs.
problem Improving RNN expressiveness and representational capacity.
method Weight-constrained delayed-RNN, equivalent to stacked-RNNs, with partial acausality.
result Delayed-RNN can approximate stacked and bidirectional RNNs, outperforming them in some tasks.
New algorithm reduces regret in delayed feedback generalised linear bandits.
problem Regret in delayed feedback generalised linear bandits.
method Adaptation of optimistic algorithm to delayed feedback.
result Achieves a regret bound independent of the horizon's delay penalty.
Adapts Exp3 to adversarial bandits with delays and data.
problem Adversarial multi-armed bandits with delayed feedback.
method Tuned Exp3 variants with step-size adaptation and implicit exploration.
result Optimal regret bounds of log ( K ) ( T K + D ) \sqrt{\log(K)(TK + D)} log ( K ) ( T K + D ) with high probability. Capacity-Constrained Online Convex Optimization with Delayed Feedback
problem Online learning with delayed feedback under a hard capacity constraint
method Reduction to a delayed and weighted OCO problem using a scheduler
result First regret guarantees for capacity-constrained OCO under convex and strongly convex losses
New algorithm tackles non-stationary delayed feedback in recommender systems.
problem Challenges in learning from delayed feedback in non-stationary environments.
method Developed a UCRL-based algorithm for non-stationary, delayed bandits with intermediate observations.
result Sublinear regret guarantees for the proposed algorithm in non-stationary delayed environments.
PCTS optimizes noisy, delayed, multi-fidelity feedbacks in black-box optimization.
problem Optimizing unknown functions with noisy, delayed, and multi-fidelity feedbacks.
method ProCrastinated Tree Search (PCTS) with DUCB1 and DUCBV algorithms.
result PCTS achieves better regret bounds for delayed, noisy, and multi-fidelity feedbacks.
Study online learning with delays and capacity constraints, achieving optimal regret bounds.
problem Online learning with delays and capacity constraints.
method Novel scheduling and preemptive techniques, matching upper and lower bounds.
result Achieves optimal regret bounds across all capacity levels.
RISE learns decisions with sensitive variables, improving worst-case outcomes.
problem Uncertainty and bias in decisions due to delayed sensitive variable data.
method Incorporates sensitive variables offline but not at deployment, using quantile or infimum optimization.
result Improves worst-case outcomes for individuals affected by unavailable sensitive variables.
Study market delay effects on contingent claims pricing.
problem Delayed market information impacts contingent claims pricing.
method Analyzes Black-Scholes and binomial models with delay.
result Scaling limit of super-replication prices equals G-expectation.
New Async-SGD and Async-SGDI methods converge for non-convex problems with unbounded delays.
problem Improving convergence of asynchronous stochastic gradient descent with unbounded delays in non-convex learning.
method Developed Async-SGD and Async-SGDI methods for non-convex optimization with unbounded gradient delays, proving convergence rates and establishing a unifying sufficient condition.
result Proved o ( 1 / k ) o(1/\sqrt{k}) o ( 1/ k ) convergence rate for Async-SGD and o ( 1 / k ) o(1/k) o ( 1/ k ) for Async-SGDI. Online learning with delayed feedback has received increasing attention recently due to its several applications in distributed, web-based learning problems. In this paper we provide a systematic study of the topic, and analyze the effect of delay on the regret of online learning algorithms. Somewhat surprisingly, it t…