On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “rare policy switch”

Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraintscs.LG

Efficient RL algorithms for linear function approximation with limited adaptivity constraints.

problem Limited adaptivity in reinforcement learning with linear function approximation.

method Proposed two efficient online RL algorithms for episodic linear Markov decision processes under batch learning and rare policy switch models.

result Achieved efficient regret bounds for both batch learning and rare policy switch models, with substantial reduction in adaptivity.

reinforcement learning linear function approximation adaptivity constraints batch learning rare policy switch

2021-01-06abs ↗pdf ↗

Achieving Limited Adaptivity for Multinomial Logistic Banditscs.LG

Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.

problem Achieving optimal regret with limited adaptivity in multinomial logistic bandits.

method Presented two algorithms, B-MNL-CB and RS-MNL, for batched and rarely-switching paradigms.

result Achieved

ilde{O}(\sqrt{T})

regret with limited adaptivity.

multinomial logistic bandits regret minimization limited adaptivity batched setting rarely-switching setting

2025-08-05abs ↗pdf ↗

Rarely-switching linear bandits: optimization of causal effects for the real worldstat.ML

Excessively changing policies in many real world scenarios is difficult, unethical, or expensive. After all, doctor guidelines, tax codes, and price lists can only be reprinted so often. We may thus want to only change a policy when it is probable that the change is beneficial. In cases that a policy is a threshold on …

2019-05-30abs ↗pdf ↗

Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processescs.LG

New RL algorithm achieves nearly optimal performance for linear MDPs.

problem Optimal reinforcement learning for episodic linear MDPs.

method Weighted linear regression with variance estimator and rare-switching policy.

result Achieves nearly minimax optimal regret

ilde O(d\sqrt{H^3K})

reinforcement learning linear MDPs regret minimization weighted regression variance estimator

2022-12-12abs ↗pdf ↗

Actor-Critics Can Achieve Optimal Sample Efficiencystat.ML

New actor-critic algorithm achieves optimal sample efficiency in RL.

problem Achieving

ε

-optimal policies with minimal samples in RL.

method Integrates optimism, off-policy critic estimation, and rare-switching policy resets.

result Sample complexity of

O(dH^5 \log|\mathcal{A}|/ε^2 + d H^4 \log|\mathcal{F}|/ ε^2)

trajectories.

Actor-Critic Reinforcement Learning Optimism Off-Policy Rare-Switching

2025-05-06abs ↗pdf ↗

Using latent representations to link disjoint longitudinal data for mixed-effects regressionstat.ML

We link disjoint longitudinal data for rare disease patients using latent representations and mixed-effects regression.

problem Analyzing treatment switches in rare diseases with limited data and changing measurement instruments.

method We embed item values into a shared latent space using variational autoencoders and apply mixed-effects regression to quantify treatment effects.

result Our approach allows for statistical inference and quantifies the impact of treatment switches in spinal muscular atrophy.

latent representations disjoint longitudinal data mixed-effects regression statistical inference rare diseases

2025-10-29abs ↗pdf ↗

Upper and Lower Bounds for Distributionally Robust Off-Dynamics Reinforcement Learningcs.LG

We develop a robust RL algorithm for off-dynamics environments with improved suboptimality bounds and computational efficiency.

problem Learning policies robust to uncertainties in transition dynamics between training and deployment environments.

method Distributionally robust Markov decision processes (DRMDPs) with a novel algorithm We-DRIVE-U.

result Improved suboptimality bound of

\widetilde{\mathcal{O}}\big({d H \cdot \min \{1/ρ, H\}/\sqrt{K} }\big)

, near-optimal up to

\mathcal{O}(\sqrt{H})

Reinforcement Learning Distributionally Robust Off-Dynamics Markov Decision Processes Suboptimality Bounds

2024-09-30abs ↗pdf ↗

To Switch or Not to Switch? Balanced Policy Switching in Offline Reinforcement Learningstat.ML

Study tackles balancing policy switching costs in offline RL.

problem Balancing the cost of policy switching in offline RL.

method Optimal transport ideas and Net Actor-Critic algorithm.

result Demonstrated efficiency on multiple RL benchmarks.

Reinforcement Learning Policy Switching Offline RL Optimal Transport Net Actor-Critic

2024-07-01abs ↗pdf ↗

Sample-Efficient Reinforcement Learning with loglog(T) Switching Costcs.LG

New RL algorithm reduces policy switching cost to loglog(T) with similar regret.

problem Low policy switching cost in real-life RL applications.

method Stage-wise exploration and adaptive policy elimination.

result Regret of

O(HSA \log\log T)

with

O(HSA \log\log T)

switching cost.

reinforcement learning policy switching regret stage-wise exploration adaptive elimination

2022-02-13abs ↗pdf ↗

Linear Bandits with Limited Adaptivity and Learning Distributional Optimal Designcs.LG

Study on adaptivity constraints in linear contextual bandits with optimal design.

problem Impact of adaptivity constraints on linear contextual bandits.

method Two models of limited adaptivity: batch learning and rare policy switches. Proposed distributional optimal design.

result Achieves minimax-optimal regret with optimal number of policy switches and batches.

linear bandits adaptivity constraints optimal design regret minimization contextual bandits

2020-07-04abs ↗pdf ↗

Evaluation-Time Policy Switching for Offline Reinforcement Learningcs.LG

A new policy switching technique improves offline RL performance.

problem Challenges in adapting off-policy algorithms to different datasets and tasks.

method Combines off-policy RL and BC, using epistemic uncertainty for policy switching.

result Outperforms individual algorithms and state-of-the-art methods on benchmarks.

offline reinforcement learning policy switching epistemic uncertainty aleatoric uncertainty behavioural cloning

2025-03-15abs ↗pdf ↗

Control randomisation approach for policy gradient and application to reinforcement learning in optimal switchingmath.OC

New framework for policy gradient methods in continuous time reinforcement learning.

problem Addressing policy gradient methods for continuous time reinforcement learning.

method Control randomisation technique to derive policy gradient representation for various Markovian control problems.

result Demonstrated application to optimal switching problems in the energy sector.

policy gradient reinforcement learning control randomisation Markovian control optimal switching

2024-04-27abs ↗pdf ↗

Stochastic Knapsack Problem Revisited: Switch-Over Policies and Dynamic Pricingq-fin.PR

The stochastic knapsack has been used as a model in wide ranging applications from dynamic resource allocation to admission control in telecommunication. In recent years, a variation of the model has become a basic tool in studying problems that arise in revenue management and dynamic/flexible pricing; and it is in thi…

2007-08-08abs ↗pdf ↗

Continuous-time reinforcement learning for optimal switching over multiple regimesmath.OC

The paper develops RL methods for optimal switching between multiple states.

problem Optimal switching between multiple states in continuous time.

method Entropy-regularized exploration, HJB equations, policy improvement, value function convergence.

result The RL algorithm converges to optimal policies as temperature parameter vanishes.

reinforcement learning optimal switching Hamilton-Jacobi-Bellman entropy regularization policy improvement

2025-12-04abs ↗pdf ↗

Logarithmic Switching Cost in Reinforcement Learning beyond Linear MDPscs.LG

New algorithm reduces switching costs in RL beyond linear MDPs.

problem Costly policy switching in reinforcement learning.

method ELEANOR-LowSwitching algorithm for linear Bellman-complete MDPs.

result Achieves near-optimal regret with logarithmic switching cost.

reinforcement learning policy switching Markov Decision Processes regret minimization logarithmic cost

2023-02-24abs ↗pdf ↗

A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximationcs.LG

New algorithm reduces RL complexity with low switching costs.

problem Exploration-exploitation dilemma in RL with complex models.

method Monotonic Q-Learning with Upper Confidence Bound (MQL-UCB) for RL with general function approximation.

result Achieves minimax optimal regret of

O(d\sqrt{HK})

and near-optimal policy switching cost.

reinforcement learning upper confidence bound function approximation exploration-exploitation minimax optimal

2023-11-26abs ↗pdf ↗

Optimal control of uncertain stochastic systems with Markovian switching and its applications to portfolio decisionsmath.OC

This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …

2014-01-11abs ↗pdf ↗

Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learningstat.ML

New RL algorithms reduce costs for single-agent and federated learning.

problem Minimizing costs in RL and federated RL settings.

method Q-EarlySettled-LowCost and FedQ-EarlySettled-LowCost algorithms.

result First algorithms to achieve low burn-in and logarithmic switching costs.

Reinforcement Learning Federated Learning Cost Reduction Regret Minimization Burn-in Cost

2025-06-05abs ↗pdf ↗

Delightful Distributed Policy Gradientcs.LG

DG separates successes and failures by gating updates with advantage and surprisal.

problem Negative learning from surprising data in distributed reinforcement learning.

method DG gates each update with the product of advantage and surprisal, suppressing failures and preserving successes.

result DG outperforms other methods in various challenging reinforcement learning tasks.

reinforcement learning distributed learning policy gradient surprisal advantage

2026-03-20abs ↗pdf ↗

Contextual Slate GLM Bandits with Limited Adaptivitycs.LG

Efficient algorithms for contextual slate bandits with limited adaptivity.

problem Contextual slate bandit problem with limited adaptivity.

method Proposed B-SlateGLinCB and RS-SlateGLinCB algorithms for batched and rarely-switching settings.

result Achieved regret bounds of O(Nd^(3/2)√T) and O(Nd√T) under diversity assumption.

contextual bandits generalized linear models limited adaptivity slate selection regret bounds

2026-06-30abs ↗pdf ↗

Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillationcs.LG

Breaks down complex nonlinear dynamics into simpler components.

problem Control of nonlinear dynamical systems remains challenging.

method Inspired by hybrid switching systems, decomposes dynamics into simpler stochastic switching linear dynamical systems.

result Extracts hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts.

nonlinear dynamics hybrid switching systems stochastic switching Markovian controllers auto-regressive

2020-05-04abs ↗pdf ↗

Non-stationary Online Learning with Memory and Non-stochastic Controlcs.LG

New algorithm reduces decision switching in dynamic environments.

problem Online learning with memory and non-stationary environments.

method Dynamic policy regret, novel ensemble approach, meta-base decomposition.

result Proves optimal dynamic policy regret for memory length, non-stationarity, and time horizon.

Online Learning Dynamic Policy Regret Memory Non-stationary Control Theory

2021-02-07abs ↗pdf ↗

Learning to Switch Among Agents in a Team via 2-Layer Markov Decision Processescs.LG

Algorithm learns to switch control among agents in a team.

problem Learning to switch control among reinforcement learning agents.

method 2-layer Markov decision process, upper confidence bounds, shared confidence bounds.

result Sublinear total regret with shared confidence bounds.

reinforcement learning team control Markov decision process upper confidence bounds shared confidence bounds

2020-02-11abs ↗pdf ↗

CTMSTOU driven markets: simulated environment for regime-awareness in trading policiescs.MA

This work models market regimes using CTMSTOU and simulates trading policies.

problem Defining and understanding market regimes in finance.

method Discrete event time multi-agent market simulation with CTMSTOU model.

result Illustrates the importance of regime-awareness in trading policies.

market regimes CTMSTOU regime-awareness trading policies market simulation

2022-02-02abs ↗pdf ↗

Off-Policy Evaluation Using Information Borrowing and Context-Based Switchingstat.ML

New DR-IC estimator reduces bias and variance in OPE.

problem Estimating value of a target policy using logged data from a different policy.

method DR-IC estimator that combines parametric reward model and context-based switching rule.

result DR-IC estimator outperforms state-of-the-art OPE algorithms.

off-policy evaluation contextual bandits doubly robust information borrowing context-based switching

2021-12-18abs ↗pdf ↗

Regime Switching Optimal Growth Model with Risk Sensitive Preferencesmath.OC

Optimizes consumption under regime-switching economic states with risk-sensitive preferences.

problem Optimizing consumption in an economy with uncertain states and random shocks.

method Risk-sensitive optimization of consumption-utility with a Markov chain model of economic states and i.i.d. random shocks.

result Existence of unique optimal policy and value function in stationary policies.

Regime Switching Optimal Growth Risk-Sensitive Euler Equation Stationary Policies

2021-10-28abs ↗pdf ↗

Deep Learning Predictive Band Switching in Wireless Networkscs.NI

In cellular systems, the user equipment (UE) can request a change in the frequency band when its rate drops below a threshold on the current band. The UE is then instructed by the base station (BS) to measure the quality of candidate bands, which requires a measurement gap in the data transmission, thus lowering the da…

2019-10-02abs ↗pdf ↗

Fast Policy Learning through Imitation and Reinforcementcs.LG

Imitation learning (IL) consists of a set of tools that leverage expert demonstrations to quickly learn policies. However, if the expert is suboptimal, IL can yield policies with inferior performance compared to reinforcement learning (RL). In this paper, we aim to provide an algorithm that combines the best aspects of…

2018-05-26abs ↗pdf ↗

Policy-Controlled Generalized Share: A General Framework with a Transformer Instantiation for Strictly Online Switching-Oracle Trackingcs.LG

PCGS-TF uses a Transformer to adaptively control expert switching in non-stationary environments.

problem Static regret is insufficient for strictly online prediction in non-stationary settings.

method Policy-Controlled Generalized Share (PCGS) with a Transformer as an update controller.

result PCGS-TF achieves the lowest dynamic regret in non-stationary families and expert pools.

Transformer non-stationary strictly online dynamic regret switching-oracle

2026-03-30abs ↗pdf ↗

Stochastic Bandits with Delay-Dependent Payoffsstat.ML

Motivated by recommendation problems in music streaming platforms, we propose a nonstationary stochastic bandit model in which the expected reward of an arm depends on the number of rounds that have passed since the arm was last pulled. After proving that finding an optimal policy is NP-hard even when all model paramet…

2019-10-07abs ↗pdf ↗

Insurance policy value and Pareto-optimal retention in the hypothesis of rare loss eventsmath.PR

In the hypothesis of rare loss events, the general expression of the policy value has been determined as a functional of the "expected frequency / loss severity" function and of the retention function. Exponential disutility has been chosen after mathematical characterization of some of its economical aspects, where fu…

1998-06-07abs ↗pdf ↗

Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcs.LG

New method improves reinforcement learning generalization.

problem Few environments lead to poor generalization in reinforcement learning.

method Integrates sequential structure into representation learning, using a policy similarity metric (PSM) and contrastive embeddings (PSEs).

result PSEs improve generalization across various benchmarks.

reinforcement learning generalization behavioral similarity representation learning contrastive embeddings

2021-01-13abs ↗pdf ↗

Regret-Optimal Model-Free Reinforcement Learning for Discounted MDPs with Short Burn-In Timecs.LG

New algorithm learns optimal policies with minimal memory and time.

problem Learning optimal policies in discounted MDPs with short burn-in time.

method Variance reduction and adaptive policy switching.

result First regret-optimal model-free algorithm with low burn-in time.

reinforcement learning discounted MDPs regret minimization policy switching variance reduction

2023-05-24abs ↗pdf ↗

Computing non-equilibrium trajectories by a deep learning approachphysics.comp-ph

A new method uses deep learning to predict rare events in complex systems.

problem Predicting rare and extreme events in non-equilibrium systems.

method A deep learning approach that minimizes the geometrical action.

result The method accurately predicts rare events in various complex systems.

deep learning rare events non-equilibrium systems machine learning Freidlin-Wentzell action

2022-10-08abs ↗pdf ↗

Reinforcement Learning based Control of Imitative Policies for Near-Accident Drivingcs.LG

H-ReIL learns to drive safely in near-accident scenarios.

problem Driving safely in high-risk near-accident situations.

method Hierarchical RL and IL approach.

result High-level policy switches between low-level policies for safe driving.

Reinforcement Learning Imitation Learning Autonomous Driving Near-Accident Scenarios Safety

2020-07-01abs ↗pdf ↗

Provably Efficient Q-Learning with Low Switching Costcs.LG

We take initial steps in studying PAC-MDP algorithms with limited adaptivity, that is, algorithms that change its exploration policy as infrequently as possible during regret minimization. This is motivated by the difficulty of running fully adaptive algorithms in real-world applications (such as medical domains), and …

2019-05-30abs ↗pdf ↗

A Provably Efficient Algorithm for Linear Markov Decision Process with Low Switching Costcs.LG

Paper presents an efficient algorithm for linear MDP with low switching cost.

problem Large state space reinforcement learning problems with low switching cost.

method First algorithm for linear MDP with low switching cost, achieving near-optimal regret and switching cost.

result Regret bound of $\widetilde{O}\left(\sqrt{d^3H^4K} ight)$ and near-optimal switching cost of $O\left(d H\log K ight)$.

reinforcement learning Markov decision process linear function approximation regret bound switching cost

2021-01-02abs ↗pdf ↗

Regime-Arrival Uncertainty in Generalization Bounds under Distribution Shiftcs.LG

Paper addresses uncertainty in model generalization under regime shifts.

problem Uncertainty in model generalization under regime changes.

method Proposes a framework to quantify and separate regime mismatch and sensitivity.

result Obtains exact decomposition and minimax lower bound for regime-aware models.

regime shifts generalization bounds distribution shifts Markov-switching beta-mixing

2026-06-01abs ↗pdf ↗

On Classifying the Effects of Policy Announcements on Volatilityq-fin.GN

The study classifies policy announcements' impact on stock market volatility.

problem Evaluating the impact of Central Bank announcements on stock market volatility.

method Proposed a model-based classification method using Markov Switching dynamics and Multiplicative Error Model.

result Successful classification of 144 European Central Bank announcements on stock market volatility.

Central Bank announcements stock market volatility Markov Switching Multiplicative Error Model classification

2020-11-28abs ↗pdf ↗

Optimal Hybrid Dividend Strategy Under The Markovian Regime-Switching Economyq-fin.MF

In this paper, we consider the optimal dividend problem for a company. We describe the surplus process of the company by a diffusion model with regime switching. The aim of the company is to choose a dividend policy to maximize the expected total discounted payments until ruin. In this article, we consider a hybrid div…

2014-06-30abs ↗pdf ↗

Indifference pricing of pure endowments in a regime-switching market modelq-fin.PM

Study indifference pricing for insurance policies in a regime-switching market model.

problem Indifference pricing of pure endowment policies in a stochastic-factor model with different economic regimes.

method Stochastic control approach based on Hamilton-Jacobi-Bellman equation, Feynman-Kac formula, and sensitivity analysis.

result Characterization of indifference price as a solution to a linear PDE and a backward PDE.

indifference pricing regime-switching stochastic control Feynman-Kac partial differential equation

2023-01-31abs ↗pdf ↗

Gap-Dependent Bounds for Q-Learning using Reference-Advantage Decompositionstat.ML

This paper improves Q-learning bounds using reference-advantage decomposition.

problem Improving Q-learning bounds in MDPs with positive suboptimality gaps.

method Develops a novel error decomposition framework to prove gap-dependent regret bounds.

result Establishes logarithmic gap-dependent regret bounds for Q-learning.

Q-learning MDPs gap-dependent bounds reference-advantage decomposition policy switching cost

2024-10-10abs ↗pdf ↗

Optimal Dividends under Markov-Modulated Bankruptcy Levelmath.OC

Optimal dividends strategy in a two-state regime-switching environment.

problem Maximizing profits from dividends until bankruptcy in a company with fluctuating cash surplus and regime changes in drift, volatility, and bankruptcy levels.

method Analyzes the optimal dividend payout strategy considering four factors: Brownian fluctuations in cash surplus, regime changes in drift, volatility, and bankruptcy levels.

result Rich structure of the optimal strategy, which can be either barrier-type or liquidation-barrier type, depending on model parameters.

dividends bankruptcy regime-switching optimal strategy profit maximization

2021-11-05abs ↗pdf ↗

Online Learning with Switching Costs and Other Adaptive Adversariescs.LG

We study the power of different types of adaptive (nonoblivious) adversaries in the setting of prediction with expert advice, under both full-information and bandit feedback. We measure the player's performance using a new notion of regret, also known as policy regret, which better captures the adversary's adaptiveness…

2013-02-18abs ↗pdf ↗

On Investment-Consumption with Regime-Switchingmath.OC

In a continuous time stochastic economy, this paper considers the problem of consumption and investment in a financial market in which the representative investor exhibits a change in the discount rate. The investment opportunities are a stock and a riskless account. The market coefficients and discount factor switches…

2011-07-10abs ↗pdf ↗

Near-Optimal Adversarial Reinforcement Learning with Switching Costscs.LG

This paper tackles near-optimal adversarial RL with switching costs, providing algorithms and matching lower bounds.

problem Adversarial RL with switching costs, where loss distribution can be non-stationary or adversarial.

method Developed novel switching-reduced algorithms with matching lower bounds for known and unknown transition functions.

result Achieved near-optimal performance in adversarial RL with switching costs, matching theoretical lower bounds.

Adversarial Reinforcement Learning Switching Costs Regret Analysis Lower Bounds Provable Efficiency

2023-02-08abs ↗pdf ↗

A general framework for adaptive two-index fusion attribute weighted naive Bayescs.LG

Adaptive framework improves NB accuracy by fusing two index categories.

problem Challenges in attribute weighted NB, especially fusion of two indexes.

method Proposes ATFNB framework using switching factor to fuse two index categories.

result ATFNB outperforms basic NB and state-of-the-art models.

Naive Bayes Attribute Weighting Two-Index Fusion Adaptive Switching Factor Data Mining

2022-02-24abs ↗pdf ↗

Gas Storage valuation with regime switchingq-fin.PR

In this paper we treat a gas storage valuation problem as a Markov Decision Process. As opposed to existing literature we model the gas price process as a regime-switching model. Such a model has shown to fit market data quite well in Chen and Forsyth (2010). Before we apply a numerical algorithm to solve the problem, …

2014-12-03abs ↗pdf ↗