As a metric to measure the performance of an online method, dynamic regret with switching cost has drawn much attention for online decision making problems. Although the sublinear regret has been provided in many previous researches, we still have little knowledge about the relation between the dynamic regret and the s…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “switching decisions”
Efficient algorithms for online convex optimization with limited switching decisions.
problem Online convex optimization with limited switching decisions.
method Presented computationally efficient algorithms for both general and strongly convex losses.
result Regret bounds of for general convex losses and for strongly convex losses.
Paper presents an efficient algorithm for linear MDP with low switching cost.
problem Large state space reinforcement learning problems with low switching cost.
method First algorithm for linear MDP with low switching cost, achieving near-optimal regret and switching cost.
result Regret bound of $\widetilde{O}\left(\sqrt{d^3H^4K}
ight)$ and near-optimal switching cost of $O\left(d H\log K
ight)$.
Study tackles balancing policy switching costs in offline RL.
problem Balancing the cost of policy switching in offline RL.
method Optimal transport ideas and Net Actor-Critic algorithm.
result Demonstrated efficiency on multiple RL benchmarks.
New algorithm reduces switching costs in RL beyond linear MDPs.
problem Costly policy switching in reinforcement learning.
method ELEANOR-LowSwitching algorithm for linear Bellman-complete MDPs.
result Achieves near-optimal regret with logarithmic switching cost.
Optimizes control of hybrid systems with multiple switching processes.
problem Optimal control of hybrid systems with multiple Markov switching processes.
method Combines two separate Markov chains into one synthetic chain, derives HJB equations, and solves the portfolio choice problem.
result Derives explicit solutions and value functions for the optimal control problem.
New algorithm reduces decision switching in dynamic environments.
problem Online learning with memory and non-stationary environments.
method Dynamic policy regret, novel ensemble approach, meta-base decomposition.
result Proves optimal dynamic policy regret for memory length, non-stationarity, and time horizon.
This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using …
We solve non-Markovian optimal switching problems in discrete time on an infinite horizon, when the decision maker is risk aware and the filtration is general, and establish existence and uniqueness of solutions for the associated reflected backward stochastic difference equations. An example application to hydropower …
Reinforcement learning agents have been mostly developed and evaluated under the assumption that they will operate in a fully autonomous manner -- they will take all actions. In this work, our goal is to develop algorithms that, by learning to switch control between agents, allow existing reinforcement learning agents …
This paper studies the impact of limited switches on resource-constrained dynamic pricing with demand learning. We focus on the classical price-based blind network revenue management problem and extend our results to the bandits with knapsacks problem. In both settings, a decision maker faces stochastic and distributio…
A Problem of Finite-Horizon Optimal Switching and Stochastic Control for Utility Maximizationmath.OC
Paper tackles utility maximization with job-switching and retirement constraints.
problem Maximizing utility with job-switching and retirement constraints.
method Dual-martingale approach and double obstacle problem theory.
result Characterization of optimal job-switching strategy and wealth boundaries.
The stochastic knapsack has been used as a model in wide ranging applications from dynamic resource allocation to admission control in telecommunication. In recent years, a variation of the model has become a basic tool in studying problems that arise in revenue management and dynamic/flexible pricing; and it is in thi…
New RL algorithm reduces policy switching cost to loglog(T) with similar regret.
problem Low policy switching cost in real-life RL applications.
method Stage-wise exploration and adaptive policy elimination.
result Regret of with switching cost.
In this paper we treat a gas storage valuation problem as a Markov Decision Process. As opposed to existing literature we model the gas price process as a regime-switching model. Such a model has shown to fit market data quite well in Chen and Forsyth (2010). Before we apply a numerical algorithm to solve the problem, …
PCGS-TF uses a Transformer to adaptively control expert switching in non-stationary environments.
problem Static regret is insufficient for strictly online prediction in non-stationary settings.
method Policy-Controlled Generalized Share (PCGS) with a Transformer as an update controller.
result PCGS-TF achieves the lowest dynamic regret in non-stationary families and expert pools.
We develop a robust RL algorithm for off-dynamics environments with improved suboptimality bounds and computational efficiency.
problem Learning policies robust to uncertainties in transition dynamics between training and deployment environments.
method Distributionally robust Markov decision processes (DRMDPs) with a novel algorithm We-DRIVE-U.
result Improved suboptimality bound of , near-optimal up to .
Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learningstat.ML
New RL algorithms reduce costs for single-agent and federated learning.
problem Minimizing costs in RL and federated RL settings.
method Q-EarlySettled-LowCost and FedQ-EarlySettled-LowCost algorithms.
result First algorithms to achieve low burn-in and logarithmic switching costs.
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
New algorithms improve on consistency and robustness in convex function chasing with black-box advice.
problem Minimizing cost in normed vector space with black-box advice for convex function chasing.
method Two novel algorithms: INTERP and BDINTERP, exploiting convexity to achieve improved consistency and robustness.
result BDINTERP achieves near-optimal consistency-robustness trade-off for α-polyhedral cost functions.
Modeling decision-making dynamics in MDD patients using RL-HMM.
problem Characterize reward learning strategies in MDD patients.
method Proposed RL-HMM framework to analyze reward-based decision-making.
result MDD patients show reduced engagement in RL compared to healthy controls.
This paper considers the problem of consumption and investment in a financial market within a continuous time stochastic economy. The investor exhibits a change in the discount rate. The investment opportunities are a stock and a riskless account. The market coefficients and discount factor switch according to a finite…
An agent-based model for financial markets has to incorporate two aspects: decision making and price formation. We introduce a simple decision model and consider its implications in two different pricing schemes. First, we study its parameter dependence within a supply-demand balance setting. We find realistic behavior…
Study when to replace machine learning models with new data.
problem When to switch machine learning models with new data sources.
method Unified economic and statistical framework linking learning-curve dynamics, data-acquisition, and retraining costs.
result Look-ahead sequential method outperforms other methods and approaches optimal value.
Efficiently optimize GPs by reusing candidate solutions multiple times.
problem High computational cost of Gaussian process optimization due to unique historical points.
method Sticking to a candidate solution for multiple evaluation steps and limiting switches.
result Improved efficiency and practicality of Gaussian process optimization algorithms.
Safe autonomous decisions made with machine learning predictions using Conformal Decision Theory.
problem Safe decisions from imperfect machine learning predictions.
method Conformal Decision Theory framework for producing safe decisions.
result Safe decisions with provable statistical guarantees of low risk.
We study a multi-armed bandit problem where the rewards exhibit regime switching. Specifically, the distributions of the random rewards generated from all arms are modulated by a common underlying state modeled as a finite-state Markov chain. The agent does not observe the underlying state and has to learn the transiti…
New RL algorithm achieves nearly optimal performance for linear MDPs.
problem Optimal reinforcement learning for episodic linear MDPs.
method Weighted linear regression with variance estimator and rare-switching policy.
result Achieves nearly minimax optimal regret .
In a continuous time stochastic economy, this paper considers the problem of consumption and investment in a financial market in which the representative investor exhibits a change in the discount rate. The investment opportunities are a stock and a riskless account. The market coefficients and discount factor switches…
New algorithm reduces switching costs in multinomial logit bandit problems.
problem Minimizing switching costs in multinomial logit bandit problems.
method Proposed AT-DUCB and FH-DUCB algorithms with low assortment switching costs.
result AT-DUCB and FH-DUCB algorithms achieve almost optimal minimax regret with low switching costs.
New polynomial invariants derived from birack and switch structures.
problem Polynomial invariants of braids.
method Switch structures, birack colorings, quiver-valued invariants.
result New polynomial invariants of braids.
Efficient RL algorithms for linear function approximation with limited adaptivity constraints.
problem Limited adaptivity in reinforcement learning with linear function approximation.
method Proposed two efficient online RL algorithms for episodic linear Markov decision processes under batch learning and rare policy switch models.
result Achieved efficient regret bounds for both batch learning and rare policy switch models, with substantial reduction in adaptivity.
FedSGM tackles constrained federated learning with unified framework.
problem Functional constraints, communication bottlenecks, local updates, and partial client participation in federated learning.
method Unified framework based on switching gradient method, incorporating bi-directional error feedback, and soft switching for stability.
result Achieves convergence rate with high-probability bounds decoupling from sampling noise.
In this paper, we study optimal switching problems under ambiguity. To characterize the optimal switching under ambiguity in the finite horizon, we use multidimensional reflected backward stochastic differential equations (multidimensional RBSDEs) and show that a value function of the optimal switching under ambiguity …
Recent years have witnessed an increased focus on interpretability and the use of machine learning to inform policy analysis and decision making. This paper applies machine learning to examine travel behavior and, in particular, on modeling changes in travel modes when individuals are presented with a novel (on-demand)…
Model analyzes how firms balance full disclosure with selective disclosure to maintain a good reputation.
problem Managing reputation in financial markets through voluntary disclosure.
method Developed a dynamic model with two disclosure strategies: candid and sparing, using a piecewise-deterministic model.
result Firms are rewarded for full disclosure but may switch to selective disclosure to avoid potential downgrades.
This work extends identifiability analysis to sequential latent variable models, focusing on Switching Dynamical Systems.
problem Identifying latent variables in sequential data models.
method Proved identifiability of Markov Switching Models and established conditions for Switching Dynamical Systems.
result Identifiability of latent variables and non-linear mappings in Switching Dynamical Systems up to affine transformations.
New algorithm learns optimal policies with minimal memory and time.
problem Learning optimal policies in discounted MDPs with short burn-in time.
method Variance reduction and adaptive policy switching.
result First regret-optimal model-free algorithm with low burn-in time.
The problem of optimal switching between nonlinear autonomous subsystems is investigated in this study where the objective is not only bringing the states to close to the desired point, but also adjusting the switching pattern, in the sense of penalizing switching occurrences and assigning different preferences to util…
Code-switching, the alternation of languages within a conversation or utterance, is a common communicative phenomenon that occurs in multilingual communities across the world. This survey reviews computational approaches for code-switched Speech and Natural Language Processing. We motivate why processing code-switched …
A Meta-Method for Portfolio Management Using Machine Learning for Adaptive Strategy Selectionq-fin.PM
MPM uses machine learning to switch between two portfolio strategies for better risk management.
problem Adaptive portfolio strategy selection for improved risk management.
method XGBoost learns to switch between HRP and NRP strategies.
result MPM outperforms both HRP and NRP in risk-reward profile and interpretability.
New algorithms improve sampling from complex distributions.
problem Sampling from complex probability distributions efficiently.
method Regime-switching Langevin dynamics and Monte Carlo algorithms.
result Convergence guarantees and iteration complexities provided.
Squirrel switches between optimizers for better performance.
problem Finding the best optimizer for a given problem.
method Switches between different optimizers based on performance.
result Improves performance on various problems.
Discrete-time weak approximation of a Black-Scholes model with drift and volatility Markov switchingmath.PR
Study approximates financial market with discrete-time models.
problem Approximating continuous-time financial market models with discrete-time.
method Constructs discrete-time market models with Markov switching and proves convergence.
result Discrete-time models converge to continuous-time Black-Scholes model with Markov switching.
This paper studies deep learning methodologies for portfolio optimization in the US equities market. We present a novel residual switching network that can automatically sense changes in market regimes and switch between momentum and reversal predictors accordingly. The residual switching network architecture combines …
Unsupervised learning of multiscale switching dynamical system models from multimodal neural datacs.LG
New algorithm learns switching dynamics from multiple neural signals.
problem Learning accurate switching dynamical system models from multimodal neural data.
method Unsupervised learning algorithm for multiscale switching dynamical system models.
result Switching multiscale dynamical system models outperform single-scale models in behavior decoding.
New algorithms for batch decision-making with high-dimensional user data.
problem Maximizing treatment efficacy in groups of users.
method Teamwork LASSO Bandit algorithm, switching between teamwork and selfish stages.
result Upper bound on expected cumulative regret for the proposed algorithm.
When Alpha Disappears: A One-Switch Benchmark for Decision-Time Leakage in Financial Backtestsq-fin.RM
Benchmark detects decision-time leakage in financial backtests.
problem Detecting decision-time leakage in financial machine-learning backtests.
method Toggles one evaluation convention at a time around a clean -open reference, holding other factors fixed.
result Inflation is highly selective, affecting specific features and execution methods.