CSER improves SGD efficiency by resetting errors and partial synchronization.
problem Limited scalability of Distributed Stochastic Gradient Descent (SGD) due to communication bottlenecks.
method Introduces 'error reset' technique and partial synchronization for gradients and models.
result Proves convergence for smooth non-convex problems and accelerates distributed training significantly.
ET-GP-UCB optimizes time-varying functions without knowing change rates.
problem Sequentially optimizing a time-varying objective function with unknown change rates.
method Event-triggered Bayesian optimization with adaptive resets based on probabilistic uniform error bounds.
result ET-GP-UCB outperforms other GP-UCB algorithms in synthetic and real-world data.
Optimizes search times by resetting agents when a threshold is reached.
problem Improving search efficiency in systems with thresholds.
method Develops a framework for correlated stochastic processes with threshold resetting.
result Optimal resetting can prevent larger losses and is applicable to various stochastic systems.
We study how resetting affects geometric Brownian motion, showing it becomes stationary but remains non-ergodic.
problem Effects of stochastic resetting on geometric Brownian motion.
method Analysis of geometric Brownian motion under stochastic resetting.
result Resetting makes geometric Brownian motion stationary but non-ergodic.
The paper solves a pricing problem for a multiple reset put option using integral equations.
problem Valuation of a multiple reset put option with reset rights.
method Formulated as a multiple optimal stopping problem, reduced to single optimal stopping problems, solved by induction and integral equations.
result Characterized optimal reset boundaries as solutions to nonlinear integral equations and derived reset premium representations.
Ridge regression linked to Poisson resetting in statistical physics.
problem Understanding and extending ridge regularization in machine learning.
method Connecting stochastic resetting from statistical physics with ridge regularization in machine learning, using renewal processes.
result Exact filter identities for ridge regularization in various reset laws, including exponential and non-exponential.
Optimal threshold resetting reduces search time for multiple diffusive searchers.
problem Optimizing search time for multiple diffusive searchers in a one-dimensional space.
method Threshold resetting (TR) is introduced as an event-driven optimization strategy, coupling resetting to the internal dynamics of searchers.
result Optimal threshold distance u significantly reduces mean first-passage time for N≥2 searchers, with a minimum at Nopt(u). Selective reinitialization improves adaptability of neural bandits in dynamic environments.
problem Loss of plasticity in neural bandits, leading to rigid neural network parameters.
method Selective Reinitialization (SeRe) framework that dynamically resets underutilized units.
result SeRe enhances adaptability of CNB algorithms, reducing cumulative regret in dynamic environments.
Resetting from checkpoints improves DNN training with noisy labels.
problem Latent gradient bias induced by noisy labels causes overfitting.
method Stochastic resetting applied to SGD to mitigate latent gradient bias.
result Resetting significantly improves DNN generalization performance.
The paper analyzes XVA reduction strategies in financial crises using Mandatory Breaks, Restructuring, and Resets.
problem Challenges in client XVA management during crises when continuous collateralization is not feasible.
method Compares multiple trade strategies including Mandatory Breaks, Restructuring, and Resets.
result Resets can be twice as effective as Mandatory Breaks/Restructuring if there is no credit recovery. When recovery is at least 1/3, Mandatory Breaks/Restructuring can be more effective.
Exact results on power-law distributions in systems with resets.
problem Understanding power-law distributions in systems with resets.
method Exact mathematical analysis of multiplicative processes with resets.
result Power-law distributions are built up over time and their moments behave quantitatively determined by parameters.
This paper tackles belief-state selection in simulators with latent states.
problem Selecting among approximate belief-state samplers for simulators with latent variables.
method Reduces belief-state selection to conditional distribution selection, develops algorithms and analyses.
result Different formulations of belief-state selection have varying guarantees under different roll-out methods.
New framework finds periodic policies in reset-free MDPs with sublinear regret.
problem Reset-free reinforcement learning with unknown dynamics and terminal law constraints.
method Periodic framework, periodic policies, periodic regret.
result First non-asymptotic guarantees for reset-free learning in multi-agent settings.
New method extends supervised learning for non-stationary control problems.
problem Optimal control in non-stationary, reset-free environments.
method Prospective Learning with Control (PLuC) using Empirical Risk Minimization (ERM).
result ERM asymptotically achieves Bayes optimal policy in non-stationary environments.
New trading strategy beats traditional grid in crypto markets.
problem Low expected return of traditional grid trading strategy.
method Dynamic Grid Trading (DGT) strategy that adapts to market conditions.
result DGT strategy outperforms traditional grid and buy-and-hold strategies.
Study proposes adaptive RL for dynamic portfolio optimization.
problem Traditional portfolio optimization models fail to adapt to regime shifts.
method Regime-aware reinforcement learning framework with hybrid observations and constrained reward functions.
result Transformer PPO achieves highest risk-adjusted returns, while LSTM variants offer a good balance.
Study shows MSE with sigmoid can match SCE in classification tasks, especially with noisy data.
problem Inconsistent errors in neural network classification tasks.
method Introduced Output Reset algorithm to use MSE with sigmoid activation.
result MSE with sigmoid activation achieves comparable accuracy and convergence rates to Softmax Cross-Entropy, especially in noisy data scenarios.
We study learning control in an online reset-free lifelong learning scenario, where mistakes can compound catastrophically into the future and the underlying dynamics of the environment may change. Traditional model-free policy learning methods have achieved successes in difficult tasks due to their broad flexibility, …
Study agnostic RL in large state spaces with weak function approximation.
problem Statistical intractability of agnostic policy learning in various environments.
method Investigates agnostic policy learning with different forms of environment access.
result Agnostic policy learning remains statistically intractable with certain forms of environment access.
Deep learning models price convertible bonds with complex reset and call features.
problem Pricing convertible bonds with path-dependent reset and call provisions.
method Formulated as a PPDE, deep learning approximates conditional expectations.
result Deep learning produces stable and accurate prices across various model specifications.
New algorithm for nonstationary multi-armed bandits with optimal performance.
problem Nonstationary multi-armed bandits with changing model parameters over time.
method Adaptive Resetting Bandit (ADR-bandit) algorithm using adaptive windowing techniques.
result ADR-bandit achieves nearly optimal performance in both abrupt and gradual changes.
A variable annuity is an equity-linked financial product typically offered by insurance companies. The policyholder makes an upfront payment to the insurance company and, in return, the insurer is required to make a series of payments starting at an agreed upon date. For a higher premium, many insurance companies offer…
UDRL fails to converge in stochastic environments with episodic resets.
problem UDRL's convergence in stochastic environments with resets is questioned.
method UDRL is a supervised learning approach that does not use value functions.
result UDRL diverges in a simple stochastic environment with resets.
New RL approach uses resets to learn complex MDPs efficiently.
problem Learning complex MDPs with high-dimensional states and function approximation.
method Local simulator access and recursive value function search.
result Proven sample-efficient learning for MDPs with low coverability.
We provide an analytically treatable model that describes in a unified manner income distribution for all income categories. The approach is based on a master equation with growth and reset terms. The model assumptions on the growth and reset rates are tested on an exhaustive database with incomes on individual level s…
Markov switching models (MSMs) are probabilistic models that employ multiple sets of parameters to describe different dynamic regimes that a time series may exhibit at different periods of time. The switching mechanism between regimes is controlled by unobserved random variables that form a first-order Markov chain. Ex…
Deep RL algorithms can overfit to early experiences, leading to poor performance.
problem Overfitting to early interactions in deep reinforcement learning.
method Proposed a mechanism to periodically reset part of the agent to mitigate overfitting.
result Periodic resetting improves performance in both discrete and continuous action domains.
Recurrent neural networks (RNNs) are powerful dynamical models for data with complex temporal structure. However, training RNNs has traditionally proved challenging due to exploding or vanishing of gradients. RNN models such as LSTMs and GRUs (and their variants) significantly mitigate these issues associated with trai…
GRUwE improves irregular time series prediction with simpler, efficient RNN-based approach.
problem Irregularly sampled multivariate time series prediction challenges.
method Gated Recurrent Unit with Exponential basis functions (GRUwE).
result GRUwE achieves competitive or superior performance compared to recent state-of-the-art methods.
Tax dynamics affects wealth distribution in a linearly growing socio-economic model.
problem Analyzing how tax policies impact wealth distribution in a stochastic resetting system.
method Analytical and numerical study of a system of agents with linear wealth growth, stochastic resetting, and tax redistribution.
result Optimal taxation leads to economic equality, while excessive taxation results in reverse disparity.
We consider the sequential Bayesian optimization problem with bandit feedback, adopting a formulation that allows for the reward function to vary with time. We model the reward function using a Gaussian process whose evolution obeys a simple Markov model. We introduce two natural extensions of the classical Gaussian pr…
The goal of a learner, in standard online learning, is to have the cumulative loss not much larger compared with the best-performing function from some fixed class. Numerous algorithms were shown to have this gap arbitrarily close to zero, compared with the best function that is chosen off-line. Nevertheless, many real…
We consider the generic approach of using an experience memory to help exploration by adapting a restart distribution. That is, given the capacity to reset the state with those corresponding to the agent's past observations, we help exploration by promoting faster state-space coverage via restarting the agent from a mo…
This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.
problem Challenges in reinforcement learning with continuous interaction and sparse delayed rewards.
method Analysis of environment shaping and dynamism properties to simplify learning.
result Properties like environment shaping and dynamism can significantly ease learning in non-episodic, sparse reward settings.
The paper evaluates three variants of the Gated Recurrent Unit (GRU) in recurrent neural networks (RNN) by reducing parameters in the update and reset gates. We evaluate the three variant GRU models on MNIST and IMDB datasets and show that these GRU-RNN variant models perform as well as the original GRU RNN model while…
This is a facsimile of the circa 1990 unpublished manuscript with the same title. All the original text, figures and tables are included; although text has been reset in \TeX, the original hand-drawn figures have been redrawn digitally, and the parameter k in the original table of lens spaces has been replaced with t…
New method designs fairer transport plans with uncertainty.
problem Designing fair and balanced mass transport plans.
method Hierarchical fully probabilistic design (HFPD) for transport plans.
result Optimal hyperprior for transport plans with uncertain marginals.
Study of a generalized geometric Brownian motion with varying entry and exit rates.
problem Understanding the long-run behavior of economic systems with growth, volatility, entry, and exit.
method Generalized geometric Brownian motion framework with varying entry and exit rates, analyzing moments and survival probability.
result Optimal exit rate minimizes mean first-passage time, influencing system outcome.
EXODUS improves training of SNNs by stabilizing gradients and reducing complexity.
problem Training SNNs using BPTT is time-consuming and numerically unstable.
method EXODUS modifies SLAYER to account for neuron reset and uses IFT for correct gradient calculation, eliminating manual scaling.
result EXODUS achieves comparable or better performance than SLAYER, especially in tasks with temporal features.
New method converts conventional ANNs to SNNs with minimal loss and efficiency.
problem Difficulty in training SNNs directly from conventional ANNs due to discreteness.
method Proposes a novel pipeline combining threshold balance and soft-reset mechanisms for efficient conversion.
result Achieves almost no accuracy loss with only 1/10 of typical SNN simulation time.
Twin-Boot integrates uncertainty estimation into optimization using parallel training of identical models.
problem Uncertainty in overparameterized models, especially in low-data regimes.
method Twin-Bootstrap Gradient Descent (Twin-Boot) trains two identical models on independent bootstrap samples and uses their divergence to guide learning.
result Improves calibration and generalization, yields interpretable uncertainty maps.
ReD improves LLM inference efficiency at fixed budget, reducing attempts and cost.
problem Improving LLM inference efficiency at a fixed budget.
method Reset-and-Discard (ReD) query method.
result ReD increases coverage@cost for a given budget, reducing attempts and cost.
From the Hamilton-Jacobi-Bellman equation for the value function we derive a non-linear partial differential equation for the optimal portfolio strategy (the dynamic control). The equation is general in the sense that it does not depend on the terminal utility and provides additional analytical insight for some optimal…
Proposes a new TS algorithm for non-stationary bandits using KS tests.
problem Non-stationary multi-armed bandit problems.
method Active detection of change points using KS tests and adaptive Thompson Sampling.
result Sub-linear regret demonstrated for the two-armed bandit case.
This paper presents numerical algorithm and results for pricing a capital protection option offered by many asset managers for investment portfolios to take advantage of market growth and protect savings. Under optimal withdrawal policyholder behaviour the pricing of such a product is an optimal stochastic control prob…
The paper proposes a machine learning framework for portfolio optimization with limited data.
problem Low data environments and regime uncertainty in portfolio optimization.
method A teacher-student learning pipeline with CVaR optimizer generating supervisory labels and neural models trained on real and synthetic data.
result Student models can match or outperform the CVaR teacher and achieve improved robustness under regime shifts.
In many environments, only a relatively small subset of the complete state space is necessary in order to accomplish a given task. We develop a simple technique using emergency stops (e-stops) to exploit this phenomenon. Using e-stops significantly improves sample complexity by reducing the amount of required explorati…
In this paper, we develop a Markovian model that deals with the volume offered at the best quote of an electronic order book. The volume of the first limit is a stochastic process whose paths are periodically interrupted and reset to a new value, either by a new limit order submitted inside the spread or by a market or…