Research predicts cryptocurrency staking rewards with high accuracy.
problem Predicting cryptocurrency staking rewards.
method Two predictive methodologies: sliding-window average and linear regression models.
result ETH staking rewards can be forecasted with RMSE within 0.7% and 1.1% of the mean value for 1-day and 7-day look-aheads respectively.
INF-clip optimizes heavy-tailed MAB problems with improved performance.
problem Optimizing multi-armed bandit problems with heavy-tailed rewards.
method INF-clip algorithm for adversarial and stochastic heavy-tailed MAB settings.
result INF-clip is optimal for linear and non-linear heavy-tailed stochastic MAB problems.
This work improves MARL exploration efficiency using predictive guidance and prioritized experience replay.
problem Challenges in multi-agent reinforcement learning, especially in exploration efficiency and reward informativeness.
method A predictive network forecasts reward, guiding MARL to accelerate exploration. Improved prioritized experience replay utilizes TD error effectively.
result The proposed algorithm outperforms existing methods in cooperative multi-agent environments.
Optimizes forecast distributions for financial risk management.
problem Improving risk management through better forecast distributions.
method Optimizes forecast distributions using scoring rules relevant to financial risk management.
result Tail-focused predictive distributions yield better outcomes in hedging strategies involving VIX futures.
This thesis proposes a derivatives hedging framework using deep learning and reinforcement learning.
problem Traditional hedging models fail in complex, uncertain markets due to assumptions like continuous trading and zero transaction costs.
method Integrates deep learning and reinforcement learning, using a spatiotemporal attention-based Transformer for probabilistic forecasting and hedging.
result The proposed method significantly outperforms traditional approaches in U.S. and Chinese financial markets.
Cash management is concerned with optimizing the short-term funding requirements of a company. To this end, different optimization strategies have been proposed to minimize costs using daily cash flow forecasts as the main input to the models. However, the effect of the accuracy of such forecasts on cash management pol…
KKR uses Koopman theory to improve forecasting in complex systems.
problem Forecasting complex, nonlinear dynamical systems in decision-making.
method Derives a universal Koopman-invariant RKHS for LTI dynamical systems.
result KKR framework provides convergence results and generalization error bounds.
Enhanced decision-making through Dreamer's anticipatory trajectories and Online Decision Transformer.
problem Efficiently integrating world models with decision transformers.
method Combining Dreamer's trajectory forecasting with Online Decision Transformer's adaptive learning.
result Notable improvements in sample efficiency and reward maximization.
Study improves trading decisions by predicting profit and loss outcomes.
problem Inconsistent profitability of machine learning forecasts in financial markets.
method Developed a novel algorithm for forecasting profit and loss outcomes, integrating with market trend predictions.
result Significantly improved performance of trading strategies, including traditional and algorithmic trading.
VTA combines verbal and latent reasoning for accurate stock time-series forecasts.
problem Challenges in combining textual analysis with time-series data for financial forecasting.
method Converts stock price data into textual annotations, optimizes reasoning trace using inverse MSE, conditions time-series model outputs on reasoning attributes.
result VTA achieves state-of-the-art forecasting accuracy and interpretable reasoning traces.
Bayes optimal algorithm under certain conditions doesn't achieve exponential simple regret.
problem Best arm identification with normal rewards over time.
method Fixed-budget best arm identification problem with rewards from normal distributions. Evaluates performance via simple regret.
result Bayes optimal algorithm does not yield exponential decrease in simple regret.
NIFTY dataset for financial forecasting models.
problem Improving financial market forecasting with LLMs.
method Two datasets for supervised and reinforcement learning.
result Demonstrates applications in stock price prediction.
Current clinical practice to monitor patients' health follows either regular or heuristic-based lab test (e.g. blood test) scheduling. Such practice not only gives rise to redundant measurements accruing cost, but may even lead to unnecessary patient discomfort. From the computational perspective, heuristic-based test …
Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed meas…
Prognosticator improves performance in non-stationary MDPs.
problem Real-world applications often violate the stationary assumption in reinforcement learning.
method A policy gradient algorithm that maximizes a forecast of future performance.
result Prognosticator outperforms online adaptation techniques in non-stationary MDPs.
Due to the limited predictability of wind power and other stochastic generation, trading this energy in competitive electricity markets is challenging. This paper derives revenue-maximising and risk-constrained strategies for stochastic generators participating in electricity markets with a single-price balancing mecha…
Proposes a new TS algorithm for non-stationary bandits using KS tests.
problem Non-stationary multi-armed bandit problems.
method Active detection of change points using KS tests and adaptive Thompson Sampling.
result Sub-linear regret demonstrated for the two-armed bandit case.
New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
A new online learning setting for autoregressive processes with sublinear regret.
problem Sequential decision-making with temporal dependence in autoregressive processes.
method Autoregressive Bandits (ARBs) and AutoRegressive Upper Confidence Bound (AR-UCB) algorithm.
result Sublinear regret of order $\widetilde{\mathcal{O}} \left( \frac{(k+1)^{3/2}\sqrt{nT}}{(1-Γ)^2}
ight)$ for optimal policy.
Machine learning improves portfolio allocation between index and risk-free assets.
problem Finding optimal portfolio rules for time-varying returns and volatility.
method Two Random Forest models: one for sign probabilities of excess return, the other for optimized volatility.
result Substantial improvements in utility, risk-adjusted returns, and maximum drawdowns over buy-and-hold.
Reward hacking exploits misspecified rewards, affecting agent capabilities and true performance.
problem Reward hacking in RL models exploiting reward misspecifications.
method Constructed four RL environments with misspecified rewards; analyzed agent capabilities and behavior.
result More capable agents exploit reward misspecifications, achieving higher proxy reward but lower true reward.
Optimal reconciliation keeps some forecasts unchanged in hierarchical forecasting.
problem Keeping some forecasts unchanged in hierarchical forecasting.
method Formulates a method to keep some forecasts unchanged in a hierarchical forecasting system.
result Preserves unbiasedness and non-negativity of forecasts.
Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.
problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.
method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.
result Algorithm proves correct and convergent for learning PRMs from decision processes.
Paper addresses reward learning issues in RL, improving both under- and over-estimation.
problem Reward learning from data can lead to reward delusions or underestimation, causing unintended behaviors.
method Connects reward learning to positive-unlabeled (PU) learning and applies a large-scale PU learning algorithm.
result Improves both GAIL and supervised reward learning without additional assumptions.
This work analyzes the value of future reward information in RL.
problem Analyzing the impact of knowing future rewards in reinforcement learning.
method Competitive analysis and worst-case reward distribution.
result Exact ratios between standard RL agents and those with future-reward lookahead.
Proposes a new Hawkes process bandit model for disaster search and rescue.
problem Forecasting and detecting spatio-temporal events with undersampled or biased data.
method Upper confidence bound algorithm using Bayesian spatial Hawkes process estimation.
result Model outperforms state-of-the-art spatial MAB algorithms in disaster search and rescue.
Self-supervised reward prediction improves RL in sparse reward settings.
problem Data efficiency and sparse reward signals in reinforcement learning.
method Learning a state representation for reward prediction and using it to shape rewards.
result Self-supervised reward prediction enhances RL algorithms in single-goal environments.
The study categorizes reward errors in reinforcement learning, finding some can be beneficial.
problem Training language models with imperfect proxy rewards.
method Theoretical analysis of policy gradient optimization and categorization of reward errors.
result Reward errors can be benign or even beneficial, preventing policy from stalling.
Reward collapse occurs when ranking-based reward models yield uniform rewards for different prompts.
problem Reward collapse in aligning large language models with human preferences.
method Introduced a prompt-aware optimization scheme to derive closed-form expressions for reward distributions.
result Our prompt-aware utility functions significantly alleviate reward collapse during training.
Short-term load forecasting is a critical element of power systems energy management systems. In recent years, probabilistic load forecasting (PLF) has gained increased attention for its ability to provide uncertainty information that helps to improve the reliability and economics of system operation performances. This…
Combining forecasts of 16 ED causes improves accuracy and stability.
problem Forecasting accuracy and stability for ED admissions is poor due to model uncertainty and limited data.
method High-dimensional forecast combinations of 16 cause-specific ED forecasts using extensive covariates.
result Forecast combinations yield forecast accuracies of 3.81%-23.54% across causes, outperforming individual models in 50% of scenarios.
Reward models need more than just accuracy for effective RLHF.
problem The effectiveness of reward models in RLHF is not fully understood.
method An optimization perspective to evaluate reward models.
result Reward models with low reward variance can lead to a flat optimization landscape, hindering performance.
Proposes a method to boost deep reinforcement learning with sparse rewards.
problem Challenges in learning complex behaviors with long horizons and sparse rewards.
method Predictive coding for reward shaping.
result Achieves better learning by providing reward signals that understand environment dynamics and emphasize useful features.
Conditional forecasts improve performative prediction accuracy.
problem Performative predictions undermine standard forecasting methods.
method Condition forecasts on covariates to make them forecast-invariant.
result Proper scoring rules fail under conditioning, but two solutions are identified.
Action guidance helps agents learn true objectives in games with sparse rewards.
problem Training agents in games with sparse rewards requires significant exploration.
method Action guidance, a novel technique that combines exploration with reward shaping.
result Action guidance enables agents to optimize true objectives efficiently.
Study improves seasonal forecasts using deep learning.
problem Challenges in generating large forecast ensembles and limited observations for verification.
method Developed a probabilistic deep neural network model.
result Demonstrated favorable skill compared to state-of-the-art dynamical forecast systems.
New RL method uses distance between states instead of rewards for sparse reward environments.
problem Sparse rewards or non-reward environments in reinforcement learning.
method Uses goal-distance gradient and bridge point planning for policy improvement.
result Significantly better performance on sparse reward and local optimal problems in complex environments.
We empirically test predictability on asset price by using stock selection rules based on maximum drawdown and its consecutive recovery. In various equity markets, monthly momentum- and weekly contrarian-style portfolios constructed from these alternative selection criteria are superior not only in forecasting directio…
Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.
problem Learning from sparse and delayed rewards in reinforcement learning.
method Randomized Return Decomposition (RRD) algorithm to redistribute rewards.
result Substantial improvement over baseline algorithms in experiments.
Enhances reward specification in RL with a novel language-based approach.
problem Reward specification in RL can lead to unintended, potentially harmful behaviours.
method Developed a novel class of language-based Reward Machines using RML's built-in memory.
result Can specify non-regular, non-Markovian reward functions for complex tasks.
Reward tweaking optimizes behavior for long-term goals by adjusting the reward function.
problem Optimizing behavior for long-term goals in reinforcement learning with unstable long planning horizons.
method Reward tweaking learns a surrogate reward function that induces optimal behavior for the original task.
result Reward tweaking guides agents towards better long-term returns while planning for short horizons.
We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…
Many reinforcement-learning researchers treat the reward function as a part of the environment, meaning that the agent can only know the reward of a state if it encounters that state in a trial run. However, we argue that this is an unnecessary limitation and instead, the reward function should be provided to the learn…
Extends reinforcement learning alignment to scalar rewards, improving math reasoning.
problem Designing reinforcement learning algorithms for general LLM alignment.
method Introduces f-GRPO and f-HAL, estimating f-divergences between reward-aligned and unaligned distributions.
result Improves math reasoning RLVR tasks and mitigates reward hacking.
This work characterizes reward function partial identifiability and its impact on policy optimization.
problem Reward function partial identifiability in complex tasks.
method Formal characterisation of partial identifiability using various reward learning data sources.
result Unified framework for comparing data sources and downstream tasks by their invariances.
This paper introduces a new reward shaping method for average-reward reinforcement learning.
problem Speeding up convergence to an optimal policy in average-reward reinforcement learning tasks.
method Developed a temporal logic-based approach to automatically generate reward shaping functions.
result The optimal policy can be recovered using the proposed reward shaping framework.
For2For combines forecasts to improve time series forecasting.
problem Improving time series forecasting accuracy.
method Combines standard forecasting methods and machine learning models using forecasts as features.
result Outperforms all submissions in the M4 competition for quarterly series and most monthly series.
Paper generalizes reward distribution in multi-armed bandits with temporally-partitioned rewards.
problem Handling partial rewards distributed over multiple rounds in multi-armed bandits.
method Introduces Beta-spread property to generalize reward distribution, derives lower bound, and provides TP-UCB-FR-G algorithm.
result Improves regret upper bound for some scenarios using Beta-spread property.