RMC uses renewal theory for online reinforcement learning with low variance and easy implementation.
problem Online reinforcement learning for infinite horizon Markov decision processes.
method RMC combines Monte Carlo methods with renewal theory to estimate performance gradients and update policies.
result RMC converges to locally optimal policies and can be generalized to post-decision state models.
We briefly review our recent studies on stochastic processes modelling internet on-line trading. We present a way to evaluate the average waiting time between the observation of the price in financial markets and the next price change, especially in an on-line foreign exchange trading service for individual customers v…
A stochastic model helps maintain insufficiently funded pension funds.
problem Maintaining pension funds that are underfunded and require external financing.
method A time-homogeneous diffusion process with a barrier is used to model the unrestricted reserves value, and a renewal-reward process models the financing effort.
result Expected values and cost evaluations of maintenance are derived, and the approach is applied to a generalized Brownian motion process.
We evaluate the average waiting time between observing the price of financial markets and the next price change, especially in an on-line foreign exchange trading service for individual customers via the internet. Basic technical idea of our present work is dependent on the so-called renewal-reward theorem. Assuming th…
Study on error probability for classification of heavy-tailed renewal processes.
problem Error probability in classification of heavy-tailed renewal processes.
method Asymptotic expressions for Bhattacharyya bound on misclassification error probabilities.
result Obtained asymptotic expressions for misclassification error probabilities.
Characterizes measures preserving compound mixed renewal process properties.
problem Preserving compound mixed renewal process properties under different probability measures.
method Characterization of progressively equivalent probability measures.
result Any compound mixed renewal process can be converted into a compound mixed Poisson process through a change of measures.
Modeling stock order book dynamics with bouncing GBMs.
problem Capturing the dynamics of order book prices in financial markets.
method Modeling order book bid and ask prices as bouncing geometric Brownian motions.
result The logarithmic trading price process converges to a standard Brownian motion as δ approaches 0.
We propose a useful approach for investigating the statistical properties of foreign currency exchange rates. Our approach is based on queueing theory, particularly, the so-called renewal-reward theorem. For the first passage processes of the Sony Bank US dollar/Japanese yen (USD/JPY) exchange rate, we evaluate the ave…
Study warranty costs using alternating geometric process models.
problem Evaluate warranty servicing costs with varying repair times and product ages.
method Model repair and operational times using alternating geometric processes and derive new results for finite horizon.
result New insights into warranty costs under different warranty types.
A new method uses Gaussian Processes to solve power flow problems with uncertain renewable and load inputs.
problem Solving power flow problems with uncertain renewable and load inputs.
method Non-parametric Bayesian inference-based uncertainty propagation using Gaussian Processes.
result The method provides reasonably accurate solutions with fewer samples and time compared to Monte-Carlo simulations.
Unified framework for intermittent demand forecasting using renewal processes.
problem Intermittency in demand forecasting.
method Unified framework based on extensions of discrete-time renewal processes.
result Efficacy demonstrated in forecasting practice with favorable predictive accuracy.
Paper develops deep models for forecasting intermittent demand.
problem Forecasting intermittent demand with sporadic occurrences.
method Uses deep neural networks to model conditional interdemand time and size distributions.
result Empirical validation of deep models for intermittent demand forecasting.
In this paper, we obtain the finite-horizon and infinite-horizon ruin probability asymptotics for risk processes with claims of subexponential tails for non-stationary arrival processes that satisfy a large deviation principle. As a result, the arrival process can be dependent, non-stationary and non-renewal. We give t…
The study analyzes a model for aggregate losses with dependent and overdispersed inter-losses times.
problem Analyzing aggregate loss models with dependent and overdispersed inter-losses times.
method The study uses a two-state Markovian arrival process (MAP2) and a Markov renewal process to model the inter-losses times. Severities are modeled using a heavy-tailed, double-Pareto Lognormal distribution. The model is estimated via direct maximization of the likelihood function.
result The model with dependence and overdispersion in inter-losses times leads to higher capital charges compared to a Poisson process.
Paper uses Gaussian processes to solve AC-OPF with renewable uncertainty.
problem Optimizing power grids with fluctuating renewable sources.
method Data-driven approach using Gaussian processes.
result Efficiently solves chance-constrained AC-OPF with uncertainty.
We analyze a method to produce pairs of non independent Poisson processes M(t),N(t) from positively correlated, self-decomposable, exponential renewals. In particular the present paper provides the family of copulas pairing the renewals, along with the closed form for the joint distribution pm,n(s,t) of the pair…
Two-stage method improves solar forecasting accuracy.
problem Inaccurate solar forecasting due to nonstationary data.
method Two-stage approach: linear and nonlinear forecasting, data processing.
result Improved accuracy of solar forecasting results.
Ridge regression linked to Poisson resetting in statistical physics.
problem Understanding and extending ridge regularization in machine learning.
method Connecting stochastic resetting from statistical physics with ridge regularization in machine learning, using renewal processes.
result Exact filter identities for ridge regularization in various reset laws, including exponential and non-exponential.
We analyze the data of the Italian and U.S. futures on the stock markets and we test the validity of the Continuous Time Random Walk assumption for the survival probability of the returns time series via a renewal aging experiment. We also study the survival probability of returns sign and apply a coarse graining proce…
The paper addresses XVA valuation under market crises using a renewal process.
problem XVA valuation without considering market crises and illiquidity.
method Using an alternating renewal process, the paper develops a framework to price XVA under a state-dependent financial regime.
result The XVA price is characterized as a solution to a backward stochastic differential equation (BSDE).
Framework for renewable energy forecasting and feature engineering.
problem Forecasting and feature extraction for multivariate processes in renewable energy.
method Derivative-free optimization, ensemble of sequence-to-sequence networks, additive resampling, Bootstrap aggregating.
result The proposed method outperforms other machine learning techniques in long-term forecasts and feature selection.
Bayesian GAN generates diverse renewable scenarios efficiently.
problem Generating diverse and accurate renewable energy scenarios.
method Bayesian GAN, a deep neural network approach.
result Generates clusters of wind and solar scenarios with different variance and mean values.
Proposes a model to update industrial data predictions based on temporal changes.
problem Improving prediction accuracy in industrial data analytics by addressing changing conditions over time.
method Integrates similarity and loss functions to estimate and update prediction models adaptively.
result The data renewal model enhances prediction accuracy by identifying and updating model changes.
The paper tackles resource allocation for arms with unknown and random rewards, achieving optimal regret bounds.
problem Allocating resources on arms with unknown and random rewards.
method Developed two algorithms with optimal regret bounds for b∈[0,1], demonstrating a phase transition at b=1/2. result Achieved optimal gap-dependent and gap-independent regret bounds for b∈[0,1]. This paper addresses reward estimation and incentive design for agents with hidden rewards.
problem Estimating and incentivizing agents with unknown rewards in a learning setting.
method Repeated adverse selection game with a self-interested learning agent and a learning principal. Introduces an estimator for consistent reward estimation and a data-driven incentive policy.
result Finite-sample consistency of the estimator and a rigorous regret bound for the principal.
Study on energy storage's impact on electricity prices and profitability.
problem Analyzing the profitability of energy storage in electricity markets.
method Characterized optimal operating strategy for storage systems, determined equilibrium price in a market with storage, renewables, and conventional producers, and characterized price process using stochastic differential equations.
result Increased average revenues and interquantile ranges for storage assets in energy transition scenarios.
GP CC-OPF solves uncertain power grid optimization with Gaussian Process.
problem Uncertainty in power grid operations due to high renewables integration.
method Data-driven Gaussian Process regression for solving non-convex CC-OPF problem.
result Effective economic dispatch optimization in uncertain power grids.
Dynamic probabilistic forecasts guide optimal decisions in uncertain processes.
problem Optimal decision making in processes influenced by uncertain random factors.
method Stochastic models for probabilistic forecast evolution, calibrated from ensemble forecasts.
result Optimal decision strategies determined using dynamic probabilistic forecasts.
Proposes a value-oriented forecast reconciliation method for renewables in electricity markets.
problem Forecast reconciliation overlooks the value of forecasts in decision-making, leading to unfair outcomes.
method Value-oriented forecast reconciliation using a Nash bargaining framework and a primal-dual algorithm for parameter estimation.
result Consistently increases profits for all agents involved in an aggregated wind energy trading problem.
We introduce a new model for describing the fluctuations of a tick-by-tick single asset price. Our model is based on Markov renewal processes. We consider a point process associated to the timestamps of the price jumps, and marks associated to price increments. By modeling the marks with a suitable Markov chain, we can…
This paper forecasts renewable energy prospects in South America through cross-border interconnection.
problem Lack of renewable energy integration across South American countries.
method Long-term scenario forecasting methodology applied to raw data from typical countries.
result Promoting cross-border interconnection towards renewables can optimize energy supply, reduce costs, and balance the energy matrix.
Develops framework for valuing and assessing risk of renewable PPAs.
problem Valuation and risk assessment of non-standard renewable PPAs.
method Formalizes payoff structures, derives fair contract prices, proposes market risk-assessment methodology.
result Fair prices and risk profiles vary across technologies and contractual structures.
We approximate the distribution of total expenditure of a retail company over warranty claims incurred in a fixed period [0, T], say the following quarter. We consider two kinds of warranty policies, namely, the non-renewing free replacement warranty policy and the non-renewing pro-rata warranty policy. Our approximati…
In this paper we will develop a methodology for obtaining pricing expressions for financial instruments whose underlying asset can be described through a simple continuous-time random walk (CTRW) market model. Our approach is very natural to the issue because it is based in the use of renewal equations, and therefore i…
Proposes a pricing agent using reinforcement learning to balance renewable energy demand.
problem Intermittent renewable energy sources challenge carbon-free electricity generation.
method Reinforcement learning approach to balance customer demand with renewable energy generation.
result Demonstrates improved electricity pricing strategy for renewable energy integration.
Early stopping methods reduce unnecessary reasoning steps in LLMs by monitoring uncertainty signals.
problem LLMs sometimes generate unnecessary reasoning steps, especially under uncertainty.
method Statistically principled early stopping methods that monitor uncertainty signals during generation.
result Uncertainty-aware early stopping improves efficiency and reliability in LLM reasoning, especially in math reasoning.
Optimizes insurance renewal tariffs considering constraints and premium loadings.
problem Optimizing renewal tariffs under business and technical constraints.
method Mathematical and algorithmic approaches, including continuous and discrete optimization.
result Presented several sub-optimal algorithmic solutions and simulation techniques.
Study shows increased renewables lead to more extreme price spikes in balancing markets.
problem Higher variability of renewable energy leads to increased ancillary service costs, especially extreme price spikes.
method Agent-based modeling to simulate fluctuations in load and renewable energy production, forecasting resulting energy prices and costs.
result The probability of extreme price events increases with an increased share of renewable energy sources.
Paper develops framework for valuing and assessing credit risk in renewable PPAs.
problem Renewable PPAs expose both parties to counterparty credit risk.
method Modelled joint dynamics of electricity prices and renewable output, incorporated default probabilities.
result Provides transparent metric for PPA valuation under counterparty risk.
MLM models match or exceed RN in generating wind power time series without location info.
problem Generating accurate long-term wind power time series without location information.
method Applied neural networks to MERRA2 wind speed data with and without location info.
result MLM models produce time series of equal or better quality than RN.
The episodic, irregular and asynchronous nature of medical data render them difficult substrates for standard machine learning algorithms. We would like to abstract away this difficulty for the class of time-stamped categorical variables (or events) by modeling them as a renewal process and inferring a probability dens…
Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.
problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.
method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.
result Algorithm proves correct and convergent for learning PRMs from decision processes.
Optimizes renewable energy mix to meet carbon-free targets at lowest cost.
problem Minimizing annual procurement costs while achieving specified carbon-free hourly performance.
method Probabilistic framework with simulation scenarios and probability constraints. Fixed set of renewable generators and load customer.
result Demonstrated that certain renewable energy portfolios can meet carbon-free targets at lower costs compared to others.
This paper optimizes a power-to-heat system using reinforcement learning for cost minimization under uncertain conditions.
problem Optimizing a power-to-heat system with fluctuating renewable energy sources.
method Stochastic optimal control, reinforcement learning (Q-learning).
result Reinforcement learning provides an efficient solution to the optimization problem.
We consider the problem of finding the optimal time to sell a stock, subject to a fixed sales cost and an exponential discounting rate ρ. We assume that the price of the stock fluctuates according to the equation dY_t=Y_t(μdt+σξ(t) dt), where (ξ(t)) is an alternating Markov renewal process with values in {\pm1}, with a…
Paper introduces normalizing flows for accurate probabilistic energy forecasting.
problem Uncertainty in renewable energy forecasting for power systems.
method Normalizing flows for direct learning of multivariate stochastic distributions.
result Normalizing flows outperform other deep learning models in probabilistic forecasting.
AI-driven sales prioritization boosts renewal bookings by 8.08%.
problem Manual sales account prioritization is inefficient and under-invested.
method Developed an AI-based Account Prioritizer using machine learning and explanation algorithms.
result Generated a +8.08% increase in renewal bookings.
The paper tackles reinforcement learning with exogenous variables and rewards.
problem Exogenous state variables and rewards slow reinforcement learning by introducing uncontrolled variation.
method Formalizes exogenous state variables and rewards, decomposes MDP into exogenous and endogenous components, and introduces algorithms to discover these components.
result Optimal policies for the endogenous MDP are also optimal for the original MDP, but the endogenous MDP is easier to solve due to reduced variance.