The basic financial purpose of a firm is to maximize its value. An inventory management system should also contribute to realization of this basic aim. Many current asset management models currently found in financial management literature were constructed with the assumption of book profit maximization as basic aim. H…
Study evaluates valuation models for UK companies using case studies.
problem Determining how accounting numbers affect business value.
method Comprehensive review of three valuation models: FCFVM, REVM, AEGM.
result Accounting numbers through valuation models can affect business value.
New issue found in value-based reinforcement learning for stochastic environments.
problem Value-based reinforcement learning struggles with stochastic state transitions.
method Demonstrated using a multiobjective Markov Decision Process (MOMDP).
result Approaches may converge to Pareto-dominated solutions instead of optimal ones.
RL agents outperform baselines in asset allocation.
problem Optimizing asset allocation using reinforcement learning.
method Model-free deep RL agents trained on real-world stock prices.
result RL agents significantly outperformed random and uniform allocation.
Overparameterized models generalize well in offline contextual bandits, but policy-based algorithms struggle.
problem The performance gap between value-based and policy-based algorithms in offline contextual bandits with overparameterized models.
method Analysis of action-stability in objectives and formal proofs of regret bounds.
result The performance gap is due to action-stability of objectives, with value-based objectives being stable and policy-based objectives unstable.
Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimizat…
Paper optimizes traffic signal control for better traffic flow.
problem Optimizing traffic signal control to reduce congestion and improve safety.
method Value-based reinforcement learning with interpretable policy functions (polynomial functions).
result Deep Regulatable Hardmax Q-learning variant reduces vehicle delay by up to 19.4%.
This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.
problem Investigating the representation complexity gap among model-based, policy-based, and value-based RL.
method Demonstrated through analysis of Markov decision processes (MDPs) and introduced new classes of MDPs.
result Representation complexity hierarchy: model-based RL > policy-based RL > value-based RL.
Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic pro…
This paper improves reinforcement learning policies in a scalable way.
problem Ensuring monotonic policy improvement in entropy-regularized RL.
method Derives an entropy-aware lower bound and proposes a novel RL algorithm.
result Demonstrates effectiveness in continuous-state tasks using a linear function approximator.
Proposes a value-based method for continuous control without an actor.
problem Computational infeasibility of evaluating Q-values in continuous action spaces.
method Structurally maximizable Q-functions, actor-free approach.
result Performance and sample efficiency comparable to actor-critic methods.
Value functions struggle to represent transition dynamics, impacting statistical efficiency.
problem Limited representational power of value functions in capturing transition dynamics.
method Case studies of various reinforcement learning problems to explore the limitations of value-based methods.
result Value-based methods can be as efficient as model-based ones in some cases but severely underperform in others due to information loss.
TF-Coder simplifies tensor manipulation programming in TensorFlow.
problem Difficulty in programming with TensorFlow due to steep learning curve.
method Bottom-up weighted enumerative search with value-based pruning and type/value filtering.
result Solves 63 out of 70 real-world tasks within 5 minutes.
Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some glo…
Unified framework for CO problems using RL, providing optimal solutions and convergence guarantees.
problem Combinatorial optimization problems
method Unified framework of Markov decision processes (MDPs) and value-based reinforcement learning (RL) techniques
result RL techniques converge to approximate solutions with a guarantee on optimality gap
Stochastic Q-learning tackles large action spaces with reduced computation.
problem Effective decision-making in complex environments with large discrete action spaces.
method Stochastic value-based RL approaches that consider a sublinear number of actions in each iteration.
result Stochastic Q-learning achieves near-optimal returns with significantly reduced computation time.
Unified view on selective credit assignment for reinforcement learning.
problem Efficient credit assignment in reinforcement learning.
method Unified temporal-difference algorithms with selective weightings.
result New algorithms for backward credit assignment and off-policy learning.
New RL algorithms adapt to time limits, improving task performance.
problem Fixed RL behaviors cannot adapt to different time restrictions.
method Introduced two algorithms for time adaptive RL: Independent Gamma-Ensemble and n-Step Ensemble.
result Zero-shot adaptation between different time restrictions.
Pretraining reinforcement learning methods with demonstrations has been an important concept in the study of reinforcement learning since a large amount of computing power is spent on online simulations with existing reinforcement learning algorithms. Pretraining reinforcement learning remains a significant challenge i…
New RL algorithm handles delayed feedback with posterior sampling.
problem Challenges of delayed feedback in reinforcement learning with linear function approximation.
method Posterior sampling with delayed feedback for value-based RL.
result Achieves optimal regret guarantee with improved computational efficiency.
Optimizes cryptocurrency trading pairs for efficiency and decentralization.
problem Finding optimal trading pairs among many cryptocurrencies without direct volume data.
method Two-stage process: 1) Fill missing values using eigenvalue decomposition with regularization, 2) Optimize pairs using branch and bound with pruning.
result Optimal trading pairs lead to more decentralized markets and better liquidity.
Centralised training with decentralised execution is an important setting for cooperative deep multi-agent reinforcement learning due to communication constraints during execution and computational tractability in training. In this paper, we analyse value-based methods that are known to have superior performance in com…
RobustTAD detects anomalies in diverse time series data.
problem Effective anomaly detection for complex time series data.
method Robust seasonal-trend decomposition + CNN architecture with data augmentation.
result RobustTAD outperforms other methods on public datasets.
New methods estimate transport-growth pairs in unbalanced optimal transport.
problem Statistical guarantees for Monge-type estimation in unbalanced optimal transport remain limited.
method Developed two estimators for transport-growth pairs under different setups.
result Achieved minimax optimal rate for estimation of transport-growth pairs.
This paper aims at theoretically and empirically comparing two standard optimization criteria for Reinforcement Learning: i) maximization of the mean value and ii) minimization of the Bellman residual. For that purpose, we place ourselves in the framework of policy search algorithms, that are usually designed to maximi…
The paper explores how different loss functions impact reinforcement learning algorithms.
problem Improving reinforcement learning algorithms by optimizing loss functions.
method Comprehensive survey on loss functions in reinforcement learning, proving the benefits of specific loss functions.
result Binary cross-entropy loss leads to first-order bounds and is more efficient than squared loss.
A new multi-agent learning method improves performance in complex games.
problem Performance gap between MAPG and value-based multi-agent approaches.
method Introduces value function decomposition into multi-agent actor-critic framework for off-policy learning.
result DOP significantly outperforms state-of-the-art multi-agent reinforcement learning algorithms.
In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifte…
New method improves off-policy critic evaluation in reinforcement learning.
problem High variance and instability in off-policy policy evaluation.
method Doubly robust estimators applied to actor-critic algorithms.
result Doubly robust estimation significantly improves performance in continuous control tasks.
While biomanufacturing plays a significant role in supporting the economy and ensuring public health, it faces critical challenges, including complexity, high variability, lengthy lead time, and very limited process data, especially for personalized new cell and gene biotherapeutics. Driven by these challenges, we prop…
The goal of imitation learning is for an apprentice to learn how to behave in a stochastic environment by observing a mentor demonstrating the correct behavior. Accurate prior knowledge about the correct behavior can reduce the need for demonstrations from the mentor. We present a novel approach to encoding prior knowl…
New method improves stability of soft FQI for offline RL.
problem Stability issues in soft FQI under function approximation.
method Stationary reweighting to align operator norms.
result Local linear convergence proved under certain conditions.
RL teaches agents to make decisions through trial and error.
problem Sequential decision-making problems in dynamic environments.
method Value-based and policy search approaches, including actor-critic methods.
result RL can learn good policies from numeric reward feedback.
Analyzes premium data of Indian non-life insurers, finding GEV distribution best fits Lognormal and GEV extremes.
problem Modeling premiums of non-life insurance companies in India.
method Empirical analysis using Lognormal, GEV, and GPD distributions.
result Generalized Extreme Value distribution best fits premium data for ten Indian non-life insurers.
This paper analyzes errors in Shapley value-based model explanations.
problem Biased or unreliable explanations from existing SVA methods.
method Error theoretical analysis framework decomposing errors into observation and structural biases.
result Trade-off between observation and structural biases in SVA explanations.
This paper analyzes the sample complexity of two timescale reinforcement learning algorithms.
problem Analyzing the sample complexity of two timescale reinforcement learning algorithms.
method Non-asymptotic analysis of linear and nonlinear TDC and Greedy-GQ algorithms under Markovian sampling with constant stepsize.
result The paper provides non-asymptotic convergence results for two timescale linear and nonlinear TDC and Greedy-GQ algorithms.
This paper explores portfolio management strategies to maximize alpha and minimize beta.
problem Maximizing returns while minimizing risk in investment portfolios.
method Examines asset allocation, diversification, active management, and risk management strategies.
result Combining these strategies optimizes portfolio performance.
This paper shows using classification instead of regression improves deep RL scalability.
problem Challenges in training value functions for large networks in deep RL.
method Used categorical cross-entropy loss instead of mean squared error regression.
result Significant improvements in performance and scalability across various domains.
Tutorial on optimizing diffusion model samples for specific metrics.
problem Optimizing diffusion model samples for specific downstream metrics.
method Review and exploration of inference-time guidance and alignment methods.
result Unified perspective on inference-time algorithms and novel methods.
Obtaining more accurate equity value estimates is the starting point for stock selection, value-based indexing in a noisy market, and beating benchmark indices through tactical style rotation. Unfortunately, discounted cash flow, method of comparables, and fundamental analysis typically yield discrepant valuation estim…
Enhances data valuation by integrating global and local statistical properties.
problem Insufficient consideration of global and local statistical properties in data valuation methods.
method Proposes a method that fuses global and local statistical properties into regularization terms for Shapley value estimation and dynamic data valuation.
result Demonstrates improved performance and efficiency of data valuation methods through integration of global and local statistical properties.
Paper introduces a framework for managing cyber risk with insurance and cybersecurity models.
problem Pervasive challenges in managing cyber risk, especially for capital allocation.
method Combines insurance frequency-severity models with cybersecurity cascade models for comprehensive cyber risk assessment. Facilitates informed capital allocation through a two-pillar framework.
result Demonstrates the necessity of comprehensive cost-benefit analysis for budget-constrained companies.
Reincarnating RL reuses prior work to accelerate RL progress.
problem Efficiency and accessibility in reinforcement learning for large-scale applications.
method Transfer of learned policies between RL agents or design iterations, focusing on value-based RL.
result Demonstrated gains in performance over tabula rasa RL on various tasks.
Paper proposes using tree-based surrogate models for efficient Shapley computation.
problem Efficient computation of Shapley values using conditional expectations.
method Surrogate model-based tree for approximating Shapley and SHAP values.
result The proposed algorithm improves accuracy and unifies global interpretation.
In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct ε-optimal stopping tim…
Study finds Indian mutual funds adjust cash holdings based on inflows, impacting stock purchases.
problem Active liquidity management by mutual funds in India.
method Examined cash holdings and stock purchases of Indian equity mutual funds.
result Funds with active liquidity choices outperform, highlighting the importance of this strategy.
Framework for managing cyber risks in networks.
problem Managing systemic cyber risks in digital networks.
method Three components: acceptable configurations, risk mitigation interventions, and cost function.
result Effective decision-making for network resilience.
In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (…