An Atlas model is a rank-based system of continuous semimartingales for which the steady-state values of the processes follow a power law, or Pareto distribution. For a power law, the log-log plot of these steady-state values versus rank is a straight line. Zipf's law is a power law for which the slope of this line is …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “state-values”
On Query-efficient Planning in MDPs under Linear Realizability of the Optimal State-value Functioncs.LG
TensorPlan algorithm finds δ-optimal policies with poly queries under linearly realizable state-value function.
problem Efficient planning in MDPs with linearly realizable state-value function.
method TensorPlan algorithm using poly simulator queries.
result First algorithm with polynomial query complexity using only linear-realizability of a single competing value function.
Adaptive Synaptic Failure Enables Sampling from Posterior Predictive Distributions in the Brainq-bio.NC
Brain uses synaptic failure to sample from posterior distributions.
problem Bayesian inference in the brain's probabilistic computations.
method Adapting synaptic failure to sample posterior predictive distributions.
result Synaptic failure enables sampling of complete posterior predictive distributions.
This paper explores a new DRL algorithm that approximates both state-value and state-action functions.
problem Overestimation bias in state-action value function.
method Developed and analyzed the Deep Quality-Value (DQV) algorithm to approximate both and functions.
result DQV and DQV-Max algorithms perform better due to less overestimation bias in function.
Generative model handles varying data dimensions using jump diffusion processes.
problem Handling data of varying dimensionality in generative models.
method Formulated as a jump diffusion process, learning to approximate the process with a novel evidence lower bound.
result Effective sampling of data of varying dimensionality, better compatibility with test-time diffusion guidance imputation tasks.
New method improves deep policy gradient algorithms by learning relative state values.
problem High sample complexity and instability in policy gradient methods.
method Uses a new state-value function approximation based on residual variance.
result Empirical improvement across diverse continuous control tasks and algorithms.
Enhances reinforcement learning safety through risk-averse exploration.
problem Safety concerns in reinforcement learning due to sub-optimal actions.
method Distributionally robust policy iteration scheme with lower bound guarantees.
result Efficient algorithm that prevents poor decisions and converges to optimal policy.
A new estimator for state values in reinforcement learning reduces complexity and improves convergence.
problem Estimating state values in reinforcement learning with Markov reward processes.
method Loop estimator exploiting regenerative structure of Markov reward processes.
result Instance-dependent convergence rate of for estimating state values.
In this work we present a modification in the conventional flow of information through a LSTM network, which we consider well suited for RNNs in general. The modification leads to a iterative scheme where the computations performed by the LSTM cell are repeated over a constant input and cell state values, while updatin…
Compressed imitation learning uses simplicity priors for efficient expert behavior copying.
problem Efficiently learn expert behaviors with minimal data.
method Utilizes policy simplicity as a prior for sample-efficient imitation learning.
result Significantly higher scores achieved with limited expert demonstrations.
Improves BC policies by generating new plausible trajectories.
problem Sub-optimal data quality in BC leads to poor policy performance.
method Trajectory Stitching (TS) generates new plausible transitions.
result TS significantly improves behavioural policies over original data.
New RL method improves on standard discounted RL for operations research.
problem Applying RL to operations research problems, especially with non-zero rewards.
method Near-Blackwell-optimal RL algorithm that assesses average reward per step.
result Proves viability on challenging queuing system problems.
Enhances financial market valuation and trading algorithms using distributional value functions.
problem Accurate valuation and optimal trading decisions in financial markets.
method Combines predictive knowledge and deep reinforcement learning to introduce CDG-Model, a flexible framework for financial market valuation and trading.
result Improved market valuation and trading performance through enhanced feature creation and integration of real-world market factors.
The width of a curve in Euclidean space is the infimum of the distances between all pairs of parallel hyperplanes which bound , while its inradius is the supremum of the radii of all spheres which are contained in the convex hull of and are disjoint from . We use a mixture of topological and…
DAC combines actor-critic methods for learning options.
problem Learning hierarchical policies in reinforcement learning.
method Double Actor-Critic (DAC) architecture using two parallel augmented MDPs.
result DAC outperforms existing methods in robot simulation tasks.
Improves LSTM performance by initializing states via manifold learning.
problem Improving LSTM performance through better initialization.
method Learning an intrinsic data manifold to initialize LSTM internal states.
result Improved LSTM performance through consistent initialization.
Bayesian optimization improves Monte-Carlo tree search for better state value estimation.
problem Slow convergence in Monte-Carlo tree search due to averaging in backpropagation.
method Softmax MCTS and Monotone MCTS, using Bayesian optimization with Gaussian process prior.
result Our framework outperforms previous methods in computer Go.
Gradient learning optimises MCMC proposal distributions.
problem Intractable targets in MCMC sampling.
method Gradient-based optimisation of proposal distributions using a maximum entropy regularised objective function.
result Our method can outperform traditional MCMC algorithms, including Hamiltonian Monte Carlo.
RWR converges to global optimum in certain settings.
problem Proving convergence of RWR to optimal policy.
method Iterative learning with return-weighted log-likelihood.
result RWR converges to global optimum under certain conditions.
We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy pro…
MoNODEs improve neural ODEs by separating dynamic states from static factors.
problem Learning non-linear dynamics with variations across trajectories.
method Introduces time-invariant modulator variables to separate dynamic states from static factors.
result Consistently improves model generalization and far-horizon forecasting.
MuZero visualizes its internal representations to stabilize planning.
problem Stability issues in MuZero's planning process.
method Visualized MuZero's latent representations and proposed regularization techniques.
result Action trajectories diverge between observation embeddings and internal state transitions, leading to instability.
ReF-ER algorithm improved performance in multi-agent reinforcement learning.
problem Improving performance in multi-agent reinforcement learning environments.
method Extended ReF-ER algorithm to include dependencies between agents and modeled environment dynamics.
result ReF-ER MARL outperforms state-of-the-art algorithms in collaborative environments.
Paper proposes using expectation models for planning in stochastic environments.
problem Intractability of learning distribution and sample models in large state and action spaces.
method Proposes using approximate expectation models for MBRL, analyzes linear and non-linear parametrizations, and presents a policy evaluation algorithm.
result Planning with an expectation model is equivalent to planning with a distribution model under certain conditions.
USFs capture dynamics for faster RL task transfer.
problem Applying knowledge from one task to another.
method Proposed Universal Successor Features (USFs) for RL.
result USFs accelerate training and transfer knowledge.
Estimates hybrid dynamical systems with polynomial expansions and Markovian switching.
problem Identifying hybrid dynamical systems with nonlinear autoregressive exogenous (NARX) components and Markovian switching.
method Probabilistic framework using Expectation Maximization for parameter estimation, including submodel coefficients, hidden state values, and transition probabilities. Disentangles mode classification and NARX regression tasks. Uses soft-labels and coordinate descent approach for parameter fitting.
result Demonstrated on a SMNARX problem with three nonlinear sub-models, achieving parsimonious models through l1-norm bridge estimation and hard-thresholding.
Algorithm finds latent structure in value functions for improved reinforcement learning.
problem Finding latent structure in value functions for efficient reinforcement learning.
method Proposes a practical algorithm using two posterior distributions over state abstractions and abstract-state values.
result Substantial performance gains in multi-task settings where tasks share a common, low-dimensional representation.
Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict. Algorithms and theory that provide strong problem-dependent bounds could help illuminate the key features of what makes a RL problem hard and reduc…
This paper presents a new meta-modeling framework to employ deep reinforcement learning (DRL) to generate mechanical constitutive models for interfaces. The constitutive models are conceptualized as information flow in directed graphs. The process of writing constitutive models are simplified as a sequence of forming g…
Current imitation learning techniques are too restrictive because they require the agent and expert to share the same action space. However, oftentimes agents that act differently from the expert can solve the task just as good. For example, a person lifting a box can be imitated by a ceiling mounted robot or a desktop…
Unified framework for solving MDPs with stochastic mirror descent.
problem Approximately solving infinite-horizon Markov decision processes (MDPs).
method Primal-dual stochastic mirror descent for MDPs with a unified framework.
result Computes ε-optimal policies with expected samples for both average-reward and discounted MDPs.
Improves RL from historical data by stitching trajectories.
problem Lack of high-quality data for offline RL.
method Trajectory Stitching (TS) to augment historical data with synthetic actions.
result Improves RL policy performance over baseline.
Unified q-learning for mean-field jump-diffusion models with unobservable population distribution.
problem Continuous-time q-learning in mean-field jump-diffusion models with unobservable population distribution.
method Proposed decoupled Iq-function for unified policy evaluation in MFG and MFC problems; unified q-learning algorithm based on test policies and averaged martingale orthogonality condition.
result Unified policy evaluation rule for MFG and MFC problems based on decoupled Iq-function.
New method finds unseen states for RL, improving performance.
problem Offline RL struggles with unseen states and actions.
method Value-informed state perturbations and filtering.
result Improved performance in offline RL tasks.
Algorithm identifies best policy in MDPs with adaptive sampling.
problem Best policy identification in discounted MDPs with limited samples.
method Derive lower and upper bounds on sample complexity, design KLB-TS algorithm.
result KLB-TS algorithm achieves nearly-optimal sample allocation.
DNAF accelerates DQL for efficient resource allocation in network slicing.
problem Efficient resource allocation in network slicing with varying demands.
method Introduced discrete normalized advantage functions (DNAF) into DQL, using a k-nearest neighbor algorithm for discrete action space.
result DNAF-based DQL converges faster through simulations.
A new algorithm finds optimal solutions for constrained decision processes.
problem Optimizing state-value functions with constraints in CMDPs.
method Gradient-Aware Search (GAS) exploiting PWLC structure.
result GAS converges faster and more reliably than existing methods.
The complex, time-dependent statistical structures observed in the Dow Jones Industrial Average on a typical trading day are modeled with Lorentzian functions. The resonant-like structures are characterized by the values of the basic ratio: the average lifetime of the individual states associated with a given structura…
Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL. On-line planning with look-ahead trees has proven successful in environments where transition models are known a priori. However, in complex environments where transition models need t…
Transformers learn to predict chess moves with surprising accuracy and strength.
problem Training transformers on chess to predict moves accurately.
method Large-scale chess dataset (10M games), supervised learning with up to 270M parameters.
result Transformers can predict action-values for novel boards with high accuracy.
New RL approach uses resets to learn complex MDPs efficiently.
problem Learning complex MDPs with high-dimensional states and function approximation.
method Local simulator access and recursive value function search.
result Proven sample-efficient learning for MDPs with low coverability.
New method learns state embeddings from demonstrations for improved reinforcement learning.
problem Difficult relationship between observed state and useful policy actions in dynamic problems.
method Variational framework for learning state embeddings that optimize trajectory linearity.
result Learning embedding spaces improves policy gradient reinforcement learning performance.
A new framework enhances IDW models for complex industrial datasets.
problem Low performance of IDW models in complex industrial datasets.
method Deep reinforcement learning network to enhance IDW models and learn hyperparameters.
result The proposed framework achieves differential spatial prediction and is more accurate than current IDW models.
GAN-powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicingcs.LG
Paper proposes GAN-DDQN for efficient resource allocation in network slicing.
problem Efficient resource allocation in network slicing with varying service demands.
method Leverage deep reinforcement learning with GAN-DDQN to minimize SSR and SE discrepancies.
result Proposed GAN-DDQN and Dueling GAN-DDQN algorithms improve resource allocation efficiency.
SDQL uses modular deep Q networks to efficiently learn multi-stage optimal control tasks.
problem Training complex deep reinforcement learning models for multi-stage control tasks is inefficient and unstable.
method Stacked Deep Q Learning (SDQL) with modular Q networks and backward training.
result SDQL efficiently learns optimal control policies for multi-stage tasks with high-dimensional state and action spaces.
Paper tackles stochastic control with mean and higher-order moments, finding Nash equilibria.
problem Time-inconsistent stochastic control problems with mean and higher-order moments.
method Developed closed-loop and open-loop Nash equilibrium controls using PDEs and maximum principles.
result Identical closed-loop and open-loop Nash equilibria controls, independent of state value and random path.
Unified analysis of TD learning using MJLS theory for linear function approximators.
problem Characterizing the exact behaviors of TD learning algorithms with linear function approximators.
method Exploiting connections to Markov jump linear systems (MJLS) theory to analyze TD learning algorithms.
result Closed-form expressions for mean and covariance matrix of TD estimation error at any time step.
Activities in reinforcement learning (RL) revolve around learning the Markov decision process (MDP) model, in particular, the following parameters: state values, V; state-action values, Q; and policy, pi. These parameters are commonly implemented as an array. Scaling up the problem means scaling up the size of the arra…