Based on Colombeau's theory of algebras of generalized functions we introduce the concepts of generalized functions taking values in differentiable manifolds as well as of generalized vector bundle homomorphisms. We study their basic properties, in particular with respect to some new point value concepts for generalize…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,786 papers · 148 categories
Trend · papers per month
48 results for “General Value Function”
Paper develops efficient RL algorithm for general value function approximation.
problem Lack of theory for RL with general value function approximation.
method Provable efficient RL algorithm using bounded eluder dimension.
result Achieves a regret bound of .
Extends Tanimoto kernel to real-valued functions.
problem Measuring similarity between real-valued functions.
method Unified representation of real-valued functions via sets, derived general form of the kernel, explicit feature representation, and smooth approximation.
result General Tanimoto kernel for real-valued functions.
This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. B…
Develops hierarchical reinforcement learning value function approximators.
problem Estimating long-term returns in reinforcement learning with multiple goals.
method Introduces hierarchical universal value function approximators (H-UVFAs) using the options framework.
result Demonstrates generalization and improved performance of H-UVFAs over UVFAs.
In this work, we study value function approximation in reinforcement learning (RL) problems with high dimensional state or action spaces via a generalized version of representation policy iteration (RPI). We consider the limitations of proto-value functions (PVFs) at accurately approximating the value function in low d…
The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.
problem Improving energy decay estimates for Dir-stationary Q-valued functions.
method Establishing improved decay estimates and applying them to derive Liouville-type theorems and continuity.
result Dir-stationary Q-valued functions exhibit the Lebesgue property and reside in a generalized Campanato-Morrey space.
We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly ineffic…
We use noncommutative localization to construct a chain complex which counts the critical points of a circle-valued Morse function on a manifold, generalizing the Novikov complex. As a consequence we obtain new topological lower bounds on the minimum number of critical points of a circle-valued Morse function within a …
Study optimality conditions for interval-valued optimization problems on Riemannian manifolds.
problem Optimizing interval-valued functions on Riemannian manifolds under a total order relation.
method Generalized Hukuhara directional differentiability to derive KKT-type optimality conditions.
result Derives optimality conditions for interval-valued optimization problems on Riemannian manifolds.
PBVFs generalize across policies using learned value functions.
problem RL algorithms forget information about old policies when updating value functions to track the learned policy.
method Introduce Parameter-Based Value Functions (PBVFs) that include policy parameters in their inputs, enabling them to generalize across different policies.
result PBVFs enable zero-shot learning of new policies that outperform any policy seen during training.
Sparse Attention Guided Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement Learningcs.LG
A new method for estimating joint value functions in multi-scene reinforcement learning.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Sparse attention mechanism over multiple value function hypotheses to approximate the true joint value function.
result Significant improvements in reward scores and enhanced navigation efficiency across OpenAI ProcGen environments.
Constructs unique bases for CY varieties over valued fields.
problem Finding unique bases for CY varieties over valued fields.
method Uses techniques from higher rank degenerations in K-stability.
result Induces canonical functions on skeletons and agrees with tropicalizations of theta functions.
New integer-valued functions for Legendrian knots.
problem Understanding Legendrian knots better.
method Using Legendrian fronts to derive integer-valued linear functions.
result Introduced new invariants similar to Arnold's basic invariant.
The Lax-Hopf formula simplifies the value function of an intertemporal optimization (infinite dimensional) problem associated with a convex transaction-cost function which depends only on the transactions (velocities) of a commodity evolution: it states that the value function is equal to the marginal fonction of a fin…
New hyperbolic 4-manifolds found with special functions.
problem Finding hyperbolic 4-manifolds with specific circle-valued Morse functions.
method Constructing hyperbolic 4-manifolds with only index 2 critical points.
result Existence of infinitely many hyperbolic 4-manifolds with bounded Betti numbers.
Paper discusses directional differentiability of interval-valued functions on Riemannian manifolds.
problem Equivalence of directional differentiability of interval-valued functions and their components.
method Analyzes directional differentiability of interval-valued functions on Riemannian manifolds.
result Directional differentiability of interval-valued functions is not equivalent to the directional differentiability of their components.
Algebraic function based Banach space valued ordinary and fractional neural network approximationsstat.ML
Researchers develop neural networks for approximating functions in Banach spaces.
problem Approximating Banach space valued continuous functions.
method Quasi-interpolation Banach space valued neural network operators using algebraic sigmoid functions.
result Jackson type inequalities for function approximation.
We analyze a notion of multiple valued sections of a vector bundle over an abstract smooth Riemannian manifold, which was suggested by W. Allard in the unpublished note "Some useful techniques for dealing with multiple valued functions" and generalizes Almgren's -valued functions. We study some relevant properties o…
We discuss the nature of structure-preserving maps of varies function algebras. In particular, we identify isomorphisms between special Colombeau algebras on manifolds with invertible manifold-valued generalized functions in the case of smooth parametrization. As a consequence, and to underline the consistency and vali…
New method for off-policy evaluation in POMDPs using future-dependent value functions.
problem Curse of horizon in off-policy evaluation for POMDPs.
method Develops future-dependent value functions and minimax learning method.
result PAC result and Bellman completeness for the proposed OPE estimator.
The paper extends portfolio theory to include contingent claim functions for option pricing.
problem Developing a method to price options using portfolio generating functions.
method Extending portfolio theory to include contingent claim functions and applying partial differential equations.
result A method to price options using portfolio generating functions and replicable contingent claim functions.
We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies wit…
Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and handle variability, particularly for off-policy learning. To understand the prop…
We establish geometric and topological properties of the space of value functions in finite state-action Markov decision processes. Our main contribution is the characterization of the nature of its shape: a general polytope (Aigner et al., 2010). To demonstrate this result, we exhibit several properties of the structu…
New method calculates Shapley values for uncertain functions.
problem Uncertain value functions in explainable machine learning.
method Definition of Shapley values using probability theory.
result Shapley values can be applied to uncertain functions.
We analyze the Gambler's problem, finding a complex, fractal value function.
problem Simple reinforcement learning problem where a gambler tries to reach a target.
method Exact formula derivation for discrete and continuous cases.
result The optimal value function is fractal, self-similar, and has high-frequency components.
Study expands classical harmonic function results to Riemannian manifolds.
problem Classical harmonic function properties in domains of Riemannian manifolds.
method Generalized classical results to Riemannian manifolds, including pinched negative curvature.
result Generalized results for Riemannian manifolds, including pinched negative curvature.
We present a fully nonparametric method to estimate the value function, via simulation, in the context of expected infinite-horizon discounted rewards for Markov chains. Estimating such value functions plays an important role in approximate dynamic programming and applied probability in general. We incorporate "soft in…
Paper tackles robust control of SDEs with ambiguity, proving value function existence and applying to investment problems.
problem Robust control of SDEs with ambiguity parameters and non-Lipschitz coefficients.
method Existence and uniqueness of value function established through BSDEs with non-linear growth conditions.
result Existence and uniqueness of value function in proper space, verified through BSDEs.
The paper develops bounds for predictive values in binary classification.
problem Lack of confidence intervals for positive and negative predictive values.
method Bi-criterion framework and distribution-free large deviation and uniform convergence bounds.
result New bounds for predictive values without relying on concentration inequalities.
A new framework for generative modeling using value-driven transport.
problem Developing efficient methods for generative modeling.
method A discrete-time stochastic control formulation of measure transport, formulated as a linear program with dual variables corresponding to the optimal value function.
result Well-trained VDT policies lead to straight transport paths that can be simulated quickly and robustly.
UA-LQE improves value function learning by selectively erasing uncertain entries in Q-matrix.
problem Improving value function learning in complex reinforcement learning tasks.
method Uncertainty-aware low-rank Q-matrix estimation (UA-LQE) algorithm.
result UA-LQE selectively erases uncertain entries in Q-matrix to improve value function approximation.
Study introduces indecomposability for varifolds, leading to geometric consequences.
problem Understanding the structure of varifolds and their connectedness properties.
method Introducing indecomposability and related concepts for varifolds.
result Substantial geometric consequences derived from the connectedness properties of varifolds.
Improves reinforcement learning agent's scene-specific value function.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Proposes dynamic value estimation (DVE) for multiple MDPs, clustering value functions across scenes.
result Lower sample variance and more accurate scene-specific value function estimates.
Paper introduces a new value function for state transitions and optimal policy learning.
problem Learning optimal policies from state transitions and actions.
method Develops a forward dynamics model to maximize a novel value function .
result Demonstrates benefits in value function transfer, redundant action spaces, and off-policy learning.
Time-variant value function transfer method for RL.
problem Transfer learning with time-variant task distributions.
method Variational approach leveraging temporal structure.
result The proposed method outperforms time-invariant approach in experiments.
Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learningcs.LG
The paper explores when and why value decomposition algorithms work in cooperative multi-agent reinforcement learning.
problem The applicability and convergence properties of value decomposition algorithms in cooperative multi-agent reinforcement learning are unclear.
method The paper introduces decomposable games and proves that applying the multi-agent fitted Q-Iteration algorithm leads to an optimal Q-function in these games.
result The paper offers theoretical insights into when and why value decomposition algorithms converge in cooperative multi-agent reinforcement learning.
New -functions for 3-manifolds connect to Witten invariants and relate to generalized Bernoulli polynomials.
problem Understanding -functions for 3-manifolds and their invariants.
method Using Mellin transforms and asymptotic techniques, proving entire functions and their values.
result Linear relations between -function values at negative integers, generalizing known zeta functions.
We approximate derivatives of functions on manifolds by embedding them and applying vector-valued operators.
problem Derivatives of manifold-valued functions are harder to approximate than vector-valued functions.
method Embed the manifold into a higher space, approximate the derivative of the vector-valued function, and project back.
result We provide error bounds for the approximation of manifold-valued function derivatives.
A novel approach uses an ensemble of Gaussian processes for robust and adaptive reinforcement learning.
problem Adaptive reinforcement learning in large or continuous state spaces.
method Online scalable (OS) approach with a weighted ensemble of Gaussian processes.
result The ensemble approach improves performance in adversarial settings.
Real valued homomorphisms on the algebra of smooth functions on a differential space are described. The concept of generators of this algebra is emphasized in this description.
The paper proposes methods to find a shared active subspace for multivariate vector-valued functions.
problem Minimizing the deviation between function evaluations in the original and reconstructed spaces.
method Manipulating gradients or SPD matrices to identify a shared structure.
result Summing SPD matrices often identifies the best shared active subspace.
Efficiently plans large MDPs with weak function approximations.
problem Planning in large MDPs with limited function approximation capabilities.
method Uses linear value function approximation with weak requirements and a generative oracle.
result Produces almost-optimal actions for any state with polynomial computation time.
New RL method learns value function for many policies using few key states.
problem Evaluate and improve policies in continuous control problems.
method Combines actor-critic architecture and policy embedding to learn a single value function for many policies.
result Value function minimizes prediction error by learning a small set of 'probing states' and their impact on policies' returns.
Enhances neural networks with prior function values to improve accuracy.
problem Improving neural network accuracy in regions without training data.
method Develops a probabilistic approach to augment BNNs with prior function values.
result Predictions rely more on prior information in uncertain regions.
Dyna is an architecture for model-based reinforcement learning (RL), where simulated experience from a model is used to update policies or value functions. A key component of Dyna is search-control, the mechanism to generate the state and action from which the agent queries the model, which remains largely unexplored. …
Paper analyzes distributional reinforcement learning with value function approximation, introducing Bellman unbiasedness and a new algorithm.
problem Improving reinforcement learning by capturing environmental stochasticity and addressing infinite dimensionality.
method Introduces Bellman unbiasedness and proposes SF-LSVI algorithm for provably efficient distributional reinforcement learning.
result Achieves a tight regret bound of O(d_E H^3/2 √K) for distributional reinforcement learning.