Based on Colombeau's theory of algebras of generalized functions we introduce the concepts of generalized functions taking values in differentiable manifolds as well as of generalized vector bundle homomorphisms. We study their basic properties, in particular with respect to some new point value concepts for generalize…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Generalized Value Functions”
Paper develops efficient RL algorithm for general value function approximation.
problem Lack of theory for RL with general value function approximation.
method Provable efficient RL algorithm using bounded eluder dimension.
result Achieves a regret bound of .
Study uses randomized value functions to enhance exploration in reinforcement learning.
problem Improving exploration in reinforcement learning algorithms.
method Injecting random noise into value functions for efficient exploration.
result Provably efficient exploration achieved through worst-case regret bounds.
Extends Tanimoto kernel to real-valued functions.
problem Measuring similarity between real-valued functions.
method Unified representation of real-valued functions via sets, derived general form of the kernel, explicit feature representation, and smooth approximation.
result General Tanimoto kernel for real-valued functions.
Improved RL value function approximation using graph-based feature learning.
problem Accurate value function approximation in high-dimensional state or action spaces.
method Representation policy iteration (RPI) with graph-based feature learning algorithms.
result Node2vec and Variational Graph Auto-Encoder outperform PVFs in low-dimensional feature space.
Develops hierarchical reinforcement learning value function approximators.
problem Estimating long-term returns in reinforcement learning with multiple goals.
method Introduces hierarchical universal value function approximators (H-UVFAs) using the options framework.
result Demonstrates generalization and improved performance of H-UVFAs over UVFAs.
The paper improves energy decay estimates for Dir-stationary Q-valued functions and applies them to Liouville-type theorems and continuity.
problem Improving energy decay estimates for Dir-stationary Q-valued functions.
method Establishing improved decay estimates and applying them to derive Liouville-type theorems and continuity.
result Dir-stationary Q-valued functions exhibit the Lebesgue property and reside in a generalized Campanato-Morrey space.
We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly ineffic…
We use noncommutative localization to construct a chain complex which counts the critical points of a circle-valued Morse function on a manifold, generalizing the Novikov complex. As a consequence we obtain new topological lower bounds on the minimum number of critical points of a circle-valued Morse function within a …
Study optimality conditions for interval-valued optimization problems on Riemannian manifolds.
problem Optimizing interval-valued functions on Riemannian manifolds under a total order relation.
method Generalized Hukuhara directional differentiability to derive KKT-type optimality conditions.
result Derives optimality conditions for interval-valued optimization problems on Riemannian manifolds.
We characterize the geometric shape of value functions in reinforcement learning.
problem Understanding the structure of value functions in reinforcement learning.
method Geometric and topological analysis of value function space.
result Value functions form a polytope, with specific properties like line segments.
PBVFs generalize across policies using learned value functions.
problem RL algorithms forget information about old policies when updating value functions to track the learned policy.
method Introduce Parameter-Based Value Functions (PBVFs) that include policy parameters in their inputs, enabling them to generalize across different policies.
result PBVFs enable zero-shot learning of new policies that outperform any policy seen during training.
Sparse Attention Guided Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement Learningcs.LG
A new method for estimating joint value functions in multi-scene reinforcement learning.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Sparse attention mechanism over multiple value function hypotheses to approximate the true joint value function.
result Significant improvements in reward scores and enhanced navigation efficiency across OpenAI ProcGen environments.
Constructs unique bases for CY varieties over valued fields.
problem Finding unique bases for CY varieties over valued fields.
method Uses techniques from higher rank degenerations in K-stability.
result Induces canonical functions on skeletons and agrees with tropicalizations of theta functions.
New integer-valued functions for Legendrian knots.
problem Understanding Legendrian knots better.
method Using Legendrian fronts to derive integer-valued linear functions.
result Introduced new invariants similar to Arnold's basic invariant.
The Lax-Hopf formula simplifies the value function of an intertemporal optimization (infinite dimensional) problem associated with a convex transaction-cost function which depends only on the transactions (velocities) of a commodity evolution: it states that the value function is equal to the marginal fonction of a fin…
New hyperbolic 4-manifolds found with special functions.
problem Finding hyperbolic 4-manifolds with specific circle-valued Morse functions.
method Constructing hyperbolic 4-manifolds with only index 2 critical points.
result Existence of infinitely many hyperbolic 4-manifolds with bounded Betti numbers.
Paper discusses directional differentiability of interval-valued functions on Riemannian manifolds.
problem Equivalence of directional differentiability of interval-valued functions and their components.
method Analyzes directional differentiability of interval-valued functions on Riemannian manifolds.
result Directional differentiability of interval-valued functions is not equivalent to the directional differentiability of their components.
Algebraic function based Banach space valued ordinary and fractional neural network approximationsstat.ML
Researchers develop neural networks for approximating functions in Banach spaces.
problem Approximating Banach space valued continuous functions.
method Quasi-interpolation Banach space valued neural network operators using algebraic sigmoid functions.
result Jackson type inequalities for function approximation.
The paper provides high-confidence error estimates for learned value functions in large state-spaces.
problem Estimating the accuracy of learned value functions in large, continuous state-spaces.
method Developed a high-confidence bound on empirical value error to true value error, and an offline sampling algorithm to repeatedly compute value error estimates.
result Demonstrated that the offline sampling algorithm can provide high-confidence estimates of value error for learned value functions.
On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learningcs.LG
Paper proposes a new adaptive multiscale value function approximation for reinforcement learning.
problem Value function approximation in reinforcement learning with varying complexity.
method Adaptive multiscale approximation using multiresolution analysis and tree approximation.
result Convergence rate of the multiscale approximation is independent of basis function regularity.
We analyze a notion of multiple valued sections of a vector bundle over an abstract smooth Riemannian manifold, which was suggested by W. Allard in the unpublished note "Some useful techniques for dealing with multiple valued functions" and generalizes Almgren's -valued functions. We study some relevant properties o…
We discuss the nature of structure-preserving maps of varies function algebras. In particular, we identify isomorphisms between special Colombeau algebras on manifolds with invertible manifold-valued generalized functions in the case of smooth parametrization. As a consequence, and to underline the consistency and vali…
New method for off-policy evaluation in POMDPs using future-dependent value functions.
problem Curse of horizon in off-policy evaluation for POMDPs.
method Develops future-dependent value functions and minimax learning method.
result PAC result and Bellman completeness for the proposed OPE estimator.
The paper extends portfolio theory to include contingent claim functions for option pricing.
problem Developing a method to price options using portfolio generating functions.
method Extending portfolio theory to include contingent claim functions and applying partial differential equations.
result A method to price options using portfolio generating functions and replicable contingent claim functions.
We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies wit…
We analyze the Gambler's problem, finding a complex, fractal value function.
problem Simple reinforcement learning problem where a gambler tries to reach a target.
method Exact formula derivation for discrete and continuous cases.
result The optimal value function is fractal, self-similar, and has high-frequency components.
New method calculates Shapley values for uncertain functions.
problem Uncertain value functions in explainable machine learning.
method Definition of Shapley values using probability theory.
result Shapley values can be applied to uncertain functions.
Study expands classical harmonic function results to Riemannian manifolds.
problem Classical harmonic function properties in domains of Riemannian manifolds.
method Generalized classical results to Riemannian manifolds, including pinched negative curvature.
result Generalized results for Riemannian manifolds, including pinched negative curvature.
Paper tackles robust control of SDEs with ambiguity, proving value function existence and applying to investment problems.
problem Robust control of SDEs with ambiguity parameters and non-Lipschitz coefficients.
method Existence and uniqueness of value function established through BSDEs with non-linear growth conditions.
result Existence and uniqueness of value function in proper space, verified through BSDEs.
We present a fully nonparametric method to estimate the value function, via simulation, in the context of expected infinite-horizon discounted rewards for Markov chains. Estimating such value functions plays an important role in approximate dynamic programming and applied probability in general. We incorporate "soft in…
The paper develops bounds for predictive values in binary classification.
problem Lack of confidence intervals for positive and negative predictive values.
method Bi-criterion framework and distribution-free large deviation and uniform convergence bounds.
result New bounds for predictive values without relying on concentration inequalities.
A new framework for generative modeling using value-driven transport.
problem Developing efficient methods for generative modeling.
method A discrete-time stochastic control formulation of measure transport, formulated as a linear program with dual variables corresponding to the optimal value function.
result Well-trained VDT policies lead to straight transport paths that can be simulated quickly and robustly.
UA-LQE improves value function learning by selectively erasing uncertain entries in Q-matrix.
problem Improving value function learning in complex reinforcement learning tasks.
method Uncertainty-aware low-rank Q-matrix estimation (UA-LQE) algorithm.
result UA-LQE selectively erases uncertain entries in Q-matrix to improve value function approximation.
Study introduces indecomposability for varifolds, leading to geometric consequences.
problem Understanding the structure of varifolds and their connectedness properties.
method Introducing indecomposability and related concepts for varifolds.
result Substantial geometric consequences derived from the connectedness properties of varifolds.
Improves reinforcement learning agent's scene-specific value function.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Proposes dynamic value estimation (DVE) for multiple MDPs, clustering value functions across scenes.
result Lower sample variance and more accurate scene-specific value function estimates.
Paper introduces a new value function for state transitions and optimal policy learning.
problem Learning optimal policies from state transitions and actions.
method Develops a forward dynamics model to maximize a novel value function .
result Demonstrates benefits in value function transfer, redundant action spaces, and off-policy learning.
A theorem for Hilbert space-valued functions simplifies machine learning models.
problem Learning functions in Hilbert space-valued spaces.
method Generalized Representer Theorem for Hilbert space-valued functions.
result Unified view of supervised and semi-supervised learning methods.
Time-variant value function transfer method for RL.
problem Transfer learning with time-variant task distributions.
method Variational approach leveraging temporal structure.
result The proposed method outperforms time-invariant approach in experiments.
Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learningcs.LG
The paper explores when and why value decomposition algorithms work in cooperative multi-agent reinforcement learning.
problem The applicability and convergence properties of value decomposition algorithms in cooperative multi-agent reinforcement learning are unclear.
method The paper introduces decomposable games and proves that applying the multi-agent fitted Q-Iteration algorithm leads to an optimal Q-function in these games.
result The paper offers theoretical insights into when and why value decomposition algorithms converge in cooperative multi-agent reinforcement learning.
New -functions for 3-manifolds connect to Witten invariants and relate to generalized Bernoulli polynomials.
problem Understanding -functions for 3-manifolds and their invariants.
method Using Mellin transforms and asymptotic techniques, proving entire functions and their values.
result Linear relations between -function values at negative integers, generalizing known zeta functions.
A novel approach uses an ensemble of Gaussian processes for robust and adaptive reinforcement learning.
problem Adaptive reinforcement learning in large or continuous state spaces.
method Online scalable (OS) approach with a weighted ensemble of Gaussian processes.
result The ensemble approach improves performance in adversarial settings.
We approximate derivatives of functions on manifolds by embedding them and applying vector-valued operators.
problem Derivatives of manifold-valued functions are harder to approximate than vector-valued functions.
method Embed the manifold into a higher space, approximate the derivative of the vector-valued function, and project back.
result We provide error bounds for the approximation of manifold-valued function derivatives.
Real valued homomorphisms on the algebra of smooth functions on a differential space are described. The concept of generators of this algebra is emphasized in this description.
The paper proposes methods to find a shared active subspace for multivariate vector-valued functions.
problem Minimizing the deviation between function evaluations in the original and reconstructed spaces.
method Manipulating gradients or SPD matrices to identify a shared structure.
result Summing SPD matrices often identifies the best shared active subspace.
Efficiently plans large MDPs with weak function approximations.
problem Planning in large MDPs with limited function approximation capabilities.
method Uses linear value function approximation with weak requirements and a generative oracle.
result Produces almost-optimal actions for any state with polynomial computation time.
New RL method learns value function for many policies using few key states.
problem Evaluate and improve policies in continuous control problems.
method Combines actor-critic architecture and policy embedding to learn a single value function for many policies.
result Value function minimizes prediction error by learning a small set of 'probing states' and their impact on policies' returns.
Enhances neural networks with prior function values to improve accuracy.
problem Improving neural network accuracy in regions without training data.
method Develops a probabilistic approach to augment BNNs with prior function values.
result Predictions rely more on prior information in uncertain regions.