New method stabilizes FQE by reweighting Bellman targets.
problem Stability guarantees for FQE often rely on Bellman completeness, which can fail with function approximation.
method Proposes stationary-weighted FQE, reweighting Bellman targets by stationary target-to-behavior density ratio.
result Proves finite-sample linear convergence to stationary projected Bellman fixed point without Bellman completeness.
New method improves stability of soft FQI for offline RL.
problem Stability issues in soft FQI under function approximation.
method Stationary reweighting to align operator norms.
result Local linear convergence proved under certain conditions.
Core-Halo solves large-scale fixed-point problems by decentralizing updates.
problem Large-scale fixed-point equations with block dependencies.
method Core-Halo decomposition separates write ownership from read-only context, aligning with block-dependence structure.
result Core-Halo achieves near-centralized performance while retaining parallelism.
The Bellman error is a poor proxy for value function accuracy, even with all state-action pairs.
problem The Bellman error is a poor proxy for the accuracy of the value function.
method Study of the Bellman equation as a surrogate objective for value prediction accuracy.
result The magnitude of the Bellman error is only weakly related to the distance to the true value function, even with all state-action pairs.
A new method calibrates value predictions in offline RL to improve reliability.
problem Difficulty in long-horizon value prediction in offline reinforcement learning.
method Bellman calibration, a weak reliability criterion, and Iterated Bellman Calibration.
result Finite-sample guarantees show that Bellman calibration error is controlled at nonparametric rates.
Algorithm converts plat to standard closure of braids in 3D and related spaces.
problem Converting plat to standard closure of braids in different spaces.
method Algorithmic approach for plat to standard closure conversion in \(\mathbb{R}^3\), handlebodies, and thickened surfaces.
result Algorithm is quadratic for plat to standard closure and linear for standard to plat closure.
Improved risk-sensitive RL with exponential Bellman equation and better regret bounds.
problem Exponential gap between upper and lower bounds in risk-sensitive RL.
method Identified and addressed deficiencies in existing algorithms and analysis; developed novel analysis and exploration mechanism.
result Improved regret upper bounds over existing ones.
New Bellman error estimator improves offline model selection performance.
problem Selecting the best policy from logged data using mean squared Bellman error.
method Developed a more accurate estimator of MSBE and analyzed conditions for successful OMS.
result New estimator achieves impressive offline model selection performance on diverse tasks.
The paper studies knitted surfaces and surface-links, showing their isotopy and closure properties.
problem Understanding the isotopy and closure properties of knitted surfaces and surface-links.
method Analyzing the structure and closure of knitted surfaces and surface-links in R4. result Any surface-link is ambient isotopic to the closure of a 2-dimensional knit.
The paper explores solutions to the distributional Bellman equation in reinforcement learning.
problem Distributional reinforcement learning considers complete return distributions, not just expected returns.
method Study existence and uniqueness of solutions to general distributional Bellman equations, linking them to multivariate affine equations.
result Any solution to a distributional Bellman equation can be derived from a multivariate affine distributional equation.
Study on knot classification using 3-braid closures and ribbon surfaces.
problem Classifying smoothly slice knots from 3-braid closures.
method Construct ribbon surfaces and use twisted Alexander polynomial.
result Classification of knots up to 20 crossings.
We obtain the classical Hanner inequalities by the Bellman function method. These inequalities give sharp estimates for the moduli of convexity of Lebesgue spaces. Easy ideas from differential geometry help us to find the Bellman function using neither "magic guesses" nor calculations.
New proof classifies orbit closures in Hodge bundle.
problem Classifying mGL+(2,R)-orbit closures in Hodge bundle. method Using deformations of flat pairs of pants.
result Short proof of absolute period foliation classification.
We study a regular closure operator in the category of quandles. We show that the regular closure operator and the pullback closure operator corresponding to the reflector from the category of quandles to its full subcategory of trivial quandles coincide, we give a simple description of this closure operator, and analy…
Paper studies offline RL with linear approx, focusing on inherent Bellman error.
problem Offline RL with linear approx, focusing on inherent Bellman error.
method Algorithm that succeeds under single-policy coverage condition, leveraging inherent Bellman error.
result Algorithm yields first known guarantee under single-policy coverage, even for linear Bellman completeness.
The study connects lamination and orbit closures in hyperbolic manifolds.
problem Understanding the geometric and dynamical properties of horocycle orbit closures in Z-covers of compact hyperbolic manifolds. method Exposes connections between distance minimizing laminations and horospherical orbit closures in Z-covers of compact hyperbolic manifolds. Provides novel constructions and explicit descriptions. result Even slight perturbations to hyperbolic metrics can drastically change horocycle orbit closures.
Study orbits in right triangles, deducing periodic billiard paths and classifying orbit closures.
problem Understanding periodic billiard paths in right triangles and orbit closures in strata of Abelian and quadratic differentials.
method Classifying orbit closures of rank at least two in hyperelliptic components of strata of Abelian and quadratic differentials.
result Computed orbit closures and deduced asymptotic number of periodic billiard trajectories in right triangles.
Richard Bellman's Principle of Optimality, formulated in 1957, is the heart of dynamic programming, the mathematical discipline which studies the optimal solution of multi-period decision problems. In this paper, we look at the main trading principles of Jesse Livermore, the legendary stock operator whose method was pu…
Proves closure for specific spacetimes with certain conditions.
problem Proving closure for globally hyperbolic spacetimes.
method Using a Bonnet-Myers type result.
result Proves closure for spacetimes with specific conditions.
Polynomial-time RL algorithm for constant actions under linear Bellman completeness.
problem Efficient online reinforcement learning with few actions.
method Polynomial-time algorithm based on linear function approximation.
result First computationally efficient algorithm for RL with constant actions under linear Bellman completeness.
The impact of softmax on the value function itself in reinforcement learning (RL) is often viewed as problematic because it leads to sub-optimal value (or Q) functions and interferes with the contraction properties of the Bellman operator. Surprisingly, despite these concerns, and independent of its effect on explorati…
Classifies Zariski closures of positive representations in Lie groups.
problem Classifying Zariski closures of positive representations in Lie groups.
method Classifies the Lie algebra of the Zariski closure of a discrete subgroup with specific properties.
result Obtains a new proof of Guichard's classification of Zariski closures of Hitchin representations.
Study proposes curvature flow model for Drosophila dorsal closure.
problem Modeling and understanding Drosophila dorsal closure during embryonic development.
method Curvature-based mathematical model, analysis of maximum-principle and integral-estimates, numerical approximation scheme.
result Established global existence and convergence for the model.
Classifies orbit closures in translation surface strata.
problem Classifying orbit closures in translation surface strata.
method Classification of extGL(2,R) orbit closures. result Applications to joinings of certain Masur-Veech measures.
We study utility maximization for power utility random fields with and without intermediate consumption in a general semimartingale model with closed portfolio constraints. We show that any optimal strategy leads to a solution of the corresponding Bellman equation. The optimal strategies are described pointwise in term…
BCRL learns a Bellman complete representation for offline RL policy evaluation.
problem Learning a Q-function efficiently from offline data.
method BCRL learns a linear Bellman complete representation directly from data, enabling efficient OPE.
result BCRL achieves competitive OPE error and outperforms FQE in certain scenarios.
Uniform Closure Method and Bayes classifier perform similarly in classifying open knots.
problem Classifying knots in open macromolecular chains.
method Used the Bayes MAP classifier and compared it to the Uniform Closure Method.
result Both methods have comparable accuracy and positive predictive value.
One-step Bellman alignment improves online RL by reducing task mismatch.
problem Online RL struggles with task similarity defined by rewards or transitions.
method One-step Bellman alignment and re-weighted targeting (RWT) to correct task mismatch.
result Regret bounds show task shift complexity, not target MDP, affects performance.
The paper introduces Bellman-consistent pessimism to improve offline reinforcement learning without overly pessimistic bias.
problem Offline reinforcement learning's challenge of discovering good policies without exhaustive exploration.
method Introduces Bellman-consistent pessimism for function approximation, improving sample complexity and adaptability.
result Improves sample complexity by O(d) in the action space finite case, and automatically adapts to bias-variance tradeoff. We discuss two different in general natural approaches to the ideal closure and ideal boundary of Busemann nonpositively curved metric space. It is shown that the identity map of the space admits surjective continuation from its coarse ideal closure to the weak one. We consider some situations when these closures coinc…
A new method predicts non-Markovian closure terms for complex systems.
problem Predicting the effect of unresolved variables on resolved dynamics in high-dimensional systems.
method Mamba-Assisted Closure (MAC) framework: sequence model trained to predict closure from resolved trajectory, coupled with reduced-order equations.
result Substantially outperforms existing methods in predictive accuracy and long-time stability.
The study bounds the number of closed geodesics in a specific orbit closure of surfaces.
problem Counting closed geodesics in a specific orbit closure of surfaces.
method Analyzes triangulations and Teichmüller geodesics to bound the number of closed geodesics.
result Obtains exponential bounds on the number of closed geodesics of length at most R.
Extends graph degree theorem to simplicial closure of Auter space.
problem Connectivity of graphs in Auter space.
method Defines degree for simplicial closure, extends Hatcher-Vogtmann theorem.
result Simplicial closure of Auter space is (d-1)-connected for degree d.
We study a certain type of braid closure which resembles the plat closure but has certain advantages; for example, it maps pure braids to knots. The main results of this note are a Markov-type theorem and a description of how Vassiliev invariants behave under this braid closure.
We propose a new objective function for finite-horizon episodic Markov decision processes that better captures Bellman's principle of optimality, and provide an expression for the gradient of the objective.
We give a Dehn-Nielsen type theorem for the homology cobordism group of homology cylinders by considering its action on the acyclic closure, which was defined by Levine, of a free group. Then we construct an additive invariant of those homology cylinders which act on the acyclic closure trivially. We also describe some…
Paper introduces dynamic strategies for multi-period investment models.
problem Optimizing investment strategies over multiple periods with risk and return considerations.
method Developed a Bellman principle for discrete time multi-period mean-variance models, leading to dynamic optimal strategies and efficient frontiers.
result Dynamic optimal strategies can achieve higher returns with lower risk compared to the 1/n strategy.
The paper defines plat closures for spherical braids and shows links in RP3 can be realized this way.
problem Defining and analyzing plat closures for spherical braids in RP3. method Defining plat closures, associating residual permutations, and presenting moves on spherical braids.
result The number of components of the plat closure link of a spherical braid is equal to the number of disjoint cycles in its residual permutation.
Model quantifies uncertainty's impact on European option prices.
problem Uncertainty in market volatility risk affects option pricing.
method Hamilton-Jacobi-Bellman framework and finite element method.
result Dependence of Delta on uncertainty is nonlinear and varied.
This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables …
The paper classifies orbit closures of symplectic Lie algebras.
problem Classifying orbit closures of symplectic Lie algebras under the action of Sp(4,R). method Analyzing the natural action of Sp(4,R) on the set of 4-dimensional Lie algebras with symplectic structures. result A complete classification of orbit closures of 4-dimensional symplectic Lie algebras.
We prove Ptolemaean Inequality and Ptolemaeus' Theorem in the closure complex hyperbolic plane endowed with the Cygan metric.
Characterizes closures of mapping class group orbits on non-orientable surfaces.
problem Understanding closures of orbits in Teichmüller spaces for non-orientable surfaces.
method Analyzes closures in ML and PML for measured laminations, projective measured laminations, and points. result Characterizes closures of weighted two-sided curves in ML. Study solves optimal portfolio selection using HJB equation.
problem Optimal portfolio selection problem.
method Maximal monotone operator method, Banach fixed-point theorem, Fourier transform, monotone operators technique.
result Existence and uniqueness of solution to HJB equation.
Deep Bellman Hedging uses reinforcement learning to optimize financial portfolio hedging.
problem Optimizing financial portfolio hedging with derivatives and trading frictions.
method Actor-critic reinforcement learning algorithm with continuous state and action spaces.
result Trained model provides optimal hedge for any initial portfolio and market state.
The paper analyzes off-policy TD-learning using generalized Bellman operators and provides finite-sample bounds.
problem High variance in off-policy TD-learning due to importance sampling.
method Derives finite-sample bounds for off-policy TD-like algorithms using generalized Bellman operators.
result First-known finite-sample guarantees for several off-policy TD algorithms.
FORE evaluates occupancy ratios without requiring Bellman completeness.
problem Offline reinforcement learning occupancy ratio estimation.
method Fitted occupancy-ratio evaluation (FORE) using adjoint Bellman recursion.
result FORE achieves convergence in KL without Bellman completeness.
The article finds equivalence moves for links in specific manifolds using plat closure of braids.
problem Finding equivalence moves for links in Dunwoody and periodic Takahashi manifolds.
method Representing manifolds with Heegaard splitting and braids, determining equivalence moves algorithmically and computing them explicitly.
result Explicit computation of equivalence moves for some cases.