arXiv research

Smooth manifolds have functions with exactly two critical values.

In value-based reinforcement learning methods such as deep Q-learning, function approximation errors are known to lead to overestimated value estimates and suboptimal policies. We show that this problem persists in an actor-critic setting and propose novel mechanisms to minimize its effects on both the actor and the cr…

MAGE optimizes policies using action gradients from model-based learning.

Study uses actor-critic method for continuous-time mean-field control with entropy regularisation.

New algorithm solves mean-field control problems using actor-critic learning with moment neural networks.

Let $F=(F_1, F_2, ..., F_m): \mathbb{C}^n \to \mathbb{C}^m$ be a polynomial dominant mapping with $n>m$. In this paper we give the relations between the bifurcation set of $F$ and the set of values where $F$ is not M-tame as well as the set of generalized critical values of $F$. We also construct explicitly a proper su…

Necessary and sufficient condition is given for a set $A\subset R^1$ to be a subset of the critical values set for a $C^k$ function $f:R^m \to R^1$.

Study magnetic geodesics on half-Lie groups, proving Hopf-Rinow theorem for energies above critical value.

New Morse theory for shapes at distances.

3D manifolds can map to a plane with specific curve patterns.

New method improves deep policy gradient algorithms by learning relative state values.

The paper explores the structure of Reeb spaces for smooth functions on manifolds.

Despite the empirical success of the actor-critic algorithm, its theoretical understanding lags behind. In a broader context, actor-critic can be viewed as an online alternating update algorithm for bilevel optimization, whose convergence is known to be fragile. To understand the instability of actor-critic, we focus o…

Extends Morse-Forman theory to vector-valued functions for multiparameter persistence.

Proposes a value-based method for continuous control without an actor.

Parastatistic distribution of a total debt owed to a large number of creditors considered in relation to the duration of these debts. The process of debt calculation depends on the fractal dimension of economic system in which this process takes place. Two actual variants of these dimensions are investigated. Critical …

Smaller actor-critic models lead to performance degradation and overfitting, highlighting the critic's role in value underestimation.

We reformulate the option framework as two parallel augmented MDPs. Under this novel formulation, all policy optimization algorithms can be used off the shelf to learn intra-option policies, option termination conditions, and a master policy over options. We apply an actor-critic algorithm on each augmented MDP, yieldi…

The time value of money is a critical factor not only in risk analysis, but also in insurance and financial applications. In this paper, we consider a special class of set-valued risk statistics by introducing the time value of money. In fact, the risk statistics established by this method is closer to financial realit…

For a finite-dimensional (but possibly noncompact) symplectic manifold with a compact group acting with a proper moment map, we show that the square of the moment map is an equivariantly perfect Morse function in the sense of Kirwan, and that the set of critical points of the square of the moment map is a countable dis…

Study on manifolds that map to lower dimensions with specific critical points.

Decouples critic chunk length from policy to improve policy reactivity and performance.

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update weighted by the action-values, with the addition of entropy regularization for soft variants. In th…

We give the classification, up to homeomorphisms, of reduced complex polynomials with 2 variables with one critical value.

New groups found with critical exponents close to but less than max.

Polynomials with distinct critical values have braid monodromy groups equal to braid groups.

Whitney type examples of maps $f\in C^k(\real^m,\real^n)$ for a maximal possible real $k$, and multidimensional space-filling curves with special properties are constructed.

We prove the existence and uniqueness of a *projectively equivariant symbol map*, which is an isomorphism between the space of bidifferential operators acting on tensor densities over $R^n$ and that of their symbols, when both are considered as modules over an imbedding of $sl(n+1,\R)$ into polynomial vector fields. Th…

We study the problem of off-policy critic evaluation in several variants of value-based off-policy actor-critic algorithms. Off-policy actor-critic algorithms require an off-policy critic evaluation step, to estimate the value of the new policy after every policy gradient update. Despite enormous success of off-policy …

We describe how to compute topological objects associated to a polynomial map of several complex variables with isolated singularities. These objects are: the affine critical values, the affine Milnor numbers for all irregular fibers, the critical values at infinity, and the Milnor numbers at infinity for all irregular…

Paper develops estimates for Lagrangian phase changes in 2D.

Study finds non-monotonic Value of Information in dynamic multi-market monopoly.

We give a global version of Le-Ramanujam mu-constant theorem for polynomials. Let f_t, (t in [0,1]), be a family of polynomials of n complex variables with isolated singularities, whose coefficients are polynomials in t. We consider the case where some numerical invariants are constant (the affine Milnor number, the Mi…

ESAC improves reinforcement learning by lookahead and intuition.

Classical Morse theory proceeds by considering sublevel sets $f^{-1}(-\infty, a]$ of a Morse function $f: M \to R$, where $M$ is a smooth finite-dimensional manifold. In this paper, we study the topology of the level sets $f^{-1}(a)$ and give conditions under which the topology of $f^{-1}(a)$ changes when passing a cri…

Short note on upper bounds for loop homology classes.

A novel Q-learning variant reduces underestimation bias in deep actor-critic methods for reinforcement learning.

WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.

Let $S$ be a set of critical points of a smooth real-valued function on a closed manifold $M$. Generalizing a well-known result of Lusternik--Schnirelmann, Reeken~[R] proved that $\cat S \geq \cat M$. Here we prove a generalization of Reeken"s inequality for gradient-like flows on compact spaces.

Gradient flows of neural networks converge to optimal values or diverge, with thresholds and asymptotic behaviors.

We use noncommutative localization to construct a chain complex which counts the critical points of a circle-valued Morse function on a manifold, generalizing the Novikov complex. As a consequence we obtain new topological lower bounds on the minimum number of critical points of a circle-valued Morse function within a …

In traditional reinforcement learning, an agent maximizes the reward collected during its interaction with the environment by approximating the optimal policy through the estimation of value functions. Typically, given a state s and action a, the corresponding value is the expected discounted sum of rewards. The optima…

We introduce a new critical value $c_\infty(L)$ for Tonelli Lagrangians $L$ on the tangent bundle of the 2-sphere without minimizing measures supported on a point. We show that $c_\infty(L)$ is strictly larger than the Mañé critical value $c(L)$, and on every energy level $e\in(c(L),c_\infty(L))$ there exist infinitely…

Flexible decentralized MARL framework for cooperative multi-agent learning.

Formula for critical points of chi fields on manifolds.

Paper proves gradient estimates for Lagrangian mean curvature equation.

We describe a mathematically rigorous differential model for B-type open-closed topological Landau-Ginzburg theories defined by a pair $(X,W)$, where $X$ is a non-compact Kählerian manifold with holomorphically trivial canonical line bundle and $W$ is a complex-valued holomorphic function defined on $X$ and whose criti…

Recently many efforts have been made to incorporate persistence diagrams, one of the major tools in topological data analysis (TDA), into machine learning pipelines. To better understand the power and limitation of persistence diagrams, we carry out a range of experiments on both graph data and shape data, aiming to de…