Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
problem Improving sample efficiency in continuous control tasks.
method Meta-critic that meta-learns an additional loss for the actor.
result Online meta-critic learning leads to improved performance in various continuous control environments.
This paper reverses a construction by merging boundary critical points into an interior one.
problem Pushing interior critical points to the boundary and splitting them into two boundary points.
method Specific assumptions allow merging two boundary critical points into one interior critical point.
result Merging two boundary critical points into a single interior critical point.
The minimal number of critical points is studied for smooth functions on closed manifolds.
problem Determining the minimal number of critical points for smooth functions on closed manifolds.
method Investigates cylindrical ball neighborhoods and exotic critical points, proving the conjecture for certain types of critical points.
result The minimal number of critical points is the same for smooth functions without exotic critical points on closed manifolds of dimension at least 6.
The study confirms a conjecture about critical points of smooth functions.
problem Understanding isolated critical points of smooth functions.
method Investigated cone-like, reasonable, and Rothe H hypothesis critical points.
result The conjecture holds true for certain critical points.
We present the first provably convergent two-timescale off-policy actor-critic algorithm (COF-PAC) with function approximation. Key to COF-PAC is the introduction of a new critic, the emphasis critic, which is trained via Gradient Emphasis Learning (GEM), a novel combination of the key ideas of Gradient Temporal Differ…
Study critical points of Laplace eigenfunctions in polygons.
problem Characterize critical points of Laplace eigenfunctions in polygonal domains.
method Analyze components of the critical set with codimension 1.
result For simply connected polygons, if a second Neumann eigenfunction has infinitely many critical points, the polygon must be a rectangle.
Smaller actor-critic models lead to performance degradation and overfitting, highlighting the critic's role in value underestimation.
problem Performance degradation and overfitting in actor-critic models with smaller actors.
method Broad empirical investigations and analyses of asymmetric actor-critic setups, exploring techniques to mitigate value underestimation.
result Value underestimation is a key cause of performance degradation in smaller actor-critic models, and the critic plays a crucial role in mitigating this.
The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.
problem Lack of theoretical understanding of target networks in actor-critic methods.
method Proposes a theoretical analysis of an online target-based actor-critic algorithm with linear function approximation.
result Establishes asymptotic convergence results and finite-time analysis for both critic and actor.
New PAC-Bayesian approach stabilizes actor-critic learning.
problem Training instability in actor-critic algorithms.
method Employing PAC-Bayesian bound as the critic training objective.
result Significant improvement in online learning performance.
Hard to approximate critical points for simple nonconvex functions.
problem Approximating critical points of nonconvex functions.
method Proving hardness results for polynomial-time approximation of critical points.
result Proving that approximating critical points is intractable for simple nonconvex functions.
We prove a version of symmetric criticality for ropelength-critical knots. Our theorem implies that a knot or link with a symmetric representative has a ropelength-critical configuration with the same symmetry. We use this to construct new examples of ropelength critical configurations for knots and links which are dif…
The paper studies critical points in overparameterized neural networks, identifying a star locus and degenerate critical points.
problem Understanding the geometry of loss functions in overparameterized neural networks.
method Identifying and analyzing components of the critical locus of the loss function L for overparameterized feedforward neural networks of depth ℓ≥4. result For very wide networks, all critical points are degenerate, and lower bounds on the number of zero eigenvalues of the Hessian are given.
WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.
problem Inherent instability in actor-critic reinforcement learning algorithms.
method Wasserstein adaptive value estimation with Sinkhorn approximation.
result Achieves $\mathcal{O}\left(\frac{1}{k}
ight)$ convergence rate for critic's mean squared error.
Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this pa…
New findings on Chern flat metrics and their criticality.
problem Understanding critical Hermitian metrics on Chern flat manifolds.
method Analyzing Chern flat manifolds as compact quotients of complex Lie groups and studying their criticality.
result Chern flat metrics on semi-simple Lie groups are torsion-critical and vice versa.
We establish regularity results for critical points to energies of immersed surfaces depending on the first and the second fundamental form exclusively. These results hold for a large class of intrinsic elliptic Lagrangians which are sub-critical or critical. They are derived using uniform ε−regularity estimates whic…
Critical surfaces are defined by Bachman as topological index 2 surfaces, generalizing incompressible surfaces and strongly irreducible surfaces. In this paper we give a condition to obtain critical Heegaard surfaces by amalgamation. As a special case, we obtain critical Heegaard surfaces by boundary stabilization. It …
Study on CR curves in 3-sphere, focusing on critical curves integration and existence.
problem Addressing the integration and existence of critical curves in the CR 3-sphere.
method Provided a procedure for the explicit integration of general critical curves and characterized closed curves.
result Existence of infinite countably many closed critical curves.
Single-timescale actor-critic finds globally optimal policy.
problem Finding globally optimal policy in reinforcement learning.
method Simultaneous actor and critic updates with linear or deep neural network approximations.
result Actor sequence converges to globally optimal policy at O(K−1/2) rate. This paper focuses on the problem of topological equivalence of functions with isolated critical points on the boundary of a compact surface M which are also isolated critical points of their restrictions to the boundary. This class of functions we denote by Ω(M). Firstly, we've obtained the topological classificat…
Symmetric critical points lead to symmetry breaking in neural networks.
problem Understanding symmetry in critical points of invariant functions.
method Analyzing the symmetry of critical points and their neighbors in invariant nonconvex functions.
result Symmetric critical points in invariant nonconvex functions are generically followed by symmetry breaking adjacent points.
Study classifies ruled surfaces critical to Dirichlet energy.
problem Identifying ruled surfaces critical to Dirichlet energy.
method Explicit parametrization of ruled surfaces.
result Classification of ruled surfaces as critical points of Dirichlet energy.
Upper bounds found for systole function critical points on surface moduli space.
problem Finding upper bounds for systole function critical points.
method Analyzing the systole function on the surface moduli space.
result Upper bounds for critical points of systole function and their systole values.
Study on critical faces convergence in a Poisson point process.
problem Convergence of point processes associated with critical faces in a Čech filtration.
method Established convergence in M0-topology for critical faces above vanishing threshold. result Obtained limit theorems for positive and negative critical faces.
Noninjective monodromy found in polynomial critical point tracking.
problem Tracking critical points in polynomials leads to noninjective monodromy.
method Monic squarefree complex polynomials with prescribed critical point multiplicities.
result Monodromy map is noninjective for polynomials with exactly two critical points.
New groups found with critical exponents close to but less than max.
problem Finding discrete isometry groups with critical exponents near maximum.
method Analyzing complex hyperbolic spaces to construct groups.
result Discrete isometry groups with critical exponents arbitrarily close to max but less.
We study the problem of off-policy critic evaluation in several variants of value-based off-policy actor-critic algorithms. Off-policy actor-critic algorithms require an off-policy critic evaluation step, to estimate the value of the new policy after every policy gradient update. Despite enormous success of off-policy …
Actor-critic methods can achieve incredible performance on difficult reinforcement learning problems, but they are also prone to instability. This is partly due to the interaction between the actor and critic during learning, e.g., an inaccurate step taken by one of them might adversely affect the other and destabilize…
Paper finds critical metrics with pinched curvature are geodesic balls.
problem Identifying critical metrics with specific curvature constraints.
method Proved isometry to geodesic balls in S^n and provided conditions for the gradient of the potential function.
result Critical metrics with pinched curvature are isometric to geodesic balls in S^n.
Critical hypersurfaces with boundary have unique shapes and properties.
problem Characterizing the shapes of hypersurfaces with boundary and zero fractional mean curvature.
method Analyzing critical points of fractional area in RN with boundary conditions. result Critical hypersurfaces with specific boundary conditions are not simple shapes like (N−1)-balls. We give an alternative proof of that a critical knot of a Morse-Bott function f:S3→R is a graph knot where the critical set of f is a link in S3. Our proof inducts on the number of index-1 critical knots of f.
In this paper we investigate complete critical metrics of the L2-norm of the scalar curvature. We prove that any complete critical metric with positive scalar curvature has constant scalar curvature and we characterize critical metrics with nonnegative scalar curvature in dimension three and four.
The paper studies the smoothness of critical points of variational integrals on Hessian spaces.
problem The study focuses on the regularity of critical points of variational integrals defined on Hessian spaces.
method The approach involves solving a fourth order nonlinear equation and analyzing the Hessian of the critical points.
result Smooth critical points with bounded Hessian are shown to be smooth provided their Hessian has small BMO.
New rigidity results for critical metrics of a quadratic curvature functional.
problem Proving uniqueness of critical metrics for a specific curvature functional.
method Analyzing complete, possibly non-compact, critical metrics of the quadratic curvature functional.
result Critical metrics with finite energy are scalar flat (global minima) for dimensions n≥10.
Study shows wealth distribution tails near criticality are not universal.
problem Understanding wealth distribution tails near criticality.
method Generalized affine wealth model with nonconstant redistribution.
result Exponential tail near criticality is not universal; depends on redistribution policy.
In this paper are studied the simplest patterns of axial curvature lines (along which the normal curvature vector is at a vertex of the ellipse of curvature) near a critical point of a surface mapped into R4. These critical points, where the rank of the mapping drops from 2 to 1, occur isolated in generic one parameter…
New analysis shows actor-critic method converges efficiently in practical settings.
problem Understanding finite-time convergence of single-timescale actor-critic methods.
method Investigated online single-timescale actor-critic algorithm with linear function approximation and Markovian sampling.
result Proved convergence to ε-approximate stationary point with sample complexity of O(ε^(-2)).
This work analyzes actor-critic methods for faster convergence.
problem Finite-time analysis and sample complexity of two-time-scale actor-critic methods.
method Non-asymptotic analysis under non-i.i.d. setting, proving convergence to first-order stationary point.
result Actor-critic method finds a first-order stationary point with ildeO(ε−2.5) sample complexity. Study on critical points in random neural networks, revealing three regimes based on activation function.
problem Investigating the expected number of critical points in random neural networks.
method Deriving asymptotic formulas for critical points under infinite-width limit and suitable regularity conditions.
result Three distinct regimes of critical points behavior depending on activation function.
Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields. Recently, the loss surfaces of deep neural networks have been explored to gain insight into outstanding questions in optimization, generalization, and network architecture design. However, the degree to wh…
Study critical metrics on manifolds with boundary using integral and boundary estimates.
problem Investigate geometry of critical metrics on compact manifolds with boundary.
method Use generalized Reilly's formula to derive integral and boundary estimates.
result Establish new boundary estimates for critical metrics of the volume functional.
Despite the empirical success of the actor-critic algorithm, its theoretical understanding lags behind. In a broader context, actor-critic can be viewed as an online alternating update algorithm for bilevel optimization, whose convergence is known to be fragile. To understand the instability of actor-critic, we focus o…
Researchers identify critical protein residues using advanced graph theory.
problem Identifying essential residues in proteins for function.
method Learning Random Geometric Graphs (RGG) with Cramer's V correlation and organic thresholding.
result Advanced RGG methods accurately identify critical residues compared to existing techniques.
The paper classifies and analyzes the stability of elastic curves with fixed endpoints.
problem Classification and stability of pinned elasticae.
method Critical points of the length-penalized elastic bending energy among planar curves with fixed endpoints.
result Explicit parametrization and classification of all critical points with a threshold parameter \(\hatλ \simeq 0.70107\).
We reformulate the option framework as two parallel augmented MDPs. Under this novel formulation, all policy optimization algorithms can be used off the shelf to learn intra-option policies, option termination conditions, and a master policy over options. We apply an actor-critic algorithm on each augmented MDP, yieldi…
This paper improves convergence bounds for AC and NAC algorithms with function approximation.
problem Improving convergence bounds for actor-critic algorithms with function approximation.
method Non-asymptotic analysis of AC and NAC algorithms with compatible function approximation.
result Eliminates the term ε_critic from the error bounds while maintaining best known sample complexities.
New proof for certain groups in higher dimensions.
problem Properties of discrete subgroups in higher dimensions.
method Proving convex-cocompactness for specific groups.
result Finitely generated Kleinian groups with small critical exponent are convex-cocompact.
We prove the existence of symmetric critical torus knots for O'Hara's knot energy family Eα, α∈(2,3) using Palais' classic principle of symmetric criticality. It turns out that in every torus knot class there are at least two smooth Eα-critical knots, which supports experimental observations using numerical …