A new method trains deep neural networks using local critic networks.
problem Training deep neural networks efficiently and effectively.
method Employing local critic networks for error gradient calculation and cascaded learning.
result The approach improves training efficiency and performance of deep neural networks.
Gradient-based methods find saddle points, not critical points, in neural networks.
problem Gradient-based optimization methods converge to saddle points rather than critical points in deep neural networks.
method Critical point-finding methods used to analyze neural network losses.
result Gradient-based methods often converge to or pass through gradient-flat regions, where gradient norm has a stationary point.
Characterizes critical points and landscapes of neural networks.
problem Understanding loss functions and critical points in neural networks.
method Full characterization of analytical forms for critical points and global minimizers of square loss functions.
result Linear networks have no spurious local minima, while ReLU networks have local minima that are not global minima.
CPGAs converge to locally optimal policies for coagent networks.
problem Training stochastic neural networks using reinforcement learning.
method Proved convergence of CPGAs and extended prior theory to asynchronous and recurrent networks.
result CPGAs converge to locally optimal policies for coagent networks, including asynchronous and recurrent networks.
This study uses local Gaussian correlation to analyze stock return tails, revealing more sensitive network properties.
problem Misleading results from Pearson correlation in financial networks.
method Local Gaussian correlation coefficient for capturing nonlinear dependence and heavy-tailed distributions.
result Local Gaussian correlation network among negative tails is more sensitive to stock market risks.
Embedding principle explains loss landscape of deep neural networks.
problem Understanding the structure of loss landscapes in deep neural networks.
method Proposed an embedding principle that critical points of narrower DNNs can be embedded to critical points of wider DNNs.
result Wide DNNs are often attracted by highly-degenerate critical points embedded from narrower DNNs.
Geometric study of linear neural networks identifies pure and spurious critical points.
problem Understanding the landscape of loss functions in linear neural networks.
method Geometric properties of functional spaces and parameterization analysis.
result Different phenomena cause the absence of bad local minima in linear networks, depending on the architecture and loss function.
Deep actor-critic learning optimizes power control in mobile networks.
problem Optimizing power control in large-scale wireless mobile networks.
method Multi-agent deep reinforcement learning with deep deterministic policy gradient.
result The algorithm maximizes a global utility function in a distributed manner.
Analytical method finds deeper optima in two-layer ReLU networks.
problem Training two-layer ReLU networks with analytical methods.
method Analytically finding critical points of the loss function for one layer while keeping the other fixed.
result Significantly smaller training loss values on real datasets compared to gradient descent methods.
Deep ResNets can have better local minima than linear predictors.
problem Understanding the optimization landscape of deep ResNets compared to linear predictors.
method Analyzing the optimization landscape of ResNets with multiple residual blocks, showing geometric conditions under which ResNets have better local minima.
result Theorem showing that any critical point in the optimization landscape of deep ResNets is either at least as good as the best linear predictor or has a strictly negative eigenvalue in its Hessian.
In relativity, the energy of a moving particle depends on the observer, and the rest mass is the minimal energy seen among all observers. The Wang-Yau quasi-local mass for a surface in spacetime introduced in [7] and [8] is defined by minimizing quasi-local energy associated with admissible isometric embeddings of the …
Paper offers local neural network explanations by considering model architecture.
problem Creating true neighbourhoods for black box models.
method Penultimate layer decoding for local neighbourhood generation.
result Local explanations are more accurate and relevant to instances.
This paper simplifies deep ReLU networks into local linear models for better interpretability.
problem Limited transparency and interpretability of deep neural networks, especially ReLU networks.
method Local linear representation and equivalent set of local linear models (LLMs).
result Simplified deep ReLU networks for better interpretability and diagnostics.
Solves local minima problems on smooth manifolds.
problem Local minima issues on smooth manifolds.
method Introducing valley functions and applying Morse's lemma.
result Eliminates critical points and reduces to 1D.
We design a non-convex second-order optimization algorithm that is guaranteed to return an approximate local minimum in time which scales linearly in the underlying dimension and the number of training examples. The time complexity of our algorithm to find an approximate local minimum is even faster than that of gradie…
New DL algorithm detects critical chest X-ray findings without manual annotations.
problem Lack of explainability and manual annotation costs for DL models in medical imaging.
method Multi-instance learning approach to jointly classify and localize critical findings in CXR.
result Competitive classification results on three CXR datasets.
New algorithm for multi-agent reinforcement learning with reduced communication.
problem Cooperative learning among multiple agents with limited communication.
method Randomized multi-agent actor-critic algorithm for directed graphs.
result Algorithm solves problem for strongly connected graphs with reduced communication.
Diff-DAC uses deep neural networks for distributed reinforcement learning across multiple tasks.
problem Learning policies for multiple tasks with limited local data.
method Distributed actor-critic algorithm approximated by deep neural networks, with parameter diffusion.
result Diff-DAC outperforms previous distributed MRL approaches and even centralized methods.
PF-net combines neural network and particle filter for robot localization.
problem Applying particle filtering to complex systems with rich sensory inputs.
method PF-net integrates system model and particle filter in a neural network.
result PF-net outperforms alternative methods in visual localization tasks.
GRAC improves reinforcement learning by self-guiding and self-regularizing.
problem Learning divergence and slow updates in reinforcement learning algorithms.
method Self-regularized TD-learning and self-guided policy improvement.
result Achieved or outperformed state-of-the-art results on OpenAI gym tasks.
This paper interprets critical scales in persistent homology for compact metric spaces.
problem Understanding critical scales in persistent homology for general compact metric spaces.
method Analyzing local minima of the distance function and their impact on persistence.
result Each decrease in zero-dimensional persistence and increase in one-dimensional persistence is induced by local minima of the distance function.
Local foliation of 3D manifolds by Willmore surfaces with curvature constraints.
problem Constructing foliations of manifolds by surfaces of Willmore type.
method Adapting a method from constant mean curvature foliations to Willmore surfaces.
result Existence of a local foliation of a 3D Riemannian manifold by Willmore critical points with area constraint.
ReNN integrates domain rules into neural networks for better interpretability and accuracy.
problem Lack of interpretability and need for large datasets in neural networks.
method ReNN combines local pattern detection with rule-based global synthesis, using a two-stage optimization strategy.
result ReNN improves inference accuracy and interpretability by incorporating domain rules.
The paper uses critical percolation to analyze deep networks and maze data.
problem Understanding and analyzing the training of deep networks and graph structured data.
method Topological classification of reachability in planar graphs (Mazes) and a suitable architecture for processing.
result The cost function around the global minimum does not depend on maze size in the large maze limit.
Proves an analytical analogue of Morse's lemma for gradient fields near critical points.
problem Understanding the behavior of gradient fields near critical points of Morse functions.
method Proves an analytical analogue of Morse's lemma showing unique linear vector fields.
result Shows that gradient fields near critical points have a natural standard form.
Neural networks solve high-dimensional HJB PDEs with asymptotic guarantees.
problem Solving high-dimensional Hamilton-Jacobi-Bellman PDEs in stochastic control theory.
method Actor-critic machine learning algorithm with a structured critic and biased gradient actor.
result The training dynamics converge to an ODE, ensuring solutions to the original problem.
We study the critical points of the renormalized volume for acylindrical geometrically finite hyperbolic 3-manifolds that include rank-1 cusps, and show that the renormalized volume is locally convex around these critical points. We give a modified definition of the renormalized volume that is additive under gluing, an…
SELD-TCN improves sound event localization and detection efficiency.
problem Efficient sound event localization and detection on embedded hardware.
method Developed a novel temporal convolutional network (TCN) architecture.
result SELD-TCN outperforms state-of-the-art SELDnet on four datasets.
DMs emerge from DenseAMs, transitioning from memorization to generalization.
problem Hindered memory retrieval in DenseAMs due to spurious states.
method Examined diffusion models through the lens of DenseAMs, focusing on their generative process.
result Identified a critical phase in DMs transitioning from memorization to generalization.
New insights into gradient descent and ascent dynamics in min-max optimization.
problem Understanding the convergence and limit points of gradient descent and ascent methods in min-max optimization problems.
method Characterization of limit points using dynamical systems perspective for GDA and OGDA.
result Both GDA and OGDA dynamics avoid unstable critical points and have a superset of local min-max solutions.
Unified method detects and localizes anomalous cliques in inhomogeneous networks.
problem Detect and localize anomalous cliques in inhomogeneous networks.
method Unified method based on egonets for detection and localization.
result Unified method can detect and localize anomalous cliques in inhomogeneous networks.
Study rigidity of Einstein metrics as critical points of curvature functionals.
problem Characterize Einstein metrics as critical points of quadratic curvature functionals.
method Analyze pointwise inequalities involving Weyl curvature and traceless Ricci curvature.
result Provide rigidity results for Einstein metrics and locally conformally flat critical metrics.
Minimal networks minimize length and mass in certain configurations.
problem Finding minimal networks that minimize length and mass.
method Global and local calibrations to prove minimization properties.
result Minimal networks minimize mass and interfaces in partitions.
New scalable MARL framework for dynamic networked systems.
problem Scalability in multi-agent reinforcement learning with dynamic dependencies.
method Scalable Actor Critic framework for non-local and stochastic dependencies.
result Finite-time error bound showing convergence rate dependence on information spread speed.
Maps with few critical points can be locally trivially fibred.
problem Understanding maps with finitely many critical points in high-dimensional manifolds.
method Analyzing maps between manifolds with specific singular points.
result Existence of locally trivial topological fibrations and smooth maps with at most one critical point.
A2C-based MARL controls large-scale traffic signals more efficiently.
problem Scalability issue in centralized RL for large-scale traffic control.
method Decentralized Multi-Agent A2C with improved observability and reduced learning difficulty.
result Optimal, robust, and sample-efficient control over other algorithms.
Proposes SE(3) equivariant graph neural networks with local frames for efficient geometric approximation.
problem Equivariance in deep learning for arbitrary transformations, especially in physics.
method Introduces SE(3) equivariant graph neural networks with complete local frames to efficiently approximate geometric quantities.
result Achieves best or competitive performance in Newton mechanics modeling and equilibrium molecule conformation generation.
Study shows DNNs can recover functions with fewer samples than model parameters at overparameterization.
problem Determining reliable function recovery in overparameterized deep neural networks.
method Introducing 'local linear recovery' (LLR) and proving upper bounds on sample sizes for recovery.
result Upper bounds on optimistic sample sizes for function recovery in overparameterized DNNs are achieved.
New foliations found for critical surfaces of Hawking energy, resolving discrepancies.
problem Finding consistent critical surfaces for the Hawking energy in non-totally geodesic spacelike hypersurfaces.
method Constructing a unique local foliation of area constrained critical surfaces of the Hawking energy in the general case of non-totally geodesic spacelike hypersurfaces.
result Discrepancy found in the small sphere limit of the Hawking energy, explained and resolved.
Study local topology of a function-germ deformation with a one-dimensional critical set.
problem Analyze the local topology of a deformation of a function-germ with a one-dimensional critical set.
method Use the Brasselet number to study the local topology of a deformation of a function-germ.
result Present a new proof of the Lê-Iomdin formula for the Brasselet number.
Distributed learning method for multi-agent reinforcement learning with policy coordination.
problem Solving multi-agent reinforcement learning problems with coordination.
method Distributed off-policy actor critic with policy consensus.
result The proposed algorithm achieves asymptotic agreement on the global optimal policy function.
The paper explores the critical point equation on Kenmotsu and almost Kenmotsu manifolds.
problem Investigating the critical point equation on specific types of manifolds.
method Analyzing Kenmotsu and almost Kenmotsu manifolds with nullity conditions.
result Complete Kenmotsu metrics satisfying the CPE are Einstein and locally isometric to H2n+1.
In this paper, we show that the Chen-Nester-Tung (CNT) quasi-local energy is closely related to the Wang-Yau (WY) quasi-local mass. As a particular example, we compute the second variation of the CNT quasi-local energy for axially symmetric Kerr-like spacetimes with axially symmetric embeddings at the obvious critical …
The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.
problem Lack of theoretical understanding of target networks in actor-critic methods.
method Proposes a theoretical analysis of an online target-based actor-critic algorithm with linear function approximation.
result Establishes asymptotic convergence results and finite-time analysis for both critic and actor.
We present an actor-critic framework for MDPs where the objective is the variance-adjusted expected return. Our critic uses linear function approximation, and we extend the concept of compatible features to the variance-adjusted setting. We present an episodic actor-critic algorithm and show that it converges almost su…
Unified framework for neural networks under general input distributions.
problem Training neural networks with non-Gaussian input distributions.
method Designing loss functions with desirable landscape properties for general input distributions.
result Stochastic gradient descent can recover true parameters with global initializations for general input distributions.
Adapted metrics found on complex manifolds.
problem Finding metrics suitable for complex manifolds.
method Characterizing adapted metrics as critical points of a functional.
result Gauduchon metric is adapted on locally conformally product manifolds.
The paper studies critical points in overparameterized neural networks, identifying a star locus and degenerate critical points.
problem Understanding the geometry of loss functions in overparameterized neural networks.
method Identifying and analyzing components of the critical locus of the loss function L for overparameterized feedforward neural networks of depth ℓ≥4. result For very wide networks, all critical points are degenerate, and lower bounds on the number of zero eigenvalues of the Hessian are given.