The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.
problem Lack of theoretical understanding of target networks in actor-critic methods.
method Proposes a theoretical analysis of an online target-based actor-critic algorithm with linear function approximation.
result Establishes asymptotic convergence results and finite-time analysis for both critic and actor.
The paper studies critical points in overparameterized neural networks, identifying a star locus and degenerate critical points.
problem Understanding the geometry of loss functions in overparameterized neural networks.
method Identifying and analyzing components of the critical locus of the loss function L for overparameterized feedforward neural networks of depth ℓ≥4. result For very wide networks, all critical points are degenerate, and lower bounds on the number of zero eigenvalues of the Hessian are given.
Critical learning periods found in deep linear networks too.
problem Understanding why critical learning periods emerge in both biological and artificial networks.
method Focused on deep linear network models, analyzed depth and data distribution, and examined multi-task learning.
result Critical periods depend on model depth and data distribution structure, and pre-training can affect transfer performance.
Symmetric critical points lead to symmetry breaking in neural networks.
problem Understanding symmetry in critical points of invariant functions.
method Analyzing the symmetry of critical points and their neighbors in invariant nonconvex functions.
result Symmetric critical points in invariant nonconvex functions are generically followed by symmetry breaking adjacent points.
Study shows critical initialisation not crucial for ReLU networks under dropout limits.
problem Effect of initialisation on training speed and generalisation in ReLU networks.
method Large-scale statistical analysis of over 12,000 trained networks.
result Non-critical initialisations perform similarly to critical initialisations in terms of performance.
Study on critical points in random neural networks, revealing three regimes based on activation function.
problem Investigating the expected number of critical points in random neural networks.
method Deriving asymptotic formulas for critical points under infinite-width limit and suitable regularity conditions.
result Three distinct regimes of critical points behavior depending on activation function.
Some neural network modules are more critical to performance than others.
problem Understanding why some neural network architectures generalize better than others.
method Introduced module criticality, a measure based on the shape of loss valleys.
result Module criticality explains superior generalization performance of some architectures.
New method diagnoses criticality in deep neural networks, improving performance.
problem Improving theoretical understanding and practical initialization of deep neural networks.
method Introducing partial Jacobians and deriving recurrence relations for their norms to analyze criticality.
result Proper stacking of LayerNorm and residual connections leads to a critical architecture for any initialization.
ASAC uses actor-critic models to optimize observation selection in medical settings.
problem Optimizing observation selection in costly sequential observation scenarios.
method ASAC framework with selector and predictor networks, using actor-critic models for training.
result ASAC significantly outperforms state-of-the-art methods in real-world medical datasets.
Neural networks trained with actor-critic algorithms converge to ODEs under weak convergence analysis.
problem Challenges in convergence analysis due to changing data distributions in online learning.
method Geometric ergodicity of data samples, Poisson equation, weak convergence techniques.
result Actor and critic networks converge to solutions of ODEs with random initial conditions.
This paper examines critical points in deep linear autoencoders.
problem Locating critical points in non-convex surfaces for optimization.
method Investigated two sub-problems: sampling and numerical tolerance.
result Numerical methods often fail to sample all critical points and require strict numerical tolerance.
This paper identifies critical cases for evaluating PV investment impacts on MV networks efficiently.
problem Challenges in maintaining and controlling voltages in MV distribution networks due to increasing PV generation.
method Clustering MV nodes based on electrical adjacency and time blocks, identifying critical cases for further study.
result A scalable method to time efficiently identify critical cases for PV investment evaluation.
This work analyzes how neural networks learn representations in actor-critic algorithms.
problem Theoretical support for neural AC algorithms is limited to linear function approximations.
method Mean-field analysis of a two-timescale learning AC algorithm with overparameterized networks.
result Neural AC finds the globally optimal policy at a sublinear rate in the continuous-time and infinite-width limiting regime.
New theory predicts deep neural networks can operate in an extended critical regime without fine-tuning.
problem Understanding the dynamics and computational principles of deep neural networks.
method Combining theories of heavy-tailed random matrices and non-equilibrium statistical physics.
result Deep neural networks can operate in an extended critical regime without fine-tuning parameters.
Modeling financial markets as gas molecules, the paper predicts phase transitions similar to water and steam.
problem Understanding the dynamics of financial markets through phase transitions.
method Developed a lattice gas model equivalent to the Ising model on a social network, analyzing critical exponents and auto-correlations.
result Financial market dynamics exhibit phase transition-like behavior, with critical exponents analogous to water and steam.
Simplified neural network EFTs reveal a single critical condition.
problem Understanding neuron statistics in neural networks at initialization.
method Diagrammatic approach to effective field theories (EFTs).
result A single condition governs criticality of all neuron preactivations.
Enhances exploration in hierarchical networks using mutual information.
problem Limited exploration in hierarchical Deep Q-Networks.
method Adversarial Soft Actor-Critic with mutual information optimization.
result Improves hierarchical network exploration through mutual information maximization.
Hybrid actor-critic learns in complex action spaces.
problem Learning in complex, structured action spaces.
method Parallel sub-actor networks and a critic network.
result Hybrid PPO outperforms previous methods in parameterized action spaces.
Method reconstructs financial networks from aggregate data, revealing critical link density.
problem Reconstructing financial networks from aggregate data is challenging due to unreconstructability phases.
method Random graph generation with desired link density and replicated constraints.
result There is a critical link density below which networks become unreconstructable.
Geometric study of linear neural networks identifies pure and spurious critical points.
problem Understanding the landscape of loss functions in linear neural networks.
method Geometric properties of functional spaces and parameterization analysis.
result Different phenomena cause the absence of bad local minima in linear networks, depending on the architecture and loss function.
Due to the success of deep learning to solving a variety of challenging machine learning tasks, there is a rising interest in understanding loss functions for training neural networks from a theoretical aspect. Particularly, the properties of critical points and the landscape around them are of importance to determine …
Study reveals different types of critical points in shallow neural networks.
problem Optimization challenges in two-layer ReLU networks with symmetry.
method Symmetry analysis, bifurcation theory, and geometric group actions.
result Different types of spurious minima have distinct loss behavior.
The paper explores multidimensional critic output in GANs, improving convergence and diversity.
problem Underexplored in GANs literature, multidimensional critic output.
method Generalized Wasserstein GAN framework, SRVT block, maximal p-centrality discrepancy.
result High-dimensional critic output improves GAN performance in convergence and diversity.
The paper identifies critical points in neural networks that overfit training data.
problem Critical points in neural networks that achieve high training accuracy but low test accuracy.
method Proposed method to identify and study these points.
result Critical points in neural networks have significant generalization errors.
Embedding principle explains loss landscape of deep neural networks.
problem Understanding the structure of loss landscapes in deep neural networks.
method Proposed an embedding principle that critical points of narrower DNNs can be embedded to critical points of wider DNNs.
result Wide DNNs are often attracted by highly-degenerate critical points embedded from narrower DNNs.
Aims to make neural networks robust to small, imperceptible perturbations.
problem Neural networks can be fooled by small, imperceptible changes.
method Proposes a Generative Adversarial Networks (GAN) framework to learn robust classifiers.
result Our method outperforms adversarial training in robustness.
New algorithm solves mean-field control problems using actor-critic learning with moment neural networks.
problem Solving mean-field control problems in continuous time reinforcement learning.
method Gradient-based policy and value function learning with moment neural networks on the Wasserstein space.
result Effective solution for diverse mean-field control problems, including multi-dimensional and nonlinear settings.
Gradient-based methods find saddle points, not critical points, in neural networks.
problem Gradient-based optimization methods converge to saddle points rather than critical points in deep neural networks.
method Critical point-finding methods used to analyze neural network losses.
result Gradient-based methods often converge to or pass through gradient-flat regions, where gradient norm has a stationary point.
We develop a topology data analysis-based method to detect early signs for critical transitions in financial data. From the time-series of multiple stock prices, we build time-dependent correlation networks, which exhibit topological structures. We compute the persistent homology associated to these structures in order…
Paper analyzes NAC with neural networks for efficient policy optimization.
problem Improving sample and iteration complexity in policy optimization.
method Entropy regularization, averaging, neural network approximation, and optimization techniques.
result Entropy regularization and averaging ensure stability and sharp sample complexity bounds.
A neural network model predicts the critical point of the Ising phase transition.
problem Predicting the critical point of the Ising phase transition using supervised learning.
method Proposed a minimal one-free-parameter neural network model to describe the supervised learning problem for the Ising model.
result Just one free parameter is enough to describe the universal finite-size-scaling function in the network output.
New method uses RBM flows to find critical temperatures in Ising models.
problem Detecting critical temperatures in RBM flows without model topology information.
method Iterative sampling from RBM mapped on Ising model temperature space using a neural network thermometer.
result Flow of RBM trained on Ising spin configurations approaches critical temperature around kBTc/J≈2.269. Critical initialisation strategies are identified for noisy ReLU networks.
problem Understanding signal propagation in noisy rectifier neural networks.
method Developed a new framework for signal propagation in stochastic regularized neural networks, incorporating various noise distributions.
result Critical initialisation strategies for multiplicative noise (e.g. dropout) are identified, but not for additive noise.
Single-timescale actor-critic finds globally optimal policy.
problem Finding globally optimal policy in reinforcement learning.
method Simultaneous actor and critic updates with linear or deep neural network approximations.
result Actor sequence converges to globally optimal policy at O(K−1/2) rate. Deep actor-critic learning optimizes power control in mobile networks.
problem Optimizing power control in large-scale wireless mobile networks.
method Multi-agent deep reinforcement learning with deep deterministic policy gradient.
result The algorithm maximizes a global utility function in a distributed manner.
Enhances fault tolerance of neural networks for security-critical applications.
problem Fault tolerance of neural networks is biased and can lead to severe consequences in security-critical scenarios.
method Proposes a revised implementation that significantly enhances the fault tolerance property of neural networks with detailed mathematical analysis.
result Significantly increased fault tolerance of neural networks for security-critical applications.
Deep learning depends on tuning layers near critical points.
problem Understanding how deep learning architectures depend on tuning parameters.
method Random energy approach to analyze statistical dependence in deep belief networks.
result Statistical dependence can propagate only if layers are tuned near critical points.
Quantum field theory connects deep neural networks to criticality.
problem Understanding the criticality and training dynamics of deep neural networks.
method Constructing quantum field theory for deep neural networks, computing corrections to correlation functions.
result Found precise analogy with O(N) vector model, providing corrections to correlation length. Study shows proper initialisation of binary weights is crucial for deep neural networks.
problem Training stochastic binary neural networks with continuous surrogates is challenging.
method Developed new surrogates based on Markov chain theory and mean field analysis.
result Critical initialisations are necessary for training deep networks with binary weights.
Deep Learning can significantly benefit cancer proteomics and genomics. In this study, we attempt to determine a set of critical proteins that are associated with the FLT3-ITD mutation in newly-diagnosed acute myeloid leukemia patients. A Deep Learning network consisting of autoencoders forming a hierarchical model fro…
Mixed integer programming identifies critical neurons in neural networks.
problem Identifying neurons critical for network performance and generalization.
method Developed a mixed integer program (MIP) to assign importance scores to neurons, guiding pruning decisions.
result The method identifies multiple 'lucky' sub-networks resulting in optimized architectures that generalize across datasets.
Neural networks solve high-dimensional HJB PDEs with asymptotic guarantees.
problem Solving high-dimensional Hamilton-Jacobi-Bellman PDEs in stochastic control theory.
method Actor-critic machine learning algorithm with a structured critic and biased gradient actor.
result The training dynamics converge to an ODE, ensuring solutions to the original problem.
Ego-CNN detects critical structures in graphs efficiently.
problem Lack of precise detection of critical structures in existing graph embedding models.
method Ego-CNN uses ego-convolutions at each layer and stacks them in an ego-centric way.
result Ego-CNN achieves comparable task performance to state-of-the-art models and can incorporate scale-free priors.
This paper proposes a novel approach to train deep neural networks by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which are used to obtain error gradients without complete feedforward …
Machine learning detects tipping points in complex systems.
problem Detecting abrupt shifts in complex dynamical systems.
method Equilibrium-informed neural networks (EINNs) trained on candidate equilibrium states.
result EINNs can identify critical thresholds in nonlinear systems.
Paper introduces a meta-critic for accelerating off-policy actor-critic learning.
problem Improving sample efficiency in continuous control tasks.
method Meta-critic that meta-learns an additional loss for the actor.
result Online meta-critic learning leads to improved performance in various continuous control environments.
RBMs learn archetypes when trained on blurred copies of them, revealing a critical sample size.
problem Determining the critical sample size for RBMs to learn archetypes.
method Formal equivalence between RBMs and Hopfield networks, statistical-mechanics of disordered systems, Monte Carlo simulations.
result A phase diagram highlights regions where learning can be accomplished.
The paper analyzes deep neural networks' expressivity and training, revealing critical expressivity issues.
problem Critical expressivity issues in deep neural networks.
method Quantitative analysis using Hilbert space and Hermite polynomials for feature mapping and activation function design.
result Deep neural networks evolve to the edge of chaos, but expressivity depends on overcoming convergence.