Complex-valued neural networks perform similarly to real-valued models for real-valued classification tasks.
problem Comparing real-valued and complex-valued neural networks for real-valued classification tasks.
method Comparison of neural networks with similar capacity sizes, using various activation functions and weight initialisation strategies.
result Complex-valued neural networks perform equal to or slightly worse than real-valued models for real-valued classification tasks.
Improved sample complexity bounds for neural networks with depth independence.
problem Understanding the sample complexity of neural networks with depth and size independence.
method New bounds on Rademacher complexity with norm constraints on parameter matrices.
result Improved sample complexity bounds that are fully independent of network size under certain assumptions.
Complex-valued neural networks avoid spurious local minima.
problem Finding spurious local minima in neural networks.
method Proved no spurious local minima for shallow complex neural networks with quadratic activations.
result Complex-valued weights eliminate spurious local minima in neural networks.
Maxout networks show similar complexity issues as ReLU networks.
problem Understanding the complexity of maxout networks and decision boundaries.
method Analyzing the parameter space and decision boundaries, obtaining lower bounds, and investigating initialization procedures.
result Maxout networks exhibit a wide range of complexity, similar to ReLU networks.
Deep neural networks have almost linear sample complexity.
problem Sample complexity of deep neural networks.
method o-minimal expansion of the real field to bound sample complexity.
result Almost linear bound on sample complexity of neural networks.
Complexity measures for neural nets with general activations using path-based norms.
problem Control complexity of neural networks with arbitrary activation functions.
method Approximate general activations with ReLU networks and derive path-based norms for complexity control.
result Preliminary analyses of function spaces and regularized estimators.
New complexity measure explains better generalization with over-parametrization in neural networks.
problem Why neural networks generalize better with over-parametrization.
method Developed a novel complexity measure based on unit-wise capacities.
result Established a tighter generalization bound for two layer ReLU networks.
This research explores complex-valued neural networks and their implementation.
problem The challenges of implementing complex-valued neural networks and their potential for non-complex data.
method Detailed theory and implementation of CVNN, including Wirtinger calculus, complex backpropagation, and modules like complex layers and activation functions. Python implementation using cvnn toolbox.
result Demonstrates the potential of CVNN for non-complex data through simulations.
Complex network theory has been applied to solving practical problems from different domains. In this paper, we present a general framework for complex network applications. The keys of a successful application are a thorough understanding of the real system and a correct mapping of complex network theory to practical …
Method reconstructs networks from contagion dynamics.
problem Fitting contagion models assumes simple dynamics, ignoring complex contagions.
method Nonparametric method to reconstruct network and dynamics from node states.
result Networks are easier to reconstruct through complex contagions in dense or saturated networks.
Active learning reduces spin network inference complexity by 10^6-fold.
problem Difficulty in inferring direct interactions in complex networks.
method Information geometry framework to quantify inference difficulty and information gain from perturbations.
result Designed perturbations reduce sampling complexity by 10^6-fold across various network architectures.
Improved sample complexity for ReLU networks with norm constraints.
problem Estimating sample complexity for ReLU networks under norm constraints.
method Refined Rademacher complexity analysis for function class.
result Often no explicit depth-dependence in sample complexity bound.
Measures neural network complexity using tangent space diversity.
problem Estimating the true complexity of neural networks.
method Entropy-based measure of tangent spaces from different inputs.
result Captures effective complexity, not just theoretical capacity.
Machine learning speeds up centrality measure calculations for large networks.
problem High computational costs of traditional centrality measures in large networks.
method Neural network learning algorithms to approximate centrality measures.
result Regression model approximates centrality measures efficiently and accurately.
New findings on depth vs. width in neural networks, showing depth can improve learnability.
problem Understanding the role of depth in neural networks, especially when width is unbounded.
method Analyzing sample complexity for learnability in norm-controlled depth-2 and depth-3 ReLU networks.
result Depth can improve learnability of functions that are otherwise unlearnable with depth-2 networks.
This paper proves a generalization bound for complex-valued neural networks scaling with spectral complexity.
problem Ensuring the performance of complex-valued neural networks on unseen data.
method Theoretical derivation using Maurey Sparsification Lemma and Dudley Entropy Integral, empirical validation on various datasets.
result The spectral complexity of weight matrices is a significant factor in the generalization ability of complex-valued neural networks.
BNs extract sparse, non-redundant features from climate data networks.
problem Redundant information in correlation networks limits physical feature extraction.
method Construct data-driven complex networks using Bayesian Networks.
result Sparse, non-redundant features reveal generalizable physical features.
Proposes CXNs for neural network computations on cell complexes.
problem Performing neural network computations on complex topological spaces.
method Introduces a message passing scheme and a unified encoder-decoder framework for cell complexes.
result Generalizes message passing to cell complexes and provides a cell2vec representation.
Residual neural networks don't help overcome sampling complexity issues.
problem Learning invertible residual neural networks from samples is hard due to the curse of dimensionality.
method Investigated invertible residual neural networks and their sampling complexity.
result Invertible residual neural networks still suffer from the curse of dimensionality in sampling complexity.
Paper proposes LANN to measure model complexity of neural networks with curve activation functions.
problem Measuring model complexity of neural networks with curve activation functions.
method Proposes LANN, a piecewise linear framework to approximate curve activation functions, and derives complexity measure based on the number of linear regions.
result Demonstrates positive correlation between overfitting and model complexity during training.
New topological complexity measures for neural networks.
problem Measuring complexity of neural network functions.
method Generalized piecewise-linear Morse theory applied to ReLU networks.
result Local complexity can be arbitrarily high.
Paper tackles complex risk in deep neural networks.
problem Complex risk in deep neural networks.
method Developed new approach for complex risk statistics.
result Derived dual representation for complex risk.
Neural networks reduce DBP complexity in fiber optics.
problem Complexity in digital backpropagation implementations.
method Neural-network-based approach to implement DBP.
result Learned DBP reduces complexity by 32x100 km fiber-optic link.
Bayesian sparsification improves complex-valued neural networks by 50-100x with minimal performance loss.
problem Efficiently compressing complex-valued neural networks for embedded systems.
method Extending Sparse Variational Dropout to complex-valued networks and conducting a numerical study.
result Achieved state-of-the-art performance on MusicNet with 50-100x compression.
The angular power spectrum characterizes neural network complexity.
problem Characterizing the complexity of deep neural networks.
method Using the angular power spectrum of the limiting field to characterize network complexity.
result Classified neural networks as low-disorder, sparse, or high-disorder.
Researchers use estimated Kolmogorov complexity for better link prediction in graphs.
problem Improving link prediction accuracy in complex networks.
method Regularization based on an approximation of Kolmogorov complexity, which is differentiable and compatible with recent link prediction algorithms.
result The regularization method shows good performance on diverse real-world networks, but the success is likely due to an aggregation method rather than actual estimation of Kolmogorov complexity.
Complex-valued neural networks can approximate any continuous function.
problem Generalizing the universal approximation theorem to complex-valued networks.
method Characterizing activation functions for complex networks to approximate any continuous function.
result Different activation functions are required for deep vs shallow complex networks to achieve universal approximation.
New bounds on ReLU networks for low-regular functions.
problem Bounding approximation error for ReLU networks on low-regular functions.
method Complexity analysis of Fourier features residual networks to ReLU networks.
result Approximation error bound proportional to target function norm and inversely proportional to network width and depth.
Low-complexity spiking networks learn complex tasks with minimal trainable parameters.
problem Training complex reinforcement learning tasks with minimal resources.
method Reinforcement learning on simple networks of spiking neurons with random connections.
result Small random spiking networks achieve learning efficiency similar to humans on complex tasks.
Paper introduces a new edge exchangeable block model for complex networks.
problem Limitations of the stochastic block model in analyzing complex networks.
method Develops a Bayesian nonparametric edge exchangeable block model.
result The new model outperforms state-of-the-art SBMs for link prediction.
Study examines how network architecture handles increasing data complexity.
problem Understanding how network architecture affects performance with complex data.
method Empirical study comparing various network architectures on an image classification task with increasing class numbers.
result Modern architectures show better generalization performance with increasing data complexity.
Proposes Neural Complexity (NC) for predicting and explaining generalization in deep neural networks.
problem Challenges in specifying a suitable complexity measure for deep neural networks to predict and explain generalization.
method A meta-learning framework that learns a scalar complexity measure through interactions with many heterogeneous tasks.
result Trained NC model can be added to standard training loss to regularize any task learner.
PINNs struggle with increasingly complex ODEs, especially when parameters control their complexity.
problem Evaluating physics-informed neural networks on complex coupled ODEs.
method Tuned benchmarks of partial differential equations and harmonic oscillators; varying network architecture and training method.
result PINNs fail to solve complex ODEs, revealing issues like insufficient capacity, poor conditioning, and high local curvature.
Complex products trade through fewer countries, making them more fragile.
problem Fragility in the global economy due to centralized trade networks for complex products.
method Used network science and product complexity theory indicators to analyze trade networks.
result Products with higher complexity trade through fewer countries, making them more fragile.
Automatically updates both network weights and architecture.
problem Manual selection of network architecture limits flexibility and efficiency.
method Continuous parameterization of network depth and automatic adjustment of architecture and weights.
result Correctly adjusts network complexity to task complexity.
Paper analyzes sample complexity of polynomial neural networks.
problem Understanding the sample complexity of polynomial neural networks.
method Extends previous literature to polynomial neural networks and analyzes sample complexity.
result Obtains novel results on sample complexity of polynomial neural networks.
Great successes of deep neural networks have been witnessed in various real applications. Many algorithmic and implementation techniques have been developed, however, theoretical understanding of many aspects of deep neural networks is far from clear. A particular interesting issue is the usefulness of dropout, which w…
New bound on neural network generalization error using geometric complexity.
problem Understanding the generalization capabilities of deep neural networks.
method Derive a new upper bound on generalization error using margin-normalized geometric complexity.
result Empirical validation of the bound for ResNet-18 on CIFAR-10 and CIFAR-100 datasets.
Unified entropy formula for real, complex, and quaternionic DLNs.
problem Deriving a formula for DLNs over different fields.
method Extending Menon and Yu's formula to complex and quaternionic DLNs.
result Unified entropy formula for DLNs over R, C, and H. New sample complexity bounds for linear predictors and neural networks, focusing on initialization.
problem Understanding sample complexity for vector-valued linear predictors and neural networks, especially under initialization-dependent conditions.
method Size-independent bounds on Frobenius norm distance from a fixed reference matrix, applying to vector-valued predictors and neural networks.
result Established new sample complexity bounds for feed-forward neural networks, resolving open questions and introducing a new learnable problem.
New algorithm trains deep neural networks without global optimization.
problem Training deep neural networks efficiently and without global optimization.
method Uses random complex exponential activation functions and Markov Chain Monte Carlo sampling.
result Consistently attains theoretical approximation rate for residual networks.
Complex neural networks simplify to a mean field model as the number of neurons increases.
problem Understanding the behavior of multilayer neural networks with many neurons.
method Developed a mean field limit formalism for multilayer neural networks under stochastic gradient descent.
result The behavior of multilayer neural networks simplifies to a mean field model as the number of neurons grows large.
New embeddings capture local structure in complex networks.
problem Embeddings cannot capture local structure in complex networks.
method Logistic Principal Component Analysis (LPCA) algorithm for exact low-rank representations.
result Exact low-rank representations of real-world networks are possible.
A new complexity measure for neural networks improves upon classical methods.
problem Lack of a refined complexity measure for comparing different neural network architectures, especially permutation-invariant ones.
method Introduced an equivalence relation among linear functions and counted them relative to this relation.
result The new complexity measure clearly distinguishes between different models and increases exponentially with depth.
Develops a complexity measure for neural networks based on quantum statistical mechanics.
problem Understanding the relationship between neural network structure and generalization ability.
method Introduces Periodic Spectral Ergodicity (PSE) and cascading PSE (cPSE) to quantify neural network complexity.
result Demonstrates the effectiveness of cPSE in quantifying complexity and guiding NAS.
Modeling structure in complex networks using Bayesian non-parametrics makes it possible to specify flexible model structures and infer the adequate model complexity from the observed data. This paper provides a gentle introduction to non-parametric Bayesian modeling of complex networks: Using an infinite mixture model …
New discrete Ricci curvature for directed networks developed.
problem Directed networks require a new curvature measure.
method Extended Forman-Ricci curvature for directed networks, incorporating vertex and edge weights, and edge direction.
result New curvature measure captures higher-order correlations in directed networks.
Paper defines untangling number to measure entanglement complexity in 3-periodic networks.
problem Measuring the complexity of entanglement in 3-periodic networks.
method Defining ground states through knot-theoretic crossing diagrams and measuring untangling number.
result Introduced untangling number as a measure of entanglement complexity.