Complex network theory has been applied to solving practical problems from different domains. In this paper, we present a general framework for complex network applications. The keys of a successful application are a thorough understanding of the real system and a correct mapping of complex network theory to practical …
This research explores complex-valued neural networks and their implementation.
problem The challenges of implementing complex-valued neural networks and their potential for non-complex data.
method Detailed theory and implementation of CVNN, including Wirtinger calculus, complex backpropagation, and modules like complex layers and activation functions. Python implementation using cvnn toolbox.
result Demonstrates the potential of CVNN for non-complex data through simulations.
New topological complexity measures for neural networks.
problem Measuring complexity of neural network functions.
method Generalized piecewise-linear Morse theory applied to ReLU networks.
result Local complexity can be arbitrarily high.
Paper introduces new neural network models and theories.
problem Understanding neural networks beyond over-parameterized regime.
method Develops two exact models and a novel representor theory.
result Provides insights into neural network training and kernel evolution.
Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
Researchers calculate spectral dimension of complex networks using renormalization group theory.
problem Understanding diffusion properties in complex systems.
method Renormalization group theory applied to graph Laplacians of simplicial complexes.
result Spectral dimension decreases with randomness in topological structure.
Unified framework for complex financial networks using lattice theory.
problem Complex financial networks with multiple currencies and dependencies.
method Recast classical financial clearing model into lattice liability networks.
result Lattice-valued clearing sections form a complete lattice, enabling tractable analysis.
Simplifies deep learning scaling analysis without sacrificing accuracy.
problem Interpreting feature learning mechanisms and determining network implicit bias in high-dimensional settings.
method Developed a heuristic approach for predicting data and width scales of feature learning patterns.
result Predictions align with known results and extend to complex architectures.
Equivariant neural networks improve performance and generalization in complex scalar field theory tasks.
problem Improving performance and generalization in neural networks for complex scalar field theory tasks.
method Incorporating translational equivariance into neural network architectures.
result Equivariant neural networks significantly outperform non-equivariant networks in various tasks, including those beyond the training set and across different lattice sizes.
Study shows how large neural networks avoid overfitting through decoupling of feature learning and complexity growth.
problem Understanding inductive bias and generalization in large neural networks.
method Dynamical mean field theory applied to large two-layer networks.
result Training dynamics of large networks exhibit a separation of timescales, decoupling feature learning and overfitting.
Geometrically analyzes why deep neural networks perform well despite high parameter complexity.
problem Explains the good performance of deep neural networks with high parameter spaces.
method Introduces geometric information-theoretic approach using Fisher information matrix and singular semi-Riemannian geometry.
result Derives model complexity measures explaining the performance of deep neural networks.
The paper explores the limits of deep neural networks in approximating various function classes.
problem Characterizing the limits of deep neural networks in function approximation.
method Develops a theory relating function complexity and network complexity, using Kolmogorov complexity.
result Deep networks are optimal approximants for various function classes and provide exponential approximation accuracy.
Koopman operator theory simplifies complex systems analysis.
problem Analyzing nonlinear dynamical systems and complex networks.
method Estimating Koopman operator from data to reveal system properties.
result Koopman operators provide insights into system characteristics.
Deep neural networks can approximate functions with varying complexity.
problem Understanding the expressivity of deep neural networks in terms of function approximation.
method Using approximation theory, the study measures complexity by connections or neurons, and defines approximation spaces.
result Deep neural networks can approximate functions with low Besov smoothness if sufficiently deep, even with ReLU activation.
Proposes IPT for modeling complex joint distributions.
problem Lack of closed-form solutions for complex continuous or mixed distributions.
method Observer-centered framework with three independence axioms; derivation of closed-form solutions.
result Closed-form solutions for complex joint distributions under IPT.
This work proves neural networks can learn certain function classes with fewer parameters than usual.
problem Understanding what neural networks can learn and why they don't overfit when overparameterized.
method Proved learning of concept classes in overparameterized neural networks using SGD.
result Overparameterized neural networks can learn with fewer parameters than usual.
New study shows exponential sample growth for ReQU neural networks.
problem Computing neural network approximations from samples is challenging.
method Information-based complexity tools.
result Functions can be approximated by ReQU neural networks at arbitrary rates but require exponentially growing samples.
Group equivariant neural networks simplify complex tasks with group representation theory.
problem Challenging tasks requiring input transformations like rotations.
method Group representation theory, non-commutative harmonic analysis, differential geometry.
result A neural network is group equivariant if and only if it has a convolutional structure.
Symmetry unifies AI learning dynamics, complexity, and representation.
problem Fragmented theories of AI learning mechanisms.
method Synthesis of parameter symmetry in AI models.
result Parameter symmetry breaking and restoration unify AI learning hierarchies.
Complex network theory models China's credit system to control systemic risk.
problem Insufficient understanding of China's credit network structure during financial crises.
method Constructed bipartite financial institution-firm network and analyzed its typological properties.
result Credit network structure can amplify local risks to the whole economy.
Introduces LLC, a new complexity measure for DNNs based on SLT.
problem Lack of effective complexity measures for DNNs.
method Uses Singular Learning Theory to define LLC and proposes scalable estimator.
result Empirical evidence shows LLC provides valuable insights into DNN complexity.
Unified theory of deep neural networks with diverse activations.
problem Understanding the relationship between depth and complexity in deep neural networks.
method Developed a unified function space theory for deep networks with various activations.
result Unified theory provides meaningful complexity for deep networks with diverse activations.
Study improves generalization bounds for equivariant networks on Markov data.
problem Challenges in integrating equivariance with Markov dependencies in neural networks.
method Applied McDiarmid's inequality and computed covering number using group theory.
result Derived upper bound on Rademacher complexity for equivariant neural networks on Markov datasets.
Graph neural networks generalize well under certain conditions, explained by learning theory.
problem Understanding why graph neural networks generalize well in transductive inference.
method Analysis of transductive Rademacher complexity to explain generalization properties of graph convolutional networks.
result Transductive Rademacher complexity can explain the generalization of graph convolutional networks for node classification in stochastic block models.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
Network theory assesses systemic risk in the insurance sector.
problem Detecting critical insurance companies in systemic risk.
method Complex network approach with weighted effective resistance centrality.
result Identifies companies with significant influence on network robustness.
Develops statistical theory for measuring network complexity.
problem Estimating network complexity with statistical guarantees.
method Graphon model and neighborhood distance metric.
result Statistical theory matches minimax lower bounds.
Neural networks approximate high-dimensional functions better than theory predicts.
problem Current theory struggles to explain why small neural networks work well in high-dimensional inverse problems.
method Bounding complexity required for neural networks to approximate Hölder or uniformly continuous functions on high-dimensional sets.
result A general theoretical framework explaining empirical successes of smaller networks in inverse problems.
We represent an exchange economy in terms of statistical ensembles for complex networks by introducing the concept of market configuration. This is defined as a sequence of nonnegative discrete random variables {wij} describing the flow of a given commodity from agent i to agent j. This sequence can be arran…
Detect anomalies in complex networks using topological subspace detectors.
problem Detect anomalies in complex networks defined by simplicial complexes.
method Formulate a hypothesis testing framework using Neyman-Pearson matched topological subspace detectors.
result Effective detection of anomalies in foreign currency exchange networks and other real-world data.
Trade networks, across which countries distribute their products, are crucial components of the globalized world economy. Their structure is strongly heterogeneous across products, given the different features of the countries which buy and sell goods. By using a diversified pool of indicators from network science and …
Researchers use estimated Kolmogorov complexity for better link prediction in graphs.
problem Improving link prediction accuracy in complex networks.
method Regularization based on an approximation of Kolmogorov complexity, which is differentiable and compatible with recent link prediction algorithms.
result The regularization method shows good performance on diverse real-world networks, but the success is likely due to an aggregation method rather than actual estimation of Kolmogorov complexity.
ACL improves robustness with unlabeled data, and we analyze its generalization using Rademacher complexity.
problem Improving robustness of deep networks against adversarial attacks using unlabeled data.
method We analyze the generalization performance of Adversarial Contrastive Learning (ACL) using Rademacher complexity.
result The average adversarial risk of the downstream tasks can be upper bounded by the adversarial unsupervised risk of the upstream task.
Paper defines untangling number to measure entanglement complexity in 3-periodic networks.
problem Measuring the complexity of entanglement in 3-periodic networks.
method Defining ground states through knot-theoretic crossing diagrams and measuring untangling number.
result Introduced untangling number as a measure of entanglement complexity.
The paper addresses instability in KL divergence estimation using a neural network discriminator.
problem Unstable estimation of KL divergence due to discriminator complexity.
method Using a Reproducing Kernel Hilbert Space (RKHS) to control discriminator complexity.
result Theoretical bound on error probability of KL estimates based on discriminator complexity in RKHS.
Deep, wide ConvResNets can approximate functions and their smoothness.
problem Function approximation and smoothness in deep networks.
method Analyzing ConvResNets, proving their ability to approximate functions and their smoothness.
result Large ConvResNets can approximate functions and exhibit sufficient first-order smoothness.
A measure of neural complexity quantifies how hard it is to access information across neurons.
problem Understanding how mutual information is distributed among neurons in neural networks.
method Partial Information Decomposition (PID) to disentangle contributions of single neurons, multiple neurons, and synergistic effects.
result Representational Complexity measures the difficulty of accessing information across multiple neurons.
In this review we establish various connections between complex networks and symmetry. While special types of symmetries (e.g., automorphisms) are studied in detail within discrete mathematics for particular classes of deterministic graphs, the analysis of more general symmetries in real complex networks is far less de…
Bayesian neural networks explore rare fluctuations for better feature learning.
problem Understanding rare but dominant fluctuations in Bayesian neural networks.
method Large-deviation theory and joint optimization over predictors and internal kernels.
result Posterior rate function optimization reveals data-dependent kernel selection.
We study the wealth distribution of the Bouchaud--Mézard (BM) model on complex networks. It has been known that this distribution depends on the topology of network by numerical simulations, however, no one have succeeded to explain it. Using "adiabatic" and "independent" assumptions along with the central-limit theore…
Develops variational Bayesian neural network for complex biomedical applications.
problem High computational cost of Markov Chain Monte Carlo in BNN.
method Variational Bayes inference for posterior consistency and classification accuracy.
result Developed statistical theory for posterior consistency and prediction accuracy.
Develops a new classical network ensemble framework.
problem Lack of information-theoretic frameworks for complex networks.
method Optimal trade-off between compressed representation and actual network ensemble.
result Power-law degree distribution is optimal for networks with only expected degrees as constraints.
The paper analyzes tensor recovery from symmetric rank-one measurements using information theory.
problem Recovering tensors with low symmetric rank from symmetric rank-one measurements.
method Covering numbers argument, Carbery-Wright inequality, orthogonal polynomials, Fano's inequality.
result Near-optimal sample complexity bounds for log-concave distributions.
The learnability of different neural architectures can be characterized directly by computable measures of data complexity. In this paper, we reframe the problem of architecture selection as understanding how data determines the most expressive and generalizable architectures suited to that data, beyond inductive bias.…
Paper establishes bounds for RNN-TPPs, showing four-layer networks can achieve vanishing errors.
problem Understanding theoretical limits of RNN-TPPs.
method Characterized RNN complexity, constructed neural approximations, applied truncation technique.
result Four-layer RNN-TPPs can achieve vanishing generalization errors.
Estimates LLC for deep linear networks up to 100M parameters.
problem Quantifying model complexity for large-scale deep learning architectures.
method Empirical estimation of LLC using a method developed for DLNs.
result LLC can be accurately measured for DLNs up to 100M parameters.
This study uses persistent homology to analyze complex transitional networks from time series data.
problem Lack of effective tools to summarize complex topology in transitional networks.
method Persistent homology from topological data analysis applied to coarse-grained state-space networks (CGSSN).
result CGSSN improves dynamic state detection and noise robustness compared to other methods.
A network of spiking agents learns complex tasks using global reward signals.
problem Solving complex reinforcement learning tasks.
method A hierarchical network of GLM spiking agents, each modulating its firing policy based on local and global reward signals.
result A network of spiking agents can learn complex action representations to solve RL tasks.