PLLay adds topological layers to deep learning models efficiently.
problem Efficiently incorporating topological features into deep learning models.
method Persistence landscapes for differentiable topological features.
result PLLay improves model learnability and robustness.
This paper introduces a new method for neural networks that doesn't need a global coordinate system.
problem The lack of a global coordinate system in neural networks limits their performance and explainability.
method Proposes a learnable topological layer that works in a general metric space (Hilbert space) without requiring a Euclidean space.
result The proposed method eliminates the need for a costly parametrization stage and achieves optimal network performance.
Topology applied to real world data using persistent homology has started to find applications within machine learning, including deep learning. We present a differentiable topology layer that computes persistent homology based on level set filtrations and edge-based filtrations. We present three novel applications: th…
Given a complete non-compact surface embedded in R^3, we consider the Dirichlet Laplacian in a layer of constant width about the surface. Using an intrinsic approach to the layer geometry, we generalise the spectral results of an original paper by Duclos et al. to the situation when the surface does not possess poles. …
Neural networks simplify complex data topologies into simpler ones.
problem Understanding why deep neural networks perform better than shallow ones and why ReLU activations are superior.
method Persistent homology analysis of neural network layers on various data sets.
result Neural networks reduce the topological complexity of input data sets, often to their simplest form.
Paper learns DAGs with quadratic variance functions efficiently.
problem Learning DAGs with quadratic variance functions.
method Introduces topological layers to reconstruct DAGs hierarchically.
result Efficient algorithm reduces computational cost.
Single wide layer followed by a pyramidal structure ensures global convergence in deep networks.
problem Ensuring global convergence in deep neural networks with limited width constraints.
method Proves that a single wide layer followed by a pyramidal structure guarantees global convergence for over-parameterized networks.
result Single wide layer of width N suffices for global convergence in deep networks with constant-width remaining layers. NeuroFabric proposes a method to optimize sparse network training topologies.
problem Long training times in deep neural networks due to high memory and compute requirements.
method Developed a new sparse neural network initialization scheme and evaluated various topologies.
result Identified a single optimal topology that maximizes accuracy across different datasets.
We simplify neural networks to 3D to study their topological changes.
problem Understanding how neural network layers affect low-dimensional topological invariants.
method Limiting each layer to a width of 3D space, tracking changes in linking numbers.
result ResNets and transformers are equally powerful in changing linking numbers.
The paper sets limits on neural network sizes based on dataset shapes.
problem Understanding the size of neural networks needed for accurate predictions.
method Examined how the shape of data influences neural network complexity.
result Established upper limits on neural network width based on dataset topology.
Smectic liquid crystals are materials formed by stacking deformable, fluid layers. Though smectics prefer to have flat, uniformly-spaced layers, boundary conditions can impose curvature on the layers. Since the layer spacing and curvature are intertwined, the problem of finding minimal configurations for the layers bec…
Efficiently learns linear non-Gaussian DAGs with noisy nodes.
problem Learning DAGs with non-Gaussian noise and diverging number of nodes.
method Proposes a novel method using topological layers for bottom-up reconstruction and consistent parent-child relations.
result Topological layers can be exactly reconstructed and parent-child relations established without faithfulness assumption.
In established network architectures, shortcut connections are often used to take the outputs of earlier layers as additional inputs to later layers. Despite the extraordinary effectiveness of shortcuts, there remain open questions on the mechanism and characteristics. For example, why are shortcuts powerful? Why do sh…
Lie groupoid equivariant neural networks are a new type of neural network.
problem Designing neural networks that respect the structure of Lie groupoids.
method Introducing Lie groupoid equivariant convolutions and layers, and showing their equivalence to Lie algebroid-equivariant networks.
result Lie groupoid equivariant neural networks are equivalent to certain Lie algebroid-equivariant networks.
Network analysis reveals distinct financial relationships among Euro Area banks.
problem Understanding complex interbank relationships in the Euro Area.
method Multi-layer network approach using granular financial data.
result A more complete picture of the Euro Area interbank market topology.
The interbank market has a natural multiplex network representation. We employ a unique database of supervisory reports of Italian banks to the Banca d'Italia that includes all bilateral exposures broken down by maturity and by the secured and unsecured nature of the contract. We find that layers have different topolog…
Combines gradient-based and competitive learning for unsupervised feature extraction.
problem Handling input data without supervision and replicating input manifold topology.
method Integrates gradient-based and competitive learning approaches to learn topological structures.
result The dual competitive layer outperforms the vanilla layer in high-dimensional datasets.
Proposes deep graph persistence to address neural persistence issues in deep learning.
problem Variance of weights and lack of spatial structure in deep neural networks impact neural persistence.
method Extends neural persistence to the whole network, considering interactions between layers.
result Deep graph persistence alleviates variance-related issues and captures persistent paths through the network.
In this paper we propose a generalization of deep neural networks called deep function machines (DFMs). DFMs act on vector spaces of arbitrary (possibly infinite) dimension and we show that a family of DFMs are invariant to the dimension of input data; that is, the parameterization of the model does not directly hinge …
Dropout and similar stochastic neural network regularization methods are often interpreted as implicitly averaging over a large ensemble of models. We propose STE (stochastically trained ensemble) layers, which enhance the averaging properties of such methods by training an ensemble of weight matrices with stochastic r…
Predictive Sparse Manifold Transform learns dynamic video sequences.
problem Learning and predicting natural dynamics in video sequences.
method Two-layer framework: sparse coding and manifold learning.
result PSMT with dynamic embedding space outperforms static baselines in future frame prediction.
An asset network systemic risk (ANWSER) model is presented to investigate the impact of how shadow banks are intermingled in a financial system on the severity of financial contagion. Particularly, the focus of this study is the impact of the following three representative topologies of an interbank loan network betwee…
TOGL adds topological info to GNNs, improving graph and node classification.
problem Graph neural networks lack substructure awareness, especially cycles.
method Integrates global topological information using persistent homology.
result Improves predictive performance for graph and node classification.
Improved bounds on neural network regions using activation histograms.
problem Bounding the number of affine regions in ReLU networks.
method Analysis of algebraic topology problem, extension of framework to subnetwork composition.
result Slightly tighter bounds and insights into parameter initialization.
Deep Learning methods, specifically convolutional neural networks (CNNs), have seen a lot of success in the domain of image-based data, where the data offers a clearly structured topology in the regular lattice of pixels. This 4-neighbourhood topological simplicity makes the application of convolutional masks straightf…
Efficient memory layer improves graph neural networks for graph classification and regression.
problem Efficiently learning node representations and graph coarsening for arbitrary graph topology.
method Introduces a memory layer for GNNs that learns node representations and graph coarsening, and two new networks: MemGNN and GMN.
result Proposed models achieve state-of-the-art results in graph classification and regression benchmarks.
Constructs dg categories from surfaces using Khovanov homology.
problem Categorify quantum topology using surfaces and Khovanov homology.
method Constructs dg categories from surfaces using structures in Khovanov homology.
result Unified perspective on various categorified quantum topology constructions.
The theoretical explanation for deep neural network (DNN) is still an open problem. In this paper DNN is considered as a discrete-time dynamical system due to its layered structure. The complexity provided by the nonlinearity in the dynamics is analyzed in terms of topological entropy and chaos characterized by Lyapuno…
This paper presents a new artificial neuron model capable of learning its receptive field in the topological domain of inputs. The model provides adaptive and differentiable local connectivity (plasticity) applicable to any domain. It requires no other tool than the backpropagation algorithm to learn its parameters whi…
Empirical investor networks (EIN) proposed by \cite{Ozsoylev-Walden-Yavuz-Bildik-2014-RFS} are assumed to capture the information spreading path among investors. Here, we perform a comparative analysis between the EIN and the cellphone communication networks (CN) to test whether EIN is an information exchanging network…
The paper examines how neural network topology affects adversarial robustness.
problem Understanding how neural network topology influences adversarial robustness.
method Investigated the graph of input traversing all layers of a neural network, comparing clean and adversarial inputs.
result Under-optimized edges in neural network graphs are a source of adversarial vulnerability and can be used to detect adversarial inputs.
Poor approximators found in neural networks and random feature models.
problem Understanding why certain neural networks and models perform poorly in approximating functions.
method Established a scale separation of Kolmogorov width type and applied it to neural networks and random feature models.
result Reproducing kernel Hilbert spaces and two-layer neural networks are poor L2-approximators for certain functions. Entropy data replaces classical charts for smooth manifolds.
problem Establishing smooth structures on topological manifolds.
method Using entropy data to define admissible coordinate functions and reconstruct smooth atlases.
result Entropy-smooth structures are equivalent to classical smooth structures and stable under perturbations.
We introduce a new function-preserving transformation for efficient neural architecture search. This network transformation allows reusing previously trained networks and existing successful architectures that improves sample efficiency. We aim to address the limitation of current network transformation operations that…
This work bridges competitive learning with gradient-based learning for faster feature extraction.
problem Lack of powerful feature extractors in competitive learning methods.
method Introduces gradient-based competitive layers for feature extraction.
result Demonstrates theoretical equivalence and faster convergence of gradient-based competitive layers.
Paper estimates neural network size needed for topology learning.
problem Estimating the smallest neural network size for topology learning.
method Using algebraic topology and Lie theory, the paper introduces a procedure based on persistent homology to determine the required dimension.
result The derived dimension is the smallest capable of capturing the topology of the data manifold.
Deep learning enhances Hamiltonian Monte Carlo for sampling gauge field configurations.
problem Sampling from complex gauge field topologies efficiently.
method Stacked neural networks to generalize Hamiltonian Monte Carlo.
result Significantly reduces computational cost for generating gauge field configurations.
Fog learning distributes ML model training across heterogeneous devices and networks.
problem Challenges with conventional federated learning in heterogeneous networks.
method Intelligent distribution of ML model training across nodes from edge devices to cloud servers.
result Enhanced federated learning with multi-layer hybrid framework considering network, heterogeneity, and proximity.
Multiplex networks, a special type of multilayer networks, are increasingly applied in many domains ranging from social media analytics to biology. A common task in these applications concerns the detection of community structures. Many existing algorithms for community detection in multiplexes attempt to detect commun…
This paper improves deep forest models with soft routing and topology learning.
problem Expensive computational costs and lack of interpretability in deep neural networks.
method Soft routing in probabilistic trees and topology learning for joint optimization.
result Empowered deep forests achieve better performance with reduced model complexity.
New method integrates topological knowledge into data embeddings.
problem Lack of general tools to incorporate prior topological knowledge into embeddings.
method Introduces new topological losses to topologically regularize data embeddings.
result Natural representation of simple models like clusters and flares.
Global convergence proved for three-layer neural networks in mean field regime.
problem Optimization efficiency of multilayer neural networks in the mean field regime.
method Developed a rigorous framework for mean field limit of three-layer networks using stochastic gradient descent and neuronal embedding.
result Global convergence guarantee for unregularized feedforward three-layer networks in the mean field regime.
The paper proves skip connections help neural networks avoid shallow local minima.
problem Understanding how skip connections affect the loss landscape of deep neural networks.
method Theoretical analysis of the topology of loss landscapes of deep ReLU neural networks with skip connections.
result Skip connections help control the connectedness of sub-level sets, avoiding shallow local minima.
L-CNNs learn gauge invariant quantities on lattices.
problem Learning gauge invariant quantities on lattices.
method Novel convolutional layer preserving gauge equivariance and forming Wilson loops.
result L-CNNs can approximate any gauge covariant function on the lattice.
The worldwide trade network has been widely studied through different data sets and network representations with a view to better understanding interactions among countries and products. Here we investigate international trade through the lenses of the single-layer, multiplex, and multi-layer networks. We discuss diffe…
We build polyhedral complexes in Rn that coincide with dyadic grids with different orientations, while keeping uniform lower bounds (depending only on n) on the flatness of the added polyhedrons including their subfaces in all dimensions. After the definitions and first properties of compact Euclidean polyhedrons and c…
We propose Sparse Neural Network architectures that are based on random or structured bipartite graph topologies. Sparse architectures provide compression of the models learned and speed-ups of computations, they can also surpass their unstructured or fully connected counterparts. As we show, even more compact topologi…
Z-GCNETs uses topological data to improve time series forecasting.
problem Improving time series forecasting accuracy.
method Integrates topological data into graph convolutional networks (GCNs) using zigzag persistence.
result Z-GCNETs outperforms 13 state-of-the-art methods in traffic forecasting and Ethereum price prediction.