Novel tRSA combines geometry and topology for brain and model analysis.
problem Traditional RSA overlooks topological information in neural representations.
method Topological RSA (tRSA) using nonlinear monotonic transforms.
result Robust model comparisons and novel insights into neural computation.
Neural persistence measures network complexity using topology.
problem Lack of measures for characterizing and monitoring structural properties of neural networks.
method Topological data analysis on weighted stratified graphs.
result Neural persistence reflects best practices and can shorten training time.
This work introduces a method to compare sparse neural network topologies using graph theory.
problem Comparing and understanding sparse neural network topologies, especially during training.
method Introducing Neural Network Sparse Topology Distance (NNSTD) to measure distances between different sparse neural networks.
result Sparse neural networks can outperform over-parameterized models without further structure optimization.
New algorithm generates diverse sparse neural network topologies.
problem Training deep neural networks requires significant hardware resources.
method Deterministic algorithm to generate sparse neural network topologies.
result Generated topologies train to the same precision as dense DNNs at lower cost.
RadiX-Net generates diverse sparse neural topologies.
problem Sparse neural networks require more efficient storage and training.
method Deterministically generates RadiX-Nets from sparse topologies.
result RadiX-Nets can train to the same precision as dense DNNs at lower cost.
Topology aids in solving machine learning classification problems.
problem Machine learning classification problems.
method Classical topology applied to neural networks.
result Topology guides neural network architecture and training.
Introduces topological deep learning for neural network classification problems.
problem Classifying neural networks using minimal topological structures.
method Formalizes classification problems in a topological setting.
result Demonstrates conditions for the feasibility of classification problems in neural networks.
This paper introduces a new method for neural networks that doesn't need a global coordinate system.
problem The lack of a global coordinate system in neural networks limits their performance and explainability.
method Proposes a learnable topological layer that works in a general metric space (Hilbert space) without requiring a Euclidean space.
result The proposed method eliminates the need for a costly parametrization stage and achieves optimal network performance.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
Graph neural network using Beltrami flow for feature and topology evolution.
problem Efficient feature learning and topology evolution on graphs.
method Discretized Beltrami flow applied to graph neural networks with positional encodings.
result Achieves state-of-the-art results on various benchmarks.
The paper sets limits on neural network sizes based on dataset shapes.
problem Understanding the size of neural networks needed for accurate predictions.
method Examined how the shape of data influences neural network complexity.
result Established upper limits on neural network width based on dataset topology.
Neural networks simplify complex data topologies into simpler ones.
problem Understanding why deep neural networks perform better than shallow ones and why ReLU activations are superior.
method Persistent homology analysis of neural network layers on various data sets.
result Neural networks reduce the topological complexity of input data sets, often to their simplest form.
Topological constraints improve neural network generalization.
problem Improving generalization in neural networks with limited data.
method Imposing topological constraints on internal representations of neural networks.
result Topological constraints lead to better mass concentration around training instances, improving generalization.
The learnability of different neural architectures can be characterized directly by computable measures of data complexity. In this paper, we reframe the problem of architecture selection as understanding how data determines the most expressive and generalizable architectures suited to that data, beyond inductive bias.…
We propose Sparse Neural Network architectures that are based on random or structured bipartite graph topologies. Sparse architectures provide compression of the models learned and speed-ups of computations, they can also surpass their unstructured or fully connected counterparts. As we show, even more compact topologi…
DCNs mimic neuronal networks for improved neural classification.
problem Lack of topological similarity between DNNs and biological neural networks.
method Developed DCNs with topologies inspired by real-world neuronal networks.
result High classification accuracy achieved by DCNs.
We use barcodes to analyze neural networks' loss surfaces, revealing important properties.
problem Understanding the topology of neural networks' loss surfaces.
method Topological data analysis using Morse complexes and barcodes.
result Barcodes of local minima are located in a small part of the loss function's range and decrease with network depth and width.
A neural network visualizes data structure and concepts.
problem Data visualization and concept understanding.
method Mixing autoencoder and classifier for multi-perspective visualization.
result The network produces different topological maps based on training as autoencoder or classifier.
Study enhances neural network predictions for wave height using topological features.
problem Challenges in predicting wave heights due to short-term and long-term factors.
method Hybridization of persistent homology with neural networks for feature engineering.
result Significant improvements in R2 score and reductions in errors for various neural network models. New topological complexity measures for neural networks.
problem Measuring complexity of neural network functions.
method Generalized piecewise-linear Morse theory applied to ReLU networks.
result Local complexity can be arbitrarily high.
Sparse neural networks can improve performance with less memory.
problem Lack of fast memory limits deep neural network performance.
method Experimented with sparse neural network topologies, including pruning-based and RadiX-Nets.
result Sparse networks achieve comparable accuracy to dense networks but suffer instability at extreme sparsity.
CT improves neural network performance on cell complex data.
problem Improving predictive performance of neural networks on complex data.
method Introducing the Cellular Transformer (CT) that generalizes graph-based transformers to cell complexes.
result CT achieves state-of-the-art performance on cell complex datasets without complex enhancements.
Enhanced neural network framework improves constraint satisfaction with topological conditioning.
problem Maintaining semantic coherence while satisfying physical and logical constraints in neuro-symbolic reasoning.
method Integrates topological conditioning with gradient stabilization mechanisms using Forman-Ricci curvature, Deep Delta Learning, and Covariance Matrix Adaptation Evolution Strategy.
result Achieves mean energy reduction to 1.15 compared to baseline values of 11.68, with 95 percent success rate.
Paper estimates neural network size needed for topology learning.
problem Estimating the smallest neural network size for topology learning.
method Using algebraic topology and Lie theory, the paper introduces a procedure based on persistent homology to determine the required dimension.
result The derived dimension is the smallest capable of capturing the topology of the data manifold.
Pruned neural networks learn digital circuits with 99% weight reduction.
problem Efficiently train deep neural networks with minimal weights.
method Constrained binarized networks to zero or one weights.
result Pruned networks achieve similar performance to standard networks with 99% weight reduction.
TVS-FNNs can approximate any continuous function on expanded input spaces.
problem Processing a broader range of inputs like sequences and matrices.
method Proving a universal approximation theorem for TVS-FNNs.
result TVS-FNNs can approximate any continuous function on expanded input spaces.
Graph neural networks improve topology control of power grids.
problem Grid congestion due to renewable energy and electrification.
method Investigated the effect of graph representation on GNN effectiveness for topology control.
result Heterogeneous graph representation outperforms homogeneous in topology control tasks.
The paper examines topological features of ReLU networks and their relation to decision boundaries and training loss.
problem Understanding the topological structure of ReLU neural network activation patterns.
method Polytope decomposition of feature space, Fiedler partition of dual graph, homology computation of cellular decomposition.
result The Fiedler partition of the dual graph correlates with decision boundaries in binary classification tasks, and similar patterns in training loss and polyhedral cell-count emerge in regression tasks.
The paper examines how neural network topology affects adversarial robustness.
problem Understanding how neural network topology influences adversarial robustness.
method Investigated the graph of input traversing all layers of a neural network, comparing clean and adversarial inputs.
result Under-optimized edges in neural network graphs are a source of adversarial vulnerability and can be used to detect adversarial inputs.
A CNN with U-Net improves structural topology optimization efficiency and generalization.
problem Structural topology optimization with reduced computation cost and improved generalization.
method Deep Convolutional Neural Network (CNN) with U-Net architecture, using SIMP-generated dataset.
result Significant reduction in computation cost with little sacrifice on design optimality.
TOGL adds topological info to GNNs, improving graph and node classification.
problem Graph neural networks lack substructure awareness, especially cycles.
method Integrates global topological information using persistent homology.
result Improves predictive performance for graph and node classification.
Neural nets learn robust geometric data representations.
problem Ensuring neural networks are robust to adversarial attacks.
method Topological Data Analysis via persistence diagrams, Lipschitz stability.
result Certified ε-robustness on ORBIT5K dataset. This paper explores a novel sparse shortcut topology in neural networks.
problem Understanding the effectiveness and characteristics of shortcut connections in neural networks.
method Investigates a novel sparse shortcut topology, demonstrating its expressivity and generalizability.
result The proposed topology enables a one-neuron-wide deep network to approximate any univariate continuous function and shows excellent generalizability.
TopoFisher learns topological summaries by maximizing Fisher information, improving parameter efficiency and inference quality.
problem Simulation-based inference misses key information in low-order statistics, especially for non-Gaussian fields.
method TopoFisher uses a differentiable persistent-homology pipeline that learns topological summaries by maximizing local Gaussian Fisher information.
result TopoFisher recovers much of the available information and outperforms fixed topological vectorizations in weak gravitational lensing.
NeuroFabric proposes a method to optimize sparse network training topologies.
problem Long training times in deep neural networks due to high memory and compute requirements.
method Developed a new sparse neural network initialization scheme and evaluated various topologies.
result Identified a single optimal topology that maximizes accuracy across different datasets.
A new method detects interactions in neural networks using topological analysis.
problem Detecting interactions between input features in neural networks.
method Topological analysis of neural network connectivity to quantify interaction strength.
result The PID algorithm outperforms state-of-the-art baselines in interaction detection tasks.
Machine learning classifies topological phases in leaky photonic lattices.
problem Classifying topological phases in leaky photonic lattices using limited data.
method A fully connected neural network trained on bulk intensity measurements.
result Accurate determination of topological properties from intensity distributions.
3D topology optimization sped up using deep learning.
problem Computationally intensive 3D topology optimization.
method 3D Convolutional Neural Network for accelerating optimization.
result Achieved 40% reduction in computation time with 96% structural accuracy.
Persistent homology reveals a topological signature of grokking in neural networks.
problem Understanding how neural networks learn and generalize from modular arithmetic tasks.
method Persistent homology on point clouds derived from embedding matrices of models trained on modular arithmetic.
result A sharp increase in first homology persistence indicates grokking, with a dominant long-lived topological feature and structured secondary features.
Neural network learns its size and structure during training.
problem Adapting neural network architecture to specific datasets.
method Flexible setup allowing neural network to learn size and topology during training.
result Trained networks achieve virtually identical performance and have learned optimal structure.
Graph Neural Networks solve topology problems in simple 3D models.
problem Deciding homeomorphism of 3-manifolds described by plumbing graphs.
method Supervised and reinforcement learning with Graph Neural Networks.
result High accuracy in determining homeomorphic 3-manifolds.
Method monitors neural networks using topological properties.
problem Detecting data shifts and out-of-distribution samples.
method Topological Uncertainty based on activation graphs.
result Validates network predictions without retraining.
Develops a new neural spike train decoding framework using topological data.
problem Decoding neural spike trains from head direction and grid cells.
method Combines simplicial complex discovery with deep learning to capture higher-order connectivity.
result Demonstrates effectiveness on head direction and trajectory prediction datasets.
This research quantifies neural networks using magnitude, a topological invariant.
problem Understanding the generalization capabilities of neural networks.
method Using a novel topological invariant called magnitude to study neural network representations.
result Magnitude dimension is theoretically connected to generalisation error and can predict it.
New framework combines simple machines into complex ones for better neural network performance.
problem Improving neural network performance with limited training data.
method Developed a framework using topology and functional analysis to combine simple machines into complex ones, and used kernel methods to find optimal architectures.
result Kernel-inspired networks can outperform classical neural networks when training data is small.
Paper proposes learnable topological features for efficient phylogenetic inference.
problem Finding appropriate topological structures for phylogenetic inference tasks requires significant design effort and domain expertise.
method Combines raw node features with graph neural networks to automatically adapt to different tasks.
result Demonstrates effectiveness and efficiency on simulated and real data phylogenetic inference tasks.
ARTree uses deep learning to infer tree topologies efficiently.
problem Efficient phylogenetic inference from tree topologies.
method Deep autoregressive model based on graph neural networks (GNNs).
result ARTree provides a flexible family of distributions over tree topologies.
New method detects uncertainty in neural networks for out-of-distribution detection.
problem Detecting out-of-distribution inputs to ensure model reliability.
method Predictive topological uncertainty (pTU) based on persistent homology.
result pTU provides a statistical framework for OOD detection.