Paper introduces a new topological loss for better convergence.
problem Optimizing topological losses for model's desired topological behavior.
method Introduces a new regularized topology-aware loss function.
result Guarantees efficient optimization of the new loss function.
Topology-enhanced loss improves 3D object reconstruction from 2D images.
problem Challenges in reconstructing 3D objects from 2D images, especially capturing shape information.
method Integrates multi-scale topological features into the reconstruction loss using cubical complexes and optimal transport distance.
result Topology-aware loss substantially improves 3D reconstruction quality.
We describe loss surfaces using topological Betti numbers.
problem Understanding the complexity and structure of loss surfaces in neural networks.
method Topological analysis using Betti numbers for multilayer neural networks.
result Loss complexity is influenced by the number of hidden units and activation function.
Proposes a new method for disentangling data representations using topological analysis.
problem Learning disentangled representations for better model explainability and robustness.
method Integrates a multi-scale topological loss term into the training of deep learning models.
result Improves disentanglement scores compared to state-of-the-art methods.
We propose to study neural networks' loss surfaces by methods of topological data analysis. We suggest to apply barcodes of Morse complexes to explore topology of loss surfaces. An algorithm for calculations of the loss function's barcodes of local minima is described. We have conducted experiments for calculating barc…
New method integrates topological knowledge into data embeddings.
problem Lack of general tools to incorporate prior topological knowledge into embeddings.
method Introduces new topological losses to topologically regularize data embeddings.
result Natural representation of simple models like clusters and flares.
We propose a novel approach for preserving topological structures of the input space in latent representations of autoencoders. Using persistent homology, a technique from topological data analysis, we calculate topological signatures of both the input and latent space to derive a topological loss term. Under weak theo…
Enhances graph embeddings by preserving graph topology.
problem Node2vec struggles to recreate the topology of input graphs.
method Introduces a topological loss term to Node2vec, aligning the persistence diagram of the embedding to that of the input graph.
result Reconstructs both geometry and topology of input graphs.
Proposes a probabilistic framework for smart contract risk quantification.
problem Quantifying financial risk of smart contract cyber attacks and failures.
method Probabilistic graph-theoretical framework using bond percolation models.
result Analytical results and numerical examples for aggregate loss distribution.
Survey on optimizing topological descriptors for machine learning.
problem Optimizing topological priors in machine learning models.
method Minimizing topologically-informed losses using gradient descent.
result Various techniques enable optimization of persistence-based loss functions.
The paper examines topological features of ReLU networks and their relation to decision boundaries and training loss.
problem Understanding the topological structure of ReLU neural network activation patterns.
method Polytope decomposition of feature space, Fiedler partition of dual graph, homology computation of cellular decomposition.
result The Fiedler partition of the dual graph correlates with decision boundaries in binary classification tasks, and similar patterns in training loss and polyhedral cell-count emerge in regression tasks.
Topology applied to real world data using persistent homology has started to find applications within machine learning, including deep learning. We present a differentiable topology layer that computes persistent homology based on level set filtrations and edge-based filtrations. We present three novel applications: th…
The paper derives upper bounds on the MLE error for BTL model under general graphs.
problem Estimating the MLE of BTL model parameters with ℓ∞-loss under general graphs. method Novel upper bounds on ℓ∞ estimation error dependent on algebraic connectivity and graph topology. result Upper bounds on ℓ∞ error are sharp and match minimax lower bounds under certain graph topologies. Paper introduces new loss functions for Siamese networks using FDA.
problem Training Siamese networks with improved loss functions.
method Proposes Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC) loss functions based on FDA.
result Shows effectiveness of FDT and FDC on MNIST and histopathology datasets.
A new method for representation and metric learning on manifolds boosts performance.
problem Improving representation and metric learning on complex, non-Euclidean data.
method Atlas-based manifold representation with a modified encoder and MMD loss.
result Substantial performance boost over baseline for low-dimensional encodings.
A new method for optimal filtration learning in time-series data analysis.
problem Finding an optimal filtration for analyzing topological properties of discrete data.
method Formulated an optimization problem and proposed an algorithm for solving it.
result Derivation of the exact formula of the gradient of the loss function with respect to filtration parameters.
This paper proposes a new method for automatically selecting the optimal kernel bandwidth in density estimation.
problem The challenge of selecting the optimal kernel bandwidth in unsupervised density estimation.
method The approach uses a topology-based loss function for automated bandwidth selection.
result Demonstrates the potential of the topology-based approach across different dimensions.
As the complexity of neural network models has grown, it has become increasingly important to optimize their design automatically through metalearning. Methods for discovering hyperparameters, topologies, and learning rate schedules have lead to significant increases in performance. This paper shows that loss functions…
We introduce a representation theory for risk operations on locally compact groups in a partition of unity on a topological manifold for Markowitz-Tversky-Kahneman (MTK) reference points. We identify (1) risk torsion induced by the flip rate for risk averse and risk seeking behaviour, and (2) a structure constant or co…
Spectral graph convolutional neural networks (CNNs) require approximation to the convolution to alleviate the computational complexity, resulting in performance loss. This paper proposes the topology adaptive graph convolutional network (TAGCN), a novel graph convolutional network defined in the vertex domain. We provi…
This paper develops a novel graph convolutional network (GCN) framework for fault location in power distribution networks. The proposed approach integrates multiple measurements at different buses while taking system topology into account. The effectiveness of the GCN model is corroborated by the IEEE 123 bus benchmark…
The paper proves skip connections help neural networks avoid shallow local minima.
problem Understanding how skip connections affect the loss landscape of deep neural networks.
method Theoretical analysis of the topology of loss landscapes of deep ReLU neural networks with skip connections.
result Skip connections help control the connectedness of sub-level sets, avoiding shallow local minima.
Consider a financial market in which an agent trades with utility-induced restrictions on wealth. For a utility function which satisfies the condition of reasonable asymptotic elasticity at −∞ we prove that the utility-based super-replication price of an unbounded (but sufficiently integrable) contingent claim i…
A novel method integrates feature and topology views for unsupervised graph representation learning.
problem Lack of mutual information across feature and topology views in graph representation learning.
method Proposes a multi-view representation learning module and a common representation learning module using mutual information maximization and reconstruction loss minimization.
result Demonstrates effectiveness in integrating feature and topology views, achieving comparable or better performance than supervised methods.
We propose a permutation-invariant loss function designed for the neural networks reconstructing a set of elements without considering the order within its vector representation. Unlike popular approaches for encoding and decoding a set, our work does not rely on a carefully engineered network topology nor by any addit…
We analyze the landscape of empirical risk minimization for high-dimensional models, predicting phase transitions and critical point properties.
problem Understanding the complexity and structure of high-dimensional empirical risk landscapes.
method Using the Kac-Rice formula, we analyze the expected number of critical points and their spectral properties, providing detailed predictions.
result We derive complete topological phase diagrams for the phase retrieval problem, predicting BBP-type transitions and critical point stability.
Researchers improve visualization of neural network loss landscapes.
problem Understanding neural network generalization performance.
method Novel 'jump and retrain' procedure, non-linear dimensionality reduction (PHATE), computational homology.
result Improved visualization and quantification of neural network generalization performance.
TopInG improves graph interpretability using persistent homology.
problem Lack of interpretability in Graph Neural Networks (GNNs).
method TopInG uses persistent homology to identify persistent rationale subgraphs in graphs.
result TopInG improves predictive accuracy and interpretability compared to state-of-the-art methods.
Improved retinal vessel segmentation with topology preservation trade-off.
problem Retinal vessel segmentation accuracy and topology preservation trade-off.
method Topology preserving term in the loss function and orientation score guided convolutional module.
result Model achieves higher topological accuracy at the expense of lower overlap metrics.
Adaptive rerouting reshapes impacts of maritime chokepoint disruptions
problem How disruptions to shipping traffic at chokepoints affect global economy
method Empirically calibrated full-scale agent-based model of global commercial shipping fleet
result Rerouting changes arrival losses under chokepoint closures
New bounds link SGD's generalization to heavy tails without topological assumptions.
problem Linking SGD's generalization error to heavy tails without additional assumptions.
method Developed Wasserstein stability bounds for heavy-tailed SDEs and their discretizations, converting to generalization bounds.
result Generalization bounds for a broader class of objective functions, including non-convex functions, without topological assumptions.
The paper explores how the depth of neural networks affects their ability to represent data accurately.
problem Understanding the implicit bias and rank of neural networks with large depth.
method Analyzing the convergence of representation cost to a notion of rank as network depth increases, and investigating conditions for recovering the true rank of data.
result There is a range of network depths where the true rank of data is recovered, and this affects the topology of class boundaries.
One of the main drawbacks of the practical use of neural networks is the long time required in the training process. Such a training process consists of an iterative change of parameters trying to minimize a loss function. These changes are driven by a dataset, which can be seen as a set of labelled points in an n-dime…
A system for Operational Risk management based on the computational paradigm of Bayesian Networks is presented. The algorithm allows the construction of a Bayesian Network targeted for each bank using only internal loss data, and takes into account in a simple and realistic way the correlations among different processe…
Among many unsolved puzzles in theories of Deep Neural Networks (DNNs), there are three most fundamental challenges that highly demand solutions, namely, expressibility, optimisability, and generalisability. Although there have been significant progresses in seeking answers using various theories, e.g. information bott…
Recent research implies that training and inference of deep neural networks (DNN) can be computed with low precision numerical representations of the training/test data, weights and gradients without a general loss in accuracy. The benefit of such compact representations is twofold: they allow a significant reduction o…
While many approaches to make neural networks more fathomable have been proposed, they are restricted to interrogating the network with input data. Measures for characterizing and monitoring structural properties, however, have not been developed. In this work, we propose neural persistence, a complexity measure for ne…
TopoFisher learns topological summaries by maximizing Fisher information, improving parameter efficiency and inference quality.
problem Simulation-based inference misses key information in low-order statistics, especially for non-Gaussian fields.
method TopoFisher uses a differentiable persistent-homology pipeline that learns topological summaries by maximizing local Gaussian Fisher information.
result TopoFisher recovers much of the available information and outperforms fixed topological vectorizations in weak gravitational lensing.
This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network. While a denser (sparser) network topology results in faster (slower) error convergence in terms of iterations, it incurs more (less) communication …
New framework for neural networks converging to low loss without overparameterization.
problem Training deep neural networks without overparameterization assumptions.
method Construction of random sparse lifts and analysis using algebraic topology and random graph theory.
result Provable convergence to low loss for large sparse neural networks.
Unified geometric flows improve deep learning efficiency and simplify neural network topologies.
problem Improving deep learning performance and simplifying neural network structures.
method Proposes a thermodynamically coupled Ricci flow that dynamically adapts parameter space geometry to loss landscape topology, enabling automated singularity resolution and providing entanglement entropy bounds.
result Demonstrates 2.1× convergence acceleration and 63% topological simplification while maintaining O(NlogN) complexity, outperforming Riemannian baselines by 15.2% in few-shot accuracy. Proposes using continuum percolation to analyze data manifolds and improve generative models.
problem Disentangling geometric support from probability distributions in high-dimensional data.
method Establishes a correspondence between topological phase transitions of random geometric graphs and data manifolds, using Percolation Shift metric.
result Demonstrates that Percolation Shift metric captures structural pathologies like mode collapse and guides training to prevent manifold shrinkage and improve fidelity.
We analyse finite-time singularities of the Teichmüller harmonic map flow -- a natural gradient flow of the harmonic map energy -- and find a canonical way of flowing beyond them in order to construct global solutions in full generality. Moreover, we prove a no-loss-of-topology result at finite time, which completes th…
CRAUM-Net improves salient object detection with context and uncertainty modeling.
problem Accurate salient object detection with precise boundary delineation.
method Contextual Recursive Attention with Uncertainty Modeling, multi-scale context aggregation, attention mechanisms, edge-aware decoder, Monte Carlo Dropout.
result Superior performance in producing accurate and reliable saliency maps.
How, and to what extent, does an interconnected financial system endogenously amplify external shocks? This paper attempts to reconcile some apparently different views emerged after the 2008 crisis regarding the nature and the relevance of contagion in financial networks. We develop a common framework encompassing seve…
DMT enhances deep neural networks to better preserve data structures.
problem Preserving geometric, topological, and distributional structures of data in NLDR.
method Deep manifold transformation (DMT) using cross-layer LGP constraints.
result DMT networks outperform existing NLDR methods in preserving data structures.
PTOPOFL uses topological descriptors to protect privacy in federated learning.
problem Privacy and data reconstruction attacks in federated learning.
method PTOPOFL replaces gradient communication with persistent homology feature vectors for privacy and topology-guided aggregation.
result PTOPOFL achieves higher AUC and reduces reconstruction risk compared to gradient sharing.
While it has become common to perform automated translations on natural language, performing translations between different representations of mathematical formulae has thus far not been possible. We implemented the first translator for mathematical formulae based on recursive neural networks. We chose recursive neural…