Proposes deep graph persistence to address neural persistence issues in deep learning.
problem Variance of weights and lack of spatial structure in deep neural networks impact neural persistence.
method Extends neural persistence to the whole network, considering interactions between layers.
result Deep graph persistence alleviates variance-related issues and captures persistent paths through the network.
While many approaches to make neural networks more fathomable have been proposed, they are restricted to interrogating the network with input data. Measures for characterizing and monitoring structural properties, however, have not been developed. In this work, we propose neural persistence, a complexity measure for ne…
Regularizes persistent homology gradients for neural network integration.
problem Ill-posed inverse problem in computing gradients of persistent homology.
method Regularization through a grouping term to define gradients for larger entities.
result Ensures gradients are defined with respect to larger entities, not individual points.
Persistent neurons improve neural network optimization by leveraging previous solutions.
problem Improving neural network optimization under different initialization and data distributions.
method Persistent neurons use information from previous converged solutions to explore new landscapes and avoid local minima.
result Persistent neurons converge to more optimal solutions and improve model performance under various initializations.
Persistent homology reveals a topological signature of grokking in neural networks.
problem Understanding how neural networks learn and generalize from modular arithmetic tasks.
method Persistent homology on point clouds derived from embedding matrices of models trained on modular arithmetic.
result A sharp increase in first homology persistence indicates grokking, with a dominant long-lived topological feature and structured secondary features.
New method enhances graph neural networks using contractions and hourglass persistence.
problem Limitations of traditional persistent homology in graph neural networks.
method Hourglass Persistence, Contraction Homology, contractions as a topological operation.
result Hourglass Persistence boosts expressivity, learnability, and stability in graph representation learning.
We outline a detection method for adversarial inputs to deep neural networks. By viewing neural network computations as graphs upon which information flows from input space to out- put distribution, we compare the differences in graphs induced by different inputs. Specifically, by applying persistent homology to these …
New method uses DTW to evaluate neural network forecasts of geomagnetic indices.
problem Evaluation metrics fail to capture persistence behavior in neural network forecasts.
method Dynamic Time Warping (DTW) to measure time series similarity, training neural networks to remove persistence.
result DTW reveals persistence behavior in neural network forecasts, confirming visual inspection.
When an online learning algorithm is used to estimate the unknown parameters of a model, the signals interacting with the parameter estimates should not decay too quickly for the optimal values to be discovered correctly. This requirement is referred to as persistency of excitation, and it arises in various contexts, s…
This paper evaluates fractal dimension and persistent homology for neural network generalization.
problem Bounding and predicting the generalization gap of neural networks.
method Empirical evaluation of fractal dimension and persistent homology as generalization measures.
result Fractal dimension and persistent homology fail to predict generalization of models trained from poor initializations.
Persistence diagrams, the most common descriptors of Topological Data Analysis, encode topological properties of data and have already proved pivotal in many different applications of data science. However, since the (metric) space of persistence diagrams is not Hilbert, they end up being difficult inputs for most Mach…
New method improves neural network robustness to adversarial attacks.
problem Improving adversarial robustness of neural networks.
method Inspired by adaptive control theory, the approach uses persistency of excitation to constrain gradient descent updates.
result Networks trained with the PoE-motivated learning rate schedule are significantly more robust to adversarial attacks.
TopInG improves graph interpretability using persistent homology.
problem Lack of interpretability in Graph Neural Networks (GNNs).
method TopInG uses persistent homology to identify persistent rationale subgraphs in graphs.
result TopInG improves predictive accuracy and interpretability compared to state-of-the-art methods.
A new method detects interactions in neural networks using topological analysis.
problem Detecting interactions between input features in neural networks.
method Topological analysis of neural network connectivity to quantify interaction strength.
result The PID algorithm outperforms state-of-the-art baselines in interaction detection tasks.
Study forecasts U.S. bond index using deep learning, finding persistence is key.
problem Forecasting U.S. aggregate bond index with deep learning methods.
method Constructed a stationary but maximally persistent representation of the bond index, evaluated using MLPs and CNNs.
result Deep learning models outperform traditional methods in short-horizon forecasting of bond indices.
PMP extends GNNs to handle past states efficiently.
problem Efficient querying of data structures dependent on previous states.
method Persistent Message Passing (PMP) which persists past states through new nodes.
result Significantly outperforms GNNs in handling out-of-distribution data.
Persistent entropy detects phase transitions in complex systems.
problem Detecting phase transitions in complex systems.
method Established a general theorem for persistent entropy to reliably detect phase transitions, introduced operational framework for finite-time computations.
result Persistent entropy exhibits an asymptotically non-vanishing gap across phases, robust numerical signatures across experiments.
A new model separates persistence and transition priors in HDP-HMM.
problem Limitation of sticky HDP-HMM in expressing different persistence strengths.
method Developed a disentangled sticky HDP-HMM (DS-HDP-HMM) with novel Gibbs sampling algorithms.
result DS-HDP-HMM outperforms sticky HDP-HMM and HDP-HMM on synthetic and real data.
Weather and atmospheric patterns are often persistent. The simplest weather forecasting method is the so-called persistence model, which assumes that the future state of a system will be similar (or equal) to the present state. Machine learning (ML) models are widely used in different weather forecasting applications, …
PHLP uses persistent homology to interpret graph link prediction.
problem Interpreting why graph neural network models perform well in link prediction.
method Employing persistent homology to analyze graph topology and extract features.
result PHLP outperforms state-of-the-art models on most benchmark datasets.
Enhances graph neural networks with spectral and topological information.
problem Improving graph neural networks' expressivity beyond Weisfeiler-Leman hierarchy.
method Integrates spectral information into Persistent Homology diagrams.
result SpectRe is strictly more expressive than PH and spectral information alone.
Paper estimates neural network size needed for topology learning.
problem Estimating the smallest neural network size for topology learning.
method Using algebraic topology and Lie theory, the paper introduces a procedure based on persistent homology to determine the required dimension.
result The derived dimension is the smallest capable of capturing the topology of the data manifold.
Language Models (LMs) are important components in several Natural Language Processing systems. Recurrent Neural Network LMs composed of LSTM units, especially those augmented with an external memory, have achieved state-of-the-art results. However, these models still struggle to process long sequences which are more li…
Novel method decorrelates neurons for better deep learning model generalization.
problem High correlations between neurons limit deep learning model generalization.
method Regularization terms from minimum spanning tree of neuron cliques, using correlation dissimilarities.
result Our regularizers outperform existing methods and minimize neuron redundancies.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
TOGL adds topological info to GNNs, improving graph and node classification.
problem Graph neural networks lack substructure awareness, especially cycles.
method Integrates global topological information using persistent homology.
result Improves predictive performance for graph and node classification.
Study enhances neural network predictions for wave height using topological features.
problem Challenges in predicting wave heights due to short-term and long-term factors.
method Hybridization of persistent homology with neural networks for feature engineering.
result Significant improvements in R2 score and reductions in errors for various neural network models. Generative neural network models, including Generative Adversarial Network (GAN) and Auto-Encoders (AE), are among the most popular neural network models to generate adversarial data. The GAN model is composed of a generator that produces synthetic data and of a discriminator that discriminates between the generator's …
PHINN: A generative model for rare-event time series using persistent homology
problem Generating rare events in time series
method Flow-matching framework with dynamic Betti curves and persistence landscape loss
result Outperforms statistical and diffusion baselines in topological fidelity and tail coverage
Finding an optimal parameter of a black-box function is important for searching stable material structures and finding optimal neural network structures, and Bayesian optimization algorithms are widely used for the purpose. However, most of existing Bayesian optimization algorithms can only handle vector data and canno…
Deep neural networks are known to be fragile to small adversarial perturbations. This issue becomes more critical when a neural network is interconnected with a physical system in a closed loop. In this paper, we show how to combine recent works on neural network certification tools (which are mainly used in static set…
Approaches for approximating persistent homology for large datasets.
problem Inability to compute persistent homology for large datasets.
method Multiple subsampling framework for statistical approximation of persistent homology.
result Derivation of finite sample convergence rates for empirical means of persistent homology.
Persistence landscapes map persistence diagrams into a function space, which may often be taken to be a Banach space or even a Hilbert space. In the latter case, it is a feature map and there is an associated kernel. The main advantage of this summary is that it allows one to apply tools from statistics and machine lea…
Paper proves k-means clustering works on persistence diagrams.
problem Complex geometry of persistence diagram space.
method Proves convergence of k-means on persistence diagram space. result Performance of k-means on persistence diagrams and measures is superior. Formula for interleaving distance of rectangle persistence modules.
problem Calculating distances between rectangle persistence modules.
method Formulas based on rectangle geometry, extended to decomposable modules.
result Closed formulas for interleaving and bottleneck distances.
Monotonic Linear Interpolation property in neural networks persists despite non-convexity.
problem Understanding the geometric properties of neural network loss landscapes.
method Tools from differential geometry to analyze the monotonicity of neural network weights.
result Sufficient conditions for the Monotonic Linear Interpolation property under mean squared error.
TopoFisher learns topological summaries by maximizing Fisher information, improving parameter efficiency and inference quality.
problem Simulation-based inference misses key information in low-order statistics, especially for non-Gaussian fields.
method TopoFisher uses a differentiable persistent-homology pipeline that learns topological summaries by maximizing local Gaussian Fisher information.
result TopoFisher recovers much of the available information and outperforms fixed topological vectorizations in weak gravitational lensing.
This paper demonstrates the flaws of co-persistence theory proposed by Bollerslev and Engle (1993) which cause the theory can hardly be applied. With the introduction of the half-life of decay coefficient as the measure of the persistence, and both the weak definition of persistence and co-persistence in variance, this…
Obstructive sleep apnea is a serious condition causing a litany of health problems especially in the pediatric population. However, this chronic condition can be treated if diagnosis is possible. The gold standard for diagnosis is an overnight sleep study, which is often unobtainable by many potentially suffering from …
Optimizes wavelets for graph classification using spectral wavelet signatures and persistence diagrams.
problem Graph classification with geometric properties encoded in persistence diagrams.
method Optimizes spectral wavelets for graph datasets to capture best-suited features for classification.
result Competitive performance in graph classification problems compared to other persistence-based architectures.
New method for analyzing multiparameter persistence modules from smooth functions.
problem Analyzing multiparameter persistence modules from smooth functions.
method Generalized Morse theory applied to cobordism and Cerf theory.
result Complete description of persistence modules as direct sums of indecomposables.
Develops robust persistence diagrams using kernel methods.
problem Persistence diagrams are sensitive to data perturbations.
method Constructs robust persistence diagrams from superlevel filtrations of robust density estimators using reproducing kernels.
result Robust persistence diagrams are consistent estimators in bottleneck distance.
This paper interprets critical scales in persistent homology for compact metric spaces.
problem Understanding critical scales in persistent homology for general compact metric spaces.
method Analyzing local minima of the distance function and their impact on persistence.
result Each decrease in zero-dimensional persistence and increase in one-dimensional persistence is induced by local minima of the distance function.
New method estimates deep neural network's intrinsic dimension for better generalization.
problem Estimating intrinsic dimension of deep neural networks for generalization.
method Topological data analysis (TDA) and persistent homology (PHD).
result Efficient algorithm to estimate PHD in deep neural networks.
MuRiT efficiently computes multi-parameter persistence barcodes.
problem Efficient computation of multi-parameter persistent homology.
method Vietoris-Rips transformation to reduce multi-parameter to single-parameter computation.
result MuRiT computes pathwise persistence barcodes for multi-filtered flag complexes.
Neural ARFIMA model improves exchange rate forecasting for BRIC economies.
problem Forecasting exchange rates for emerging markets with long-term memory and nonlinear dynamics.
method Integrates ARFIMA for long-memory with neural networks for nonlinear approximation.
result NARFIMA model outperforms benchmarks in BRIC exchange rate forecasting.
Persistent homology can recognize knotting in curves.
problem Recognizing knotting in curves
method Compute one-dimensional persistent homology, extract cycle representatives, and assign a hypergraph curvature-based score.
result Systematic differences between knotted and unknotted structures are revealed.
This article analyzes the relationship between co-persistence and hedging which indicates co-persistence ratio is just the long-term hedging ratio. The new method of exhaustive search algorithm for deriving co-persistence ratio is derived in the article. And we also develop a new hedging strategy of combining co-persis…