A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This work introduces a method to compare sparse neural network topologies using graph theory.
NeuroFabric proposes a method to optimize sparse network training topologies.
The sizes of deep neural networks (DNNs) are rapidly outgrowing the capacity of hardware to store and train them. Research over the past few decades has explored the prospect of sparsifying DNNs before, during, and after training by pruning edges from the underlying topology. The resulting neural network is known as a …
The sizes of deep neural networks (DNNs) are rapidly outgrowing the capacity of hardware to store and train them. Research over the past few decades has explored the prospect of sparsifying DNNs before, during, and after training by pruning edges from the underlying topology. The resulting neural network is known as a …
Improvements in the performance of deep neural networks have often come through the design of larger and more complex networks. As a result, fast memory is a significant limiting factor in our ability to improve network performance. One approach to overcoming this limit is the design of sparse neural networks, which ca…
We propose Sparse Neural Network architectures that are based on random or structured bipartite graph topologies. Sparse architectures provide compression of the models learned and speed-ups of computations, they can also surpass their unstructured or fully connected counterparts. As we show, even more compact topologi…
Method introduces topological regularization using information filtering networks.
Many applications require sparse neural networks due to space or inference time restrictions. There is a large body of work on training dense networks to yield sparse networks for inference, but this limits the size of the largest trainable sparse model to that of the largest trainable dense model. In this paper we int…
We address the issue of estimating the topology and dynamics of sparse linear dynamic networks in a hyperparameter-free setting. We propose a method to estimate the network dynamics in a computationally efficient and parameter tuning-free iterative framework known as SPICE (Sparse Iterative Covariance Estimation). The …
The paper develops a scalable method to infer GRNs from sparse data.
Bayesian method infers network topology and dynamics from noisy, sparse measurements.
A new model for sparse networks improves consistency of score estimators.
In established network architectures, shortcut connections are often used to take the outputs of earlier layers as additional inputs to later layers. Despite the extraordinary effectiveness of shortcuts, there remain open questions on the mechanism and characteristics. For example, why are shortcuts powerful? Why do sh…
Large-scale recurrent networks have drawn increasing attention recently because of their capabilities in modeling a large variety of real-world phenomena and physical mechanisms. This paper studies how to identify all authentic connections and estimate system parameters of a recurrent network, given a sequence of node …
USN improves neural networks with uniform sparse connectivity.
New framework for neural networks converging to low loss without overparameterization.
Correlation Networks (CNs) inherently suffer from redundant information in their network topology. Bayesian Networks (BNs), on the other hand, include only non-redundant information (from a probabilistic perspective) resulting in a sparse topology from which generalizable physical features can be extracted. We advocate…
A distributed algorithm for training graph convolutional networks.
Novel algorithm learns sparse signal representations over topological spaces.
This paper begins with considering the identification of sparse linear time-invariant networks described by multivariable ARX models. Such models possess relatively simple structure thus used as a benchmark to promote further research. With identifiability of the network guaranteed, this paper presents an identificatio…
Exact causal network discovery is polynomial for sparse networks.
SVGP KAN integrates sparse variational GP with KANs for scalable probabilistic inference.
The paper uses TDA to select stocks for a sparse portfolio, improving performance across market scenarios.
SVGP KAN integrates uncertainty quantification into Kolmogorov-Arnold networks.
New pruning methods improve dynamic sparse training performance.
PEAR dynamically reconfigures agent roles to prevent persistent biases in multi-agent debates.
Paper proposes new Bayesian neural network models for efficient learning.
A new method tracks index using topological data analysis for sparse portfolios.
Grid security and open markets are two major smart grid goals. Transparency of market data facilitates a competitive and efficient energy environment, yet it may also reveal critical physical system information. Recovering the grid topology based solely on publicly available market data is explored here. Real-time ener…
This paper introduces a way to learn cross-modal convolutional neural network (X-CNN) architectures from a base convolutional network (CNN) and the training data to reduce the design cost and enable applying cross-modal networks in sparse data environments. Two approaches for building X-CNNs are presented. The base app…
Topological data analysis (TDA) has emerged as one of the most promising techniques to reconstruct the unknown shapes of high-dimensional spaces from observed data samples. TDA, thus, yields key shape descriptors in the form of persistent topological features that can be used for any supervised or unsupervised learning…
Network science provides valuable insights across numerous disciplines including sociology, biology, neuroscience and engineering. A task of major practical importance in these application domains is inferring the network structure from noisy observations at a subset of nodes. Available methods for topology inference t…
Proposes a method to infer complex network topologies from multiple graphs.
New research shows sparse topologies can lead to faster convergence in distributed optimization.
The thesis explores how to integrate machine learning with hardware constraints.
New algorithms detect communities in sparse graphs with labeled data.
High dimensional piecewise stationary graphical models represent a versatile class for modelling time varying networks arising in diverse application areas, including biology, economics, and social sciences. There has been recent work in offline detection and estimation of regime changes in the topology of sparse graph…
Predictive Sparse Manifold Transform learns dynamic video sequences.
Recent research on Bitcoin Transaction Networks reveals a growing, sparse, and core-periphery structure.
Among many unsolved puzzles in theories of Deep Neural Networks (DNNs), there are three most fundamental challenges that highly demand solutions, namely, expressibility, optimisability, and generalisability. Although there have been significant progresses in seeking answers using various theories, e.g. information bott…
Sparse curves on surfaces grow at a specific intermediate rate.
The representations learned by deep neural networks are difficult to interpret in part due to their large parameter space and the complexities introduced by their multi-layer structure. We introduce a method for computing persistent homology over the graphical activation structure of neural networks, which provides acc…
Link prediction is one of the fundamental problems in network analysis. In many applications, notably in genetics, a partially observed network may not contain any negative examples of absent edges, which creates a difficulty for many existing supervised learning approaches. We develop a new method which treats the obs…
Real world systems typically feature a variety of different dependency types and topologies that complicate model selection for probabilistic graphical models. We introduce the ensemble-of-forests model, a generalization of the ensemble-of-trees model. Our model enables structure learning of Markov random fields (MRF) …
Proposes GNAN model for detecting various network structures.
The power law has been observed in the degree distributions of many biological neural networks. Sparse deep neural networks, which learn an economical representation from the data, resemble biological neural networks in many ways. In this paper, we study if these artificial networks also exhibit properties of the power…
This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.
We present a new, far simpler family of counter-examples to Kushnirenko's Conjecture. Along the way, we illustrate a computer-assisted approach to finding sparse polynomial systems with maximally many real roots, thus shedding light on the nature of optimal upper bounds in real fewnomial theory. We use a powerful recen…