Understanding properties of deep neural networks is an important challenge in deep learning. In this paper, we take a step in this direction by proposing a rigorous way of verifying properties of a popular class of neural networks, Binarized Neural Networks, using the well-developed means of Boolean satisfiability. Our…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “network properties”
Aggregate network properties such as cluster cohesion and the number of bridge nodes can be used to glean insights about a network's community structure, spread of influence and the resilience of the network to faults. Efficiently computing network properties when the network is fully observed has received significant …
Monotonic Linear Interpolation property in neural networks persists despite non-convexity.
problem Understanding the geometric properties of neural network loss landscapes.
method Tools from differential geometry to analyze the monotonicity of neural network weights.
result Sufficient conditions for the Monotonic Linear Interpolation property under mean squared error.
Dynamical analysis of financial stocks network: improving forecasting using network propertiesq-fin.ST
Network analysis improves stock return forecasting.
problem Improving stock return forecasting using network properties.
method Network analysis of stock return correlations, using individual and global properties of stocks.
result 50% improvement in R2 score for long-term stock returns forecasting, 3% for short-term.
We present a novel condition, which we term the net- work nullspace property, which ensures accurate recovery of graph signals representing massive network-structured datasets from few signal values. The network nullspace property couples the cluster structure of the underlying network-structure with the geometry of th…
The staircase property aids deep learning by guiding hierarchical feature learning.
problem Understanding how hierarchical structure influences deep learning performance.
method Defined and proved the staircase property for Boolean hypercube functions, and showed its learnability by layerwise stochastic coordinate descent.
result Staircase functions can be learned in polynomial time using layerwise stochastic coordinate descent on regular neural networks.
The paper analyzes deep ReLU CNNs' approximation properties in 2D space.
problem Establishing approximation properties for deep ReLU CNNs.
method Analysis based on decomposition theorem for convolutional kernels, properties of ReLU activation, and connections with one-hidden-layer ReLU NNs.
result Universal approximation theorem for deep ReLU CNNs with classic structure.
Deep neural networks are powerful machine learning approaches that have exhibited excellent results on many classification tasks. However, they are considered as black boxes and some of their properties remain to be formalized. In the context of image recognition, it is still an arduous task to understand why an image …
Graph networks struggle with multi-task learning due to varying property loss surface curvatures.
problem Graph networks underperform in multi-task learning for crystal and molecule properties.
method Assessed curvature of property loss surfaces via spectral properties of Hessians, matrix-free using randomized numerical linear algebra.
result Varying curvature of property loss surfaces explains graph networks' multi-task learning inefficiency.
How can we model networks with a mathematically tractable model that allows for rigorous analysis of network properties? Networks exhibit a long list of surprising properties: heavy tails for the degree distribution; small diameters; and densification and shrinking diameters over time. Most present network models eithe…
Generative neural network designs novel 3D molecules with specified properties.
problem Designing molecules with desired properties in chemistry.
method Conditional generative neural network for 3D molecular structures.
result Demonstrated utility in generating novel molecules with specified motifs or composition.
Neural networks are increasingly deployed in real-world safety-critical domains such as autonomous driving, aircraft collision avoidance, and malware detection. However, these networks have been shown to often mispredict on inputs with minor adversarial or even accidental perturbations. Consequences of such errors can …
This paper improves hierarchical community detection efficiency using local structural properties.
problem Efficiency of hierarchical community detection methods in large networks.
method Use of local structural network properties as proxies to improve efficiency.
result Achieves competitive results in modularity with improved efficiency.
Automates GNN design for molecular property prediction.
problem Designing and tuning GNN architectures for molecular property prediction is labor-intensive.
method Developed a NAS approach to automatically discover high-performing GNN architectures for MPNNs.
result Automatically discovered MPNNs outperform manually designed GNNs in molecular property prediction.
Network embedding aims to learn the low-dimensional representations of vertexes in a network, while structure and inherent properties of the network is preserved. Existing network embedding works primarily focus on preserving the microscopic structure, such as the first- and second-order proximity of vertexes, while th…
New method filters large networks from financial data to reveal key subnetworks.
problem Filtering large dimensional networks to isolate key constituents.
method Exploits spectral properties of high-dimensional data networks, tuning for sparsity and consistency.
result Shows method can interpolate between zero and maximal filtering, preserving spectral properties.
Comparison of Atom Representations in Graph Neural Networks for Molecular Property Predictionphysics.chem-ph
Study compares atom representations in graph neural networks for molecular properties.
problem Incorrect attribution of results in molecular property prediction due to varying atom features.
method Evaluated multiple atom representations on free energy, solubility, and metabolic stability predictions.
result Different atom representations can lead to varying predictive performance in graph neural networks.
Lectures on deep learning properties in infinite and large-width networks.
problem Understanding deep neural networks in extreme width conditions.
method Analysis of random deep neural networks, connections to linear models, kernels, and Gaussian processes, perturbative and non-perturbative treatments.
result Properties and behaviors of deep neural networks in the infinite-width limit and large-width regime.
Extends neural net safety guarantees by proving structural properties.
problem Proving formal guarantees for complex DNN architectures.
method Proves structural properties related to neural net structure to infer safety properties.
result Identifies a larger region of input space for safety properties.
Topological Properties of Stock Networks Based on Random Matrix Theory in Financial Time Seriesq-fin.ST
We investigated the topological properties of stock networks through a comparison of the original stock network with the estimated stock network from the correlation matrix created by the random matrix theory (RMT). We used individual stocks traded on the market indices of Korea, Japan, Canada, the USA, Italy, and the …
Rotationally equivariant convolutions improve molecular property prediction.
problem Predicting molecular properties using graph neural networks.
method Ablation study with rotationally equivariant and invariant convolutions on QM9 data set.
result Rotationally equivariant layers decrease test error by an average of 23%.
It has been noted in existing literature that over-parameterization in ReLU networks generally improves performance. While there could be several factors involved behind this, we prove some desirable theoretical properties at initialization which may be enjoyed by ReLU networks. Specifically, it is known that He initia…
Graph neural networks leverage graph filters to learn from network data.
problem Learning from network data with graph structure.
method Characterize graph neural networks using graph signal processing and graph convolutional filters.
result Graph neural networks have permutation equivariance and stability to topology changes.
Study examines dependence properties of Bayesian neural network units in finite-width networks.
problem Understanding dependence properties of hidden units in practical finite-width Bayesian neural networks.
method Theoretical analysis and empirical evaluation of depth and width impacts.
result Hidden units in finite-width Bayesian neural networks are dependent, contrary to the infinite-width limit assumption.
Improved molecular property prediction using WL embedding in GNNs.
problem Limited performance of GNNs in predicting molecular properties.
method Explored Weisfeiler-Lehman (WL) embedding to replace GNN layers, enhancing representability and performance.
result WL embedding consistently improves GNN performance across multiple datasets.
In all empirical-network studies, the observed properties of economic networks are informative only if compared with a well-defined null model that can quantitatively predict the behavior of such properties in constrained graphs. However, predictions of the available null-model methods can be derived analytically only …
Study shows infoGAN's generalization error bound for two-layer networks.
problem Understanding generalization error in infoGAN for two-layer neural networks.
method Analyzes the difference between empirical and population objective functions, derives Rademacher complexity bounds.
result Derives error bound for infoGAN's generalization error in a two-layer network.
Paper verifies RNNs using automata learning and model checking.
problem Verifying the correctness of RNNs is challenging.
method Learn a deterministic finite automaton from RNN, use model checking for verification.
result Can discover and generalize counterexamples to faulty flows.
Understanding biological network dynamics is a fundamental issue in various scientific and engineering fields. Network theory is capable of revealing the relationship between elements and their propagation; however, for complex collective motions, the network properties often transiently and complexly change. A fundame…
Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
PG-EVIKAL refines molecular property predictions using neighbor fusion and evidential neural networks.
problem Improving molecular property predictions using test-time neighbor fusion.
method Adapting evidential neural networks to refine predictions by re-ranking structurally similar neighbors.
result PG-EVIKAL reduces RMSE on 14 out of 16 molecular datasets, improving calibration and sequential refinement.
Topological constraints improve neural network generalization.
problem Improving generalization in neural networks with limited data.
method Imposing topological constraints on internal representations of neural networks.
result Topological constraints lead to better mass concentration around training instances, improving generalization.
The paper analyzes network models with binary values and sub-Gamma noise, deriving asymptotic properties.
problem Analyzing network models with binary values and sub-Gamma noise.
method Derives asymptotic properties of network models with binary values and sub-Gamma noise.
result Established asymptotic consistency and normality of parameter estimators in network models.
The study investigates how data variability impacts the generalization of neural networks.
problem Understanding the impact of data variability on neural network generalization.
method Developed a field-theoretic formalism to compute generalization properties of neural networks, focusing on data variability.
result Data variability leads to non-Gaussian action, affecting the learning curve and generalization properties of neural networks.
Molecule property prediction is a fundamental problem for computer-aided drug discovery and materials science. Quantum-chemical simulations such as density functional theory (DFT) have been widely used for calculating the molecule properties, however, because of the heavy computational cost, it is difficult to search a…
Unified platform SOCRATES for neural network analysis.
problem Analyzing neural networks for bugs and fairness.
method Standardized format, assertion language, and multiple analysis algorithms.
result Unified platform for neural network analysis.
Hardness proven for neural networks with natural weights.
problem Difficulty in learning neural networks with weights from natural distributions.
method Proved hardness for depth-2 networks with natural weights distributions.
result Most networks are hard to learn with natural weights.
Study on hidden units in finite Bayesian neural networks and their tail properties.
problem Understanding the behavior of hidden units in finite Bayesian neural networks.
method Introduced a generalized Weibull-tail property to describe hidden units tails.
result Unit priors become heavier-tailed going deeper, providing insights into finite Bayesian neural networks.
GRAND ensures node-level differential privacy for network data.
problem Lack of node-level differential privacy for network data.
method Proposes GRAND, the first mechanism for releasing networks with node-level differential privacy and preserving structural properties.
result GRAND releases networks while ensuring node-level differential privacy and preserving structural properties.
The paper discusses regularization properties of artificial data for deep learning. Artificial datasets allow to train neural networks in the case of a real data shortage. It is demonstrated that the artificial data generation process, described as injecting noise to high-level features, bears several similarities to e…
Machine Learning Partners in Criminal Networksphysics.soc-ph
Machine learning predicts criminal networks' missing partnerships and future behavior.
problem Predicting and understanding criminal networks' properties and future behavior.
method Combining graph representation learning and machine learning methods.
result Outstanding accuracy in predicting missing criminal partnerships and future behavior.
Deep ResNets exhibit distinct scaling properties with depth, challenging neural ODE models.
problem Understanding the scaling properties of deep ResNets and their relation to neural ODEs.
method Detailed numerical experiments on weights trained by stochastic gradient descent.
result Deep ResNets can exhibit different scaling regimes, including stochastic differential equations or neither, challenging the neural ODE model.
Graph hypernetworks improve molecule property prediction and classification.
problem Improving molecule property prediction and classification using graph neural networks.
method Replacing underlying networks with hypernetworks and addressing training instability.
result Demonstrated state-of-the-art performance in various benchmarks.
This paper proposes a new algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i.e., networks that provably satisfy some desired input-output properties. The key idea is to simultaneously train two networks: a predictor network that performs the task at hand,e.g., predicting…
New research shows graph embeddings fail to capture key network properties.
problem Graph embeddings fail to capture salient properties of complex networks.
method Mathematical proof and empirical study of various embedding techniques.
result Any successful graph embedding must have a rank nearly linear in the number of vertices.
This study reveals a Min-Max property in LeNet's convolutional layers, enhancing adversarial robustness.
problem Uncertainty in the connection weights of convolutional layers in neural networks.
method Demonstrates the Min-Max property through back propagation-based training and a simplified convolution formulation.
result The Min-Max property improves adversarial robustness, indicating a stronger uncertainty in the model parameters.
In recent years, methods from network science are gaining rapidly interest in economics and finance. A reason for this is that in a globalized world the interconnectedness among economic and financial entities are crucial to understand and networks provide a natural framework for representing and studying such systems.…
We demonstrate that graphs embedded on surfaces are a powerful and practical tool to generate, characterize and simulate networks with a broad range of properties. Remarkably, the study of topologically embedded graphs is non-restrictive because any network can be embedded on a surface with sufficiently high genus. The…