This paper analyzes the generalization risk of unrolled neural networks using Stein's Unbiased Risk Estimator.
problem Analyzing the generalization risk of unrolled neural networks and its relationship to network design and train sample size.
method Using Stein's Unbiased Risk Estimator (SURE), the paper analyzes the generalization risk with bias and variance components for recurrent unrolled networks, focusing on the degrees-of-freedom (DOF) component and the trace of the end-to-end network Jacobian.
result DOF is well-approximated by the weighted path sparsity of the network under incoherence conditions on the trained weights, and DOF increases with train sample size and converges to the generalization risk for both recurrent and non-recurrent schemes.
Vehicular Ad-hoc NETworks (VANET) can efficiently detect traffic congestion, but detection is not enough because congestion can be further classified as recurrent and non-recurrent congestion (NRC). In particular, NRC in an urban network is mainly caused by incidents, workzones, special events and adverse weather. We p…
TriTPP models enable faster and more flexible event data modeling.
problem Inflexibility and slow sampling in traditional TPP models.
method Triangular Maps and Normalizing Flows for parallel sampling and likelihood computation.
result TriTPP models achieve orders of magnitude faster sampling while maintaining flexibility.
New findings show modern neural networks have finite sample complexity in o-minimal structures.
problem Understanding the learnability of modern neural networks in a broad context.
method Analyzing feedforward neural networks definable in o-minimal structures.
result Modern neural networks, including MLPs, CNNs, GNNs, and transformers, have finite sample complexity in the agnostic PAC setting.
New method for Transformer models to encode position information without sequential bias.
problem Lack of flexible and learnable position encoding for Transformer models.
method Continuous dynamical model to learn position encoding.
result Consistent improvements over baselines in various NLP tasks.
The present paper deals with the proper existence of a generalized class of recurrent manifolds, namely, hyper-generalized recurrent manifolds. We have established the proper existence of various generalized notions of recurrent manifolds. For this purpose we have presented a metric and computed its curvature propertie…
Neural Physicist learns physical dynamics from images.
problem Learning meaningful physical state representations and accurate state transitions from image sequences.
method Neural Physicist uses VAE for state extraction, NP for parameters, and SSM for dynamics.
result Achieves long-term predictions and identifies system degrees of freedom.
Regularizes RNNs to be invariant to input order.
problem Making RNNs invariant to input order.
method Stochastic regularization to enforce permutation invariance.
result Improves model performance on permutation invariant tasks.
Transformers simulate finite-state automata with fewer layers.
problem How do shallow, non-recurrent Transformers simulate complex computations?
method Hierarchical reparameterization of recurrent dynamics to simulate automata.
result Polynomial-sized, O(logT)-depth solutions exist and are common. Study invariant measures on measured laminations for subgroups of mapping class group.
problem Classify invariant Radon measures on space of measured laminations for subgroups of mapping class group.
method Geometric approach, focusing on recurrent measured laminations, explicitly constructing ergodic measures.
result Show uniquely ergodic for divergence-type subgroups, generalize results for full mapping class group.
This paper proposes a real-time signal plan recommendation system for traffic incidents.
problem Limited effectiveness of traffic incident management due to late response and workload.
method Decomposes recommendation task into real-time traffic prediction and plan association, learning from historical data and metric learning.
result Precision score of 96.75% and recall of 87.5% on testing plan, with 22.5 minutes lead time ahead of Waze alerts.
The study connects geodesic flows on Riemann surfaces to random walks on their dual graphs.
problem Understanding ergodicity of geodesic flows on infinite Riemann surfaces.
method Analyzing random walks on the dual graph of pants decompositions.
result Equivalence between ergodicity of geodesic flows and recurrence of random walks.
RFN improves GCNs for road networks, outperforming state-of-the-art by 21%-40%.
problem Leveraging the structure of road networks effectively in machine learning tasks.
method Introducing RFN, a novel GCN specifically designed for road networks.
result RFN outperforms state-of-the-art GCNs by 21%-40% on road network tasks.
This survey clarifies dynamic network terminology and reviews GNN models for dynamic networks.
problem Ambiguity in dynamic network terminology and lack of GNN models for dynamic networks.
method Established consistent terminology and notation for dynamic networks, reviewed GNN models.
result Comprehensive survey of dynamic graph neural network models.
From the perspective of network analysis, the ubiquitous networks are comprised of regular and irregular components, which makes uncovering the complexity of network structures to be a fundamental challenge. Exploring the regular information and identifying the roles of microscopic elements in network data can help us …
DCNs mimic neuronal networks for improved neural classification.
problem Lack of topological similarity between DNNs and biological neural networks.
method Developed DCNs with topologies inspired by real-world neuronal networks.
result High classification accuracy achieved by DCNs.
DANE adapts network embeddings across multiple domains.
problem Learning embeddings for multiple networks without transferability.
method Graph Convolutional Network with adversarial learning.
result DANE achieves superior performance in cross-network domain adaptation.
Capsule networks are vulnerable to adversarial attacks, similar to convolutional neural networks.
problem Vulnerability of capsule networks to adversarial attacks.
method Compared capsule networks to convolutional neural networks using various adversarial attacks.
result Capsule networks are vulnerable to adversarial attacks, similar to convolutional neural networks.
Chemical networks outperform spiking neural networks in classification tasks.
problem Learning tasks with spiking neural networks require hidden layers, which are computationally expensive.
method Used deterministic mass-action kinetics to prove chemical reaction networks without hidden layers can solve tasks previously solved by spiking neural networks.
result A chemical reaction network without hidden layers outperforms a spiking neural network with hidden layers in a handwritten digit classification task.
Deep ReLU networks can be simplified to a three-layer model.
problem Understanding the behavior of deep neural networks.
method Constructive proof and algorithm to transform deep networks into shallow ones.
result Deep ReLU networks can be represented by a simpler three-layer structure.
Tackles network structure inference from time series data using GNN.
problem Inferring network structure from incomplete or no information.
method Gumbel Graph Network (GGN) model for network reconstruction and completion.
result GGN can reconstruct up to 100% network structure and infer missing parts with up to 90% accuracy.
Network embedding helps predict speed limits on incomplete Danish road network.
problem Incomplete speed limit data on Danish roads limits machine learning applications.
method Applied node2vec network embedding to Danish road network.
result Network embedding can derive useful features for predicting speed limits.
This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.
problem Understanding the loss landscape of sparse neural networks, especially one-hidden-layer networks.
method Analyzes sparse networks with dense and sparse final layers, focusing on linear and non-linear models.
result Sparse networks can have no spurious valleys under certain conditions, but spurious valleys and minima can exist for wide sparse networks.
New approach learns latent motifs in networks for mesoscale structure analysis.
problem Understanding large-scale behavior in complex systems through mesoscale structures.
method Network dictionary learning (NDL) combining network sampling and nonnegative matrix factorization.
result Networks can be approximated using a small set of latent motifs.
A challenging problem in complex networks is the network reconstruction problem from data. This work deals with a class of networks denoted as conserved networks, in which a flow associated with every edge and the flows are conserved at all non-source and non-sink nodes. We propose a novel polynomial time algorithm to …
Social network analysis is an important problem in data mining. A fundamental step for analyzing social networks is to encode network data into low-dimensional representations, i.e., network embeddings, so that the network topology structure and other attribute information can be effectively preserved. Network represen…
Paper links network Lasso to network flow optimization.
problem Joint clustering and optimization of networked data.
method Exploration of duality between network Lasso and network flow optimization.
result nLasso is equivalent to a minimum-cost flow problem on the data network structure.
Taking inspiration from biological evolution, we explore the idea of "Can deep neural networks evolve naturally over successive generations into highly efficient deep neural networks?" by introducing the notion of synthesizing new highly efficient, yet powerful deep neural networks over successive generations via an ev…
SyNGLER generates synthetic networks efficiently while preserving key structural properties.
problem Efficiently generating realistic synthetic networks with preserved structural properties.
method SyNGLER uses latent space network models to learn and reconstruct node embeddings, then generates synthetic networks.
result SyNGLER produces synthetic networks that better preserve key network characteristics than existing approaches.
Paper introduces method to make neural networks symmetrical.
problem Creating symmetrical neural networks for data with inherent symmetries.
method Introduces a method for modifying neural networks to enforce equivariance.
result Group convolutional neural networks are a special case of the introduced framework.
Deep networks better approximate functions with compositional structure.
problem Approximating functions with complex structures.
method Design deep networks with compositional structure, leveraging the blessing of compositionality.
result Deep networks can approximate functions better than shallow networks when the function has a compositional structure.
Secret neural networks hidden within trained models.
problem Excess capacity in neural networks allows embedding secret models.
method Novel framework for hiding secret neural networks within carrier networks.
result Detection of hidden networks is computationally infeasible.
GCNs adapted for road networks improve performance on edge prediction tasks.
problem Improving machine learning on road networks for edge prediction tasks.
method Introducing Relational Fusion Network (RFN) for road networks.
result RFN outperforms state-of-the-art GCNs on road segment regression and classification tasks.
The structure of complex networks has been of interest in many scientific and engineering disciplines over the decades. A number of studies in the field have been focused on finding the common properties among different kinds of networks such as heavy-tail degree distribution, small-worldness and modular structure and …
Natural graph networks are a new class of graph neural networks that are more flexible and scalable.
problem Traditional graph neural networks are limited by equivariance to node permutations.
method Introduced natural graph networks, which are more flexible and scalable than conventional graph neural networks.
result Natural graph networks are as scalable as conventional message passing graph neural networks but more flexible.
Convolutional networks outperform fully-connected ones in certain tasks.
problem Understanding the computational advantage of convolutional networks over fully-connected networks.
method Demonstrated a computational advantage through a specific problem class.
result Convolutional networks can solve certain problems that fully-connected networks cannot, even with gradient descent.
Proposes a graph neural network for traffic forecasting in WANs.
problem Traffic forecasting challenges in WANs due to dynamic and large data volumes.
method Dynamic diffusion convolutional recurrent neural networks for multistep traffic forecasting.
result Significant improvements in forecasting accuracy compared to classical methods.
Researchers derive exact priors for finite Bayesian neural networks.
problem Understanding non-Gaussian priors in finite Bayesian neural networks.
method Analytical derivation of function space priors for finite fully-connected feedforward networks.
result Exact solutions for priors of finite networks, including Meijer G-function for linear networks and mixtures for ReLU networks.
i-cNRL learns network differences with interpretability.
problem Comparing unique network characteristics.
method Contrastive network representation learning (cNRL) integrating machine learning schemes.
result i-cNRL reveals unique network patterns with interpretability.
Deep and wide networks are shown to be equivalent in terms of their capability.
problem The relationship between the width and depth of neural networks.
method Formulated transforms to map networks, used polynomial representations.
result Deep and wide networks are quasi-equivalent with an arbitrarily small error.
Self-teaching networks improve deep neural networks' generalization.
problem Improving deep neural networks' generalization capacity.
method Generates soft supervision labels to train lower layers, using an auxiliary loss to mimic the output layer.
result Self-teaching network achieves consistent improvements and outperforms existing methods in speech recognition tasks.
DCGANs generate drainage networks quickly from samples.
problem High computational costs in generating large numbers of drainage networks.
method DCGANs trained with connectivity-informed directional information.
result Connectivity-informed DCGANs outperform other methods in reproducing accurate drainage networks.
A network embedding consists of a vector representation for each node in the network. Its usefulness has been shown in many real-world application domains, such as social networks and web networks. Directed networks with text associated with each node, such as software package dependency networks, are commonplace. Howe…
Quantum neural network and tensor network models outperform classical models in Japanese stock market predictions.
problem Improving stock return predictions using quantum and quantum-inspired machine learning.
method Evaluation of quantum neural network and tensor network models against classical models like linear and neural networks.
result Tensor network model outperforms classical models in Japanese stock market, including linear and neural network models.
Study deep maxout networks and their equivalence to Gaussian processes.
problem Understanding neural networks with infinite width.
method Derive equivalence between deep maxout networks and Gaussian processes, characterize maxout kernel, and provide efficient numerical implementation.
result Bayesian inference based on deep maxout network kernel leads to competitive results compared to finite-width counterparts and deep neural network kernels.
Machine learning improves network classification and model selection.
problem Quantifying suitability of generative models for network structures.
method Interpretable machine learning to classify simulated networks based on features and interactions.
result Specific network features and their interactions are crucial for distinguishing generative models.
Study shows effective resistance distance yields more accurate network barycenter than Hamming distance.
problem Identifying the best metric for computing the Fréchet mean network.
method Compared the effectiveness of Hamming distance and effective resistance distance in capturing network topology.
result Effective resistance distance produces a more accurate Fréchet mean network.
Model-based neural networks generalize better than ReLU networks for sparse recovery.
problem Understanding and quantifying the superior generalization of model-based neural networks.
method Using complexity measures like global and local Rademacher complexities, the paper provides theoretical bounds on generalization and estimation errors.
result Model-based neural networks exhibit higher generalization capabilities for sparse recovery problems compared to ReLU networks.