SyNGLER generates synthetic networks efficiently while preserving key structural properties.
problem Efficiently generating realistic synthetic networks with preserved structural properties.
method SyNGLER uses latent space network models to learn and reconstruct node embeddings, then generates synthetic networks.
result SyNGLER produces synthetic networks that better preserve key network characteristics than existing approaches.
DeepWeightFlow generates diverse neural network weights efficiently.
problem Generating complete neural network weights efficiently and accurately.
method Flow Matching in weight space with Git Re-Basin and TransFusion.
result DeepWeightFlow generates high-accuracy neural networks without fine-tuning.
Model-based neural networks generalize better than ReLU networks for sparse recovery.
problem Understanding and quantifying the superior generalization of model-based neural networks.
method Using complexity measures like global and local Rademacher complexities, the paper provides theoretical bounds on generalization and estimation errors.
result Model-based neural networks exhibit higher generalization capabilities for sparse recovery problems compared to ReLU networks.
DCGANs generate drainage networks quickly from samples.
problem High computational costs in generating large numbers of drainage networks.
method DCGANs trained with connectivity-informed directional information.
result Connectivity-informed DCGANs outperform other methods in reproducing accurate drainage networks.
Machine learning improves network classification and model selection.
problem Quantifying suitability of generative models for network structures.
method Interpretable machine learning to classify simulated networks based on features and interactions.
result Specific network features and their interactions are crucial for distinguishing generative models.
Proves generic existence of spectral networks for many cases.
problem Existence of spectral networks for a broad range of spectral data.
method Generic existence proof for spectral networks.
result Proves existence for a large class of spectral data.
The paper analyzes how low-rank layers in neural networks improve generalization.
problem Understanding how low-rank layers affect generalization in neural networks.
method Applying Maurer's chain rule for Gaussian complexity to analyze rank and spectral norm constraints.
result Deep networks with low-rank layers achieve better generalization than those with full-rank layers.
Paper tackles optimal network compression for financial systems.
problem Optimal network compression for financial systems under shocks.
method Formulated as an NP-hard problem, studied systemic risk measures, and analyzed specific networks.
result Systemic fragility results no longer hold generally under shocks and heterogeneous networks.
We develop a convex relaxation method for analyzing neural network generalization.
problem Analyzing the generalization of parallel positively homogeneous networks.
method Linking non-convex ERM to a convex optimization problem over prediction functions.
result Achieved generalization bounds with almost linear sample complexity in network width.
Improved deep neural network generalization through noise resilience.
problem Understanding and predicting generalization error of deep neural networks.
method Noise resilience measures to predict generalization error.
result Secured 5th position in the PGDL competition at NeurIPS 2020.
Gradient descent methods for deep ReLU networks achieve optimal generalization rates.
problem Generalization of gradient descent methods for deep neural networks
method Establishing minimax-optimal rates for GD and SGD with deep ReLU networks
result Gradient descent methods for deep ReLU networks achieve optimal generalization rates
We propose a novel technique to make neural network robust to adversarial examples using a generative adversarial network. We alternately train both classifier and generator networks. The generator network generates an adversarial perturbation that can easily fool the classifier network by using a gradient of each imag…
Real networks exhibit nontrivial topological features such as heavy-tailed degree distribution, high clustering, and small-worldness. Researchers have developed several generative models for synthesizing artificial networks that are structurally similar to real networks. An important research problem is to identify the…
PBN combines generative and discriminative capabilities in a neural network.
problem Combining generative and discriminative capabilities in neural networks.
method Convolutional PBN, sharing FF-NN embodiment, combining generative and discriminative qualities.
result PBN shows excellent qualities from either generative or discriminative viewpoint.
Modular neural networks generalize better with less data.
problem Theoretical and practical understanding of how modularity improves neural network generalization.
method Theoretical analysis of sample complexity, development of a novel learning rule.
result Modular networks require fewer samples to generalize compared to nonmodular networks, especially in high-dimensional tasks.
We establish a margin based data dependent generalization error bound for a general family of deep neural networks in terms of the depth and width, as well as the Jacobian of the networks. Through introducing a new characterization of the Lipschitz properties of neural network family, we achieve significantly tighter g…
Improves stability in hyperbolic neural networks for complex data generation.
problem Numerical instability in hyperbolic neural networks hinders complex architecture development.
method Proposes a novel hyperbolic AE-GAN architecture with stable layers.
result Demonstrates state-of-the-art performance in generating complex data.
DFM model detects communities in weighted networks without distributional assumptions.
problem Challenges in community detection for weighted networks.
method Distribution-Free Model (DFM) for weighted networks, using spectral clustering.
result Spectral clustering yields consistent community detection under DFM.
CnGAN generates synthetic user preferences for non-overlapped users in cross-network recommender systems.
problem Cross-network recommender solutions ignore non-overlapped users, limiting their applicability.
method Multi-task learning, encoder-GAN architecture, user-based pairwise loss function.
result Generated user preferences improve recommendations for non-overlapped users, achieving superior performance.
Generative graph models create instances of graphs that mimic the properties of real-world networks. Generative models are successful at retaining pairwise associations in the underlying networks but often fail to capture higher-order connectivity patterns known as network motifs. Different types of graphs contain diff…
Affine spiking neural networks learn efficiently and generalize well.
problem Learning with spiking neural networks, especially with positive weights.
method Affine encoders and decoders, continuous parameter dependence, gradient-based training.
result Affine spiking neural networks can approximate shallow ReLU networks and generalize well.
We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process. The generation network maps samples from stochastic latent variables to the data space while the inference network maps training examples in data space to the sp…
Generative adversarial networks (GANs) have demonstrated to be successful at generating realistic real-world images. In this paper we compare various GAN techniques, both supervised and unsupervised. The effects on training stability of different objective functions are compared. We add an encoder to the network, makin…
Flow-based data sets are necessary for evaluating network-based intrusion detection systems (NIDS). In this work, we propose a novel methodology for generating realistic flow-based network traffic. Our approach is based on Generative Adversarial Networks (GANs) which achieve good results for image generation. A major c…
Deep generative models can learn to generate realistic-looking images, but many of the most effective methods are adversarial and involve a saddlepoint optimization, which requires a careful balancing of training between a generator network and a critic network. Maximum mean discrepancy networks (MMD-nets) avoid this i…
DAMNETS generates complex network dynamics models.
problem Generating flexible and scalable models for network time series is challenging.
method Deep autoregressive model for Markovian network time series.
result DAMNETS outperforms other methods in sample quality.
New method improves neural network robustness by identifying functions rather than parameters.
problem Neural networks' lack of robustness to distribution shifts.
method Identify the function represented by quadratic networks, not their parameters.
result Obtain robust generalization bounds for neural networks.
The generalization error of deep neural networks via their classification margin is studied in this work. Our approach is based on the Jacobian matrix of a deep neural network and can be applied to networks with arbitrary non-linearities and pooling layers, and to networks with different architectures such as feed forw…
Study bounds graph neural networks' over-parameterized error.
problem Understanding graph neural networks' performance in over-parameterized regimes.
method Developed mean-field regime bounds for graph convolutional and message passing neural networks.
result Established upper bounds with a convergence rate of O(1/n) for generalization error. Complexity measures for neural nets with general activations using path-based norms.
problem Control complexity of neural networks with arbitrary activation functions.
method Approximate general activations with ReLU networks and derive path-based norms for complexity control.
result Preliminary analyses of function spaces and regularized estimators.
A recent Cell paper [Chang and Tsao, 2017] reports an interesting discovery. For the face stimuli generated by a pre-trained active appearance model (AAM), the responses of neurons in the areas of the primate brain that are responsible for face recognition exhibit strong linear relationship with the shape variables and…
Unified analysis of neural networks for sparse signal recovery.
problem Sparse signal recovery from few linear measurements.
method Introduces a general class of neural networks with weight-sharing, analyzes their Rademacher complexity, and derives generalization bounds.
result Derives generalization bounds that depend linearly on the number of parameters and depth, applicable to various neural network types.
Improved graph neural network bounds using graph diffusion matrix.
problem Empirical performance of graph neural networks on real-world graphs.
method Unified model of graph neural networks, focusing on feature diffusion matrix stability.
result Generalization bounds scale with largest singular value of feature diffusion matrix, smaller than prior bounds.
Generative adversarial networks fix aliasing issues by making signals continuous.
problem Alias-free generation in GANs to prevent unwanted information leakage.
method Interpreting all signals as continuous, deriving small architectural changes.
result Generative models match FID of StyleGAN2 but have better internal representations.
We use multiple measures of graph complexity to evaluate the realism of synthetically-generated networks of human activity, in comparison with several stylized network models as well as a collection of empirical networks from the literature. The synthetic networks are generated by integrating data about human populatio…
Neural networks generalize on simple data generated by a programming language.
problem Generalization of neural networks on low complexity data.
method Minimum description length (MDL) approach for feedforward neural networks.
result MDL feedforward neural networks generalize with high probability on simple data.
CNN-F uses generative feedback to improve neural networks' robustness to perturbations.
problem Neural networks' vulnerability to input perturbations like noise and attacks.
method Enforces self-consistency in neural networks by incorporating generative recurrent feedback.
result CNN-F shows significantly improved adversarial robustness compared to conventional CNNs.
Generative diffusion models mimic biological memory networks, encoding associative dynamics in deep neural weights.
problem Understanding long-term memory mechanisms in neuroscience and AI.
method Interpreting generative diffusion models as energy-based models and comparing them to Hopfield networks.
result Generative diffusion models can encode associative dynamics of Hopfield networks in deep neural weights.
The study provides a generalization bound for a family of implicit networks.
problem Theoretical understanding of implicit networks' generalization is limited.
method A generalization bound is derived for a family of implicit networks using a covering number argument for Rademacher complexity.
result A theoretical generalization bound is established for implicit networks.
Despite existing work on ensuring generalization of neural networks in terms of scale sensitive complexity measures, such as norms, margin and sharpness, these complexity measures do not offer an explanation of why neural networks generalize better with over-parametrization. In this work we suggest a novel complexity m…
Neural networks' slope controls their generalization.
problem Controlling overfitting in neural networks.
method Control the slope of neural networks to avoid memorization.
result The slope of a well-trained neural network is generally independent of network width and architecture.
Study shows infoGAN's generalization error bound for two-layer networks.
problem Understanding generalization error in infoGAN for two-layer neural networks.
method Analyzes the difference between empirical and population objective functions, derives Rademacher complexity bounds.
result Derives error bound for infoGAN's generalization error in a two-layer network.
The skip-connections used in residual networks have become a standard architecture choice in deep learning due to the increased training stability and generalization performance with this architecture, although there has been limited theoretical understanding for this improvement. In this work, we analyze overparameter…
Quantum neural networks generalize better due to flatter parameter space.
problem Generalization in quantum neural networks.
method Mapped feature data to a quantum state, applied unitary evolution, and measured for classification.
result Quantum neural networks have better generalization than classical networks.
Deep neural networks can grok better than shallow ones, showing multi-stage generalization.
problem Understanding the generalization behavior of deep neural networks.
method Empirical replication and analysis of grokking phenomenon in deep MLP models.
result Deep neural networks exhibit multi-stage generalization, with a secondary surge in test accuracy.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
The accuracy of deep learning, i.e., deep neural networks, can be characterized by dividing the total error into three main types: approximation error, optimization error, and generalization error. Whereas there are some satisfactory answers to the problems of approximation and optimization, much less is known about th…
We present Generative Adversarial Capsule Network (CapsuleGAN), a framework that uses capsule networks (CapsNets) instead of the standard convolutional neural networks (CNNs) as discriminators within the generative adversarial network (GAN) setting, while modeling image data. We provide guidelines for designing CapsNet…