Study finds root vertex in large networks with high probability.
problem Finding the root vertex in large growing networks.
method Constructs confidence sets for the root vertex in various random network models.
result Confidence sets of size independent of the number of vertices contain the root vertex with high probability.
GraphMoE generates random graphs using neural networks and graphlets.
problem Learning generative models for random graphs.
method GraphMoE uses a neural network trained with graphlets and subgraph counts to match the distribution of random graphs.
result GraphMoE can generate graphs that mimic various real-world datasets and fool graph classifiers.
A number of recent papers have provided evidence that practical design questions about neural networks may be tackled theoretically by studying the behavior of random networks. However, until now the tools available for analyzing random neural networks have been relatively ad-hoc. In this work, we show that the distrib…
Random convolutional networks can be fooled with adversarial examples.
problem Existence of adversarial examples for random convolutional networks.
method Utilizing isoperimetric inequalities on the special orthogonal group so(d). result Adversarial examples exist for various random convolutional networks.
The weights of a neural network are typically initialized at random, and one can think of the functions produced by such a network as having been generated by a prior over some function space. Studying random networks, then, is useful for a Bayesian understanding of the network evolution in early stages of training. In…
NetOTC compares and aligns directed or undirected networks via random walk transitions.
problem Comparing and aligning networks of different types and sizes.
method NetOTC uses a transport-based approach to find optimal transition couplings of random walks.
result NetOTC quantifies network differences and provides vertex and edge alignments.
New algorithm learns random neural networks efficiently.
problem Learning random constant-depth neural networks efficiently.
method Presented a PTAS (Polynomial-Time Approximation Scheme) for learning random Xavier networks of fixed depth.
result For any fixed ε and depth i, there is a poly-time algorithm that learns random Xavier networks up to an additive error of ε.
Neural networks learn patterns in random data, improving downstream performance.
problem Understanding what deep networks learn with random labels.
method Analytical and empirical study of convolutional and fully connected networks pre-trained on random labels.
result Pre-trained networks on random labels transfer faster to real datasets, despite specialization effects.
Sparse random networks reduce communication in federated learning.
problem Large communication cost in federated learning.
method Freeze random weights, train stochastic binary mask to sparsify.
result Improves accuracy, reduces communication, speeds convergence.
Randomized neural networks use fixed connections for efficiency.
problem Efficiency in deep learning models.
method Fixed connections in neural networks.
result Deep randomized neural networks achieve state-of-the-art results.
Improved neural network training in low-dimensional random bases.
problem Inefficient optimization in large-scale neural networks.
method Re-draw random subspace at each training step, apply independent projections to different network parts.
result Significantly better optimization performance and efficiency.
Higher-order proximity preserved network embedding has attracted increasing attention. In particular, due to the superior scalability, random-walk-based network embedding has also been well developed, which could efficiently explore higher-order neighborhoods via multi-hop random walks. However, despite the success of …
This paper studies node embeddings of networks, revealing their geometric properties.
problem Understanding the geometric properties of node embeddings in random networks.
method Characterization of ergodic limits, generalization, and convex relaxations of random walk node embedding objectives.
result The optimal node embedding Grammians have rank 1 for a nuclear norm relaxation of the non-randomized objective.
Study evaluates neural networks based on random graph structures and finds key performance indicators.
problem Understanding and optimizing neural network architectures using graph theory.
method Evaluation of neural networks with random graph structures, focusing on structural and numerical properties.
result A new numerical graph characteristic selects a set of quasi-1-dimensional graphs that perform well.
Study finds no significant difference in neural network weights with quantum random numbers.
problem Effects of biased quantum random numbers on neural network initialization.
method Empirical study using quantum hardware and classical pseudo-random numbers.
result No statistically significant difference found between quantum random numbers and other types.
Random feature models approximate functions in Banach spaces efficiently.
problem Approximating functions in Banach spaces efficiently.
method Randomly initialized feature maps and linear readout training.
result Universal approximation in Bochner spaces for Banach space-valued models.
UniNet efficiently learns network representations from large graphs.
problem Efficiently learning network representations from large graphs.
method Metropolis-Hastings sampling for efficient edge sampling and random walk model abstraction.
result UniNet outperforms existing NRL models on billion-edge networks.
Study on random matrices in deep neural networks with IID entries.
problem Distribution of singular values in product of random matrices for deep neural networks.
method Random matrix theory with a streamlined approach for non-Gaussian data.
result Generalization of macroscopic universality property to non-Gaussian data.
Single gradient step finds adversarial examples in random neural networks.
problem Finding adversarial examples in neural networks with random architectures.
method Gradient descent approach applied to random undercomplete and overcomplete two-layers neural networks.
result A single gradient step is sufficient to find adversarial examples in random neural networks.
The study examines Fisher information matrices and neural tangent kernels for simple ReLU networks with random weights.
problem Understanding the relationship between Fisher information matrices and neural tangent kernels for 2-layer ReLU networks.
method Analyzes Fisher information matrices and neural tangent kernels for 2-layer ReLU networks with random hidden weights, focusing on spectral decomposition and eigenfunctions.
result Obtained an approximation formula for functions represented by 2-layer neural networks.
Random neural nets learn Black-Scholes PDEs without dimensionality issues.
problem Learning Black-Scholes type PDEs efficiently in high dimensions.
method Random feature neural networks applied to Kolmogorov PDEs.
result Random neural nets avoid the curse of dimensionality for Black-Scholes PDEs.
Gradient descent amplifies random features in neural networks to useful ones.
problem Generalization in neural networks trained on corrupted data.
method Characterization of feature-learning process in two-layer ReLU networks trained by gradient descent.
result Gradient descent amplifies random features to useful ones, achieving near optimal generalization error.
Random matrix analysis reveals that neural network weights are mostly random, with some indicating learned information.
problem Understanding how neural networks store information needed for tasks.
method Random matrix theory (RMT) applied to weight matrices of trained deep neural networks.
result Most singular values and eigenvectors of trained neural networks follow universal RMT predictions, suggesting they are random and do not contain system-specific information.
Multivariate splines linked to infinitely-wide neural networks with improved numerical performance.
problem Understanding the relationship between multivariate splines and neural networks.
method Showed multivariate splines can be represented as random features in infinitely-wide neural networks with a homogeneous activation function.
result The function space of multivariate splines is a Sobolev space on a Euclidean ball with explicit norm bounds on derivatives.
Dropout neural networks can approximate any function with high probability.
problem Approximating functions with dropout neural networks.
method Two universal approximation theorems for dropout neural networks in random and deterministic modes.
result Dropout neural networks can approximate any function in probability and in Lq. Bounds on Gaussian approximation for neural networks with novel smoothing techniques.
problem Approximating the distribution of wide random neural networks.
method Stein's method, Gaussian smoothing, Laplacian operators, Cameron-Martin space.
result First bounds on Gaussian approximation of wide random neural networks.
Graph Neural Networks struggle on random graphs without node identifiers.
problem Graph Neural Networks' limitations on random graphs without node identifiers.
method Study of Graph Neural Networks and Structural Graph Neural Networks convergence on large random graphs.
result Structural Graph Neural Networks are more powerful and universal than Graph Neural Networks on random graphs.
Bandlimited random neural networks may not approximate all functions perfectly.
problem Expressive power of shallow neural networks with bandlimited random weights.
method Ridgelet analysis for deriving approximation error lower bounds.
result Bandlimited random weights can lead to non-zero approximation error.
RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.
problem Improving performance of deep residual neural networks (RFNNs) while preserving convex optimization benefits.
method Random Feature Representation Boosting (RFRBoost) using boosting theory and random features at each layer.
result RFRBoost significantly outperforms RFNNs and end-to-end trained MLP ResNets in small- to medium-scale tabular datasets.
Study improves prediction accuracy and uncertainty for mobile sensor data using randomized neural networks.
problem Improving prediction accuracy and uncertainty for mobile sensor data.
method Cross-validation and uncertainty determination for randomized neural networks.
result Improved out-of-sample performance and confidence intervals for prediction error.
This paper analyzes DeepWalk and node2vec for community detection in large networks.
problem Community detection in large, sparse networks.
method Low-dimensional network embedding algorithms (DeepWalk and node2vec) applied to random walk segments.
result The performance of DeepWalk and node2vec in recovering communities depends on the length of random walk segments and sparsity of the network.
Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields. Nevertheless, it remains unclear how probabilistic inference can be implemented by a network of spiking neurons in th…
In network embedding, random walks play a fundamental role in preserving network structures. However, random walk based embedding methods have two limitations. First, random walk methods are fragile when the sampling frequency or the number of node sequences changes. Second, in disequilibrium networks such as highly bi…
Introduces epistemic deep learning for better uncertainty estimation in neural networks.
problem Uncertainty quantification in deep neural networks.
method Random-set convolutional neural networks with belief function-based loss functions.
result Epistemic approach produces better performance in uncertainty estimation.
Deep neural nets approximate random dynamical system trajectories uniformly in time.
problem Approximating trajectories of random dynamical systems over infinite time horizons.
method Recurrent neural networks with simple feedback structures.
result Certain random trajectories can be approximated uniformly in time to any desired accuracy.
Randomized spectral co-clustering speeds up large-scale directed networks.
problem Co-clustering directed networks efficiently for large-scale data.
method Randomized spectral co-clustering algorithms using random-projection and random-sampling techniques.
result Theoretical and numerical validation of approximation and misclustering error rates.
RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.
problem Efficient reinforcement learning in environments with continuous state and action spaces.
method RANDPOL uses randomized function approximation to represent policy and value functions, providing finite time guarantees and improved numerical performance.
result RANDPOL achieves better numerical performance and provides finite time guarantees compared to deep neural network based algorithms.
G-Net constructs binary neural networks with high accuracy using randomized binary embeddings.
problem Creating high-accuracy binary neural networks with theoretical guarantees.
method Proposes a novel floating-point G-Net family with randomized binary embeddings and theoretical accuracy guarantees.
result Empirically, G-Net achieves almost 30% higher accuracy on CIFAR-10 compared to prior HDC models.
Adding random features to GNNs improves their performance.
problem Limitations of GNNs in distinguishing graphs and learning efficient algorithms.
method Adding random features to each node in GNNs.
result Random features enable GNNs to learn optimal algorithms for graph problems.
We introduce a new framework for comparing parametric network families.
problem Comparing and analyzing data modeled as parameterized families of networks.
method A Gromov-Wasserstein variant of optimal transport for defining distances.
result Established foundational properties and theoretical approximation guarantees for the new distances.
We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features…
Heterogeneous information network (HIN) embedding has gained increasing interests recently. However, the current way of random-walk based HIN embedding methods have paid few attention to the higher-order Markov chain nature of meta-path guided random walks, especially to the stationarity issue. In this paper, we system…
Random learning rate improves neural network training without extra cost.
problem Optimizing neural networks with stochastic gradient descent.
method Uniformly-distributed random learning rate in SGD.
result Random learning rate yields better regularization without extra computational cost.
Recently, a spate of papers have provided positive theoretical results for training over-parameterized neural networks (where the network size is larger than what is needed to achieve low error). The key insight is that with sufficient over-parameterization, gradient-based methods will implicitly leave some components …
Neural networks approximate random utility models for choice prediction.
problem Approximating random utility models with neural networks.
method RUMnets, a neural network-based model inspired by RUM framework.
result RUMnets can approximate any RUM model arbitrarily closely and vice versa.
A new method to prune neural networks with iterative randomization improves efficiency.
problem Efficiency in pruning randomly initialized neural networks.
method Iteratively randomizing weight values to reduce parameter requirements.
result The method achieves remarkable performance with fewer parameters.
We propose reinforcement learning on simple networks consisting of random connections of spiking neurons (both recurrent and feed-forward) that can learn complex tasks with very little trainable parameters. Such sparse and randomly interconnected recurrent spiking networks exhibit highly non-linear dynamics that transf…
Bounds neural network output distribution to Gaussian for random initialization.
problem Quantifying the distribution of randomly initialized deep neural networks.
method Quantitative Gaussian approximation using quadratic Wasserstein distance.
result Explicit inequalities show how network sizes affect Gaussian behavior.