arXiv research

New method improves random parameter generation in neural networks.

Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping a…

RVFL NNs perform well without direct links and output bias for regression.

This paper studies node embeddings of networks, revealing their geometric properties.

ELMs benefit from using equal number of hidden nodes to training samples.

New method constructs neural network architecture iteratively to optimize learning.

The concept of SCN offers a fast framework with universal approximation guarantee for lifelong learning of non-stationary data streams. Its adaptive scope selection property enables for proper random generation of hidden unit parameters advancing conventional randomized approaches constrained with a fixed scope of rand…

Dropout is often used in deep neural networks to prevent over-fitting. Conventionally, dropout training invokes \textit{random drop} of nodes from the hidden layers of a Neural Network. It is our hypothesis that a guided selection of nodes for intelligent dropout can lead to better generalization as compared to the tra…

This paper is concerned with the sparsification of the input-hidden weights of ELM (Extreme Learning Machine). For ordinary feedforward neural networks, the sparsification is usually done by introducing certain regularization technique into the learning process of the network. But this strategy can not be applied for E…

RBMs model binary interactions with hidden node activation effects.

Study of deep neural networks with dependent weights leading to new model limits and properties.

EPFGNN models graph connections for better node classification.

We examine the effect of the Group Lasso (gLasso) regularizer in selecting the salient nodes of Deep Neural Network (DNN) hidden layers by applying a DNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of gLasso regularization, one for outgoing weight vectors and another for incoming weight vec…

New model detects hidden group structures in criminal networks.

Enhances graph neural networks with random walks to improve performance.

We analyze the dynamics of training deep ReLU networks and their implications on generalization capability. Using a teacher-student setting, we discovered a novel relationship between the gradient received by hidden student nodes and the activations of teacher nodes for deep ReLU networks. With this relationship and th…

Vanishing nodes cause hidden nodes to behave similarly, complicating deep neural network training.

CoulGAT interprets GAT models by analyzing node interactions.

We investigate the problem of factorizing a matrix into several sparse matrices and propose an algorithm for this under randomness and sparsity assumptions. This problem can be viewed as a simplification of the deep learning problem where finding a factorization corresponds to finding edges in different layers and valu…

Optimal rates for learning hidden tree structures are determined.

The autoencoder is an artificial neural network model that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar…

New algorithm reduces sample complexity for learning CNNs.

Improved covariate shift handling with node-based Bayesian neural networks.

New method uncovers hidden causal connections in multivariate point process networks.

Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve de…

New results on inferring hidden states in trackable weak models.

Two new inverse-free ELM algorithms for incremental and decremental learning are proposed.

Spectral clustering is a celebrated algorithm that partitions objects based on pairwise similarity information. While this approach has been successfully applied to a variety of domains, it comes with limitations. The reason is that there are many other applications in which only \emph{multi}-way similarity measures ar…

We analyze the joint probability distribution on the lengths of the vectors of hidden variables in different layers of a fully connected deep network, when the weights and biases are chosen randomly according to Gaussian distributions, and the input is in $\{ -1, 1\}^N$. We show that, if the activation function $φ$ sat…

Stochastic configuration networks (SCNs) as a class of randomized learner model have been successfully employed in data analytics due to its universal approximation capability and fast modelling property. The technical essence lies in stochastically configuring hidden nodes (or basis functions) based on a supervisory m…

Study finds significant instability in node embeddings due to randomness.

We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require $N$ hidden nodes to memorize/interpolate arbitrary $N$ data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with $Ω(\sqrt{N})$ hidden nodes can perfectly memorize most datasets with $N$ po…

Paper develops a new model for dynamic graph representation learning.

Study compares classification techniques to predict customer churn in banking.

The paper shows how random ReLU networks converge to smooth splines.

Study reconstructs hidden perfect matchings in random graphs with specific edge weights.

LIFE framework improves model accuracy and interpretability.

Many works have been proposed in the literature to capture the dynamics of diffusion in networks. While some of them define graphical markovian models to extract temporal relationships between node infections in networks, others consider diffusion episodes as sequences of infections via recurrent neural models. In this…

DAG models with hidden variables present many difficulties that are not present when all nodes are observed. In particular, fully observed DAG models are identified and correspond to well-defined sets ofdistributions, whereas this is not true if nodes are unobserved. Inthis paper we characterize exactly the set of dist…

Proposes a block-based model for attributed network embedding.

Algorithm predicts performance of learning in multi-layer networks with matrix-valued hidden variables.

ARGEW improves node embeddings for weighted homophilous graphs by emphasizing strong edge weights.

The technological applications of hidden Markov models have been extremely diverse and successful, including natural language processing, gesture recognition, gene sequencing, and Kalman filtering of physical measurements. HMMs are highly non-linear statistical models, and just as linear models are amenable to linear a…

Paper combines matrix factorization and random walk models for node embeddings.

RVFL networks can efficiently approximate Lipschitz functions in L∞ norm.

Deep belief networks can approximate any multivariate density with binary hidden units.

Paper introduces a new method to compute pseudoinverse for ELM with large datasets.

Randomly chosen primary hidden units and derived secondary units reduce neural network complexity.