A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Vanishing nodes cause hidden nodes to behave similarly, complicating deep neural network training.
This paper is concerned with the sparsification of the input-hidden weights of ELM (Extreme Learning Machine). For ordinary feedforward neural networks, the sparsification is usually done by introducing certain regularization technique into the learning process of the network. But this strategy can not be applied for E…
RBMs model binary interactions with hidden node activation effects.
We examine the effect of the Group Lasso (gLasso) regularizer in selecting the salient nodes of Deep Neural Network (DNN) hidden layers by applying a DNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of gLasso regularization, one for outgoing weight vectors and another for incoming weight vec…
CoulGAT interprets GAT models by analyzing node interactions.
The autoencoder is an artificial neural network model that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar…
ELMs benefit from using equal number of hidden nodes to training samples.
Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping a…
New method uncovers hidden causal connections in multivariate point process networks.
Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve de…
New results on inferring hidden states in trackable weak models.
Two new inverse-free ELM algorithms for incremental and decremental learning are proposed.
RBM models reveal how hidden unit tail behavior affects pattern reconstruction.
We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require hidden nodes to memorize/interpolate arbitrary data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with hidden nodes can perfectly memorize most datasets with po…
Many works have been proposed in the literature to capture the dynamics of diffusion in networks. While some of them define graphical markovian models to extract temporal relationships between node infections in networks, others consider diffusion episodes as sequences of infections via recurrent neural models. In this…
The standard method of generating random weights and biases in feedforward neural networks with random hidden nodes, selects them both from the uniform distribution over the same fixed interval. In this work, we show the drawbacks of this approach and propose a new method of generating random parameters. This method en…
DAG models with hidden variables present many difficulties that are not present when all nodes are observed. In particular, fully observed DAG models are identified and correspond to well-defined sets ofdistributions, whereas this is not true if nodes are unobserved. Inthis paper we characterize exactly the set of dist…
Chemical networks outperform spiking neural networks in classification tasks.
Deep belief networks can approximate any multivariate density with binary hidden units.
Paper introduces a new method to compute pseudoinverse for ELM with large datasets.
We extend the Bayesian Information Criterion (BIC), an asymptotic approximation for the marginal likelihood, to Bayesian networks with hidden variables. This approximation can be used to select models given large samples of data. The standard BIC as well as our extension punishes the complexity of a model according to …
Investor selects portfolios based on news attention in a hidden Markov model.
RVFL NNs perform well without direct links and output bias for regression.
Dropout is often used in deep neural networks to prevent over-fitting. Conventionally, dropout training invokes \textit{random drop} of nodes from the hidden layers of a Neural Network. It is our hypothesis that a guided selection of nodes for intelligent dropout can lead to better generalization as compared to the tra…
Training an artificial neural network involves an optimization process over the landscape defined by the cost (loss) as a function of the network parameters. We explore these landscapes using optimisation tools developed for potential energy landscapes in molecular science. The number of local minima and transition sta…
Graph Ricci flow reveals hidden hierarchies in stock market correlations.
A colored graph is a directed graph in which nodes or edges have been assigned colors that are not necessarily unique. Observability problems in such graphs consider whether an agent observing the colors of edges or nodes traversed on a path in the graph can determine which node they are at currently or which nodes wer…
Theoretical study shows adversarial training improves robustness in deep learning models.
CoMGNN models heterogeneous graphs with evolving nodes and edges.
Machine learning uncovers hidden correlations in granular material behavior.
The majority of real-world networks are dynamic and extremely large (e.g., Internet Traffic, Twitter, Facebook, ...). To understand the structural behavior of nodes in these large dynamic networks, it may be necessary to model the dynamics of behavioral roles representing the main connectivity patterns over time. In th…
Study develops a new model for predicting individual mobility based on activity patterns.
Gene expression profiles have been widely used to characterize patterns of cellular responses to diseases. As data becomes available, scalable learning toolkits become essential to processing large datasets using deep learning models to model complex biological processes. We present an autoencoder to capture nonlinear …
Algorithmic approaches endow deep learning systems with implicit bias that helps them generalize even in over-parametrized settings. In this paper, we focus on understanding such a bias induced in learning through dropout, a popular technique to avoid overfitting in deep learning. For single hidden-layer linear neural …
We provide high probability finite sample complexity guarantees for hidden non-parametric structure learning of tree-shaped graphical models, whose hidden and observable nodes are discrete random variables with either finite or countable alphabets. We study a fundamental quantity called the (noisy) information threshol…
A stochastic model with hidden discrete Markov processes is constructed to understand the behavior of debtors.
We analyze the dynamics of training deep ReLU networks and their implications on generalization capability. Using a teacher-student setting, we discovered a novel relationship between the gradient received by hidden student nodes and the activations of teacher nodes for deep ReLU networks. With this relationship and th…
In many real-world networks, nodes have class labels, attributes, or variables that affect the network's topology. If the topology of the network is known but the labels of the nodes are hidden, we would like to select a small subset of nodes such that, if we knew their labels, we could accurately predict the labels of…
New method embeds dynamic networks with stability for node behavior.
We present and implement two algorithms for analytic asymptotic evaluation of the marginal likelihood of data given a Bayesian network with hidden nodes. As shown by previous work, this evaluation is particularly hard for latent Bayesian network models, namely networks that include hidden variables, where asymptotic ap…
New model detects hidden group structures in criminal networks.
Feedforward neural networks with random hidden nodes suffer from a problem with the generation of random weights and biases as these are difficult to set optimally to obtain a good projection space. Typically, random parameters are drawn from an interval which is fixed before or adapted during the learning process. Due…
We study the problem of learning a latent tree graphical model where samples are available only from a subset of variables. We propose two consistent and computationally efficient algorithms for learning minimal latent trees, that is, trees without any redundant hidden nodes. Unlike many existing methods, the observed …
DIFNET tackles the suspended animation problem in deep graph neural networks.
This paper studies node embeddings of networks, revealing their geometric properties.
Study of deep neural networks with dependent weights leading to new model limits and properties.
New method discovers concepts in hidden feature layers using sparse subspace clustering.
New algorithm reduces sample complexity for learning CNNs.