In-network learning outperforms Federated and Split learning in wireless networks.
problem Efficiently using distributed features for inference in wireless networks.
method Proposes 'in-network learning' architecture, uses neural networks for optimization, compares with Federated and Split learning.
result In-network learning offers better accuracy and bandwidth savings.
i-cNRL learns network differences with interpretability.
problem Comparing unique network characteristics.
method Contrastive network representation learning (cNRL) integrating machine learning schemes.
result i-cNRL reveals unique network patterns with interpretability.
Three-layer networks learn more complex features than two-layer networks.
problem Understanding feature learning in deep neural networks.
method Analysis of three-layer neural networks trained with gradient descent.
result Three-layer networks can learn functions that two-layer networks cannot.
Fog learning distributes ML model training across heterogeneous devices and networks.
problem Challenges with conventional federated learning in heterogeneous networks.
method Intelligent distribution of ML model training across nodes from edge devices to cloud servers.
result Enhanced federated learning with multi-layer hybrid framework considering network, heterogeneity, and proximity.
Chemical networks outperform spiking neural networks in classification tasks.
problem Learning tasks with spiking neural networks require hidden layers, which are computationally expensive.
method Used deterministic mass-action kinetics to prove chemical reaction networks without hidden layers can solve tasks previously solved by spiking neural networks.
result A chemical reaction network without hidden layers outperforms a spiking neural network with hidden layers in a handwritten digit classification task.
Hybrid tensor networks improve machine learning by combining quantum and classical methods.
problem Limitations of regular tensor networks in machine learning.
method Quantum-classical hybrid tensor networks (HTN) combining tensor networks and classical neural networks.
result HTN overcomes limitations of regular tensor networks and enables deep learning training.
New findings show learning deeper neural networks is hard even with Gaussian inputs and non-degenerate weights.
problem The computational complexity of learning neural networks, especially deeper ones.
method Smoothed analysis framework and local pseudorandom generators.
result Learning depth-3 ReLU networks under Gaussian input distribution is hard even if weight matrices are non-degenerate.
With the widespread use of information technologies, information networks are becoming increasingly popular to capture complex relationships across various disciplines, such as social networks, citation networks, telecommunication networks, and biological networks. Analyzing these networks sheds light on different aspe…
Recent works reveal that network embedding techniques enable many machine learning models to handle diverse downstream tasks on graph structured data. However, as previous methods usually focus on learning embeddings for a single network, they can not learn representations transferable on multiple networks. Hence, it i…
A powerful network teaches a weak one, improving its performance.
problem Improving the performance of a weak neural network using a more powerful one.
method During training, a weak network learns features from a strong network to minimize feature distance.
result A weak neural network can increase its performance by learning from a more powerful network.
These are lecture notes for a course on machine learning with neural networks for scientists and engineers that I have given at Gothenburg University and Chalmers Technical University in Gothenburg, Sweden. The material is organised into three parts: Hopfield networks, supervised learning of labeled data, and learning …
Meta-learning with network pruning reduces overfitting and improves few-shot learning.
problem Overfitting in meta-learning models with over-parameterized neural networks.
method Network pruning to control capacity and explicitly reduce generalization gap.
result Uniform concentration analysis shows the benefit of network capacity constraint.
Probabilistic deep learning uses neural networks and models to handle uncertainty.
problem Handling uncertainty in deep learning models.
method Two approaches: probabilistic neural networks and deep probabilistic models.
result TensorFlow Probability library supports both approaches.
In this paper, we introduce the problem of jointly learning feed-forward neural networks across a set of relevant but diverse datasets. Compared to learning a separate network from each dataset in isolation, joint learning enables us to extract correlated information across multiple datasets to significantly improve th…
New approach learns latent motifs in networks for mesoscale structure analysis.
problem Understanding large-scale behavior in complex systems through mesoscale structures.
method Network dictionary learning (NDL) combining network sampling and nonnegative matrix factorization.
result Networks can be approximated using a small set of latent motifs.
Transfer learning improves highway traffic forecasting using graph neural networks.
problem Lack of historical data for traffic forecasting on large highway networks.
method Developed a transfer learning approach for DCRNN, a graph neural network for highway forecasting.
result TL-DCRNN can forecast traffic on unseen regions of the highway network with high accuracy.
New learning algorithm mimics biological neural networks.
problem Biologically implausible backpropagation for directed neural networks.
method Introduces new neuronal dynamics and learning rule for arbitrary architectures, sparsity-inducing pruning method, and dynamical-systems characterization.
result Prunes irrelevant connections and improves learning efficiency.
Neural network learns its size and structure during training.
problem Adapting neural network architecture to specific datasets.
method Flexible setup allowing neural network to learn size and topology during training.
result Trained networks achieve virtually identical performance and have learned optimal structure.
New method uses MHN for associative learning in network embedding.
problem Represent nodes in networks as low-dimensional vectors while incorporating topological and structural information.
method Introduces Modern Hopfield Networks (MHN) for associative learning between node content and neighbors.
result Competitive performance on node classification and linkage prediction tasks.
Hardness proven for neural networks with natural weights.
problem Difficulty in learning neural networks with weights from natural distributions.
method Proved hardness for depth-2 networks with natural weights distributions.
result Most networks are hard to learn with natural weights.
Deep neural networks bring in impressive accuracy in various applications, but the success often relies on the heavy network architecture. Taking well-trained heavy networks as teachers, classical teacher-student learning paradigm aims to learn a student network that is lightweight yet accurate. In this way, a portable…
Deep Belief Network predicts lncRNA-disease associations with high accuracy.
problem Accurately identifying lncRNA-disease associations to understand lncRNA functionality and disease mechanism.
method Proposes a DBN-based model using heterogeneous networks and DBN for feature learning.
result Obtained AUC of 0.96 and AUPR of 0.967 on standard dataset.
Feature networks link ML features via graph structure for enhanced learning.
problem Enhancing feature expressiveness and learning efficiency in machine learning.
method Graph representation of feature vectors, leveraging Fourier and functional analysis.
result Feature networks enable novel, complex feature dependencies.
Neural networks learn task-specific features, influenced by nonlinearity.
problem Understanding the nature of task-dependent feature learning in neural networks.
method Investigation of fully-connected, wide neural networks using Bayesian framework.
result The nature of internal representations depends on neuronal nonlinearity, leading to analog, redundant, or sparse coding schemes.
We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process. The generation network maps samples from stochastic latent variables to the data space while the inference network maps training examples in data space to the sp…
We study how finite Bayesian neural networks adapt their hidden representations.
problem Understanding how finite Bayesian neural networks differ from infinite ones.
method We analyze the asymptotics of learned feature kernels for various network architectures.
result The leading finite-width corrections to feature kernels have a universal form.
Sparse routing networks with co-training prevent catastrophic forgetting in continual learning.
problem Catastrophic forgetting in neural networks trained on a sequence of tasks.
method Sparse routing networks with co-training to minimize interference between dissimilar tasks.
result Sparse routing networks with co-training outperform densely connected networks on benchmarks.
Wide neural networks can learn complex functions like gravitational force law.
problem Learning complex functions like gravitational force law with neural networks.
method Extending theoretical bounds to analytic functions on the sphere using SGD and ReLU networks.
result Wide ReLU networks can learn analytic functions efficiently with proportional number of samples.
Neural networks outperform kernels by learning features better.
problem Current theories of feature learning do not adequately assess feature quality.
method Introduced feature quality metric and examined existing theories empirically.
result Current theories of feature learning do not provide a sufficient foundation for neural network generalization.
Machine learning improves network classification and model selection.
problem Quantifying suitability of generative models for network structures.
method Interpretable machine learning to classify simulated networks based on features and interactions.
result Specific network features and their interactions are crucial for distinguishing generative models.
The study examines deep convolutional neural networks and their learning ability.
problem Understanding the learning ability of deep convolutional neural networks (DCNNs).
method Examines DCNNs under both underparameterized and overparameterized settings, using a novel network deepening scheme.
result Establishes the first learning rates of underparameterized DCNNs and shows how adding layers can create interpolating DCNNs with good learning rates.
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
In recent years, there is a growing interest in learning Bayesian networks with continuous variables. Learning the structure of such networks is a computationally expensive procedure, which limits most applications to parameter learning. This problem is even more acute when learning networks with hidden variables. We p…
Paper introduces a noise-robust classification method using hypergraph neural networks.
problem Noisy label learning problem in image datasets.
method PCA for dimensionality reduction, then applies graph-based semi-supervised learning methods including hypergraph neural network.
result Our proposed hypergraph neural network achieves the best performance when noise level increases.
New neural network designs learn contact dynamics efficiently.
problem Learning contact dynamics in robotics from noisy data.
method Physically structured neural networks.
result Data-efficient learning of discontinuous contact events.
Social network analysis is an important problem in data mining. A fundamental step for analyzing social networks is to encode network data into low-dimensional representations, i.e., network embeddings, so that the network topology structure and other attribute information can be effectively preserved. Network represen…
Network representation learning has exploded recently. However, existing studies usually reconstruct networks as sequences or matrices, which may cause information bias or sparsity problem during model training. Inspired by a cognitive model of human memory, we propose a network representation learning scheme. In this …
We introduce a principled approach for unsupervised structure learning of deep neural networks. We propose a new interpretation for depth and inter-layer connectivity where conditional independencies in the input distribution are encoded hierarchically in the network structure. Thus, the depth of the network is determi…
Ensembles of neural networks learn better by sharing information.
problem Improving performance of neural networks through collective learning.
method Modeling neural networks as socially interacting agents aiming to maximize their own performance and functional relations to others.
result Optimal collective performance emerges from local interactions between networks, leading to specialization and higher confidence.
Lectures on deep learning properties in infinite and large-width networks.
problem Understanding deep neural networks in extreme width conditions.
method Analysis of random deep neural networks, connections to linear models, kernels, and Gaussian processes, perturbative and non-perturbative treatments.
result Properties and behaviors of deep neural networks in the infinite-width limit and large-width regime.
L2GMOM learns financial networks and optimizes momentum strategies.
problem Expensive databases and financial expertise limit network construction accessibility.
method End-to-end machine learning framework (L2GMOM) that learns networks and optimizes trading signals.
result Significant improvement in portfolio profitability and risk control with Sharpe ratio of 1.74.
New algorithm learns random neural networks efficiently.
problem Learning random constant-depth neural networks efficiently.
method Presented a PTAS (Polynomial-Time Approximation Scheme) for learning random Xavier networks of fixed depth.
result For any fixed ε and depth i, there is a poly-time algorithm that learns random Xavier networks up to an additive error of ε.
New neural networks learn distribution functions using quantiles and moments.
problem Approximating functions of distributions in probability spaces.
method Quantile and moment neural networks, mixing quantile and moment features.
result Moment neural network outperforms others for bivariate distributions.
Deep reinforcement learning boosts throughput in RF-powered cognitive radio networks.
problem Maximizing throughput in large-scale, decentralized RF-powered cognitive radio networks.
method Proposes deep reinforcement learning to find optimal policies for network throughput maximization.
result Deep reinforcement learning outperforms existing techniques in large-scale RF-CRN environments.
Methods for learning Bayesian network structure can discover dependency structure between observed variables, and have been shown to be useful in many applications. However, in domains that involve a large number of variables, the space of possible network structures is enormous, making it difficult, for both computati…
DyHATR learns dynamic heterogeneous networks for better link prediction.
problem Learning effective representations of dynamic heterogeneous networks for link prediction.
method Hierarchical attention for heterogeneous information and temporal RNN for evolutionary patterns.
result DyHATR significantly outperforms state-of-the-art baselines on link prediction tasks.
Gradient descent struggles to learn equivariant neural networks, even with symmetries.
problem Learning equivariant neural networks via gradient descent is hard.
method Lower bounds for various equivariant neural network classes.
result Gradient descent struggles to learn equivariant neural networks, even with symmetries.
Recent years have seen an increasing popularity of learning the sparse \emph{changes} in Markov Networks. Changes in the structure of Markov Networks reflect alternations of interactions between random variables under different regimes and provide insights into the underlying system. While each individual network struc…