arXiv research

A novel multi-layer architecture for one-class classification using graph-embedded kernel ridge regression.

FMMNN combines sine activations with multi-component, multi-layer structure for high-frequency function approximation.

Relational Networks (RN) as introduced by Santoro et al. (2017) have demonstrated strong relational reasoning capabilities with a rather shallow architecture. Its single-layer design, however, only considers pairs of information objects, making it unsuitable for problems requiring reasoning across a higher number of fa…

Parsimonious representations are ubiquitous in modeling and processing information. Motivated by the recent Multi-Layer Convolutional Sparse Coding (ML-CSC) model, we herein generalize the traditional Basis Pursuit problem to a multi-layer setting, introducing similar sparse enforcing penalties at different representat…

Lookahead pruning extends single-layer optimization to multi-layer, outperforming magnitude-based pruning.

Paper reviews ranking systems in multi-layered architectures.

Modern data analysis pipelines are becoming increasingly complex due to the presence of multi-view information sources. While graphs are effective in modeling complex relationships, in many scenarios a single graph is rarely sufficient to succinctly represent all interactions, and hence multi-layered graphs have become…

GCNs improve multi-layer network classification by expanding the distance between means.

SRR detects early signs of financial crises using multi-layer graphs.

How can local-search methods such as stochastic gradient descent (SGD) avoid bad local minima in training multi-layer neural networks? Why can they fit random labels even given non-convex and non-smooth architectures? Most existing theory only covers networks with one hidden layer, so can we go deeper? In this paper, w…

Improved recommendation systems using multi-layer embeddings reduce model size while maintaining accuracy.

New method initializes sigmoidal MLPs for interpretable shapes.

Transformers can learn spectral methods and perform unsupervised learning.

Despite their impressive performance, deep convolutional neural networks (CNNs) have been shown to be sensitive to small adversarial perturbations. These nuisances, which one can barely notice, are powerful enough to fool sophisticated and well performing classifiers, leading to ridiculous misclassification results. In…

AMEIR automates recommender system design using NAS.

Linear attention in Transformers can be interpreted as dynamic VAR models.

Deep neural networks solve Raven's Progressive Matrices with high accuracy.

CascadeXML improves multi-resolution learning for XMC with transformer features.

New view: Deep GCNs learn to anti-oversmooth during training.

Theoretical analysis of the error landscape of deep neural networks has garnered significant interest in recent years. In this work, we theoretically study the importance of noise in the trajectories of gradient descent towards optimal solutions in multi-layer neural networks. We show that adding noise (in different wa…

In this paper, a multi-layer architecture (in a hierarchical fashion) by stacking various Kernel Ridge Regression (KRR) based Auto-Encoder for one-class classification is proposed and is referred as MKOC. MKOC has many layers of Auto-Encoders to project the input features into new feature space and the last layer was r…

This study compares mtl architectures for renewable power generation forecasting.

DeepAM optimizes deep neural networks for image super-resolution.

In this work, we contribute a new multi-layer neural network architecture named ONCF to perform collaborative filtering. The idea is to use an outer product to explicitly model the pairwise correlations between the dimensions of the embedding space. In contrast to existing neural recommender models that combine user em…

Two spectral clustering methods for multi-layer networks are analyzed and compared.

When using deep, multi-layered architectures to build generative models of data, it is difficult to train all layers at once. We propose a layer-wise training procedure admitting a performance guarantee compared to the global optimum. It is based on an optimistic proxy of future performance, the best latent marginal. W…

Recurrent Neural Networks (RNNs) achieve state-of-the-art results in many sequence-to-sequence modeling tasks. However, RNNs are difficult to train and tend to suffer from overfitting. Motivated by the Data Processing Inequality (DPI), we formulate the multi-layered network as a Markov chain, introducing a training met…

New methods estimate mixed memberships in multi-layer networks.

TWIST algorithm detects communities in multi-layer networks with tensor decomposition.

Two-stage neural network for few-shot image recognition.

This work tackles asymmetric community estimation in multi-layer directed networks.

Proposes a balanced multi-component and multi-layer neural network for efficient function approximation.

New AMP algorithms improve multi-layer signal reconstruction.

New model for multi-layer categorical data improves latent class analysis.

Learned data models based on sparsity are widely used in signal processing and imaging applications. A variety of methods for learning synthesis dictionaries, sparsifying transforms, etc., have been proposed in recent years, often imposing useful structures or properties on the models. In this work, we focus on sparsif…

Smooth KANs improve model reliability in computational biomedicine.

New neural networks combine additive regression with traditional architectures.

Graph embedding is an important approach for graph analysis tasks such as node classification and link prediction. The goal of graph embedding is to find a low dimensional representation of graph nodes that preserves the graph information. Recent methods like Graph Convolutional Network (GCN) try to consider node attri…

New framework for neural networks converging to low loss without overparameterization.

Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve le…

Bayesian Optimisation (BO) refers to a class of methods for global optimisation of a function $f$ which is only accessible via point evaluations. It is typically used in settings where $f$ is expensive to evaluate. A common use case for BO in machine learning is model selection, where it is not possible to analytically…

Efficient neural network invariant to symmetry subgroups.

New method detects OOD samples using neural network trajectories.

Paper presents ML-VAMP for efficient multi-layer inference with exact performance analysis.

In recent years there has been an increased interest in statistical analysis of data with multiple types of relations among a set of entities. Such multi-relational data can be represented as multi-layer graphs where the set of vertices represents the entities and multiple types of edges represent the different relatio…

Polynomial-time convex optimization for CNNs with ReLU activations.

Deep learning using multi-layer neural networks (NNs) architecture manifests superb power in modern machine learning systems. The trained Deep Neural Networks (DNNs) are typically large. The question we would like to address is whether it is possible to simplify the NN during training process to achieve a reasonable pe…

A new model improves the performance of knowledge distillation in GNNs.