The paper bounds the complexity of GCNs using Rademacher complexity.
problem Understanding the sample complexity of GCNs.
method Derived tight upper and lower bounds of Rademacher complexity for GCN models.
result The derived bounds depend on the largest eigenvalue of the graph filter and the degree distribution.
TaLK Convolutions improve sequence modeling efficiency.
problem Efficiently modeling sequences with limited time complexity.
method Adaptive convolution operation that learns kernel size.
result Time complexity reduced to O(n), making sequence encoding linear. Alternative to convolutions using decision trees for neural networks.
problem Replacing complex convolutions with simpler decision-based layers.
method Binary decisions as indices to conditional distributions, trained using backpropagation.
result Performance similar to conventional neural networks, with runtime improvements.
Efficient Winograd convolution for INT8 networks using RNS.
problem Difficulty in applying Winograd algorithm to low-precision quantized networks.
method Extends Winograd algorithm to Residue Number System (RNS) for efficient INT8 convolution.
result Arithmetic complexity reduction up to 7.03x with performance improvement up to 2.30x-4.69x.
Convolution and pooling improve kernel methods in image classification.
problem Understanding the interplay between approximation and generalization in convolutional architectures.
method Characterized RKHS of kernels with convolution, pooling, and downsampling, computed generalization error.
result Convolution and pooling operations trade off approximation with generalization power.
GCN improved for large graphs with LCF to reduce complexity and noise.
problem Efficiency and effectiveness of graph convolution in large graphs.
method Proposed Low-pass Collaborative Filter (LCF) to simplify graph convolution.
result Significant improvement in effectiveness and efficiency of GCN.
Graph Convolutional Networks improve performance on complex data.
problem Processing high-dimensional, graph-based data for automation.
method Enhanced existing Graph Convolutional Network models with four improvements.
result Significant performance improvements on four benchmark datasets.
We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution la…
DeepcomplexMRI uses deep residual networks for faster MRI imaging.
problem Accelerating parallel MR imaging with high accuracy.
method Deep complex convolutional neural network with residual connections and k-space consistency.
result The method can accurately reconstruct multi-channel MRI images.
A hybrid model reduces graph complexity for improved classification accuracy.
problem High computational complexity and large number of parameters in higher-order graph convolutional networks.
method Weight sharing mechanism and novel fusion pooling layer to reduce parameters and complexity.
result The proposed model achieves highest classification accuracy with fewer trainable parameters.
Spectral graph convolutional neural networks (CNNs) require approximation to the convolution to alleviate the computational complexity, resulting in performance loss. This paper proposes the topology adaptive graph convolutional network (TAGCN), a novel graph convolutional network defined in the vertex domain. We provi…
Protein-ligand scoring is an important step in a structure-based drug design pipeline. Selecting a correct binding pose and predicting the binding affinity of a protein-ligand complex enables effective virtual screening. Machine learning techniques can make use of the increasing amounts of structural data that are beco…
Improved singular value approximation for convolutional layers.
problem Improving accuracy of singular value approximation for linear convolutional layers.
method Developed a new spectral density matrix method for singular value approximation with improved accuracy and reduced computational complexity.
result Obtained moderate improvement in singular value distribution compared to circular approximation.
Novel graph model forecasts urban traffic with reduced spatial complexity.
problem Challenges in traffic forecasting due to spatio-temporal complexity, especially in urban environments.
method MW-TGC network model that combines spatial and temporal dependencies using multi-weighted adjacency matrices and graph convolution operations.
result MW-TGC network outperforms other models in urban-core and urban-mix sites, reducing variance in heterogeneous environments.
Convolutional neural network for probabilistic time series forecasting.
problem Forecasting multiple related time series with complex patterns.
method Temporal convolutional neural network with stacked residual blocks and dilated causal convolution.
result Outperforms state-of-the-art methods in accuracy and efficiency.
Complex-valued neural networks improve seismic data analysis by preserving phase information.
problem Low-frequency aliasing in seismic data due to discarded phase information.
method Developed complex-valued deep convolutional networks to leverage phase information in deterministic physical data.
result Complex-valued networks outperform real-valued networks in training and inference from deterministic physical data.
BScNets expands graph learning to higher-order interactions.
problem Link prediction on graphs with higher-order interactions.
method Proposes a new SNN using block Hodge-Laplacian and block simplicial complexes.
result BScNets outperforms state-of-the-art models in link prediction.
Maps geometric deformations to algebraic classes in Lie groupoids and algebroids.
problem Deformation theory of Lie groupoids and algebroids.
method Defining a morphism between deformation complexes and Hochschild complexes, applying to adiabatic groupoids.
result Induced van Est map from geometric to algebraic deformation cohomology.
A complex-valued convolutional network (convnet) implements the repeated application of the following composition of three operations, recursively applying the composition to an input vector of nonnegative real numbers: (1) convolution with complex-valued vectors followed by (2) taking the absolute value of every entry…
We present a logarithmic-scale efficient convolutional neural network architecture for edge devices, named WaveletNet. Our model is based on the well-known depthwise convolution, and on two new layers, which we introduce in this work: a wavelet convolution and a depthwise fast wavelet transform. By breaking the symmetr…
Improved model for non-smooth signals with complex spectra.
problem Current models struggle with non-smooth signals and complex spectral structures.
method CGPCM and RGPCM models with causality and Bayesian nonparametric interpretations, improved variational inference.
result Proposed models show better performance on synthetic and real-world data.
Yes, they do. This paper provides the first empirical demonstration that deep convolutional models really need to be both deep and convolutional, even when trained with methods such as distillation that allow small or shallow models of high accuracy to be trained. Although previous research showed that shallow feed-for…
Graph convolutional networks adapt the architecture of convolutional neural networks to learn rich representations of data supported on arbitrary graphs by replacing the convolution operations of convolutional neural networks with graph-dependent linear operations. However, these graph-dependent linear operations are d…
Simplicial neural networks extend graph neural networks to handle higher-order interactions.
problem Handling higher-order interactions in complex data structures.
method Define a convolution operation for simplicial complexes and use it to construct convolutional neural networks.
result SNNs effectively impute missing data in coauthorship complexes.
The success of deep convolutional architectures is often attributed in part to their ability to learn multiscale and invariant representations of natural signals. However, a precise study of these properties and how they affect learning guarantees is still missing. In this paper, we consider deep convolutional represen…
Proposes a new method to improve CNNs' shift invariance and accuracy.
problem Improving CNNs' shift invariance and prediction accuracy.
method Replaces RMax with CMod, a Gabor-like structure, to increase shift invariance and accuracy.
result Achieves superior accuracy on ImageNet and CIFAR-10 classification tasks.
ConEx learns complex embeddings for knowledge graphs, improving link prediction.
problem Predicting missing links in knowledge graphs.
method 2D convolution with Hermitian inner product of complex-valued embeddings.
result ConEx outperforms state-of-the-art methods on various benchmarks.
The paper addresses instability in CNNs' first layer by proving max pooling's shift invariance.
problem Instability in CNNs' first layer, leading to sensitivity to small input shifts.
method Establishing conditions for max pooling's shift invariance and deriving a measure of stability.
result Max pooling approximates a nearly shift-invariant complex modulus under certain conditions.
ROCKET speeds up time series classification without sacrificing accuracy.
problem High computational complexity and intractability of existing time series classification methods.
method Simple linear classifiers using random convolutional kernels.
result Achieves state-of-the-art accuracy with significantly reduced computational expense.
Proposes a new normalization method using convolutional neural networks.
problem Slow and inefficient training of deep neural networks.
method Uses depth-wise convolutional neural networks to approximate statistics.
result Learned coefficients improve the approximation of statistics.
CDEFs reduce model complexity and uncover time correlations.
problem Model complexity and data efficiency in probabilistic modeling.
method Builds on deep exponential families, ties weights for reduced parameters.
result CDEFs uncover time correlations with fewer parameters.
It is widely believed that the practical success of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) owes to the fact that CNNs and RNNs use a more compact parametric representation than their Fully-Connected Neural Network (FNN) counterparts, and consequently require fewer training examples to…
Paper introduces signal processing on cell complexes.
problem Processing signals on non-Euclidean domains.
method Signal processing on abstract regular cell complexes.
result Hodge Laplacians for cell complexes enable convolutional filters.
Previous research has shown that computation of convolution in the frequency domain provides a significant speedup versus traditional convolution network implementations. However, this performance increase comes at the expense of repeatedly computing the transform and its inverse in order to apply other network operati…
Convolutional Neural Networks (CNNs) filter the input data using spatial convolution operators with compact stencils. Commonly, the convolution operators couple features from all channels, which leads to immense computational cost in the training of and prediction with CNNs. To improve the efficiency of CNNs, we introd…
L-GCNs learn from complex multigraphs, improving node classification performance.
problem Learning from complex multigraphs with rich edge labels.
method Latent-Graph Convolutional Networks (L-GCNs) that propagate information to a latent adjacency tensor.
result L-GCNs improve node classification performance, especially with nonlinear interactions.
New neural model processes 2D data with long-range dependencies efficiently.
problem Limited receptive field of convolutions for complex 2D tasks.
method Proposes Matrix Shuffle-Exchange network with O(logn) layers and O(n2logn) complexity. result Exceeds convolutional and graph neural network baselines in long-range dependency modeling.
In this work, we are interested in generalizing convolutional neural networks (CNNs) from low-dimensional regular grids, where image, video and speech are represented, to high-dimensional irregular domains, such as social networks, brain connectomes or words' embedding, represented by graphs. We present a formulation o…
Empirical scoring functions based on either molecular force fields or cheminformatics descriptors are widely used, in conjunction with molecular docking, during the early stages of drug discovery to predict potency and binding affinity of a drug-like molecule to a given target. These models require expert-level knowled…
New findings on how convolutional architectures approximate time series data.
problem Understanding the approximation properties of convolutional architectures in time series modeling.
method Mathematical analysis of convolutional architectures applied to time series modeling.
result A new definition of spectrum-based regularity for measuring temporal relationships under convolutional approximation.
Study explains Graph Convolutional Networks decisions.
problem Difficulty in understanding Graph Network decisions.
method Gradient-based and decomposition-based techniques.
result Sets groundwork for future explainability development.
Improved generalization bounds for CNNs using Rademacher complexity.
problem Establishing non-vacuous generalization bounds for deep learning models.
method Rademacher complexity framework with novel contraction lemmas for high-dimensional mappings.
result Enhanced generalization bounds for a broader class of activation functions.
Enhances speech emotion recognition by adapting to varying time scales.
problem Robust emotion recognition from speech audio with temporal variations.
method Introduces multi-time-scale (MTS) convolutional layers to CNNs.
result MTS layers improve generalization, especially on smaller datasets.
We present our ongoing work on understanding the limitations of graph convolutional networks (GCNs) as well as our work on generalizations of graph convolutions for representing more complex node attribute dependencies. Based on an analysis of GCNs with the help of the corresponding computation graphs, we propose a gen…
New algorithm speeds up spike sorting for large datasets.
problem Numerical complexity limits processing large scale neuroscience datasets.
method Windowed active set Lasso algorithm for convolutional spike sorting.
result Linear complexity ensures scalability and opens online sorting.
A new method for comparing image probability measures using convolution operators.
problem Efficiently comparing images using conventional sliced Wasserstein methods.
method Proposed convolution sliced Wasserstein (CSW) methods with stride, dilation, and non-linear activation.
result CSW demonstrates favorable performance over conventional sliced Wasserstein in image comparison and deep generative modeling.
Deep convolutional neural networks (CNNs) have been shown to be able to fit a random labeling over data while still being able to generalize well for normal labels. Describing CNN capacity through a posteriori measures of complexity has been recently proposed to tackle this apparent paradox. These complexity measures a…
ie-HGCN addresses HIN challenges by efficiently learning node representations.
problem Lack of flexibility in exploring meta-paths and high computational complexity in HIN GCN methods.
method Hierarchical aggregation architecture that automatically extracts useful meta-paths and reduces computational cost.
result ie-HGCN outperforms state-of-the-art methods on real network datasets.