ConvNets can be translated into CKNs that perform similarly.
problem The distinction between ConvNets and kernel-based methods.
method Translation of ConvNets into CKNs using a new gradient algorithm.
result CKNs perform as well as ConvNets, supporting the translation.
Graph convolutional kernel networks generalize CNNs to graph data.
problem Representing graph-structured data for machine learning.
method Convolutional kernel networks applied to graph data.
result Competitive performance on graph classification benchmarks.
Proposes a method to constrain singular values of convolutional kernels in neural networks.
problem Avoiding exploding/vanishing gradient problems and improving generalizability in neural networks.
method Introduces a penalty function to constrain singular values of convolutional kernels around 1, and derives an algorithm for optimization.
result Demonstrates the effectiveness of the method through numerical examples.
VC dimensions of group CNNs are infinite for certain kernels and groups.
problem Estimating the generalization capacity of group convolutional neural networks.
method Identifying precise VC dimension estimates for simple sets of group CNNs.
result Two-parameter families of convolutional neural networks have an infinite VC dimension for infinite groups and certain kernels.
Improved CNN with general image processing kernels reduces training time and achieves high accuracy.
problem Training time and accuracy of CNNs.
method Used 41 general-purpose kernels for the first layer of CNNs.
result GFNN reduces training time by 30% and achieves 99.56% accuracy.
New image representation using supervised convolutional kernel networks.
problem Learning optimal kernel representations for image classification.
method Proposed supervised convolutional kernel networks (CKNs) with backpropagation for supervised learning.
result Achieves competitive performance on image classification and super-resolution tasks.
Enhanced LSTM with multiple kernels and attention improves video action recognition.
problem Improving motion understanding in video analysis.
method Proposed a Network-in-LSTM approach with multiple convolutional kernels and layers, and an attention-based mechanism.
result Improves accuracy in supervised classification on UCF-101 and Sports-1M datasets.
New method bounds singular values of convolutional kernels to stabilize gradients.
problem Stable gradients in convolutional neural networks.
method Frobenius norm regularization for convolutional kernels.
result Bounded singular values of transformation matrices.
Deep convolutional networks can be understood through kernel methods, providing insights into their inductive bias.
problem Understanding the functional space and inductive bias of deep convolutional networks.
method Using kernel methods to analyze simple hierarchical kernels with convolution and pooling layers.
result The RKHS consists of additive models of interaction terms between patches, and pooling layers encourage spatial similarities.
Convolution and pooling improve kernel methods in image classification.
problem Understanding the interplay between approximation and generalization in convolutional architectures.
method Characterized RKHS of kernels with convolution, pooling, and downsampling, computed generalization error.
result Convolution and pooling operations trade off approximation with generalization power.
Kernel Quantization improves CNN compression without sacrificing performance.
problem Efficiently compressing CNN models without significant performance loss.
method Quantizes convolution kernels as the unit, learning a codebook for low-bit indexes.
result Significant compression ratio achieved with minimal accuracy loss.
A new convolutional spectral kernel network learns hierarchical and local features.
problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.
We describe convolutional networks using harmonic functions.
problem Understanding the function space and smoothness of convolutional networks.
method Using reproducing kernel Hilbert spaces and functional ANOVA decomposition.
result Convolutional networks can be decomposed into a sum of elementary functions.
Proposes flexible dilation networks for better time series analysis.
problem Fixed dilation limits flexibility in time series analysis.
method End-to-end learnable dilation layers and independent kernels.
result Improves efficiency and flexibility in training.
Smooth kernel regularizer improves deep neural networks' performance with less data.
problem Deep neural networks need large datasets for effective learning.
method Proposes a smooth kernel regularizer that encourages spatial correlations in convolution kernel weights, learned from previous experience.
result The smooth kernel regularizer improves visual recognition models over an L2 regularization baseline.
Convolutional neural networks for graphs improve feature extraction efficiency.
problem Learning from graph data, especially for complex attributes.
method Proposed a framework for learning convolutional neural networks on arbitrary graphs.
result Learned feature representations are competitive with graph kernels and efficient.
ROCKET speeds up time series classification without sacrificing accuracy.
problem High computational complexity and intractability of existing time series classification methods.
method Simple linear classifiers using random convolutional kernels.
result Achieves state-of-the-art accuracy with significantly reduced computational expense.
The paper analyzes deep convolutional representations and their properties.
problem Understanding the properties of deep convolutional architectures.
method Introducing a multilayer kernel based on convolutional kernel networks and studying the geometry induced by the kernel mapping.
result Characterization of the RKHS and its relation to model complexity and generalization.
Convolutional DKMs improve kernel methods on MNIST, CIFAR-10, and CIFAR-100.
problem Improving kernel methods for image classification.
method Developed a novel inter-domain inducing point approximation and introduced various techniques to extend DKMs to convolutional networks.
result Achieved state-of-the-art performance on image classification benchmarks.
HKConv learns hyperbolic features by aggregating kernel points.
problem Challenges in learning good hyperbolic representations using Euclidean operations.
method Proposes HKConv, a trainable hyperbolic convolution that correlates local features with kernel points and aggregates them.
result HKConv learns expressive local features according to hyperbolic geometry and enjoys equivariance to permutation and invariance to parallel transport.
DeepTD learns CNN weights from non-overlapping patches using tensor decomposition.
problem Learning weights of a deep convolutional neural network (CNN).
method Deep Tensor Decomposition (DeepTD) based on rank-1 tensor decomposition.
result DeepTD is data-efficient and works as soon as sample size exceeds total number of weights.
The paper proposes an ensemble of convolution-based methods for fault detection in gearboxes.
problem Fault detection in planetary gearboxes using vibration signals.
method Ensemble of three convolution kernel-based methods (ROCKET, 1D CNN with ResNet, FCN).
result Outperforms other approaches with over 98.8% accuracy.
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
A guide simplifies convolutional neural network properties.
problem Understanding and manipulating convolutional neural network architectures.
method Clarifying relationships between properties of convolutional, pooling, and transposed convolutional layers.
result Intuitive relationships between various convolutional and transposed convolutional layers.
Researchers aim to understand TextCNN's learning on NLP datasets.
problem Interpreting TextCNN is challenging due to its black box nature.
method Deep visualization tools are used to understand TextCNN's functions and correlations.
result Functions of different convolutional kernels and correlations between them are explored.
Empirical study shows standard CNNs deviate from NTK predictions.
problem Understanding how standard finite-width CNNs behave compared to their infinite-width NTK counterparts.
method Empirical analysis of AlexNet and LeNet architectures.
result Standard CNNs deviate significantly from their NTK counterparts, but deviation decreases with wider networks.
Study the inductive bias of neural networks using neural tangent kernels.
problem Understanding the generalization properties of over-parameterized neural networks.
method Analysis of the neural tangent kernel and its corresponding function space (RKHS).
result Stability properties of functions with finite norm, including stability to image deformations in convolutional networks.
New mechanism discovered for feature learning in CNNs.
problem Understanding how CNNs learn features from images.
method Proposed Convolutional Neural Feature Ansatz linking filter covariances to patch-based AGOPs.
result Deep ConvRFM algorithm learns features similar to deep CNNs, improving performance.
Symmetric CNNs improve sequential recommendation and protein structure prediction.
problem Improving prediction accuracy in sequential recommendation and protein structure inference.
method Developed a CNN architecture that preserves symmetry in convolutional layers, using parameterized convolutional kernels.
result Symmetric structured CNNs achieve better performance with fewer parameters.
CNNs become Gaussian processes with many filters, achieving state-of-the-art performance.
problem Training deep CNNs is computationally expensive.
method Showed that CNNs with many filters can be approximated by Gaussian processes, and computed the equivalent kernel efficiently.
result The kernel equivalent of a 32-layer ResNet achieves 0.84% classification error on MNIST.
New kernel-based models improve on traditional neural methods in sequence modeling.
problem Sequence modeling challenges in natural language processing and neuroscience.
method Kernel-based recurrent neural networks and convolutional neural networks.
result Kernel-based models perform on par or better than traditional neural methods.
Efficiently implements large-scale CNNs using memristors.
problem Implementing high-dimensional convolution in deep neural networks efficiently.
method Developed a method to convert CNN kernels to memristor-based circuits, optimized for accuracy and minimized error.
result 8-bit ADC/DAC is necessary to preserve software level classification accuracy in memristor-based CNNs.
End-to-end graph SVM with graph convolutions and RKHS.
problem Graph classification with complex feature spaces.
method End-to-end training of graph convolutions, kernel function, and SVM parameters.
result Outperforms existing deep learning models on graph classification tasks.
Improved deep learning for action recognition using multi-kernel SVM and deep neural networks.
problem Challenges in video understanding, especially action recognition, despite deep neural networks' success in image understanding.
method Combining multi-kernel SVM with a multi-stream deep convolutional neural network, including hand-crafted features.
result Achieved close to state-of-the-art performance on the HMDB-51 dataset.
Locality helps in learning from high-dimensional data.
problem Understanding how convolutional neural networks learn from high-dimensional data.
method Teacher-student framework for kernel regression with convolutional kernels.
result Locality is key to determining the learning curve exponent in high-dimensional data.
Network deconvolution removes redundant data to improve neural network performance.
problem Redundant data in neural networks makes training challenging.
method Network deconvolution optimally removes pixel-wise and channel-wise correlations before each layer.
result Network deconvolution improves performance across various datasets.
SKN learns multi-layer nonlinear features using kernel methods.
problem Limited representational power of classic kernel methods.
method Interleaves layers of nonlinear and linear transformations in a hierarchy of RKHS-based features.
result SKN and SKCN outperform competitive methods on various datasets.
New filter bank regularization improves DCNNs by incorporating image priors.
problem Improving DCNNs' robustness and generality.
method Structured filter bank regularization of DCNN kernels.
result Filter bank regularization leads to faster convergence and better generalization.
Structured sparsity improves deep neural networks for real-time applications.
problem High computational complexity and memory access issues in deep learning.
method Structured sparsity at various scales (channel, kernel, intra kernel) using particle filtering for importance weighting.
result Significant reduction in network size and storage requirements for real-time applications.
Unified theory for adaptive image convolutions using metric perspectives.
problem Fixed kernels in convolutions limit adaptability in image processing.
method Metric perspective on images as 2D manifolds with local distances, proposing metric convolutions.
result Metric convolutions provide better generalisation and competitive performance.
Graph convolutional deep kernel machine learns representations for graph tasks.
problem Limited representation learning in infinite-width neural networks.
method Developed a graph convolutional deep kernel machine as an infinite-width limit.
result Representation learning improves performance for heterophilous node classification tasks.
A CNN-based method detects and counts corn kernels from images.
problem Manual counting of corn kernels is labor-intensive and prone to error.
method Sliding window approach with CNN for detection and NMS for overlapping removal.
result The method successfully detects and counts kernels with low error.
Randomly shuffled kernels can be compressed efficiently.
problem Reducing storage cost of CNN parameters on resource-limited platforms.
method Randomly-shuffled tensor decomposition (RsTD) to embed kernels into random low-rank subspaces.
result CNNs can be significantly compressed even with randomly shuffled kernels, achieving more stable accuracy.
Efficiently learns 3D convolutions with less data.
problem High parameter and data costs in 3D convolutions.
method Temporal factorization of 3D kernels.
result Significantly reduces training data requirement and parameter count.
New IP analysis for deep neural networks using Rényi's entropy and tensor kernels.
problem Estimating mutual information in high-dimensional hidden layers of deep neural networks.
method Matrix-based Rényi's entropy coupled with tensor kernels for convolutional layers.
result First comprehensive IP analysis of large-scale DNNs and CNNs.
The paper verifies deep neural networks' ability to approximate functions on spheres.
problem Theoretical verification of deep neural networks' performance on spherical functions.
method Spherical analysis using reproducing kernels and convolutional factorizations.
result Rates of uniform approximation for functions in Sobolev spaces and additive ridge forms.
KM method reduces ConvNet parameters to 9% higher accuracy with minimal additional memory.
problem Expensive memory usage for training ConvNets on embedded devices.
method Kernel Modulation (KM) method that adapts all network parameters for each task.
result KM delivers up to 9% higher accuracy than other parameter-efficient methods.
AutoQ automatically optimizes quantization for CNNs, reducing latency and energy.
problem Efficiently quantizing CNN weights for low-power mobile devices.
method Hierarchical-DRL for kernel-wise quantization bitwidth selection.
result Reduces inference latency and energy consumption by 54.06% and 50.69% respectively.