Study combines CNNs and LSTMs for ECG classification, improving performance with attention mechanisms.
problem Improving ECG classification accuracy for weakly supervised tasks.
method Investigates ConvLSTM networks and various attention mechanisms to capture both morphological and temporal features.
result Attention mechanisms improve classification performance and provide interpretable saliency maps.
Simpler CNN model with spatial attention and temporal pooling outperforms complex models.
problem Emotion recognition from videos with small face deformations and identity variations.
method Spatial attention mechanism and temporal softmax pooling applied to a pre-trained CNN.
result The approach achieves higher accuracy than state-of-the-art methods on the EmotiW dataset.
Deep CNN detects mind wandering from EEG data.
problem Detecting mind wandering to reorient attention.
method Channel-wise deep convolutional neural network (CNN) model.
result 91.78% accuracy in detecting mind wandering.
Proposes GPCA module for channel attention in CNNs using Gaussian processes.
problem Improving performance in visual tasks through effective channel selection.
method Integrates Gaussian processes into channel attention mechanisms for probabilistic modeling of channel correlations.
result Demonstrates improved performance of GPCA module in end-to-end CNN training.
Improved action detection for multi-person videos using attention filtering.
problem Difficulty in distinguishing relevant parts of multi-person videos for action detection.
method Fovea attention filtering and generalized binary loss function.
result 20% relative improvement in mAP over baseline in AVA dataset.
A new channel locality block improves CNN performance.
problem Improving the performance of convolutional neural networks.
method Proposed a variant of Squeeze-and-Excitation block using convolutional layers to learn nearby channel correlation.
result Our C-Local block achieved higher accuracy than the standard SE block on the cifar-10 dataset.
A new neural network separates vocals from music accompaniment.
problem Separating vocals from music accompaniment in recordings.
method Self-attention convolutional neural network (CNN) with densely-connected blocks.
result 19.5% relative improvement in vocals separation.
Self-attention can replace convolutional layers in vision tasks.
problem The supremacy of convolutional layers in vision tasks.
method Analysis and experiments with self-attention layers compared to convolutional layers.
result Self-attention layers can perform as well as convolutional layers and learn to do so.
Self multi-head attention improves speaker recognition for long utterances.
problem Speaker recognition for long speech segments using Deep Learning.
method Convolutional Neural Network (CNN) for short-term features, self multi-head attention for long-term embeddings.
result Self multi-head attention outperforms other pooling methods by 18% relative EER on VoxCeleb1 dataset.
Graph-augmented CNN predicts drug interactions with high accuracy.
problem Predicting drug-drug interactions (DDIs) with high accuracy.
method Combining graph CNN with an attentive pooling network to extract structural relations between drug pairs.
result Desirable performance with ROC 0.988, F1-score 0.956, and AUPR 0.986.
Hybrid model predicts stock prices with high accuracy.
problem Complex volatility of stock market makes traditional models unsatisfactory.
method Attention-based CNN-LSTM and XGBoost integrated model.
result Hybrid model improves prediction accuracy.
In recent years, deep learning poses a deep technical revolution in almost every field and attracts great attentions from industry and academia. Especially, the convolutional neural network (CNN), one representative model of deep learning, achieves great successes in computer vision and natural language processing. How…
This paper describes the participation of Amobee in the shared sentiment analysis task at SemEval 2018. We participated in all the English sub-tasks and the Spanish valence tasks. Our system consists of three parts: training task-specific word embeddings, training a model consisting of gated-recurrent-units (GRU) with …
Proposes a novel framework for multi-label text classification.
problem Lack of coherent consideration of non-consecutive and long-distance semantics and hierarchical relations among labels.
method Hierarchical taxonomy-aware and attentional graph capsule recurrent CNNs framework.
result Significantly improves multi-label text classification performance.
The paper investigates how Global Self-attention improves GCNs.
problem Improving expressive power and addressing overfitting in GCNs.
method Applying Global Self-attention mechanism over graph features.
result GSA mechanism enhances expressive power and mitigates overfitting and over-smoothing in GCNs.
UAG defends GNNs against adversarial attacks by quantifying and explaining uncertainties.
problem Lack of uncertainty quantification in GNNs makes them vulnerable to adversarial attacks.
method UAG uses Bayesian Uncertainty Technique (BUT) and Uncertainty-aware Attention Technique (UAT).
result UAG outperforms state-of-the-art solutions in defending adversarial attacks on GNNs.
Capsule networks improve performance on image classification tasks with fewer parameters.
problem Improving performance of capsule networks with fewer parameters.
method Inverted dot-product attention routing, Layer Normalization, concurrent iterative routing.
result Improves performance on benchmark datasets CIFAR-10 and CIFAR-100, and performs at-par with ResNet-18.
Study improves fraud detection in e-commerce with a stacked model combining CNNs, GNNs, and confidence gating.
problem Detecting credit card fraud in online transactions.
method Stacking approach with attention and confidence-driven layers, using DOWA and IOWA operators.
result The method achieves high accuracy and robust generalization in CCF detection.
The wide adoption of Convolutional Neural Networks (CNNs) in applications where decision-making under uncertainty is fundamental, has brought a great deal of attention to the ability of these models to accurately quantify the uncertainty in their predictions. Previous work on combining CNNs with Gaussian processes (GPs…
iSTFTNet2 improves iSTFTNet's speed and lightness with 1D-2D CNN.
problem Efficiently synthesizing high-fidelity speech.
method Improved iSTFTNet using 1D-2D CNNs for temporal and spectrogram structures.
result iSTFTNet2 is faster and more lightweight with comparable speech quality.
A new type of End-to-End system for text-dependent speaker verification is presented in this paper. Previously, using the phonetically discriminative/speaker discriminative DNNs as feature extractors for speaker verification has shown promising results. The extracted frame-level (DNN bottleneck, posterior or d-vector) …
ConViT combines CNN and ViT strengths, improving image classification.
problem Combining the strengths of CNNs and ViTs while avoiding their limitations.
method Introducing GPSA, a form of positional self-attention with a soft convolutional inductive bias.
result ConViT outperforms DeiT on ImageNet while offering improved sample efficiency.
Attention-based CNNs improve band selection in hyperspectral images.
problem Selecting informative bands from hyperspectral images for accurate classification.
method Attention-based convolutional neural networks reusing activations at different depths.
result Deep models with attention mechanisms achieve high-quality classification and identify significant bands.
Paper attacks CNNs in EEG BCIs with adversarial examples.
problem Vulnerability of CNN classifiers in EEG-based BCIs.
method UFGSM to attack three popular CNN classifiers in BCIs.
result Demonstrates effectiveness of adversarial attacks on CNNs in BCIs.
Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP share the same learned (and static) set of filters for all input sentences. In this paper, we consider an approach of using a smal…
Model classifies environment sounds using multiple feature channels and attention mechanisms.
problem Environment sound classification task.
method Multiple feature channels (MFCC, GFCC, CQT, Chromagram) and attention mechanism in a deep CNN.
result Achieves state-of-the-art performance on three benchmark datasets.
CNNPred uses CNNs to predict stock market movements across multiple markets.
problem Feature extraction and prediction in financial markets.
method CNN-based framework for multiple markets, using various data sources.
result Significant improvement in prediction performance compared to baseline algorithms.
Scale-equivariant CNNs handle scale changes for improved performance.
problem Translation equivariance is not sufficient for handling scale changes in CNNs.
method Developed scale-equivariant convolutional networks with steerable filters.
result Demonstrated state-of-the-art results on MNIST-scale and STL-10 datasets.
Vision Transformers show different internal representations compared to CNNs.
problem Understanding how Vision Transformers solve image classification tasks.
method Comparative analysis of ViT and CNN architectures on image classification benchmarks.
result ViT has more uniform representations across all layers, while CNNs have more varied representations.
Morph accelerates 3D CNNs for video recognition, reducing energy consumption and improving performance.
problem Efficiently accelerating 3D CNNs for video recognition is challenging due to their large memory footprint and higher dimensionality.
method Designing a flexible accelerator called Morph that adapts to different spatial and temporal tiling strategies, and codesigning a software infrastructure to control the hardware.
result Morph achieves up to 3.4x reduction in energy consumption and up to 5.1x improvement in performance/watt compared to a baseline 3D CNN accelerator.
Convolutional Neural Networks (CNN) outperform traditional classification methods in many domains. Recently these methods have gained attention in neuroscience and particularly in brain-computer interface (BCI) community. Here, we introduce a CNN optimized for classification of brain states from magnetoencephalographic…
Proposes CNN-based analog CSI feedback for FDD MIMO-OFDM systems.
problem High CSI feedback overhead in FDD MIMO systems.
method AnalogDeepCMC: maps downlink CSI to uplink channel input, reconstructs channel estimate.
result Significantly improves downlink spectral efficiency and simplifies operation.
Method generates visual explanations for similarity models without classification.
problem Lack of visual explanations for similarity models trained without classification loss.
method Gradient-based visual attention using learned feature embeddings.
result Attention maps improve model performance and can be used as constraints.
A new CNN-based code generator outperforms RNNs by 5%.
problem Capturing long sequences in code generation using RNNs.
method Grammar-based structural CNN with tree-based and pre-order convolution modules.
result Significantly outperforms previous state-of-the-art methods by 5 percentage points.
Numerous pattern recognition applications can be formed as learning from graph-structured data, including social network, protein-interaction network, the world wide web data, knowledge graph, etc. While convolutional neural network (CNN) facilitates great advances in gridded image/video understanding tasks, very limit…
CTLNet predicts Shanghai Composite Index with improved accuracy.
problem Predicting Shanghai Composite Index for investors and researchers.
method Proposes CNN-Transformer-LSTM Networks (CTLNet) for multivariate time series forecasting.
result CTLNet outperforms state-of-the-art baselines in Shanghai Composite Index prediction.
Novel GNN model tackles few-shot learning with improved performance.
problem Few-shot learning with GNN suffers from over-fitting and over-smoothing.
method Proposes Attentive GNN with triple-attention mechanism.
result Improves GNN performance for few-shot learning tasks.
New neural network improves audio classification accuracy.
problem Challenging audio classification problem in pattern recognition.
method Introduces a Classifier-Attention-Based Convolutional Neural Network (CAB-CNN) with an attention mechanism to reduce classifier complexity.
result Significantly improves audio classification performance, achieving more than 10% improvements.
Transformers use a unique Hessian structure that differs from classical networks, affecting optimization.
problem Understanding the unique optimization landscape of Transformers.
method Theoretical Hessian analysis of a single self-attention layer in Transformers.
result Transformers have a highly non-linear Hessian structure, distinguishing them from classical networks.
Transformers predict price movements from limit order books.
problem Predicting price movements from limit order books.
method Causal convolutional network with masked self-attention.
result Significantly outperforms existing architectures on FI-2010 dataset.
Convolutional neural networks (CNN) have led to many state-of-the-art results spanning through various fields. However, a clear and profound theoretical understanding of the forward pass, the core algorithm of CNN, is still lacking. In parallel, within the wide field of sparse approximation, Convolutional Sparse Coding…
CNNs help diagnose diabetic retinopathy by localizing lesions.
problem Diabetic retinopathy diagnosis requires identifying lesions in fundus images.
method Post-attention technique (Grad-CAM) on deep learning models' penultimate layer.
result InceptionV3 model achieves best performance and localizes lesions better.
New approach reduces malware detection memory requirements and speeds up training.
problem Efficiently classifying long sequences of malware detection data.
method Developed a new temporal max pooling method and global channel gating design.
result 116x more memory efficient and 25.8x faster training on original dataset.
The paper applies Information Bottleneck theory to CNNs and finds compression phase not always present.
problem Understanding the behavior of convolutional neural networks.
method Employed Information Bottleneck theory to analyze CNNs.
result Compression phase not observed in all CNN cases.
A model for hit song prediction can be used in the pop music industry to identify emerging trends and potential artists or songs before they are marketed to the public. While most previous work formulates hit song prediction as a regression or classification problem, we present in this paper a convolutional neural netw…
Two KG-based methods explain transfer learning in CNN and ZSL.
problem Uninterpretable transfer learning for non-ML experts.
method Knowledge Graph-based explanation for transferability and model justification.
result Rich, human-understandable explanations for transfer learning.
Deep learning models compare performance on Limit Order Book tasks.
problem Comparing Deep Learning models for High Frequency Trading.
method Reviewed and compared state-of-the-art models on the same dataset.
result Multilayer Perceptrons perform comparably to CNN-LSTM architectures.
GA-Net selectively attends to parts of a sequence for text classification.
problem Inefficient global attention mechanisms on long sequences.
method Gated Attention Network (GA-Net) using an auxiliary network to dynamically select and attend to important parts of the sequence.
result GA-Net achieves better performance with less computation and interpretability.