Direct convolution eliminates memory overhead and improves performance.
problem Memory overhead and suboptimal performance in convolution layers.
method Implementing direct convolution without additional memory overhead.
result Performance improvement between 10% to 400% on various architectures.
Extends GCNs to directed graphs for better performance.
problem Limited application of GCNs to undirected graphs.
method Uses first- and second-order proximity to extend GCNs to directed graphs.
result DGCN outperforms state-of-the-art methods on citation and co-purchase datasets.
Improved spectral-based GCN for directed graphs.
problem Cannot directly work on directed graphs.
method Redefined Laplacians to improve propagation model.
result Outperforms state-of-the-art methods on directed graph datasets.
Novel Haar-Laplacian for directed graphs enhances spectral graph applications.
problem Lack of suitable Laplacian for directed graphs in spectral graph theory.
method Inspired by Haar-like transformation, introduces a Hermitian matrix preserving direction and weight.
result HaarNet outperforms in weight prediction and denoising on directed graphs.
Paper proposes a method to speed up DNNs by quantizing Winograd/Toom-Cook convolutions.
problem Speeding up convolution computations in DNNs with reduced time consumption and improved accuracy.
method Application of base change technique for quantized Winograd-aware training model.
result 8-bit quantized network achieves nearly the same accuracy as direct quantized convolution with minimal additional operations.
Optimizes CNNs by directing gradients along output channels.
problem Improving generalization error in CNNs.
method Output-channel directed re-weighted L2 or Sobolev metrics.
result Improves generalization error by optimizing gradients.
Develops neural network for directed hypergraphs for node classification.
problem Irregular data structure, particularly directed graphs.
method Directed hypergraph neural network and semi-supervised learning method.
result Novel directed hypergraph neural network achieves highest accuracies on node classification tasks.
Graph diffusion convolution improves graph learning by leveraging generalized graph diffusion.
problem Noisy and arbitrarily defined edges in real graphs.
method Graph diffusion convolution (GDC) using generalized graph diffusion like heat kernel and personalized PageRank.
result Replacing message passing with graph diffusion convolution leads to significant performance improvements.
Study reveals how neural network architectures bias their learning based on feature directions.
problem Understanding how neural network architectures bias their learning based on feature directions.
method Defined neural anisotropy directions (NADs) to encapsulate the directional inductive bias of architectures and provided an efficient method to identify them.
result NADs characterize the features used by CNNs to discriminate between different classes for the CIFAR-10 dataset.
We study the problem of explaining a rich class of behavioral properties of deep neural networks. Distinctively, our influence-directed explanations approach this problem by peering inside the network to identify neurons with high influence on a quantity and distribution of interest, using an axiomatically-justified in…
Graph convolutional networks adapt the architecture of convolutional neural networks to learn rich representations of data supported on arbitrary graphs by replacing the convolution operations of convolutional neural networks with graph-dependent linear operations. However, these graph-dependent linear operations are d…
A new CNN approach for time series forecasting outperforms traditional RNNs.
problem Time series forecasting using conventional RNNs.
method Temporally folded convolutional neural networks (TFC's) for sequence forecasting.
result TFC's outperform conventional RNNs on sequential MNIST and JSB chorals datasets.
The paper improves DFA for CNN and RNN training to match BP accuracy.
problem Low accuracy in CNN and RNN training using DFA.
method Divide network into modules, apply DFA within, use sparse backward weight, and incorporate dilated convolution and sparse matrix multiplication.
result Achieves BP-level accuracy in CNN and RNN training.
Deep learning predicts fluid flow in porous media, accelerating simulations by orders of magnitude.
problem Accurate simulation of fluid flow in complex porous media requires excessive computational resources.
method Combining deep learning with direct simulation, using Gated U-Net CNNs trained on datasets of 2D and 3D porous media.
result Deep learning predictions can reach over 90% accuracy for permeability estimation and accelerate simulations by orders of magnitude.
Numerous important problems can be framed as learning from graph data. We propose a framework for learning convolutional neural networks for arbitrary graphs. These graphs may be undirected, directed, and with both discrete and continuous node and edge attributes. Analogous to image-based convolutional networks that op…
Winograd convolution improves DNN accuracy in fp16 and bf16 formats.
problem Improving DNN accuracy in floating point formats.
method Investigated a wider range of Winograd algorithms for DNNs.
result Significant improvement in FP accuracy in fp16 and bf16 formats.
Convolutional neural network improves MRE image reconstruction.
problem Reconstructing MRE images from displacement data is computationally intensive and costly.
method Proposes a CNN architecture to directly map MRE displacement data into elastograms, introducing a secondary loss for training.
result CNN-generated images compare favorably with nonlinear inversion methods.
Deep learning predicts S&P 500 index direction.
problem Accurate stock price prediction remains challenging.
method Convolutional neural network model for S&P 500 index forecasting.
result Model achieves over 55% accuracy in predicting index direction.
Trellis networks improve sequence modeling performance.
problem Sequence modeling challenges.
method Temporal convolutional network with weight tying and direct input injection.
result Trellis networks outperform state-of-the-art methods on benchmarks.
Generating novel graph structures that optimize given objectives while obeying some given underlying rules is fundamental for chemistry, biology and social science research. This is especially important in the task of molecular graph generation, whose goal is to discover novel molecules with desired properties such as …
Minimalistic model captures head direction system properties.
problem Representing head direction system in a high-dimensional space.
method A minimalistic representation model of the rotation group U(1), including fully connected and convolutional versions.
result Emergence of Gaussian-like tuning profiles and 2D circle geometry in both model versions.
Develops a new neural spike train decoding framework using topological data.
problem Decoding neural spike trains from head direction and grid cells.
method Combines simplicial complex discovery with deep learning to capture higher-order connectivity.
result Demonstrates effectiveness on head direction and trajectory prediction datasets.
NAS helps find best neural network designs.
problem Designing optimal neural network architectures.
method Optimization algorithms and search spaces.
result Introduction to major advances in NAS for CNNs.
Image denoising is always a challenging task in the field of computer vision and image processing. In this paper, we have proposed an encoder-decoder model with direct attention, which is capable of denoising and reconstruct highly corrupted images. Our model consists of an encoder and a decoder, where the encoder is a…
New training method improves neural network performance.
problem Improving neural network performance with direct feedback alignment.
method Direct feedback alignment with best practices and observations.
result Characterization of bottleneck effect in narrow layers.
DenseNets improve accuracy and efficiency in convolutional networks.
problem Improving accuracy and efficiency in deep convolutional networks.
method Introducing Dense Convolutional Networks (DenseNet) with direct connections between all layers.
result DenseNets achieve significant improvements over state-of-the-art networks on object recognition benchmarks.
A new 2.5D U-net for 3D segmentation reduces memory constraints.
problem Large storage requirements for 3D convolutions in neural networks.
method Transform volumetric data into sequences of 2D images, apply 2D convolutions, and reconstruct.
result Outperforms existing methods in volumetric segmentation tasks.
MeshCNN analyzes 3D shapes using edges, overcoming irregularities.
problem Irregularities in mesh representations hinder neural network analysis.
method MeshCNN uses specialized convolution and pooling layers on mesh edges, collapsing them to focus on important features.
result MeshCNN effectively analyzes 3D shapes, learning which edges to collapse.
We solve the compressive sensing problem via convolutional factor analysis, where the convolutional dictionaries are learned {\em in situ} from the compressed measurements. An alternating direction method of multipliers (ADMM) paradigm for compressive sensing inversion based on convolutional factor analysis is develope…
We give the first provably efficient algorithm for learning a one hidden layer convolutional network with respect to a general class of (potentially overlapping) patches. Additionally, our algorithm requires only mild conditions on the underlying distribution. We prove that our framework captures commonly used schemes …
DCGANs generate drainage networks quickly from samples.
problem High computational costs in generating large numbers of drainage networks.
method DCGANs trained with connectivity-informed directional information.
result Connectivity-informed DCGANs outperform other methods in reproducing accurate drainage networks.
This work compares NN architectures for spectrum sensing.
problem Choosing the best neural network architecture for spectrum sensing.
method Comparison of fully-connected NN (FC), CNN, RNN, and BiRNN.
result CNN, RNN, and BiRNN achieve similar performance.
Efficient inference for adaptive data with directional stability condition.
problem Efficient inference on scalar targets after adaptive data collection.
method Introduces directional stability, a weaker condition than i.i.d. data, and shows asymptotic normality and efficiency of estimators.
result Estimators remain asymptotically normal and semiparametrically efficient under directional stability.
Machine learning predicts critical points for directed percolation models.
problem Determining critical points for directed percolation models.
method Supervised and unsupervised machine learning algorithms (CNN and DBSCAN) were used.
result Machine learning accurately predicts critical points for both models.
Paper explores offline and online image recognition using neural networks.
problem Challenges in evolving image recognition through different settings.
method Used Convolutional Neural Networks and Multi-layer Perceptrons.
result Encouraging preliminary results in offline and online image classification.
AngularGrad optimizes CNNs by considering gradient direction, improving convergence.
problem Dying gradient problem and inefficiency in exploiting gradient curvature.
method AngularGrad considers gradient direction/angle, generating a score for step size control.
result AngularGrad outperforms state-of-the-art optimizers in benchmark tests.
Model projection transfers convolutional network properties to feedforward networks.
problem Transferring properties between feedforward and convolutional networks.
method Unified node-level framework with tensor-valued activations, model projection.
result Projected CNN nodes inherit GFFN-style trainable structure.
Improved CNN training with BDFA reduces computational cost and improves accuracy.
problem Low training performance of DFA in CNN.
method Combining DFA with BP, introducing feedback weight initialization, and proposing BDFA.
result BDFA shows better performance than conventional BP, especially in small datasets.
Automated method finds meaningful directions in neural network activations.
problem Mixed selectivity in neurons makes interpretation challenging.
method Automated quantification of interpretability and discovery of meaningful directions.
result Meaningful directions in neural network activations are more interpretable than individual neurons.
NeuralArTS categorizes neural ops in a type system for NAS.
problem Manual optimization of search spaces for NAS is inefficient.
method Developed NeuralArTS, a type system for categorizing network ops.
result NeuralArTS can be applied to convolutional layers.
Evolutionary method constructs CNNs for data compression and classification.
problem Creating efficient CNNs for data compression and classification.
method Two-step approach using evolutionary algorithms: 1) Convolutional autoencoder, 2) Convolutional neural network. Compression trade-off considered.
result Framework achieves comparable accuracy to hand-crafted networks, demonstrating effectiveness.
Unified deep architecture for domain-invariant network alignment.
problem Eliminate domain representation bias in network alignment.
method DANA (Domain Adversarial Network Alignment) using graph convolutional networks and semi-supervised learning.
result Achieves state-of-the-art alignment results on real-world social networks.
TSAM predicts directed temporal links using GCN and self-attention.
problem Predicting links in directed temporal networks.
method GCN, self-attention mechanism, autoencoder architecture, graph attentional layers, graph convolutional layers, graph recurrent unit layer.
result TSAM outperforms benchmarks on four realistic networks.
Vector-valued neural learning has emerged as a promising direction in deep learning recently. Traditionally, training data for neural networks (NNs) are formulated as a vector of scalars; however, its performance may not be optimal since associations among adjacent scalars are not modeled. In this paper, we propose a n…
CompGCN embeds nodes and relations in multi-relational graphs.
problem Handling multi-relational graphs with direction and labels.
method CompGCN uses entity-relation composition operations from KG embedding.
result CompGCN achieves superior results on node classification, link prediction, and graph classification.
A new method for comparing image probability measures using convolution operators.
problem Efficiently comparing images using conventional sliced Wasserstein methods.
method Proposed convolution sliced Wasserstein (CSW) methods with stride, dilation, and non-linear activation.
result CSW demonstrates favorable performance over conventional sliced Wasserstein in image comparison and deep generative modeling.
We embed KKT points in neural networks of different sizes.
problem Classifying data using homogeneous neural networks.
method Introducing KKT point embedding principle and proving it for different network types.
result KKT points of a smaller network can be mapped to those of a larger network via linear transformations.
Graph convolutional networks refine organ segmentation using uncertainty analysis.
problem Challenges in organ segmentation due to variability and tissue similarity.
method Uncertainty analysis of graph convolutional networks for semi-supervised learning.
result Improved segmentation accuracy (1% for pancreas, 2% for spleen) compared to state-of-the-art methods.