Improved student networks boost image accuracy by 10% with same inference time.
problem Improving neural network efficiency on general hardware.
method Attention transfer and channel saliency metrics combined with runtime performance observations.
result Performance-enhanced student networks achieve up to 10% higher ImageNet accuracy.
Binary neural networks trained from scratch achieve state-of-the-art results.
problem Training accurate binary neural networks from scratch is challenging.
method No prior knowledge and simple training strategy used.
result Achieved state-of-the-art results on standard benchmark datasets.
MS-FCN improves cardiac left ventricle segmentation accuracy.
problem Accurate segmentation of left ventricle from MRI images.
method Multi-scale fully convolutional network with dense connectivity.
result MS-FCN achieves state-of-the-art Dice scores (0.93 on endocardium, 0.96 on epicardium).
Nontrivial connectivity has allowed the training of very deep networks by addressing the problem of vanishing gradients and offering a more efficient method of reusing parameters. In this paper we make a comparison between residual networks, densely-connected networks and highway networks on an image classification tas…
DenseNets improve accuracy and efficiency in convolutional networks.
problem Improving accuracy and efficiency in deep convolutional networks.
method Introducing Dense Convolutional Networks (DenseNet) with direct connections between all layers.
result DenseNets achieve significant improvements over state-of-the-art networks on object recognition benchmarks.
Detects anomalous behavior in social media users by analyzing content and connections.
problem Identifying disruptive patterns in user behavior on social media platforms.
method Joint representation learning of content and connection to detect anomalous behavior.
result Observed densely connected users engaging in local politics and exhibiting troll-like behavior.
Improves audio source separation using dilated convolutions and dense connections.
problem Optimizing feature extraction in audio source separation models.
method Adaptive dilated convolutions and dense connections in U-Net architecture.
result Improved performance on MUSDB test dataset.
Finding "densely connected clusters" in a graph is in general an important and well studied problem in the literature \cite{Schaeffer}. It has various applications in pattern recognition, social networking and data mining \cite{Duda,Mishra}. Recently, Ames and Vavasis have suggested a novel method for finding cliques i…
A new deep learning method for tissue-cleared image registration.
problem Efficient registration of high-resolution tissue-cleared images.
method Densely connected convolutional architecture for deformable image registration, unsupervised training.
result Comparable and superior registration performance to state-of-the-art methods, especially at higher resolutions.
Deep neural network predicts traffic flow on city maps.
problem Short-term traffic flow prediction on high-resolution city maps.
method UNet-based deep convolutional neural network with densely connected layers.
result Best performance on the Traffic4cast challenge 2019.
New neural network architectures use signed permutation representations for finite groups, improving performance.
problem Designing and optimizing neural networks for finite groups with signed permutation representations.
method Introduces G-invariant deep neural networks with densely connected layers and signed permutation representations. result Signed permutation representations lead to significantly better performance in classification tasks.
Capsule Networks improve clothing retrieval without landmark info.
problem In-shop clothing retrieval performance improvement.
method Triplet-based Capsule Network architecture with SC and RC blocks.
result Triplet Capsule Networks outperform FashionNet and SOTA architectures.
Novel graph theory for neural networks improves understanding of their structure and performance.
problem Understanding the structural benefits and generalization power of neural networks.
method Developed a novel graph theoretical formulation and extended error analysis for neural networks.
result Similar a priori estimates can be obtained for neural networks under certain conditions, independent of input dimension.
Dense neural networks can't approximate all functions.
problem Approximation capabilities of dense neural networks.
method Model compression approach combining weak regularity lemma and graph neural networks.
result Existence of Lipschitz continuous functions not approximable by dense neural networks.
Networks, which represent agents and interactions between them, arise in myriad applications throughout the sciences, engineering, and even the humanities. To understand large-scale structure in a network, a common task is to cluster a network's nodes into sets called "communities", such that there are dense connection…
Method integrates functional data into neural networks for better interpretability.
problem Integrating functional data into neural networks for improved interpretability.
method Integrates functional data into deep neural networks, visualizes dynamic weights.
result Model performs well in prediction and recovery of functional weights.
New method selects neural network architectures without needing data.
problem Choosing efficient deep neural network architectures.
method Developed the deep frame potential to quantify network capacity.
result Deep frame potential correlates with generalization error.
In this work we introduce a time- and memory-efficient method for structured prediction that couples neuron decisions across both space at time. We show that we are able to perform exact and efficient inference on a densely connected spatio-temporal graph by capitalizing on recent advances on deep Gaussian Conditional …
A new neural network separates vocals from music accompaniment.
problem Separating vocals from music accompaniment in recordings.
method Self-attention convolutional neural network (CNN) with densely-connected blocks.
result 19.5% relative improvement in vocals separation.
A new algorithm speeds up neural network derivative calculations.
problem Exponential runtime of autodifferentiation for high-order derivatives in neural networks.
method n-TangentProp, a quasilinear algorithm for computing higher-order derivatives.
result Computes exact derivatives in quasilinear time, not exponential.
FineMorphs models smooth transformations for multivariate regression.
problem Efficiently modeling complex transformations for multivariate regression.
method Optimal control of affine and diffeomorphic transformations using smooth vector fields.
result FineMorphs can reduce dimensionality and adapt to large datasets.
Sparse routing networks with co-training prevent catastrophic forgetting in continual learning.
problem Catastrophic forgetting in neural networks trained on a sequence of tasks.
method Sparse routing networks with co-training to minimize interference between dissimilar tasks.
result Sparse routing networks with co-training outperform densely connected networks on benchmarks.
GHNet improves graph learning by balancing homogeneity and heterogeneity.
problem Over-smoothing in GCN leads to similar node representations.
method GHNet uses gating units to balance homogeneity and heterogeneity in feature propagation.
result GHNet achieves larger receptive fields without over-smoothing.
Achieving international food security requires improved understanding of how international trade networks connect countries around the world through the import-export flows of food commodities. The properties of food trade networks are still poorly documented, especially from a multi-network perspective. In particular,…
DVNet efficiently segments large neurovascular datasets using skip connections.
problem Challenges in segmenting large neurovascular datasets from high-throughput microscopy data.
method A fully-convolutional, deep, and densely-connected encoder-decoder network with skip connections.
result DVNet achieves superior performance in semantic segmentation of neurovascular datasets.
CAggNet improves medical image segmentation by fusing coarse and fine features.
problem Medical image segmentation accuracy and efficiency.
method Crossing Aggregation Network with nested skip connections and weighted aggregation.
result CAggNet achieves more accurate and efficient segmentation compared to existing methods.
In this work, we investigate the value of employing deep learning for the task of wireless signal modulation recognition. Recently in [1], a framework has been introduced by generating a dataset using GNU radio that mimics the imperfections in a real wireless channel, and uses 10 different modulation types. Further, a …
Economies grow by upgrading the type of products they produce and export. The technology, capital, institutions and skills needed to make such new products are more easily adapted from some products than others. We study the network of relatedness between products, or product space, finding that most upscale products a…
Networks are a convenient way to represent complex systems of interacting entities. Many networks contain "communities" of nodes that are more densely connected to each other than to nodes in the rest of the network. In this paper, we investigate the detection of communities in temporal networks represented as multilay…
We present a new notion of probabilistic duality for random variables involving mixture distributions. Using this notion, we show how to implement a highly-parallelizable Gibbs sampler for weakly coupled discrete pairwise graphical models with strictly positive factors that requires almost no preprocessing and is easy …
RDL-Net improves speech enhancement with fewer parameters and better performance.
problem Improving speech enhancement with fewer parameters and better performance.
method Proposes RDL-Net, a CNN combining residual and dense aggregations without over-allocating parameters.
result RDL-Net achieves higher speech enhancement performance with fewer parameters and lower computational requirements.
The state of the art lung nodule detection studies rely on computationally expensive multi-stage frameworks to detect nodules from CT scans. To address this computational challenge and provide better performance, in this paper we propose S4ND, a new deep learning based method for lung nodule detection. Our approach use…
This paper proposes a new randomized strategy for adaptive MCMC using Bayesian optimization. This approach applies to non-differentiable objective functions and trades off exploration and exploitation to reduce the number of potentially costly objective function evaluations. We demonstrate the strategy in the complex s…
SWNets optimize DL architectures for faster convergence.
problem Excessive training parameters in deep learning models.
method Transforms network topology to reach Small-World Network boundary.
result SWNets achieve faster convergence with fewer parameters.
The paper uses neural networks to segment brain tumors and predict patient survival.
problem Brain tumor segmentation and survival prediction.
method Fully convolutional neural network with encoder-decoder architecture, radiomic features, and random forest regression.
result 55.4% classification accuracy for predicting survival of patients with gross-total resection.
This paper introduces a hierarchical associative memory model with multiple layers.
problem Limitations of traditional associative memory models with only one hidden layer.
method Develops a fully recurrent model with arbitrary layers, including locally connected ones, and a corresponding energy function.
result The model can dynamically assemble memories using weights from lower layers and higher layers' rules.
Real-world networks usually have community structure, that is, nodes are grouped into densely connected communities. Community detection is one of the most popular and best-studied research topics in network science and has attracted attention in many different fields, including computer science, statistics, social sci…
Efficiently processes dynamic inputs in AI writing assistants with incremental computation.
problem Efficiently updating AI models in real-time with dynamic inputs.
method Incremental computing using vector quantization to filter and reuse intermediate values in neural networks.
result Comparable accuracy with 12.1X fewer operations for processing dynamic inputs.
Graph energy helps detect communities in networks better than traditional methods.
problem Detecting communities in sparse networks where traditional methods fail.
method Using graph energy based on the full spectrum of adjacency matrices.
result The difference in graph energy between a planted partition model and an Erdős--Rényi network has a distinct transition at the detectability threshold.
Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep n…
RobNets improve robustness of deep neural networks against adversarial attacks.
problem Vulnerability of deep neural networks to adversarial attacks.
method One-shot neural architecture search, finetuning sub-networks, and analyzing robustness indicators.
result RobNets achieve superior robustness performance across various datasets.
Chaos in cerebellar cells enhances complexity of neural patterns.
problem Understanding how cerebellar granular layer represents complex information.
method Constructed a model of cerebellar granular layer with gap junctions, evaluated using reservoir computing.
result Chaotic dynamics in the cerebellar granular layer produce complex and diverse output patterns.
New models explain residual and dilated dense neural networks using sparse coding.
problem Lack of theoretical understanding of residual and dilated dense neural networks.
method Proposed Res-CSC and MSD-CSC models, derived mathematical relationships, implemented ISTA.
result Mathematical understanding of residual and dilated dense neural networks.
Study reveals how illiquidity network signals Chinese stock market crashes.
problem Understanding and predicting Chinese stock market crashes.
method Established an illiquidity network to model market dynamics.
result Market crashes are preceded by a more densely connected illiquidity network.
This paper proposes a principled information theoretic analysis of classification for deep neural network structures, e.g. convolutional neural networks (CNN). The output of convolutional filters is modeled as a random variable Y conditioned on the object class C and network filter bank F. The conditional entropy (CENT…
Graph kernel uses Ricci curvature for comparison.
problem Graph comparison without node attributes.
method Edge curvature distribution for graph kernel.
result Graphs can be compared using topology alone.
The development of new technology such as wearables that record high-quality single channel ECG, provides an opportunity for ECG screening in a larger population, especially for atrial fibrillation screening. The main goal of this study is to develop an automatic classification algorithm for normal sinus rhythm (NSR), …
Partitioning a graph into groups of vertices such that those within each group are more densely connected than vertices assigned to different groups, known as graph clustering, is often used to gain insight into the organisation of large scale networks and for visualisation purposes. Whereas a large number of dedicated…