Deep neural network predicts traffic flow on city maps.
problem Short-term traffic flow prediction on high-resolution city maps.
method UNet-based deep convolutional neural network with densely connected layers.
result Best performance on the Traffic4cast challenge 2019.
This work proposes hyperbolic deep convolutional neural networks for better pattern recognition.
problem The limitations of Euclidean deep convolutional neural networks in capturing intricate patterns.
method Developed Hyperbolic DCNN based on Poincaré Disc, analyzing expansive convolution in non-Euclidean space.
result Hyperbolic convolutional architecture outperforms Euclidean ones in pattern recognition tasks.
Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. The involved deep neural network architectures and computational issues have been well studied in machine learning. But there lacks a theoretical foundation for understanding the approximation…
Proves DCNNs with expansive convolution are strongly universally consistent.
problem Theoretical consistency of deep convolutional neural networks (DCNNs).
method Empirical risk minimization on DCNNs with expansive convolution (with zero-padding).
result DCNNs with expansive convolution are strongly universally consistent.
The study examines deep convolutional neural networks and their learning ability.
problem Understanding the learning ability of deep convolutional neural networks (DCNNs).
method Examines DCNNs under both underparameterized and overparameterized settings, using a novel network deepening scheme.
result Establishes the first learning rates of underparameterized DCNNs and shows how adding layers can create interpolating DCNNs with good learning rates.
The paper verifies deep neural networks' ability to approximate functions on spheres.
problem Theoretical verification of deep neural networks' performance on spherical functions.
method Spherical analysis using reproducing kernels and convolutional factorizations.
result Rates of uniform approximation for functions in Sobolev spaces and additive ridge forms.
The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network parameters. Reducing the number of parameters while preserving essentially the same predictive performance is critically important …
Channel normalization prevents vanishing gradients in convolutional neural networks.
problem Vanishing gradients in convolutional neural networks during optimization.
method Channel normalization, which centers and normalizes each channel individually.
result Channel normalization avoids vanishing gradients, enabling efficient optimization.
The paper explains implicit regularization in hierarchical tensor factorization and deep CNNs.
problem Understanding implicit regularization in complex neural network architectures.
method Theoretical analysis using dynamical systems to overcome challenges in hierarchy.
result Established implicit regularization towards low hierarchical tensor rank, equivalent to locality in CNNs.
Enhanced deep CNNs improve cardiac abnormality diagnosis from ECGs.
problem Diagnosing cardiac abnormalities from 12-lead ECGs.
method Training an enhanced deep convolutional neural network with hand-crafted features, data preprocessing, and augmentation.
result Promising generalization performance in ECG diagnosis.
Spiking neural networks perform similarly to deep networks on occluded images.
problem Robust object recognition in partially occluded images.
method Developed a two-layer spiking neural network trained on natural scenes with a biologically plausible learning rule, compared to deep convolutional networks.
result Spiking neural networks achieve good accuracy and robustness on stepwise pixel erasement tasks.
Proves bounds on deep CNN generalization error.
problem Understanding how well deep CNNs perform on unseen data.
method Bounds on generalization error based on training loss, parameters, Lipschitz constant, and weight distance.
result Bounds are independent of input size and feature map dimensions.
Convolutional networks struggle with repeating patterns in ECGs.
problem Modeling repeating patterns in electrocardiogram signals.
method Demonstrated through ECG examples, highlighting systemic issues in deep learning.
result Counterintuitive effects on generalization in deep networks.
The paper analyzes deep ReLU CNNs' approximation properties in 2D space.
problem Establishing L2 approximation properties for deep ReLU CNNs. method Analysis based on decomposition theorem for convolutional kernels, properties of ReLU activation, and connections with one-hidden-layer ReLU NNs.
result Universal approximation theorem for deep ReLU CNNs with classic structure.
AdaGCN uses AdaBoost to efficiently integrate high-order neighbor knowledge in graph neural networks.
problem Efficiently exploring and exploiting knowledge from different hops of neighbors in graph neural networks.
method Incorporates AdaBoost into graph convolutional networks to integrate knowledge from high-order neighbors.
result AdaGCN achieves state-of-the-art prediction performance across different graphs and label rates.
New mechanism discovered for feature learning in CNNs.
problem Understanding how CNNs learn features from images.
method Proposed Convolutional Neural Feature Ansatz linking filter covariances to patch-based AGOPs.
result Deep ConvRFM algorithm learns features similar to deep CNNs, improving performance.
Review of neural network expressivity and architectures.
problem Understanding neural network expressivity across different architectures.
method Comprehensive overview of approximation results for various neural network types.
result Deep neural networks offer advantages over shallow ones for specific function classes.
We introduce a new deep convolutional neural network, CrescendoNet, by stacking simple building blocks without residual connections. Each Crescendo block contains independent convolution paths with increased depths. The numbers of convolution layers and parameters are only increased linearly in Crescendo blocks. In exp…
In this paper, some patterns of the Neuron Response of deep Convolutional Neural Networks were observed.
CMDRNN predicts user location using WiFi fingerprints with deep learning.
problem Predicting user activity with WiFi fingerprints is challenging due to high dimensionality.
method Combines CNN, RNN, and MDN to model high-dimensional time-series data.
result CMDRNN effectively predicts user location using WiFi fingerprints.
This paper removes the finite variance assumption for deep convolutional neural networks.
problem Removing the finite variance assumption for deep convolutional neural networks.
method Assuming iid parameters distributed according to a stable distribution, the paper shows that the infinite-channel limit of a deep feed-forward convolutional neural network is a multivariate stable stochastic process.
result The infinite-channel limit of a deep feed-forward convolutional neural network, under suitable scaling, is a multivariate stable stochastic process.
A modified VDCNN model reduces size and latency for mobile platforms.
problem Memory and processing constraints on mobile platforms.
method Temporal Depthwise Separable Convolutions and Global Average Pooling.
result The squeezed model (SVDCNN) is 10x-20x smaller with minimal accuracy loss.
In recent work, it was shown that combining multi-kernel based support vector machines (SVMs) can lead to near state-of-the-art performance on an action recognition dataset (HMDB-51 dataset). This was 0.4\% lower than frameworks that used hand-crafted features in addition to the deep convolutional feature extractors. I…
Deep convolutional neural networks comprise a subclass of deep neural networks (DNN) with a constrained architecture that leverages the spatial and temporal structure of the domain they model. Convolutional networks achieve the best predictive performance in areas such as speech and image recognition by hierarchically …
A hybrid method improves Convolutional Neural Networks training.
problem Training Convolutional Neural Networks efficiently and avoiding local minima.
method Combines backpropagation with evolutionary strategies.
result Improves accuracy by 0.61% on CIFAR-10 image classification.
Complex-valued neural networks improve seismic data analysis by preserving phase information.
problem Low-frequency aliasing in seismic data due to discarded phase information.
method Developed complex-valued deep convolutional networks to leverage phase information in deterministic physical data.
result Complex-valued networks outperform real-valued networks in training and inference from deterministic physical data.
Convolutional neural networks outperform other architectures in streaming time series classification.
problem Efficient deep learning models for real-time data streams.
method Asynchronous dual-pipeline deep learning framework for real-time predictions.
result Convolutional architectures achieve higher accuracy and efficiency in streaming time series classification.
Deep neural network learns robust acoustic models from speech waveforms.
problem Robustness in speech recognition systems using standard feature extraction techniques.
method Deep convolutional neural network with stochastic variational inference and cosine modulated filters.
result Superior performance compared to baseline waveform-based models and deep CNNs with FBANK features.
Proposes a method to constrain singular values of convolutional kernels in neural networks.
problem Avoiding exploding/vanishing gradient problems and improving generalizability in neural networks.
method Introduces a penalty function to constrain singular values of convolutional kernels around 1, and derives an algorithm for optimization.
result Demonstrates the effectiveness of the method through numerical examples.
Deep neural networks are a powerful tool for feature learning and extraction given their ability to model high-level abstractions in highly complex data. One area worth exploring in feature learning and extraction using deep neural networks is efficient neural connectivity formation for faster feature learning and extr…
New bounds for neural networks ensure robustness and accuracy.
problem Ensuring robustness of neural networks by computing Lipschitz constants.
method Analyzed and proposed new bounds for l1 and l∞ norms, using explicit and implicit methods for convnets. result One of the new bounds is optimal and more accurate than existing ones.
Deep learning predicts stock market trends using candlestick charts.
problem Predicting stock market prices with multiple influencing factors.
method Used Deep Convolutional Neural Networks and candlestick charts.
result 92.2% and 92.1% accuracy for Taiwan and Indonesian stock markets.
Paper proposes a quantum deep clustering framework with improved performance.
problem Improving clustering performance in quantum machine learning.
method Quantum deep SVM, deep convolutional neural networks, and quantum K-Means clustering.
result The proposed quantum deep clustering framework shows significant performance gains over classical methods.
Deep neural networks solve optimal risk sharing problems.
problem Optimally sharing financial positions among agents with different risk measures.
method Neural network-based framework to compute inf-convolution and optimal allocations.
result Convergence of neural network approximations to theoretical values.
Image understanding using deep convolutional network has reached human-level performance, yet a closely related problem of video understanding especially, action recognition has not reached the requisite level of maturity. We combine multi-kernels based support-vector-machines (SVM) with a multi-stream deep convolution…
Recently the generalization error of deep neural networks has been analyzed through the PAC-Bayesian framework, for the case of fully connected layers. We adapt this approach to the convolutional setting.
Deep Learning Library (DLL) is a new library for machine learning with deep neural networks that focuses on speed. It supports feed-forward neural networks such as fully-connected Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs). It also has very comprehensive support for Restricted Boltzmann …
Study uses CNNs to estimate BMI from photos.
problem Estimating BMI from photos with limited data.
method Used deep learning (CNNs) to process silhouette images.
result High correlation between estimated and actual BMI.
Two deep learning models improve indoor location prediction from WiFi fingerprints.
problem Indoor location prediction from WiFi fingerprints.
method Convolutional mixture density recurrent neural network and VAE-based semi-supervised learning model.
result Proposed models outperform existing methods in real-world datasets.
Model based iterative reconstruction (MBIR) algorithms for low-dose X-ray CT are computationally expensive. To address this problem, we recently proposed a deep convolutional neural network (CNN) for low-dose X-ray CT and won the second place in 2016 AAPM Low-Dose CT Grand Challenge. However, some of the texture were n…
PSTN improves traffic condition forecasting with deep neural networks.
problem Challenges in accurately forecasting traffic conditions due to complex spatiotemporal correlations.
method Proposes PSTN with three modules: graph convolutional network, temporal convolutional network, and gated recurrent unit framework.
result Significantly outperforms state-of-the-art benchmarks in short-term traffic conditions forecasting.
Deep neural networks are powerful learning models that achieve state-of-the-art performance on many computer vision, speech, and language processing tasks. In this paper, we study a fundamental question that arises when designing deep network architectures: Given a target network architecture can we design a smaller ne…
Visualizes deep neural networks for speech recognition using learned topographic filter maps.
problem Unintuitive internal structure of deep neural networks complicates activation visualization.
method Trains a convolutional speech recognition model with filters arranged in a 2D grid, highlighting similar filters.
result Topographic filter maps visualize artificial neuron activations more intuitively.
In recent years, China, the United States and other countries, Google and other high-tech companies have increased investment in artificial intelligence. Deep learning is one of the current artificial intelligence research's key areas. This paper analyzes and summarizes the latest progress and future research direction…
Paper explores offline and online image recognition using neural networks.
problem Challenges in evolving image recognition through different settings.
method Used Convolutional Neural Networks and Multi-layer Perceptrons.
result Encouraging preliminary results in offline and online image classification.
Paper studies a universal logical operator for deep networks, improving interpretability.
problem Learning a universal logical operator for deep convolution networks without manual prescription.
method Exploration of different logical operators (AND, OR, XOR) and learning a universal one.
result Insightful observations lead to a novel logical interpretation of deep convolution networks.
Deep neural nets predict aircraft flight paths from weather data.
problem Accurate prediction of aircraft trajectories for aviation efficiency.
method Deep generative convolutional recurrent neural network with tree-based matching.
result Model accurately predicts aircraft flight paths from weather data.
Paper improves speech separation by using deep neural networks for more accurate density priors.
problem Improving the accuracy of source priors for independent vector analysis in speech separation.
method Estimating the derivative of speech density using deep neural networks to optimize performance indices.
result Neural network density priors outperform previous ones in convergence speed and SIR.