Deep convolutional nets are essential for accurate learning on CIFAR-10.
problem Training shallow models to mimic deep convolutional nets on CIFAR-10.
method Used distillation to train shallow feed-forward nets on CIFAR-10, demonstrating the necessity of multiple convolutional layers.
result Accurate models on CIFAR-10 require multiple convolutional layers, even when trained with distillation.
New algorithm improves kernel-based deep net performance.
problem Understanding deep learning and its mysteries.
method Exact computation of Convolutional NTK (CNTK) for infinite-width convolutional nets.
result CNTK results in 10% higher performance on CIFAR-10 than previous methods. New deep learning methods improve CT image quality from few projections.
problem Sparse-view CT images suffer from streaking artifacts due to limited projections.
method Inspired by deep convolutional framelets, propose new U-Net variants that satisfy the frame condition.
result New U-Net variants provide better reconstruction performance for sparse-view CT.
Proposes Π-Nets, polynomial neural networks, for improved representation power.
problem Improving representation power in deep learning models.
method Introduces Π-Nets, a new class of deep polynomial neural networks. result Demonstrates Π-Nets outperform standard DCNNs and achieve state-of-the-art results. Z-Net improves 3D CT volume segmentation for surgical planning.
problem Discontinuities and class-imbalances in 3D CT volume segmentation.
method Z-Net uses anisotropic spatial separable convolutions to preserve full field-of-view.
result Z-Net achieves up to 12.6% improvement in IoU for CT segmentation.
Neural net reconstructs dark matter density from halo velocities.
problem Reconstructing local dark matter density from halo velocities.
method Hybrid architecture combining U-Net and DeepSets.
result Hybrid network recovers density amplitudes and phases better than U-Net.
Convolutional attractor nets improve image completion and super-resolution.
problem Construct coherent neural states from noisy data.
method Revisit and extend attractor networks with convolutional bipartite architecture.
result Demonstrates potential for image completion and super-resolution.
Researchers shrink U-Net to find limits of retinal vessel segmentation.
problem Improving retinal vessel segmentation with deep learning.
method Modified U-Net with functional blocks, then simplified to extreme conditions.
result U-Net does not degrade until very minimal configurations.
DGC-SPNs improve SPNs for image data by combining CNNs and SPNs.
problem SPNs struggle with complex spatial relationships in images.
method Integrates CNNs with SPNs, using novel parameterization for dilations and strides.
result Significantly improved feature coverage and resolution compared to existing SPN architectures.
HAR-Net combines deep features with traditional hand-crafted features for better human activity recognition.
problem Challenges in traditional HAR methods, especially feature extraction.
method Combines deep learning and traditional feature engineering.
result Performance improvement of 0.9% compared to traditional SVM.
TCNs can approximate complex input-output maps with limited memory.
problem Approximating complex input-output maps with limited memory.
method Proved TCNs can approximate a wide class of input-output maps with arbitrary error tolerance.
result Deep ReLU TCNs can approximate input-output maps with finite memory to arbitrary error.
RDL-Net improves speech enhancement with fewer parameters and better performance.
problem Improving speech enhancement with fewer parameters and better performance.
method Proposes RDL-Net, a CNN combining residual and dense aggregations without over-allocating parameters.
result RDL-Net achieves higher speech enhancement performance with fewer parameters and lower computational requirements.
Infinitely wide neural nets perform well on small datasets.
problem Performing well on small datasets with limited training samples.
method Using Neural Tangent Kernels (NTKs) for kernel regression.
result NTK SVM outperforms Random Forests and Convolutional NTK on CIFAR-10 with 10-640 training samples.
A new meta-learning method improves deep neural net training efficiency.
problem Efficient training of complex deep neural networks with long training processes.
method Meta-learning with Hessian-Free (MLHF) approach based on Hessian-Free optimization.
result MLHF shows good and continuous training performance in deep convolution neural nets.
Butterfly-Net improves CNN performance with structured connections and initialization.
problem Improving the performance of convolutional neural networks (CNNs).
method Butterfly-Net introduces structured and sparse cross-channel connections, and Butterfly initialization strategy.
result Butterfly-Net approximates Fourier representations with exponentially decaying error as depth increases.
Paper tackles dermoscopy lesion boundary detection using deep learning.
problem Automatic detection of lesion boundaries in dermoscopy images.
method Adapted U-net convolutional neural network with skip connections.
result Demonstrates effectiveness of deep learning in lesion boundary segmentation.
NAIS-Net stabilizes deep networks using non-autonomous dynamical systems.
problem Stabilizing deep neural networks to prevent vanishing/exploding gradients.
method NAIS-Net uses non-autonomous dynamical systems with skip connections to enforce stability.
result NAIS-Net proves to be globally asymptotically stable and reduces generalization gap.
U-Time uses a fully convolutional network for sleep stage classification.
problem Challenges in tuning and optimizing recurrent neural networks for sleep data.
method U-Time is a fully feed-forward deep learning approach based on U-Net architecture.
result U-Time outperforms state-of-the-art models for sleep stage classification.
BCD-Net uses identical CNN structures for image recovery in undersampled imaging.
problem Challenges in obtaining accurate images from undersampled or noisy measurements.
method Incorporates image mapping CNN into BCD signal recovery method using alternating direction method of multipliers.
result Significantly more accurate image recovery compared to existing methods.
PS8-Net improves eight-state protein secondary structure prediction accuracy.
problem Precise prediction of eight-state protein secondary structure (PSS) is crucial in bioinformatics.
method PS8-Net is a new deep convolutional neural network (DCNN) that uses a PS8 module with skip connections to enhance accuracy.
result PS8-Net achieves 76.89% Q8 accuracy on benchmark datasets.
DEMO-Net improves graph neural networks by focusing on node degree.
problem Limited analysis of graph convolution properties and lack of degree-specific graph structure.
method Proposes DEMO-Net, a degree-specific graph neural network that recursively identifies 1-hop neighborhood structures and uses multi-task learning for node representation learning.
result Demonstrates effectiveness and efficiency of DEMO-Net on node and graph classification benchmarks.
Lat-Net compresses Lattice Boltzmann simulations using neural networks.
problem High computational and memory demands in fluid simulations.
method Convolutional autoencoders and residual connections in a fully differentiable scheme.
result Computational and memory efficient neural network that generalizes to large grid sizes and complex geometries.
PDE-Net 2.0 learns PDEs from data without prior knowledge.
problem Discovering PDEs from empirical data without detailed prior knowledge.
method Numeric-symbolic hybrid deep network combining numerical approximations and symbolic neural networks.
result PDE-Net 2.0 can uncover hidden PDEs and predict dynamics in noisy environments.
TR-Nets compress deep networks by 11x for LeNet-5 and 243x for Wide ResNet.
problem Large neural networks require excessive memory and computation.
method Tensor Ring factorization to compress fully connected and convolutional layers.
result TR-Nets can compress LeNet-5 by 11x and Wide ResNet by 243x with minimal accuracy loss.
Convolutional nets require fewer samples than fully-connected nets for image classification.
problem Understanding why convolutional nets are more sample-efficient than fully-connected nets.
method Construction of a natural distribution and target function to demonstrate a sample complexity gap.
result Convolutional nets require O(1) samples for a single target function, while fully-connected nets require Ω(d2) samples. Deep neural networks improve PET attenuation correction from MR images.
problem Challenges in PET attenuation correction from MR images in PET/MR hybrid systems.
method Deep neural networks, specifically U-net and GroupU-net structures, to derive continuous attenuation coefficients.
result Proposed neural network methods outperform standard methods in PET quantification accuracy.
Deepr learns features from medical records to predict patient risk.
problem Feature engineering bottleneck in creating predictive systems from medical records.
method Transforms medical records into sequences, uses convolutional neural nets to detect and combine clinical motifs.
result Deepr achieves superior accuracy in predicting patient risk compared to traditional techniques.
Deep learning models reconstruct volatility surfaces from noisy data under no-arbitrage constraints.
problem Reconstructing implied volatility surfaces from sparse and noisy option quotes.
method Compared multiple neural architectures including Transformers, U-Nets, and variational autoencoders.
result Transformer and U-Net architectures achieve strong reconstruction accuracy, especially under sparse observation regimes.
IVUS-Net automatically segments IVUS images for quicker diagnosis of cardiovascular diseases.
problem Automatically delineating lumen and media-adventitia borders in IVUS images.
method Proposes IVUS-Net, a fully convolutional network followed by post-processing.
result IVUS-Net outperforms state-of-the-art methods by 4% to 20% in HD distance.
A CNN with U-Net improves structural topology optimization efficiency and generalization.
problem Structural topology optimization with reduced computation cost and improved generalization.
method Deep Convolutional Neural Network (CNN) with U-Net architecture, using SIMP-generated dataset.
result Significant reduction in computation cost with little sacrifice on design optimality.
Deep learning improves mammography assessment with high accuracy.
problem Challenges in mammography assessment due to noise, resolution, and lack of ground truths.
method Proposes a classification approach using multi-scale deep tissue classifiers.
result Highest AUC of 0.9 achieved in classifying suspicious tissue patches.
Gradient descent globally optimizes a neural net with Gaussian inputs.
problem Training deep neural networks with gradient descent is hard in general.
method Used gradient descent on a specific neural network architecture with Gaussian inputs.
result Gradient descent converges to the global optimum in polynomial time for Gaussian inputs.
Hybrid approach combines ASTs and deep learning for PowerShell malware detection.
problem Detecting malicious PowerShell scripts effectively.
method Combining static program analysis (ASTs) with deep learning.
result Preliminary results show promising classification of PowerShell scripts by family type.
This work improves optic disc and cup segmentation for glaucoma detection.
problem Automatic segmentation of optic disc and cup on eye fundus images for glaucoma diagnosis.
method Modification of U-Net convolutional neural network.
result Our method achieves comparable quality to state-of-the-art methods, with faster prediction times.
BCD-Net improves low-dose CT image reconstruction.
problem Challenges in obtaining accurate low-dose CT images.
method Modified iterative regression CNN, BCD-Net, with faster numerical solvers.
result BCD-Net achieves better image quality and generalization than state-of-the-art methods.
FCC-GAN combines fully connected and convolutional layers for improved GAN performance.
problem Lack of understanding in choosing GAN network architectures.
method Proposes FCC-GAN, a hybrid architecture combining fully connected and convolutional layers.
result FCC-GAN outperforms traditional GAN architectures in terms of learning speed and sample quality.
Deep learning system diagnoses AVNFH from plain radiographs.
problem Challenging AVNFH diagnosis from plain radiographs.
method Deep convolutional neural networks for end-to-end diagnosis.
result AVN-net achieves state-of-the-art AUC of 0.97 in AVNFH detection.
Novel method uses U-net for seismic data reconstruction without large datasets.
problem Reconstruction of seismic data with missing traces.
method Unsupervised learning with U-net exploiting deep seismic prior.
result DSPRecon algorithm outperforms SSA and Cadzow methods in reconstruction performance.
G-Net constructs binary neural networks with high accuracy using randomized binary embeddings.
problem Creating high-accuracy binary neural networks with theoretical guarantees.
method Proposes a novel floating-point G-Net family with randomized binary embeddings and theoretical accuracy guarantees.
result Empirically, G-Net achieves almost 30% higher accuracy on CIFAR-10 compared to prior HDC models.
Deep learning models complex dependencies using neural networks.
problem Understanding the new characteristics and theoretical foundations of deep learning.
method Introduction of neural network models and training techniques from a statistical perspective.
result Highlight new characteristics of deep learning like depth and over-parametrization.
Deep learning model predicts subsurface flow dynamics.
problem Predicting dynamic subsurface flow in channelized geological systems.
method Residual U-Net and Convolutional LSTM networks trained on pressure and saturation maps.
result Surrogate model accurately predicts pressure, saturation, and well rates for new realizations.
Seq-U-Net improves sequence modeling efficiency with dilated U-Net.
problem Efficiently modeling long-term dependencies in sequences.
method Causal U-Net architecture with dilated filters and slow feature hypothesis.
result Seq-U-Net achieves comparable performance with speed-ups of over 4x in audio generation.
Distill-Net creates efficient CNNs for IoT by distilling complex models.
problem Efficient inference of deep CNNs on resource-constrained IoT platforms.
method Application-specific distillation of deep CNNs.
result Efficient inference on resource-constrained platforms with high accuracy.
Study examines noise sensitivity of DNNs for binary classification.
problem Understanding non-robustness of DNN classifiers under noise.
method Defined and extended noise sensitivity and stability concepts for Boolean functions, applied to DNN models.
result Sorted out the relation between definitions and properties of DNN architectures under noise.
FEA-Net uses physics knowledge to predict material responses efficiently.
problem Predicting material mechanical responses accurately and efficiently.
method Physics-guided deep learning with FEA integration.
result FEA-Net accurately predicts mechanical responses under external loading.
Res-SE-Net boosts Resnet performance by enhancing bridge-connections.
problem Reduced accuracy in Resnet due to lack of feature map contribution from bridge-connections.
method Proposed Res-SE-Net architecture with Squeeze-and-Excitation (SE) block to quantify and weight feature map importance.
result Res-SE-Net generalizes better than Resnet and SE-Resnet on CIFAR-10 and CIFAR-100 datasets.
Framework improves CT image segmentation robustness with domain-specific cues.
problem Challenges in CT image segmentation by deep learning models.
method Combines domain-specific preprocessing and augmentation with CNN architectures.
result Framework stabilizes prediction performance on varying CT volumes.
PDE-Net learns PDEs from data using neural networks.
problem Learning PDEs from complex system dynamics.
method Proposes PDE-Net, a feed-forward deep network to learn differential operators and nonlinear responses.
result PDE-Net can accurately predict dynamics and uncover hidden PDE models.