New deep learning architecture learns martingales efficiently.
problem Efficiently learning martingales in financial derivatives pricing.
method High-order weak approximation algorithms of Runge-Kutta type.
result Deep neural networks based on this architecture learn martingales effectively.
New method finds best neural architecture during learning.
problem Active learning of deep neural networks with known architectures.
method Neural architecture search during active learning.
result Outperforms fixed architecture active learning.
Deep learning improves crime prediction accuracy.
problem Improving crime prediction accuracy using deep learning.
method Comparative study of 10 deep learning methods on crime data.
result Deep learning methods outperform existing methods in crime prediction.
Deep learning boosts building energy load forecasting.
problem Short-term load forecasting in buildings.
method Stacked Boosters Network architecture with sparse interactions, parameter sharing, and equivariant representations.
result Outperforms state-of-the-art models in short-term load forecasting tasks.
Improved robustness in multi-modal sensor fusion with deep learning.
problem Inconsistency in fusion weights leading to poor performance under sensor failures.
method Proposes deep multi-modal sensor fusion architectures with fusion weight regularization and target learning.
result Proposed architectures outperform existing deep learning methods under sensor failures.
GTN improves multi-task RL with a novel deep neural network.
problem Limited deep neural network for multi-task reinforcement learning.
method GTN architecture with horizontal and vertical streams.
result GTN achieves state-of-the-art multi-task reinforcement learning.
Survey of neural architecture search methods.
problem Automating the selection of neural network architectures.
method Comprehensive analysis of existing methods using reinforcement learning, evolutionary algorithms, and surrogate models.
result Unified formalism for categorizing and comparing architecture search methods.
DeepRAM evaluates and selects the best deep learning architecture for DNA/RNA binding specificity prediction.
problem Selecting the best deep learning architecture for predicting DNA/RNA binding specificity.
method Systematic exploration of various deep learning architectures using deepRAM, an end-to-end deep learning tool.
result A k-mer embedding convolutional layer and recurrent layer architecture outperforms other methods.
Paper introduces privacy-preserving deep-learning service.
problem Privacy loss in deep-learning-as-a-service.
method Homomorphic Encryption tailored for CNNs.
result Effectiveness of proposed privacy-preserving architecture.
Hybrid deep architectures with reasoning layers show promising convergence and generalization properties.
problem Understanding the theoretical foundations of hybrid deep architectures with reasoning layers.
method Analyzing the interplay between algorithm layers and neural components in deep architectures.
result Properties of algorithm layers are closely related to the approximation and generalization abilities of end-to-end models.
The paper optimizes dynamic scheduling for ring architectures in deep learning training.
problem Optimizing deep learning training times with ring architectures.
method Formulated a non-convex, non-linear, NP-hard integer programming problem and developed a doubling heuristic.
result Dynamic scheduling can significantly reduce job completion times in ring architectures.
Automates deep learning model development for cancer data.
problem Manual design of high-performing deep learning models for cancer data is time-consuming and requires expertise.
method Reinforcement-learning-based neural architecture search with custom building blocks.
result Automated discovery of deep neural network architectures with similar or higher accuracy.
Optimized deep learning architectures improve sensor fusion performance.
problem Sensor fusion in autonomous systems.
method Proposed two optimized architectures: coarser-grained and two-stage gated.
result Significant performance improvements and robustness in noisy conditions.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
New method uses graph neural networks for neural architecture search.
problem Finding optimal neural architectures efficiently.
method Bayesian graph neural network for feature extraction and graph Bayesian optimization.
result Significantly outperforms existing methods in benchmark tasks.
Study compares shallow and deep learning for MS lesion segmentation.
problem Automated segmentation of white matter lesions in early-stage MS patients.
method Training and testing shallow and deep learning architectures on 32 patients.
result Combining shallow and deep architectures improves lesion-wise metrics.
Survey of 1000 NAS papers, automating neural architecture design.
problem Designing high-performing neural architectures for various tasks.
method Taxonomy of search spaces, algorithms, and speedup techniques.
result NAS has surpassed human-designed architectures on many tasks.
Proposes a model encoder to recommend deep learning architectures for unknown datasets.
problem Choosing an appropriate deep learning architecture for unknown datasets is time-consuming and laborious.
method Proposes a model encoder approach to learn fixed-length representations of architectures and hyperparameters in an unsupervised manner.
result Predicted accuracy of recommended architectures is a good estimator of actual accuracy on unknown datasets.
Method detects neural network equivalence via matrix ensembles and spectral analysis.
problem Detecting equivalence among different deep learning architectures.
method Generating Mixed Matrix Ensembles (MMEs) and matching to conjugate circular ensembles.
result Empirical evidence shows vanishing differences in spectral densities with long tail decay rates.
Deep learning models predict generalization gaps without specific task or architecture.
problem Predicting when deep learning works across different tasks and architectures.
method Created a dataset of 13,500 neural networks trained on various spiral datasets and parameters. Used this dataset to train predictors for generalization gaps.
result DNNs and RNNs outperform linear models in predicting generalization gaps, with RNNs achieving R2=0.584. Proposes deep learning methods for handling random vectors.
problem Handling flexible input data like probability measures in deep learning.
method Develops deep architectures to handle permutation invariances, varying weights, and cardinality.
result Demonstrates the effectiveness of deep architectures on measures for classification, reduction, and prediction.
Direct Feedback Alignment performs well on diverse deep learning tasks and architectures.
problem The limitations of backpropagation in parallelizing and scaling to modern deep learning tasks.
method Direct Feedback Alignment approach applied to neural view synthesis, recommender systems, geometric learning, and natural language processing.
result Direct Feedback Alignment successfully trains a wide range of state-of-the-art deep learning architectures with performance close to fine-tuned backpropagation.
Deep learning uses complex networks for high-dimensional data.
problem Computational inefficiency in training deep learning models.
method Use of hierarchical latent variables, efficient linear algebra, SGD optimization, and batch sampling.
result Efficient training and inference possible with optimized algorithms.
Study compares deep learning and superlearner architectures for classification problems.
problem Assessing performance and computational efficiency of different machine-learning methods.
method Comparative analysis of superlearner and deep learning architectures across various datasets.
result Superlearner outperforms deep neural networks on nonlinear predictor relationship datasets, suggesting faster convergence and interpretability.
This study examines how model architecture affects deep learning model privacy.
problem Privacy concerns in deep learning models due to potential leakage of sensitive information.
method Investigation of CNNs and Transformers, focusing on activation layers, stem layers, LN layers, and attention modules.
result Transformers generally exhibit higher vulnerability to privacy attacks than CNNs.
Law explains how deep networks separate data for classification.
problem Black-box nature of deep learning limits architecture design and interpretation.
method Studied how deep neural networks process data in intermediate layers.
result Law of geometric data separation emerges in various architectures and datasets.
ForecastNet uses a time-variant deep feed-forward neural network for better multi-step-ahead time series forecasting.
problem Time-invariant architectures limit multi-step-ahead forecasting.
method ForecastNet employs a deep feed-forward architecture with time-variant parameters and interleaved outputs.
result ForecastNet outperforms other models on multi-step-ahead time series forecasting tasks.
Deep learning rules simplified with implicit methods.
problem Complexity and interpretability in deep learning models.
method Simplified notation through fixed-point equations and implicit features.
result Simplified deep learning models with enhanced interpretability.
This thesis aims to automate deep neural network design for efficiency and complexity reduction.
problem Manual design of deep neural networks is inefficient and complex.
method Examines and proposes automated approaches to neural network design.
result Creation of less complex models with good performance through automation.
New math for deep learning tackles key questions about neural networks.
problem Understanding the exceptional performance of deep learning models.
method Analyzing overparametrized neural networks, depth, optimization, feature learning, and architecture effects.
result Partial answers to deep learning's outstanding generalization and optimization performance.
New algorithms for deep learning mimic brain's learning but struggle with complex images.
problem Evaluating biologically inspired deep learning algorithms on complex image datasets.
method Implemented and compared various biologically inspired algorithms (TP, FA, DTP) on MNIST, CIFAR-10, and ImageNet.
result Biologically inspired algorithms perform well on MNIST but poorly on CIFAR and ImageNet, suggesting new architectures or algorithms are needed.
Framework evaluates deep learning EEG architectures on 100 datasets.
problem Evaluating different deep learning architectures for EEG signal decoding.
method Large-scale evaluation framework with 100 EEG datasets and multiple decoders.
result Comparison of three CNN architectures on different EEG tasks.
A new method corrects flaws in comparing deep learning architectures.
problem Flaws in comparing deep learning architectures using best single model performance.
method Proposes Boo_n method to correct stochasticity in model performance.
result Corrects flaws in comparing deep learning architectures.
Automatically finds strong neural network topologies for continuous control tasks.
problem Handcrafted neural network architectures limit the performance of Deep Reinforcement Learning.
method Combines Neuroevolution with off-policy training and proposes a novel architecture mutation operator.
result The proposed Actor-Critic Neuroevolution algorithm often outperforms strong baseline methods.
EmbraceNet improves robustness in multimodal classification.
problem Robust multimodal classification with partial data loss.
method Deep learning architecture for multimodal fusion.
result EmbraceNet outperforms other models in partial data scenarios.
Survey of automated neural architecture search methods.
problem Manual development of neural architectures is time-consuming and error-prone.
method Categorizes existing automated neural architecture search methods.
result Growing interest in automated methods due to manual development's limitations.
BioNAS optimizes deep learning models for biomedical research, revealing new knowledge.
problem Building interpretable deep learning models for biomedical research.
method Neural architecture search with knowledge dissimilarity functions for joint optimization of predictive power and biological knowledge.
result BioNAS optimal models reveal novel knowledge in both simulated and real functional genomics data.
A new loss function improves deep learning performance without class separation constraints.
problem Training deep learning architectures for classification.
method Minimizing smoothness of label signals on similarity graphs.
result The proposed loss function leads to similar classification performance as cross-entropy, with added robustness.
We propose using category theory to unify deep learning architectures.
problem Lack of a coherent bridge between model constraints and implementations.
method Apply category theory to unify neural network design.
result Theory recovers constraints from geometric deep learning and encodes standard constructs.
Develops reversible neural networks for deep learning, improving efficiency and performance.
problem Training and deploying deep neural networks with limited resources and memory.
method Interprets deep residual networks as ODEs, derives reversible architectures, and analyzes stability.
result Reversible neural networks can be trained deeper with less memory and computational resources.
Adapts deep learning with kernel methods for efficient learning.
problem Combining kernel methods and deep learning for efficient learning.
method Nyström approximation of kernel functions in neural networks.
result Performance comparable to standard architectures on datasets like SVHN and CIFAR100.
Transfer Neural AutoML speeds up deep learning architecture design.
problem High computational cost in Neural AutoML.
method Transfer learning to speed up architecture design.
result Reduces convergence time by over an order of magnitude.
NAT optimizes neural architectures to improve performance without extra cost.
problem Redundant operations in neural architectures consume memory and degrade performance.
method Transformed Markov Decision Process (MDP) and reinforcement learning to replace redundant operations with more efficient ones.
result Transformed architectures outperform original and existing methods on CIFAR-10 and ImageNet datasets.
Introduces Deep Genetic Network for optimizing neural network hyperparameters.
problem Optimizing neural network hyperparameters is a tedious and time-consuming task.
method Uses genetic algorithms within a deep neural network architecture to optimize hyperparameters during training.
result Deep Genetic Network successfully optimizes hyperparameters in various types of neural network layers.
New method selects neural network architectures without needing data.
problem Choosing efficient deep neural network architectures.
method Developed the deep frame potential to quantify network capacity.
result Deep frame potential correlates with generalization error.
Book reviews multimodal deep learning approaches and models.
problem Understanding and integrating different data types in deep learning.
method Examined current state-of-the-art approaches, discussed transformation and enhancement models, introduced simultaneous handling models, and covered other modalities.
result Unified architectures for handling multiple modalities in deep learning.
The paper extends deep multi-task learning to diverse domains, finding shared functionality.
problem General problem solving from diverse deep learning tasks.
method Decomposes tasks into subproblems, optimizing sharing through stochastic algorithm.
result Joint learning across diverse domains and architectures improves performance.
A deep model learns to stop early based on variational stopping policy.
problem Varying optimal depth for different inputs in deep architectures.
method A steerable architecture learns a feed-forward deep model and a variational stopping policy together.
result The learned deep model and stopping policy improve diverse tasks.