New deep learning architecture learns martingales efficiently.
problem Efficiently learning martingales in financial derivatives pricing.
method High-order weak approximation algorithms of Runge-Kutta type.
result Deep neural networks based on this architecture learn martingales effectively.
Deep learning improves crime prediction accuracy.
problem Improving crime prediction accuracy using deep learning.
method Comparative study of 10 deep learning methods on crime data.
result Deep learning methods outperform existing methods in crime prediction.
Deep learning boosts building energy load forecasting.
problem Short-term load forecasting in buildings.
method Stacked Boosters Network architecture with sparse interactions, parameter sharing, and equivariant representations.
result Outperforms state-of-the-art models in short-term load forecasting tasks.
Improved robustness in multi-modal sensor fusion with deep learning.
problem Inconsistency in fusion weights leading to poor performance under sensor failures.
method Proposes deep multi-modal sensor fusion architectures with fusion weight regularization and target learning.
result Proposed architectures outperform existing deep learning methods under sensor failures.
Survey of neural architecture search methods.
problem Automating the selection of neural network architectures.
method Comprehensive analysis of existing methods using reinforcement learning, evolutionary algorithms, and surrogate models.
result Unified formalism for categorizing and comparing architecture search methods.
DeepRAM evaluates and selects the best deep learning architecture for DNA/RNA binding specificity prediction.
problem Selecting the best deep learning architecture for predicting DNA/RNA binding specificity.
method Systematic exploration of various deep learning architectures using deepRAM, an end-to-end deep learning tool.
result A k-mer embedding convolutional layer and recurrent layer architecture outperforms other methods.
Paper introduces privacy-preserving deep-learning service.
problem Privacy loss in deep-learning-as-a-service.
method Homomorphic Encryption tailored for CNNs.
result Effectiveness of proposed privacy-preserving architecture.
Hybrid deep architectures with reasoning layers show promising convergence and generalization properties.
problem Understanding the theoretical foundations of hybrid deep architectures with reasoning layers.
method Analyzing the interplay between algorithm layers and neural components in deep architectures.
result Properties of algorithm layers are closely related to the approximation and generalization abilities of end-to-end models.
The paper optimizes dynamic scheduling for ring architectures in deep learning training.
problem Optimizing deep learning training times with ring architectures.
method Formulated a non-convex, non-linear, NP-hard integer programming problem and developed a doubling heuristic.
result Dynamic scheduling can significantly reduce job completion times in ring architectures.
Deep learning (DL) advances state-of-the-art reinforcement learning (RL), by incorporating deep neural networks in learning representations from the input to RL. However, the conventional deep neural network architecture is limited in learning representations for multi-task RL (MT-RL), as multiple tasks can refer to di…
Automates deep learning model development for cancer data.
problem Manual design of high-performing deep learning models for cancer data is time-consuming and requires expertise.
method Reinforcement-learning-based neural architecture search with custom building blocks.
result Automated discovery of deep neural network architectures with similar or higher accuracy.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
Survey of 1000 NAS papers, automating neural architecture design.
problem Designing high-performing neural architectures for various tasks.
method Taxonomy of search spaces, algorithms, and speedup techniques.
result NAS has surpassed human-designed architectures on many tasks.
New method uses graph neural networks for neural architecture search.
problem Finding optimal neural architectures efficiently.
method Bayesian graph neural network for feature extraction and graph Bayesian optimization.
result Significantly outperforms existing methods in benchmark tasks.
In this article we review computational aspects of Deep Learning (DL). Deep learning uses network architectures consisting of hierarchical layers of latent variables to construct predictors for high-dimensional input-output models. Training a deep learning architecture is computationally intensive, and efficient linear…
Proposes a model encoder to recommend deep learning architectures for unknown datasets.
problem Choosing an appropriate deep learning architecture for unknown datasets is time-consuming and laborious.
method Proposes a model encoder approach to learn fixed-length representations of architectures and hyperparameters in an unsupervised manner.
result Predicted accuracy of recommended architectures is a good estimator of actual accuracy on unknown datasets.
Deep learning models predict generalization gaps without specific task or architecture.
problem Predicting when deep learning works across different tasks and architectures.
method Created a dataset of 13,500 neural networks trained on various spiral datasets and parameters. Used this dataset to train predictors for generalization gaps.
result DNNs and RNNs outperform linear models in predicting generalization gaps, with RNNs achieving R2=0.584. Method detects neural network equivalence via matrix ensembles and spectral analysis.
problem Detecting equivalence among different deep learning architectures.
method Generating Mixed Matrix Ensembles (MMEs) and matching to conjugate circular ensembles.
result Empirical evidence shows vanishing differences in spectral densities with long tail decay rates.
Direct Feedback Alignment performs well on diverse deep learning tasks and architectures.
problem The limitations of backpropagation in parallelizing and scaling to modern deep learning tasks.
method Direct Feedback Alignment approach applied to neural view synthesis, recommender systems, geometric learning, and natural language processing.
result Direct Feedback Alignment successfully trains a wide range of state-of-the-art deep learning architectures with performance close to fine-tuned backpropagation.
This study examines how model architecture affects deep learning model privacy.
problem Privacy concerns in deep learning models due to potential leakage of sensitive information.
method Investigation of CNNs and Transformers, focusing on activation layers, stem layers, LN layers, and attention modules.
result Transformers generally exhibit higher vulnerability to privacy attacks than CNNs.
Law explains how deep networks separate data for classification.
problem Black-box nature of deep learning limits architecture design and interpretation.
method Studied how deep neural networks process data in intermediate layers.
result Law of geometric data separation emerges in various architectures and datasets.
Deep learning rules simplified with implicit methods.
problem Complexity and interpretability in deep learning models.
method Simplified notation through fixed-point equations and implicit features.
result Simplified deep learning models with enhanced interpretability.
ForecastNet uses a time-variant deep feed-forward neural network for better multi-step-ahead time series forecasting.
problem Time-invariant architectures limit multi-step-ahead forecasting.
method ForecastNet employs a deep feed-forward architecture with time-variant parameters and interleaved outputs.
result ForecastNet outperforms other models on multi-step-ahead time series forecasting tasks.
This thesis aims to automate deep neural network design for efficiency and complexity reduction.
problem Manual design of deep neural networks is inefficient and complex.
method Examines and proposes automated approaches to neural network design.
result Creation of less complex models with good performance through automation.
We consider active learning of deep neural networks. Most active learning works in this context have focused on studying effective querying mechanisms and assumed that an appropriate network architecture is a priori known for the problem at hand. We challenge this assumption and propose a novel active strategy whereby …
New math for deep learning tackles key questions about neural networks.
problem Understanding the exceptional performance of deep learning models.
method Analyzing overparametrized neural networks, depth, optimization, feature learning, and architecture effects.
result Partial answers to deep learning's outstanding generalization and optimization performance.
Automatically finds strong neural network topologies for continuous control tasks.
problem Handcrafted neural network architectures limit the performance of Deep Reinforcement Learning.
method Combines Neuroevolution with off-policy training and proposes a novel architecture mutation operator.
result The proposed Actor-Critic Neuroevolution algorithm often outperforms strong baseline methods.
EmbraceNet improves robustness in multimodal classification.
problem Robust multimodal classification with partial data loss.
method Deep learning architecture for multimodal fusion.
result EmbraceNet outperforms other models in partial data scenarios.
BioNAS optimizes deep learning models for biomedical research, revealing new knowledge.
problem Building interpretable deep learning models for biomedical research.
method Neural architecture search with knowledge dissimilarity functions for joint optimization of predictive power and biological knowledge.
result BioNAS optimal models reveal novel knowledge in both simulated and real functional genomics data.
A new loss function improves deep learning performance without class separation constraints.
problem Training deep learning architectures for classification.
method Minimizing smoothness of label signals on similarity graphs.
result The proposed loss function leads to similar classification performance as cross-entropy, with added robustness.
We propose using category theory to unify deep learning architectures.
problem Lack of a coherent bridge between model constraints and implementations.
method Apply category theory to unify neural network design.
result Theory recovers constraints from geometric deep learning and encodes standard constructs.
Adapts deep learning with kernel methods for efficient learning.
problem Combining kernel methods and deep learning for efficient learning.
method Nyström approximation of kernel functions in neural networks.
result Performance comparable to standard architectures on datasets like SVHN and CIFAR100.
NAT optimizes neural architectures to improve performance without extra cost.
problem Redundant operations in neural architectures consume memory and degrade performance.
method Transformed Markov Decision Process (MDP) and reinforcement learning to replace redundant operations with more efficient ones.
result Transformed architectures outperform original and existing methods on CIFAR-10 and ImageNet datasets.
New method selects neural network architectures without needing data.
problem Choosing efficient deep neural network architectures.
method Developed the deep frame potential to quantify network capacity.
result Deep frame potential correlates with generalization error.
Book reviews multimodal deep learning approaches and models.
problem Understanding and integrating different data types in deep learning.
method Examined current state-of-the-art approaches, discussed transformation and enhancement models, introduced simultaneous handling models, and covered other modalities.
result Unified architectures for handling multiple modalities in deep learning.
The paper extends deep multi-task learning to diverse domains, finding shared functionality.
problem General problem solving from diverse deep learning tasks.
method Decomposes tasks into subproblems, optimizing sharing through stochastic algorithm.
result Joint learning across diverse domains and architectures improves performance.
A deep model learns to stop early based on variational stopping policy.
problem Varying optimal depth for different inputs in deep architectures.
method A steerable architecture learns a feed-forward deep model and a variational stopping policy together.
result The learned deep model and stopping policy improve diverse tasks.
The backpropagation of error algorithm (BP) is impossible to implement in a real brain. The recent success of deep networks in machine learning and AI, however, has inspired proposals for understanding how the brain might learn across multiple layers, and hence how it might approximate BP. As of yet, none of these prop…
End-to-end trainable graph matching using improved combinatorial solvers.
problem Graph matching in deep learning.
method Combining deep learning with optimized combinatorial solvers.
result Advances state-of-the-art on deep graph matching benchmarks.
We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods. Specifically, we transform the inputs of a spectral mixture base kernel with a deep architecture, using local kernel interpolation, inducing points, and struc…
In this work, we present a comparison of a shallow and a deep learning architecture for the automated segmentation of white matter lesions in MR images of multiple sclerosis patients. In particular, we train and test both methods on early stage disease patients, to verify their performance in challenging conditions, mo…
Deep learning models reconstruct volatility surfaces from noisy data under no-arbitrage constraints.
problem Reconstructing implied volatility surfaces from sparse and noisy option quotes.
method Compared multiple neural architectures including Transformers, U-Nets, and variational autoencoders.
result Transformer and U-Net architectures achieve strong reconstruction accuracy, especially under sparse observation regimes.
Improved software flaw detection using NAS on multimodal DL models.
problem Software flaw detection in multimodal deep learning models.
method Adapted NAS framework for multimodal learning, combined with multimodal deep learning models.
result Improved performance on the Juliet Test Suite.
PETRA enables parallel training of deep models with reversible architectures.
problem Challenges in parallelizing deep model training.
method Introduces PETRA, a novel approach for parallelizing gradient computations in reversible architectures.
result Achieves competitive accuracies on CIFAR-10, ImageNet32, and ImageNet using ResNet models.
Deep learning improves human affect recognition in natural settings.
problem Automatic recognition of human emotions in real-world scenarios.
method Review and analysis of 233 studies using deep neural networks.
result Deep learning significantly enhances affect recognition from multimodal sensor data.
Specialized Deep Learning (DL) acceleration stacks, designed for a specific set of frameworks, model architectures, operators, and data types, offer the allure of high performance while sacrificing flexibility. Changes in algorithms, models, operators, or numerical systems threaten the viability of specialized hardware…
In this paper, we evaluate the accuracy of deep learning approaches on geospatial vector geometry classification tasks. The purpose of this evaluation is to investigate the ability of deep learning models to learn from geometry coordinates directly. Previous machine learning research applied to geospatial polygon data …
TAAN model learns optimal network architecture for MTL tasks.
problem Improving generalization performance of MTL by finding flexible and accurate shared architecture.
method TAAN model with flexible activation functions and functional regularization.
result TAAN and regularization methods improve MTL performance.