Deep learning detects malware from imbalanced cyber data.
problem Detecting malware from imbalanced cyber data.
method End-to-end Tree-Shaped Deep Neural Network (TSDNN) with Quantity Dependent Backpropagation (QDBP).
result TSDNN outperforms state-of-the-art methods in imbalanced learning.
New method identifies causal parameters in tree-shaped linear models using cycles.
problem Identifying causal parameters from correlations in tree-shaped linear models.
method Investigates tree-shaped linear models, uses missing cycles to identify causal parameters, solves quadratic equations.
result Shows how missing cycles can be combined to obtain a unique solution for causal parameters.
Paper develops a new method to analyze 3D tree-like objects.
problem Analyzing complex geometrical and topological variations in 3D tree-like objects.
method Extended SRVF representation and new metric for tree-shaped 3D objects.
result Captures full elasticity and topological variations of branches.
This research develops a new model for cyber risk and insurance pricing.
problem Accurate calculation of aggregate losses in cyber insurance pricing.
method A path-based k-generation risk contagion model in a tree-shaped network structure.
result Explicit expressions for mean and variance of local loss on a single path.
Probabilistic deep learning uses neural networks and models to handle uncertainty.
problem Handling uncertainty in deep learning models.
method Two approaches: probabilistic neural networks and deep probabilistic models.
result TensorFlow Probability library supports both approaches.
Unified framework for deep neural networks using capsule networks.
problem Unified description of deep neural networks.
method Formalized neural networks mathematically, set up a unified capsule framework.
result Unified framework simplifies existing deep neural networks and provides theoretical basis.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
GA improves deep neural networks by creating sparser models.
problem Training deep neural networks efficiently and sparsely.
method Genetic algorithms for evolving deep neural networks.
result GA-assisted deep autoencoder produces sparser neural networks.
Study of infinitely deep but narrow neural networks using NTK theory.
problem Analyzing the role of depth in deep learning with overparameterized networks.
method Infinite-depth limit analysis of MLP and CNN using Neural Tangent Kernel (NTK) theory.
result Established trainability guarantee for infinitely deep but narrow neural networks.
Deep neural networks can approximate rough functions with high accuracy.
problem Approximating rough functions with neural networks.
method Proved that ENO interpolation can be cast as a deep ReLU neural network, transferring ENO's high-order accuracy.
result Deep neural networks can achieve high-order accuracy in approximating Lipschitz functions.
This paper shows overparameterized deep neural networks are convex and learn useful features.
problem Analyzing fully trained overparameterized deep neural networks.
method Generalized neural feature repopulation technique.
result Overparameterized deep neural networks are inherently convex and learn useful features.
Study of deep neural networks' NTK evolution during training.
problem Understanding the performance gap between deep neural networks and kernel regression.
method Derive an infinite hierarchy of ordinary differential equations (NTH) to capture gradient descent dynamics of deep neural networks.
result Truncated NTH approximates the dynamic of the NTK up to arbitrary precision under certain conditions.
The paper proposes a factorization method to interpret deep neural networks.
problem Interpreting and debugging deep neural networks.
method Factorization based approach to understand deep neural networks.
result Identified patterns linking factorization rank to network training quality.
Study deep maxout networks and their equivalence to Gaussian processes.
problem Understanding neural networks with infinite width.
method Derive equivalence between deep maxout networks and Gaussian processes, characterize maxout kernel, and provide efficient numerical implementation.
result Bayesian inference based on deep maxout network kernel leads to competitive results compared to finite-width counterparts and deep neural network kernels.
Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.
problem Achieving sup-norm convergence for deep neural network estimators in nonparametric regression.
method Developed an adversarial training scheme to address the sup-norm convergence issue.
result Deep neural network estimators achieve optimal sup-norm convergence with the proposed adversarial training.
Deep CNNs can approximate any function, proving their efficiency.
problem Understanding the approximation and generalization ability of deep learning methods.
method Demonstrated universality of deep convolutional neural networks (CNNs).
result Deep CNNs can approximate any continuous function to arbitrary accuracy.
Deep networks are shown to be equivalent to a new type of kernel chain.
problem Identifying an appropriate function space for deep neural networks.
method Extending Reproducing Kernel Banach Spaces (RKBS) to chain RKBS (cRKBS), which composes kernels rather than functions.
result Any deep neural network function is a neural cRKBS function, and conversely, any neural cRKBS function corresponds to a deep neural network.
Paper verifies properties of binarized neural networks using SAT solvers.
problem Verifying properties of deep neural networks.
method Exact Boolean encoding of binarized neural networks, SAT solvers, counterexample-guided search.
result Demonstrates scalability to medium-size deep neural networks for robustness verification.
Deep neural nets converge exponentially fast for analytic functions.
problem Analytic function approximation in low dimensions.
method Exponential convergence rate of deep neural networks for analytic functions.
result Deep neural nets achieve exponential convergence for analytic functions.
Bayesian Neural Networks help quantify uncertainty in deep learning predictions.
problem Uncertainty quantification in deep learning predictions.
method Bayesian statistics applied to neural networks.
result Design, implementation, training, and evaluation of Bayesian Neural Networks.
Sparse deep neural networks follow a power law in their connectivity.
problem Understanding the connectivity patterns in sparse deep neural networks.
method Experimentally tested multilayer perceptrons and convolutional neural networks, proposed an internal preferential attachment model.
result Sparse deep neural networks exhibit a power law in their connectivity, similar to biological neural networks.
Study binary activated deep neural networks using PAC-Bayesian theory.
problem Generalization bounds for binary activated deep neural networks.
method Developed an end-to-end framework and provided PAC-Bayesian generalization bounds.
result Nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks.
DCNs mimic neuronal networks for improved neural classification.
problem Lack of topological similarity between DNNs and biological neural networks.
method Developed DCNs with topologies inspired by real-world neuronal networks.
result High classification accuracy achieved by DCNs.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
This work makes deep neural networks more resilient to adversarial attacks.
problem Making deep neural networks more resilient to adversarial attacks in real-time systems.
method Used GTSRB dataset to craft adversarial samples, then attacked a Deep Convolutional Neural Network to build a more resilient network.
result Built a more robust deep neural network that can resist adversarial attacks.
Gradient descent efficiently finds global minima in deep neural networks.
problem Training deep neural networks efficiently and reliably.
method Gradient descent, leveraging the stability of the Gram matrix induced by the network architecture.
result Gradient descent achieves zero training loss in polynomial time for deep over-parameterized neural networks with residual connections.
Deep neural networks can efficiently classify Gaussian mixture models with fewer neurons.
problem Classifying Gaussian mixture models efficiently.
method Used deep neural networks with two hidden layers to classify GMMs optimally.
result Deep neural networks require significantly fewer neurons than shallow ones for optimal classification of GMMs.
Proposes deep graph persistence to address neural persistence issues in deep learning.
problem Variance of weights and lack of spatial structure in deep neural networks impact neural persistence.
method Extends neural persistence to the whole network, considering interactions between layers.
result Deep graph persistence alleviates variance-related issues and captures persistent paths through the network.
Global optimality conditions found for deep neural networks.
problem Understanding the error landscape of deep neural networks.
method Analyzing deep linear and nonlinear neural networks with squared error loss.
result Necessary and sufficient conditions for global optimality in deep linear networks, with an efficiently checkable test.
DeepDIG generates samples near decision boundaries of deep neural networks for better understanding.
problem Limited knowledge of how deep neural networks make decisions.
method Adversarial example generation to create samples near decision boundaries.
result Characterized decision boundaries of various deep neural network models.
DQV Learning uses neural networks to improve reinforcement learning performance.
problem Improving reinforcement learning algorithms for better performance.
method Temporal-difference learning with Value and Quality-value networks, using Deep Convolutional Neural Networks, Experience Replay, and Target Neural Networks.
result DQV learns faster and better than Deep Q-Learning and Double Deep Q-Learning.
Bayesian Neural Networks improve uncertainty estimation in deep learning.
problem Lack of robustness and sensitivity to out-of-distribution samples in DNNs.
method Empirical evaluation of Bayesian Neural Networks against point estimate DNNs.
result Bayesian Neural Networks provide better uncertainty quantification and performance.
The paper proves deep ReLU networks avoid spurious local minima in NTK regime.
problem The existence of spurious local minima in deep ReLU neural networks.
method Theoretical proof under Neural Tangent Kernel regime.
result Deep ReLU networks do not lie in spurious local minima in NTK regime.
This paper surveys deep learning techniques for non-neural classifiers.
problem Improving performance and generality of non-neural classifiers using deep learning methods.
method Reviews feature learning, optimization, and regularization methods from deep networks for non-neural classifiers.
result Many opportunities and challenges remain for expanding deep learning to non-neural classifiers.
Deep neural networks maximize variation when few nodes change activation.
problem Maximizing variation in deep neural networks.
method Theoretical analysis of ReLU activation function and layer node numbers.
result Maximal variation occurs when few nodes change activation.
Deep neural networks' infinite-width behavior approximated by Gaussian models.
problem Understanding the behavior of deep neural networks in the limit of infinite width.
method Using the Lindeberg exchange principle to approximate weights by Gaussian random variables.
result Quantitative bounds on the 2-Wasserstein distance between deep neural networks and Gaussian limits.
Deep neural networks with various activation functions can approximate Hölder smooth functions.
problem Expressivity of deep neural networks with general activation functions.
method Investigates approximation ability of deep neural networks with a broad class of activation functions, including Hölder smooth functions.
result Derives the required depth, width, and sparsity of deep neural networks to approximate Hölder smooth functions.
Deep neural networks with piecewise-polynomial activations can approximate smooth functions and their derivatives.
problem Approximating smooth functions and their derivatives with neural networks.
method Derives the depth, width, and sparsity required for approximation in Hölder norms.
result Deep neural networks with bounded weights can approximate Hölder smooth functions and their derivatives.
Synaptic cluster-driven evolution improves deep neural networks by reducing synapses and clusters.
problem Efficiently synthesizing deep neural networks with fewer synapses and clusters.
method Synaptic cluster-driven genetic encoding scheme.
result Significantly smaller number of synapses and clusters in offspring networks.
Embeds sparsity in deep neural networks, allowing exact zero parameters.
problem Learning sparse structures in deep networks.
method Embeds sparsity into neural network structure, allowing exact zero parameters during training.
result Can learn both structured and unstructured sparsity.
SelectiveNet optimizes deep neural networks with a built-in reject option.
problem Selective prediction in deep neural networks.
method End-to-end training of a deep neural architecture to optimize both classification and rejection.
result Improved risk-coverage trade-off over various datasets.
New framework explains deep neural networks using variational spline theory.
problem Understanding functions learned by deep neural networks.
method Developed a variational framework and function space.
result Deep ReLU networks are solutions to regularized data fitting problems over the proposed function space.
Deep neural network detects heart murmur with high accuracy.
problem Detecting heart murmur from heart sound recordings.
method Parallel combination of RNN-BiLSTM and CNN.
result 96-100% sensitivity and specificity, 98% F1 score.
Deep neural networks have almost linear sample complexity.
problem Sample complexity of deep neural networks.
method o-minimal expansion of the real field to bound sample complexity.
result Almost linear bound on sample complexity of neural networks.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
DLL accelerates deep learning models up to 10x faster.
problem Improving deep learning training and inference speed.
method Novel software engineering strategies for accelerating DLL.
result DLL is up to 10x faster than other frameworks on CPU and GPU.
DeepESN models efficiently design deep neural networks for temporal data.
problem Designing efficient deep neural networks for temporal data.
method Hierarchical compositions of recurrent layers.
result Intrinsic properties of state dynamics in deep RNNs.