Probabilistic deep learning uses neural networks and models to handle uncertainty.
problem Handling uncertainty in deep learning models.
method Two approaches: probabilistic neural networks and deep probabilistic models.
result TensorFlow Probability library supports both approaches.
InferPy simplifies probabilistic modeling with deep neural networks in Python.
problem Complex probabilistic models with deep neural networks.
method User-friendly API for defining, learning, and evaluating models.
result Compact and simple way to define general hierarchical probabilistic models.
Characterizes deep neural network weight space for adversarial attacks.
problem Poor performance of deep learning models in adversarial examples.
method Characterizes deep neural network solution space using two paradigms.
result Adversarial attacks are less successful against Associative Memory Models.
Deep neural networks can efficiently classify Gaussian mixture models with fewer neurons.
problem Classifying Gaussian mixture models efficiently.
method Used deep neural networks with two hidden layers to classify GMMs optimally.
result Deep neural networks require significantly fewer neurons than shallow ones for optimal classification of GMMs.
GA improves deep neural networks by creating sparser models.
problem Training deep neural networks efficiently and sparsely.
method Genetic algorithms for evolving deep neural networks.
result GA-assisted deep autoencoder produces sparser neural networks.
Bayesian methods enhance deep learning models by improving reliability and uncertainty.
problem Improving reliability and uncertainty awareness in deep learning models.
method Approximate Bayesian inference techniques, including SG-MCMC and VI, applied to deep learning models.
result Enhanced posterior inference for deep learning models, particularly in neural networks and generative models.
Unified framework for deep neural networks using capsule networks.
problem Unified description of deep neural networks.
method Formalized neural networks mathematically, set up a unified capsule framework.
result Unified framework simplifies existing deep neural networks and provides theoretical basis.
Deep learning models outperform classical methods in forecasting neural activity.
problem Improving forecasting of neural activity using deep learning models.
method Systematic evaluation of eight probabilistic deep learning models against classical statistical models and baseline methods.
result Several deep learning models consistently outperform classical approaches in forecasting neural activity.
New deep learning model tackles graph data learning challenges.
problem Handling graph structured data challenges in deep learning models.
method Introduces a deep loopy neural network with extensive connections and a new learning algorithm based on spanning trees.
result Demonstrates effectiveness on real-world graph datasets.
DeepDIG generates samples near decision boundaries of deep neural networks for better understanding.
problem Limited knowledge of how deep neural networks make decisions.
method Adversarial example generation to create samples near decision boundaries.
result Characterized decision boundaries of various deep neural network models.
Deep networks become equivalent to linear models in large data regimes.
problem Understanding the behavior of deep neural networks in large data regimes.
method Information-theoretic analysis of fully-trained neural networks in proportional scaling regime.
result Proves deep Gaussian equivalence principle, showing deep networks can be simplified to linear models.
Gradient amplification boosts deep learning model performance without increasing training time.
problem Vanishing gradients in deep neural networks.
method Gradient amplification approach to prevent vanishing gradients and training strategy to enable/disable across epochs.
result Improves performance of deep learning models with reduced training time.
Deep neural networks blend mechanistic and phenomenological NLP approaches.
problem Combining theory-driven and data-driven NLP methods.
method Using deep neural networks to integrate mechanistic and phenomenological models.
result Deep learning can effectively model language and perception in spatial cognition.
New method speeds up deep neural networks inference.
problem Inference speed of deep neural networks.
method Maximum volume algorithm for reduced-order modeling.
result Convolutional layers can be replaced with smaller fully-connected layers with minimal accuracy loss.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
Theoretical analysis of deep neural networks for time series data.
problem Theoretical development for deep neural networks on temporally dependent observations is lacking.
method Established non-asymptotic bounds for prediction error of deep neural networks under mixing-type assumptions.
result Deep neural networks can model non-linear time series data with additional logarithmic factors due to dependence.
Deep SSMs use neural networks to identify complex systems.
problem Identifying nonlinear systems with high uncertainty.
method Deep state space models with neural networks.
result Deep SSMs outperform traditional methods on benchmarks.
Our paper explains deep neural collapse in multiple layers.
problem Understanding deep neural collapse in multi-layered neural networks.
method Generalized unconstrained features model for deep networks.
result Deep unconstrained features model exhibits deep neural collapse.
The paper studies how regularization parameters affect sparsity in deep neural networks.
problem Reducing the complexity of deep neural networks by promoting sparsity.
method Derives ℓ1-norm sparsity-promoting models, characterizes sparsity levels, and develops algorithms for selecting optimal regularization parameters. result Developed algorithms to select regularization parameters for desired sparsity levels in neural networks.
DLL accelerates deep learning models up to 10x faster.
problem Improving deep learning training and inference speed.
method Novel software engineering strategies for accelerating DLL.
result DLL is up to 10x faster than other frameworks on CPU and GPU.
Sparse deep neural networks follow a power law in their connectivity.
problem Understanding the connectivity patterns in sparse deep neural networks.
method Experimentally tested multilayer perceptrons and convolutional neural networks, proposed an internal preferential attachment model.
result Sparse deep neural networks exhibit a power law in their connectivity, similar to biological neural networks.
Deep neural networks are optimizable due to their multilayered structure.
problem Understanding why deep neural networks are easily optimizable despite their non-convex loss functions.
method Analysis of a spin glass model of deep neural networks using random matrix theory and algebraic geometry.
result The multilayered structure of deep neural networks leads to fewer stationary points, more clustered minima, and less severe tradeoffs between depth and width of minima.
Paper studies deep diagonal circulant neural networks and introduces training techniques.
problem Understanding and training deep neural networks with structured weight matrices.
method Theoretical analysis and practical training techniques including initialization and non-linearity use.
result Deep diagonal circulant networks outperform other structured models in accuracy and weight efficiency.
New findings show DNC is not optimal for deep models, revealing a low-rank bias.
problem Theoretical limitations of DNC in non-linear models and multi-class classification.
method Analysis of non-linear models of arbitrary depth in multi-class classification.
result DNC stops being optimal for DUFM when going beyond two layers or two classes, due to a low-rank bias.
Paper visualizes and analyzes deep neural networks for CTR prediction.
problem Interpreting deep neural networks for CTR prediction is challenging.
method Visualizing and analyzing deep neural networks at neuron level, measuring layer-wise performance, and calculating saliency scores.
result Insights into the model's performance and mechanism.
Deep learning models complex dependencies using neural networks.
problem Understanding the new characteristics and theoretical foundations of deep learning.
method Introduction of neural network models and training techniques from a statistical perspective.
result Highlight new characteristics of deep learning like depth and over-parametrization.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
The rise and fall of artificial neural networks is well documented in the scientific literature of both computer science and computational chemistry. Yet almost two decades later, we are now seeing a resurgence of interest in deep learning, a machine learning algorithm based on multilayer neural networks. Within the la…
This paper investigates the impact of normalization on deep neural networks for click-through rate prediction.
problem The effect of normalization on deep neural network models for CTR estimation.
method Systematic study of various normalization approaches applied to feature embedding and MLP part of DNN models.
result Correct normalization significantly enhances model performance, as demonstrated by extensive experiments on real-world datasets.
This paper reviews deep learning's latest progress and applications.
problem Challenges in deep learning models and applications.
method Analysis of existing models and new emerging models.
result Summarizes deep learning's applications in various AI fields.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
Extends deep learning with interpretable additive models.
problem Identifiability issues between neural networks and additive models.
method Orthogonalization cell to separate deep neural network and structured model parts.
result Stable estimation and interpretability of structured model parts.
Proposes Neural Complexity (NC) for predicting and explaining generalization in deep neural networks.
problem Challenges in specifying a suitable complexity measure for deep neural networks to predict and explain generalization.
method A meta-learning framework that learns a scalar complexity measure through interactions with many heterogeneous tasks.
result Trained NC model can be added to standard training loss to regularize any task learner.
Extracts salient concepts from CNNs for explaining deep neural networks.
problem Explaining the opaque behavior of deep neural networks in safety-critical domains.
method Uses autoencoders to extract salient concepts and builds a Bayesian causal model.
result Identifies and visualizes features influencing deep neural network classifications.
Metrics assess uncertainty structure and distribution for regression models.
problem Quantifying uncertainty in high-dimensional and nonlinear regression tasks.
method Two bounded comparison metrics for uncertainty structure and distribution.
result DNNs and DNOs provide encouraging uncertainty metric values in high dimensions.
Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.
problem Achieving sup-norm convergence for deep neural network estimators in nonparametric regression.
method Developed an adversarial training scheme to address the sup-norm convergence issue.
result Deep neural network estimators achieve optimal sup-norm convergence with the proposed adversarial training.
Deep neural network predicts multiphase flow in heterogeneous domains.
problem Predicting multiphase flow in complex, heterogeneous systems.
method Deep neural network model for handling permeability heterogeneity and learning interplay of forces.
result Highly accurate predictions of CO2 saturation distribution with computational efficiency.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
DeepESN models efficiently design deep neural networks for temporal data.
problem Designing efficient deep neural networks for temporal data.
method Hierarchical compositions of recurrent layers.
result Intrinsic properties of state dynamics in deep RNNs.
DIFNET tackles the suspended animation problem in deep graph neural networks.
problem Deep graph neural networks suffer from the suspended animation problem.
method DIFNET uses neural gates and graph residual learning for node hidden state modeling, and includes an attention mechanism for node neighborhood information diffusion.
result DIFNET effectively addresses the suspended animation problem and improves learning performance.
The paper proposes a consensus algorithm to improve deep neural network interpretability and accuracy in mortality prediction.
problem The black-box nature and overgeneralization of deep neural networks in healthcare applications.
method An (n, k) consensus algorithm that is insensitive to adversarial examples and can reliably reject out-of-distribution samples. result The consensus algorithm improves both prediction accuracy and interpretability of deep neural network models in mortality prediction.
DyMoN models complex systems from short snapshots using deep neural networks.
problem Modeling high-dimensional stochastic systems from limited snapshot data.
method Dynamics Modeling Network (DyMoN) as a deep generative Markov model trained on current and next-state pairs.
result DyMoN outperforms shallow and deep models in capturing system dynamics and generating longitudinal trajectories.
DNDT combines neural networks and decision trees for tabular data.
problem Tabular data processing with interpretability and efficiency.
method Deep Neural Decision Trees (DNDT) using neural networks to model decision trees.
result DNDT achieves both interpretability and efficiency in tabular data processing.
New method uses mutual info and network science to explain deep learning models.
problem Interpreting deep neural networks for understanding their decision-making process.
method Coupling mutual information with network science to quantify information flow in deep learning models.
result Proposed NIF technique for codifying information flow in deep learning models.
Lipschitz networks bound distributional robustness for deep neural networks.
problem Improving robustness of deep neural networks to adversarial perturbations.
method Bounding distributional robust risk with Lipschitz constant of the model.
result Distributional robustness upperbounds adversarial training risk.
Use simplified layerwise linear models to understand neural dynamics.
problem Complex neural network dynamics are hard to grasp.
method Apply simplified layerwise linear models to explain neural phenomena.
result Simplified models explain neural collapse, emergence, etc.
Embeds sparsity in deep neural networks, allowing exact zero parameters.
problem Learning sparse structures in deep networks.
method Embeds sparsity into neural network structure, allowing exact zero parameters during training.
result Can learn both structured and unstructured sparsity.
This thesis aims to automate deep neural network design for efficiency and complexity reduction.
problem Manual design of deep neural networks is inefficient and complex.
method Examines and proposes automated approaches to neural network design.
result Creation of less complex models with good performance through automation.