Improved animated faces using audiovisual and modality dropout.
problem Creating realistic animated faces using speech and visual cues.
method Training a deep learning model with modality dropout to balance audio and visual inputs.
result Modality dropout improves viewer preference for audiovisual-driven animation.
LRMM learns to recommend with missing modalities, improving robustness to data sparsity and cold-start issues.
problem Learning to recommend with missing modalities and cold-start problems.
method LRMM uses modality dropout and multimodal sequential autoencoder to learn multimodal representations and impute missing modalities.
result LRMM achieves state-of-the-art performance on rating prediction tasks and is more robust to data sparsity and cold-start issues.
MHVAE learns cross-modality inference inspired by human cognition.
problem Cross-modality inference in multimodal data.
method Hierarchical multimodal generative model with modality-specific and joint-modality distributions.
result MHVAE performs on par with state-of-the-art models on multimodal datasets.
Enhances uncertainty estimation in medical image segmentation.
problem Frequency-related noise in medical imaging leads to biased uncertainty estimates.
method Extends MC-Dropout to the frequency domain for better uncertainty estimation.
result MC-Frequency Dropout improves calibration and uncertainty in semantic segmentation.
Paper introduces Prob-SSI for robust OMA in noisy data.
problem Challenges in estimating modal parameters from noisy data.
method Probabilistic formulation of SSI, robust Prob-SSI algorithm.
result Robust Prob-SSI outperforms conventional SSI in corrupted data.
Bayesian approach improves activity recognition accuracy and uncertainty quantification.
problem Lack of predictive uncertainty in multimodal audiovisual activity recognition.
method Uncertainty aware multimodal Bayesian fusion framework combining deterministic and variational layers.
result Improved precision-recall AUC by 10.2% on MiT dataset.
New method quantifies uncertainty in physics-informed neural networks for stochastic problems.
problem Uncertainty in physics-informed neural networks for solving stochastic PDEs.
method Combining DNNs for residual and data mismatch, using stochastic data and dropout for uncertainty quantification, and active learning.
result Effective uncertainty quantification for both parametric and approximation uncertainties in PINNs.
Guided dropout improves neural network performance by selectively dropping nodes.
problem Overfitting in deep neural networks.
method Guided selection of nodes for dropout based on their strength.
result Guided dropout leads to better generalization compared to conventional dropout.
Advanced Dropout improves DNN performance without requiring model-specific dropout techniques.
problem Overfitting in deep neural networks due to lack of data.
method Model-free, end-to-end training with adaptive dropout rate optimization.
result Advanced Dropout outperforms other techniques on various datasets.
Enhanced dropout technique improves training speed and generalization.
problem Improving generalization and training speed of deep neural networks.
method Multi-sample dropout technique, creating multiple dropout samples and averaging their losses.
result Multi-sample dropout accelerates training and achieves lower error rates.
Continuous dropout mimics brain neuron firing rates to prevent feature detector co-adaptation.
problem Preventing overfitting in deep neural networks.
method Extending binary dropout to continuous dropout, inspired by brain neuron firing rates.
result Continuous dropout improves test performance by preventing feature detector co-adaptation.
Inserts proximal mapping into deep networks for better regularization.
problem Effective regularization of deep learning models to handle adversarial perturbations and correlations between modalities.
method Proposes a new layer that directly produces regularized hidden layer outputs using proximal mapping.
result Outperforms state-of-the-art methods in robust temporal learning and multiview modeling.
Dropout regularizes against high-order interactions by canceling interaction rates.
problem Overfitting to high-order interactions in neural networks.
method Analyzes Dropout through the lens of interaction effects, showing how it effectively cancels out the probability of surviving interactions of different orders.
result Dropout regularizes against high-order interactions by effectively canceling out the probability of surviving interactions of different orders.
Integrates global information into dropout for better text classification.
problem Improving neural networks for text classification.
method GI-Dropout, a novel dropout method integrating global information.
result Demonstrates the effectiveness of GI-Dropout on seven text classification tasks.
We analyze dropout in deep networks with rectified linear units and the quadratic loss. Our results expose surprising differences between the behavior of dropout and more traditional regularizers like weight decay. For example, on some simple data sets dropout training produces negative weights even though the output i…
Tabu Dropout improves performance of standard Dropout by generating more diverse neural network architectures.
problem Preventing co-adaptation of neurons in deep neural networks.
method Integrates a diversification strategy into dropout, marking units from the last forward propagation for re-selection in the current forward propagation.
result Improves performance of standard Dropout on MNIST and Fashion-MNIST datasets.
Dropout is one of the key techniques to prevent the learning from overfitting. It is explained that dropout works as a kind of modified L2 regularization. Here, we shed light on the dropout from Bayesian standpoint. Bayesian interpretation enables us to optimize the dropout rate, which is beneficial for learning of wei…
Adversarial dropout improves RNNs' performance on sequential data tasks.
problem Improving generalization performance of RNNs for sequential data.
method Adversarial dropout technique for RNNs using intentionally generated dropout masks.
result Adversarial dropout improves RNNs' effectiveness on sequential tasks.
Dropout is a simple but effective technique for learning in neural networks and other settings. A sound theoretical understanding of dropout is needed to determine when dropout should be applied and how to use it most effectively. In this paper we continue the exploration of dropout as a regularizer pioneered by Wager,…
Dropout has been witnessed with great success in training deep neural networks by independently zeroing out the outputs of neurons at random. It has also received a surge of interest for shallow learning, e.g., logistic regression. However, the independent sampling for dropout could be suboptimal for the sake of conver…
Rate-In dynamically adjusts dropout rates during inference to improve uncertainty estimation in neural networks.
problem Static dropout rates lead to suboptimal uncertainty estimates in neural networks.
method Rate-In dynamically adjusts dropout rates using information-theoretic principles.
result Rate-In improves calibration and sharpens uncertainty estimates compared to fixed or heuristic dropout rates.
QSD enhances deep network performance through biologically plausible dropout.
problem Overfitting in deep networks.
method Quantal Synaptic Dilution (QSD) model based on neuronal synapses.
result QSD outperforms standard dropout in various deep network architectures.
Analyzes dropout in neural networks using differential equations.
problem Lack of theoretical understanding of dropout's effectiveness.
method Analytical study of dropout in two-layer neural networks with online stochastic gradient descent.
result Dropout reduces detrimental correlations and optimal probability increases with data noise.
Develops a multilevel Monte Carlo framework with dropout for efficient uncertainty quantification.
problem Efficiently quantify uncertainty in complex models using dropout.
method Integrates multilevel Monte Carlo with Monte Carlo dropout, creating coupled estimators to reduce variance.
result Demonstrates significant variance reduction and efficiency gains over single-level Monte Carlo dropout.
Dropout introduces both explicit and implicit regularization effects.
problem Understanding the full impact of dropout regularization.
method Disentangled explicit and implicit regularization effects through experiments and analytic simplifications.
result Explicit and implicit regularization effects of dropout are distinct and can be characterized analytically.
Structured dropout improves confidence calibration in convolutional neural networks.
problem Poor confidence calibration in convolutional neural networks using dropout.
method Structured dropout to promote model diversity and improve calibration.
result Structured dropout reduces calibration error compared to naive dropout.
Dropout technique is analyzed using information geometry.
problem Understanding the regularization performance of dropout in neural networks.
method Unified analysis from information geometry viewpoint.
result Dropout flattens the model manifold and its performance depends on curvature.
The paper analyzes SGD with dropout regularization in linear models, proving asymptotic properties and providing inference tools.
problem Analyzing the behavior of SGD with dropout regularization in linear models.
method Establishing geometric-moment contraction (GMC) and proving quenched central limit theorems (CLT).
result The existence of a unique stationary distribution and asymptotic normality results for SGD with dropout.
Deep learning is the state-of-the-art in fields such as visual object recognition and speech recognition. This learning uses a large number of layers and a huge number of units and connections. Therefore, overfitting is a serious problem with it, and the dropout which is a kind of regularization tool is used. However, …
Variational dropout and ARD are shown to be equivalent, improving sparsity in neural networks.
problem Improper priors in variational dropout models and theoretical issues.
method Applying ARD to Bayesian DNNs with Gaussian approximations and exploring hierarchical priors.
result Variational dropout and ARD objectives are equivalent, with improved sparsity.
Dropout training improves neural networks' performance.
problem Improving neural network convergence and generalization.
method Two-layer neural networks with ReLU activations, overparametrization, and positive margin assumption.
result Dropout training achieves ε-suboptimality in test error in O(1/ε) iterations.
The paper improves dropout's utility by reducing interactions in deep neural networks.
problem Over-fitting problem in deep learning.
method Game-theoretic interactions analysis and interaction loss.
result Interaction loss improves dropout's utility and boosts DNN performance.
Study compares dropout and l2 regularization in linear models.
problem Understanding the statistical behavior of dropout and l2 regularization in linear models.
method Derives non-asymptotic bounds for gradient descent iterates with dropout and compares them to l2 regularization.
result Indicates a more subtle relationship between dropout and l2 regularization, highlighting interactions between dynamics and randomness.
Qualitative analysis of MC dropout for NN model uncertainty.
problem Measuring uncertainty in neural network models.
method Mathematical formulation of Monte Carlo dropout and its benefits/costs in NN models.
result Potential benefits and associated costs of using MC dropout in NN models.
Dropout is typically interpreted as bagging a large number of models sharing parameters. We show that using dropout in a network can also be interpreted as a kind of data augmentation in the input space without domain knowledge. We present an approach to projecting the dropout noise within a network back into the input…
Dropout controls model capacity in deep learning and matrix completion.
problem Controlling model capacity in deep learning and matrix completion problems.
method Investigates dropout's effect on model capacity and Rademacher complexity.
result Dropout induces a regularizer that controls model capacity in expectation.
Characterizes dropout's regularizer in deep linear networks.
problem Understanding dropout's regularization effect in deep learning.
method Formal characterization of dropout's regularizer, showing it is composed of an ℓ2-path regularizer and the squared nuclear norm. result For large dropout rates, the global optima of the dropout objective can be characterized.
Continuum Dropout improves neural differential equations by preventing overfitting.
problem Overfitting in Neural Differential Equations (NDEs).
method Introduces Continuum Dropout, a regularization technique based on alternating renewal processes.
result Continuum Dropout outperforms existing methods in various tasks, improving generalization and uncertainty quantification.
This paper introduces defensive dropout to protect deep neural networks from adversarial attacks.
problem Vulnerability of deep neural networks to adversarial attacks.
method Proposes using dropout at test time to enhance robustness, modeling the problem as a game between attacker and defender.
result Defensive dropout reduces adversarial attack success rate significantly, from 100% to 13.89% on MNIST.
The paper improves marine buoy placement to detect ships robustly against disruptions.
problem Detecting fishing vessels in the presence of natural and man-made disruptions.
method Formulated as a clustering problem, used dropout k-means and k-median to improve buoy placement robustness.
result Improved ship detection probability with dropout k-means compared to classic methods.
Dropout and other feature noising schemes control overfitting by artificially corrupting the training data. For generalized linear models, dropout performs a form of adaptive regularization. Using this viewpoint, we show that the dropout regularizer is first-order equivalent to an L2 regularizer applied after scaling t…
Study on dropout in neural networks using percolation theory.
problem Understanding dropout's effect on neural network training.
method Investigates percolation models mimicking dropout in neural networks.
result Dropout can cause a breakdown in neural networks.
We explore a recently proposed Variational Dropout technique that provided an elegant Bayesian interpretation to Gaussian Dropout. We extend Variational Dropout to the case when dropout rates are unbounded, propose a way to reduce the variance of the gradient estimator and report first experimental results with individ…
Proposes LBD for more robust deep learning models.
problem Improving robustness and uncertainty in deep learning models.
method Model-agnostic learnable Bernoulli dropout with joint optimization of dropout rates.
result Superior performance compared to other dropout schemes.
Wasserstein Dropout improves uncertainty estimation in neural networks.
problem Estimating neural uncertainties for safe machine learning.
method A purely non-parametric approach using dropout-based sub-network distributions and Wasserstein distance.
result Wasserstein Dropout outperforms state-of-the-art methods in uncertainty estimation.
Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks. This paper proposes a theoretical explanation for this phenomenon: we show that, under a generative Poisson topic model with long documents, dropout training improves the exponent…
Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimates, a grid-search over the dropout probabilities is necessary - a prohibitive operation with large models, and an impossible one with RL. We …
Dropout improves regularization in flexible models for rare features.
problem Understanding theoretical properties of dropout in generalized linear models.
method Theoretical analysis and application to adaptive smoothing with B-splines.
result Dropout prefers rare features in mean and dispersion parameters.