NRWS improves training of SBNs and HMs using natural gradient.
problem Training Sigmoid Belief Networks and Helmholtz Machines efficiently.
method Exploits block-diagonal structure of Fisher Information Matrices to use natural gradient.
result NRWS and NBiHM achieve better log-likelihood and faster convergence.
A new method for training generative models with sparse supervision.
problem Training deep generative models with sparse and varying supervision.
method Caffeinated Wake-Sleep (CWS) method, combining reweighted wake-sleep and teacher-forcing.
result The CWS method is robust to variable length supervision and performs well on various datasets.
Deep learning detects microsleep episodes in EEG data.
problem Automatic detection of microsleep episodes (MSEs) in EEG data.
method Convolutional neural networks (CNNs) and LSTM networks were implemented to analyze MWT data.
result Deep learning algorithms showed good performance close to human experts in detecting MSEs.
A new method learns latent variable updates directly, not approximating the posterior.
problem Intractable maximum-likelihood learning for complex latent-variable models.
method Amortised learning using wake-sleep Monte-Carlo strategy.
result Demonstrated effectiveness on various complex models.
Fluctuations in heart rate are intimately tied to changes in the physiological state of the organism. We examine and exploit this relationship by classifying a human subject's wake/sleep status using his instantaneous heart rate (IHR) series. We use a convolutional neural network (CNN) to build features from the IHR se…
New method improves variational inference for better posterior approximation.
problem Challenges in minimizing inclusive KL divergence for amortized variational inference.
method Likelihood-tempered sequential Monte Carlo samplers to estimate inclusive KL gradient.
result SMC-Wake method fits variational distributions more accurately than existing methods.
TVO tightens variational inference bounds for deep models.
problem Improving variational inference bounds for deep models.
method Introduces thermodynamic variational objective (TVO) connecting variational inference and thermodynamic integration.
result TVO provides tighter lower bound to log marginal likelihood than ELBO.
Stochastic control-flow models (SCFMs) are a class of generative models that involve branching on choices from discrete random variables. Amortized gradient-based learning of SCFMs is challenging as most approaches targeting discrete variables rely on their continuous relaxations---which can be intractable in SCFMs, as…
Framework for lifelong learning using eigentasks to avoid forgetting and transfer knowledge.
problem Avoiding forgetting and transferring knowledge in lifelong learning.
method Eigentask framework: skills paired with generative models, wake-sleep cycle for learning and consolidation.
result Improved performance in supervised continual learning, evidence of forward knowledge transfer.
Study automates detection of visitation disruptions in ICU patients.
problem Difficulty in detecting frequent visitation disruptions in ICU patients.
method Used DensePose R-CNN model to count people in video frames, analyzed disruptions and patient outcomes.
result Automated method detects visitation disruptions, impacts on pain and length of stay examined.
DDVI uses diffusion models for variational inference, improving latent variable model performance.
problem Improving variational inference in latent variable models.
method Introduces diffusion-based variational posteriors trained with a regularized ELBO.
result Outperforms alternative variational posteriors on various benchmarks and a biology task.
Sleep-based regularization stabilizes STDP in recurrent neural networks.
problem Pathological weight dynamics in recurrent SNNs.
method Periodic offline phases with stochastic decay and spontaneous activity.
result Sleep-based renormalization prevents weight saturation and preserves learned structure.
NAS-X improves inference and model learning for SLVMs.
problem Challenges in analytic inference and model learning for flexible SLVMs.
method NAS-X combines reweighted wake-sleep and smoothing sequential Monte Carlo.
result NAS-X provides low-bias and low-variance gradient estimates.
A new approach to learning in brain-like networks using adversarial algorithms.
problem Complex inter-dependencies in brain-like networks not compatible with conditional independence assumptions.
method Adversarial algorithm for learning models of perceptual processing.
result The approach can mimic known neural phenomena and yields testable hypotheses.
Massively parallel RWS improves inference in complex models.
problem Exponential sample requirement for effective importance weighting.
method Draws K samples of all n latent variables and individually reasons through all combinations. result Significant improvements over standard RWS.
QEM uses parallel importance weighting for fast approximate Bayesian inference.
problem Bayesian inference challenges in large models with many observations and latent variables.
method Expectation Maximization (EM) with massively parallel importance weighting.
result QEM is faster and more scalable than RWS and VI.
We study a variant of decision-theoretic online learning in which the set of experts that are available to Learner can shrink over time. This is a restricted version of the well-studied sleeping experts problem, itself a generalization of the fundamental game of prediction with expert advice. Similar to many works in t…
In this work, a dense recurrent convolutional neural network (DRCNN) was constructed to detect sleep disorders including arousal, apnea and hypopnea using Polysomnography (PSG) measurement channels provided in the 2018 Physionet challenge database. Our model structure is composed of multiple dense convolutional units (…
We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables. IPs are therefore highly flexible implicit priors over functions, with examples including data simulators, Bayesian neural networks and non-linear tr…
In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method. To mitigate the monotonic alignment requirement of SWAN, we introduce a new …
This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, an…
Highly expressive directed latent variable models, such as sigmoid belief networks, are difficult to train on large datasets because exact inference in them is intractable and none of the approximate inference methods that have been applied to them scale well. We propose a fast non-iterative approximate inference metho…
This paper describes InfoGAN, an information-theoretic extension to the Generative Adversarial Network that is able to learn disentangled representations in a completely unsupervised manner. InfoGAN is a generative adversarial network that also maximizes the mutual information between a small subset of the latent varia…
We introduce a new approach to learning in hierarchical latent-variable generative models called the "distributed distributional code Helmholtz machine", which emphasises flexibility and accuracy in the inferential process. In common with the original Helmholtz machine and later variational autoencoder algorithms (but …
Deep generative models have achieved impressive success in recent years. Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), as emerging families for generative model learning, have largely been considered as two distinct paradigms and received extensive independent studies respectively. This pa…
Sleep plays a vital role in human health, both mental and physical. Sleep disorders like sleep apnea are increasing in prevalence, with the rapid increase in factors like obesity. Sleep apnea is most commonly treated with Continuous Positive Air Pressure (CPAP) therapy. Presently, however, there is no mechanism to moni…
Paper improves variance control in importance weighted variational bounds.
problem Improving the variance of gradient estimators for IWAE.
method Develops a novel control variate that grows SNR as √K for large K.
result Empirically, the method yields superior variance reduction for generative models.
Deep learning detects sleep state fluctuations in neonates from single EEG channel.
problem Monitoring sleep state fluctuations in neonatal intensive care units.
method Deep learning-based algorithm trained on 53 EEG recordings, validated on 30 polysomnography recordings.
result High accuracy (90%) in detecting quiet sleep states from single EEG channel, generalizing well to external dataset.
Deep transfer learning improves automatic sleep staging accuracy.
problem Small cohort data variability and inefficiency in sleep studies.
method Deep transfer learning approach using a large dataset to a small cohort.
result Significant performance improvement on automatic sleep staging.
Evidence suggests Rapid-Eye-Movement (REM) Sleep Behaviour Disorder (RBD) is an early predictor of Parkinson's disease. This study proposes a fully-automated framework for RBD detection consisting of automated sleep staging followed by RBD identification. Analysis was assessed using a limited polysomnography montage fr…
A deep neural network detects sleep events in polysomnograms with high accuracy.
problem Manual scoring of sleep events in clinical analysis is inconsistent and time-consuming.
method A single deep neural network architecture trained on 1653 recordings for joint detection of arousals, leg movements, and sleep disordered breathing.
result Joint detection of sleep events yields higher accuracy compared to separate models, and correlates well with manual annotations.
Deep learning improves sleep apnea diagnosis accuracy.
problem Manual sleep expert scoring is tedious, time-consuming, and variable.
method Adapted deep learning method DOSED for automatic sleep event detection in PSG.
result Automatic approach achieved 81% accuracy for sleep apnea severity diagnosis.
SLEEPER combines deep learning with expert rules for accurate sleep staging.
problem Manual sleep staging is tedious and requires expert time.
method SLEEPER uses convolutional neural networks and expert rules to generate interpretable models.
result SLEEPER achieves comparable accuracy to human experts and deep neural networks.
Solves the Sleeping Beauty problem as a 'thirder' using the Kelly Criterion.
problem Solving the Sleeping Beauty problem with imperfect recall.
method Using the Kelly Criterion under multiplicative dynamics to maximize wealth growth rate.
result Sleeping Beauty maximizes expected growth rate as a 'thirder' and is impervious to Dutch books.
New method predicts wind farm power and wakes using weather patterns.
problem Inefficient and computationally intensive wind energy resource assessment.
method Unsupervised clustering of ERA5 data on wind velocity, WRF simulations at cluster centers, and post-processing.
result Accurate long-term predictions of power and wakes with reduced computational time.
Much attention has been given to automatic sleep staging algorithms in past years, but the detection of discrete events in sleep studies is also crucial for precise characterization of sleep patterns and possible diagnosis of sleep disorders. We propose here a deep learning model for automatic detection and annotation …
Model learns association between text and speech without paired data.
problem Learning association between text and speech with limited paired data.
method Noisy channel generative model with variational inference and KL encoder loss.
result Tiny paired data (5 minutes) sufficient to learn association.
The paper rethinks the use of exponential averaging in machine learning optimization.
problem The inefficiency of using exponential averaging in optimization algorithms.
method The paper connects EA-CM algorithms to Wake of Quadratic regularized models and proposes new algorithms, KLD-WRM.
result The new algorithms outperform existing methods like K-FAC on MNIST.
Study benchmarks automated sleep staging against human scorers, achieving human-level performance.
problem Lack of standardized comparison between human and automated sleep staging.
method Developed multi-scored datasets and a framework to compare multiple human scorers' consensus.
result Many automated methods can match human scorers' performance, with SimpleSleepNet achieving high F1 scores.
RobustSleepNet automates sleep stage classification for any PSG montage.
problem Manual sleep staging is tedious and expensive; automatic methods are limited by PSG montage and demographic differences.
method RobustSleepNet is a deep learning model that handles arbitrary PSG montages and is robust to demographic changes.
result RobustSleepNet achieves 97% F1 score on unseen data, outperforming specific training datasets by 2%.
Electroencephalography (EEG) is a method to record the electrical signals in the brain. Recognizing the EEG patterns in the sleeping brain gives insights into the understanding of sleeping disorders. The dataset under consideration contains EEG data points associated with various physiological conditions. This study at…
Sleep studies are important for diagnosing sleep disorders such as insomnia, narcolepsy or sleep apnea. They rely on manual scoring of sleep stages from raw polisomnography signals, which is a tedious visual task requiring the workload of highly trained professionals. Consequently, research efforts to purse for an auto…
Simple machine learning models outperform deep learning for sleep scoring.
problem Limited applicability and lack of interpretability of deep learning solutions for sleep scoring.
method Revisited sleep stage classification using classical machine learning, including preprocessing, feature extraction, and simple machine learning models.
result Competitive performance achieved with conventional machine learning models on public datasets.
New algorithm minimizes inclusive KL for VI, improving accuracy.
problem Improving variational inference accuracy with KL(p||q).
method Markovian score climbing (MSC) using stochastic gradients.
result MSC converges to local optimum of inclusive KL without bias.
RED detects sleep EEG events using deep neural networks, outperforming previous methods.
problem Manual detection of sleep EEG events is time-consuming and variable.
method Deep Recurrent Neural Networks (RNNs) with convolutional and recurrent components.
result RED outperforms state-of-the-art methods in sleep spindle and K-complex detection.
New deep learning method validated across multiple sleep staging databases.
problem Improving automatic sleep scoring accuracy across different datasets.
method Ensemble of local models using deep learning for automatic sleep staging.
result Good general performance compared to human experts and state-of-the-art methods.
Improved sleep apnea detection using sensor fusion and backward shortcut connections.
problem Untreated sleep apnea leads to severe health consequences; automated detection is needed.
method Late sensor fusion using backward shortcut connections to improve deep learning models.
result Significant improvement in predictive performance over single sensor methods.
This paper proposes a practical approach for automatic sleep stage classification based on a multi-level feature learning framework and Recurrent Neural Network (RNN) classifier using heart rate and wrist actigraphy derived from a wearable device. The feature learning framework is designed to extract low- and mid-level…