A new speech enhancement method using variational autoencoders.
problem Improving speech quality in noisy environments.
method Using a variational autoencoder as a speech model, trained with unsupervised noise modeling.
result The method outperforms existing techniques in speech enhancement.
A new TTS method uses diffusion and VAE for better speech synthesis.
problem Improving text-to-speech synthesis for better speech quality and robustness.
method Combines diffusion probabilistic model and variational autoencoder for latent variable conversion.
result The method is robust to poor orthography and alignment errors.
New RFs reduce kernel approximation variance and improve Transformer performance.
problem Efficient approximation of Gaussian and softmax kernels for kernel methods and Transformers.
method Parameterized, positive, non-trigonometric RFs optimized for variance reduction.
result Significant variance reduction in practice, outperforming previous methods.
In training speech recognition systems, labeling audio clips can be expensive, and not all data is equally valuable. Active learning aims to label only the most informative samples to reduce cost. For speech recognition, confidence scores and other likelihood-based active learning methods have been shown to be effectiv…
Unsupervised beamforming improves ASR in noisy conditions.
problem Improving ASR in unknown noisy environments.
method Multichannel NMF-Informed Beamforming (MVDR) for noise-robust ASR.
result The proposed method outperformed DNN-based beamforming in unknown environments.
SapAugment learns adaptive augmentation policies for better model training.
problem Fixed data augmentation methods often apply the same augmentation to all samples, ignoring sample difficulty.
method SapAugment adapts augmentation parameters based on training loss, learning a sample-adaptive policy.
result SapAugment achieves up to 21% relative reduction in word error rate on LibriSpeech dataset.
Blockwise bootstrap improves ASR performance testing for correlated data.
problem Testing reliability of WER improvements between ASR systems.
method Divide evaluation utterances into nonoverlapping blocks and resample these blocks.
result The variance estimator of absolute WER difference is consistent under mild conditions.
Paper shows continuous speech recognition with EEG features, no speech input.
problem Continuous speech recognition with limited vocabulary and noisy/no speech input.
method Connectionist temporal classification (CTC) model, EEG features, new deep learning architecture.
result Continuous speech recognition achieved on limited vocabulary with noisy/no speech input.
Improved visual speech synthesis using adapted ASR acoustic models.
problem Lack of synchronized audio, video, and depth data for speaker-independent speech-driven visual speech synthesis.
method Adapted an ASR acoustic model trained on audio-only data to the visual speech synthesis domain.
result Viewers significantly prefer animations generated from the adapted ASR acoustic model.
End-to-end speech recognition using EEG without speech input.
problem Speech recognition without direct speech input.
method Implemented attention model and CTC-based ASR systems for EEG signals; fused EEG with noisy speech features.
result Demonstrated end-to-end speech recognition using EEG signals.
AudioPaLM combines text and speech models to improve speech processing and translation.
problem Improving speech understanding and generation models.
method Unified multimodal architecture combining text and speech models.
result AudioPaLM significantly outperforms existing speech translation systems.
Graphical lasso models ASR utterance dependencies for consistent WER estimation.
problem Modeling dependent structure among ASR utterances for accurate significance analysis.
method Graphical lasso for dependency modeling, followed by blockwise bootstrap resampling.
result Statistically consistent variance estimator of WER under mild conditions.
Improved speech enhancement using diffusion models with MSE loss.
problem Efficient incorporation of noisy speech in generative speech enhancement.
method Augmented diffusion-based generative model with a MSE loss for enhanced speech.
result Proposed method improves speech enhancement performance compared to original diffusion model.
Enhances spoken speech quality using EEG signals.
problem Improves speech clarity in noisy environments.
method Generative adversarial network (GAN), gated recurrent unit (GRU), temporal convolutional network (TCN) regression models.
result Significant improvement in speech enhancement quality compared to traditional methods.
Diffusion models enhance speech without supervision.
problem Challenges in generalizing supervised speech enhancement methods to unseen conditions.
method Unsupervised speech enhancement using diffusion-based generative models.
result Demonstrates promising results compared to supervised and unsupervised baselines.
This paper presents a statistical method of single-channel speech enhancement that uses a variational autoencoder (VAE) as a prior distribution on clean speech. A standard approach to speech enhancement is to train a deep neural network (DNN) to take noisy speech as input and output clean speech. Although this supervis…
Synthetic speech data improves keyword spotting models with fewer real examples.
problem Training models for recognizing spoken keywords with limited real data.
method Used a pre-trained speech embedding model to extract features for training a small keyword spotting model.
result A model trained on synthetic speech data can detect 10 keywords with the same accuracy as a model trained on over 500 real examples.
AV-CPL uses continuous pseudo-labels for AVSR combining labeled and unlabeled data.
problem Improving AVSR performance with labeled and unlabeled data.
method Semi-supervised method using continuous pseudo-labels generated by the same AVSR model.
result Significant improvements in VSR performance on LRS3 dataset.
Paper explores EEG-based speech recognition using transformers, showing faster training and better performance for smaller vocabularies.
problem Continuous speech recognition using EEG features.
method Transformer-based ASR model compared to RNN-based models.
result Transformer models perform better for smaller vocabularies but RNN models outperform them for larger vocabularies.
WaveCycleGAN2 improves speech synthesis quality by reducing aliasing.
problem Human ear can still distinguish synthesized speech from natural speech.
method WaveCycleGAN2 uses generators without down/up-sampling modules and combines discriminators from waveform and acoustic parameter domains.
result WaveCycleGAN2 achieves high-quality speech synthesis with comparable mean opinion scores to natural speech.
This paper presents sampling-based speech parameter generation using moment-matching networks for Deep Neural Network (DNN)-based speech synthesis. Although people never produce exactly the same speech even if we try to express the same linguistic and para-linguistic information, typical statistical speech synthesis pr…
VoiceFilter-Lite separates speech from background in real-time for on-device speech recognition.
problem Separate speech from background in real-time for on-device speech recognition.
method Asymmetric loss, adaptive runtime suppression, quantization to 8-bit.
result VoiceFilter-Lite achieves real-time speech separation and maintains speech recognition performance.
AeGAN improves speech clarity in noisy environments.
problem Improving speech recognition in crowded noisy environments.
method Generative adversarial networks (GAN) with a novel architecture.
result The proposed framework outperforms traditional and learning-based methods.
Detects AI-synthesized speech using cepstral and bispectral analysis.
problem Validating the authenticity of speech from AI-generated content.
method Integrates cepstral and bispectral analysis for distinguishing human from AI-synthesized speech.
result Higher-order statistics show less correlation for human speech compared to AI-synthesis, and cepstral analysis reveals unique power components.
This paper benchmarks speech LVMs against deterministic models and adapts a video model to speech.
problem Speech generation models are inferior to deterministic models.
method Developed a speech benchmark of LVMs and compared them against deterministic models.
result The Clockwork VAE outperforms previous LVMs and reduces the gap to deterministic models.
This work disentangles speech and non-speech components from found data.
problem Building robust acoustic models from found data with non-standard variations.
method Latent Stochastic Models and Multinode Latent Space Variational Autoencoders (VAE).
result Speech and music can be separated in the latent space of a VAE, independent of the language.
Enhances speech quality in noisy environments using symbolic sequential modeling.
problem Improving speech quality in noisy conditions.
method Incorporates symbolic sequential modeling into speech enhancement framework.
result Significant improvement in speech quality metrics (PESQ, STOI) on TIMIT dataset.
Speech enhancement model is used to map a noisy speech to a clean speech. In the training stage, an objective function is often adopted to optimize the model parameters. However, in most studies, there is an inconsistency between the model optimization criterion and the evaluation criterion on the enhanced speech. For …
Speech enhancement improved using variational autoencoders and alpha-stable noise models.
problem Improving speech quality in noisy environments.
method Deep generative model using variational autoencoders with alpha-stable noise model.
result The proposed method outperforms conventional approaches in speech intelligibility and quality.
Bayesian SHMM models speech units from unannotated speech.
problem Discover latent speech units from unannotated data.
method Bayesian Subspace Hidden Markov Model (SHMM).
result System performs well on synthesis quality, lower bitrate.
This paper proposes a new loss using short-time Fourier transform (STFT) spectra for the aim of training a high-performance neural speech waveform model that predicts raw continuous speech waveform samples directly. Not only amplitude spectra but also phase spectra obtained from generated speech waveforms are used to c…
End-to-end model detects articulatory features from speech data.
problem Detecting articulatory features from speech data for various applications.
method Apply Listen, Attend and Spell (LAS) architecture and attention models.
result End-to-end training of manners and places of articulation detectors.
Continuous speech recognition from brain activity without vocalization.
problem Recognizing silent speech from EEG signals.
method Implemented a CTC ASR model using EEG signals.
result Demonstrated feasibility of EEG for continuous silent speech recognition.
Improved speech recognition using EEG and video.
problem Enhancing continuous speech recognition systems.
method Implemented a CTC-based ASR model using EEG features.
result EEG features improve continuous visual speech recognition.
New dataset for evaluating speech recognition fairness across demographics.
problem Lack of fairness metrics in speech recognition datasets.
method Developed Fair-Speech dataset with diverse demographic information.
result Helps evaluate ASR models for fairness across demographics.
Speech enhancement improved by adapting to unknown speakers without auxiliary signals.
problem Improving speech enhancement accuracy for unknown speakers.
method Adopting multi-task learning for speech enhancement and speaker identification, using multi-head self-attention.
result Achieved state-of-the-art performance and improved subjective quality.
Improved speech emotion recognition using pre-trained language models.
problem Challenging task of speech emotion recognition for natural human-machine interaction.
method Fine-tuning pre-trained language models for text emotion recognition, combining with speech emotion recognition.
result 73.5% accuracy in speech emotion recognition on a subset of IEMOCAP dataset.
WaveCRN improves E2E speech enhancement with efficient CNN and SRU.
problem Efficiently modeling speech locality and sequential properties for E2E speech enhancement.
method WaveCRN uses a CNN for speech locality and SRU for temporal sequential modeling, with RFM for noise suppression.
result WaveCRN outperforms state-of-the-art approaches with reduced complexity and inference time.
Speech synthesis from EEG features using RNN.
problem Speech synthesis from EEG data.
method Recurrent Neural Network (RNN) regression model to predict acoustic features from EEG features.
result Feasibility of synthesizing speech directly from EEG features demonstrated.
ADReSS Challenge at INTERSPEECH 2020 benchmarks speech recognition for Alzheimer's dementia.
problem Automated recognition of Alzheimer's dementia from spontaneous speech.
method Provides a benchmark speech dataset, defines two tasks (classification and regression), and presents baseline models.
result Demonstrates the feasibility of automated speech recognition for Alzheimer's dementia.
This paper improves speech synthesis using a DGP with SRU for naturalness.
problem Improving naturalness in synthetic speech.
method Deep Gaussian process with a recurrent architecture using SRU.
result SRU-DGP outperforms other models in naturalness of synthetic speech.
Speech emotion recognition system using features and text.
problem Improving accuracy in emotion recognition from speech.
method Used speech features (Spectrogram, MFCC) and text, trained Deep Neural Networks.
result Combined MFCC-Text CNN model achieved highest accuracy.
An ability to model a generative process and learn a latent representation for speech in an unsupervised fashion will be crucial to process vast quantities of unlabelled speech data. Recently, deep probabilistic generative models such as Variational Autoencoders (VAEs) have achieved tremendous success in modeling natur…
Enhanced FastMNMF for better speech separation.
problem Improving blind source separation for speech.
method Gaussian scale mixture (GSM) for heavy-tailed distributions.
result GSM-FastMNMF outperforms existing methods in speech enhancement.
New model predicts hate speech with uncertainty estimates.
problem Detect and explain hate speech without restricting free speech.
method Adapted deep neural networks with Monte Carlo dropout.
result Models can reliably assess when texts are borderline.
We present a recurrent encoder-decoder deep neural network architecture that directly translates speech in one language into text in another. The model does not explicitly transcribe the speech into text in the source language, nor does it require supervision from the ground truth source language transcription during t…
Paper shows EEG can improve ASR in noisy speech.
problem ASR performance drops in noisy conditions.
method Used EEG to train ASR models and improve performance.
result ASR accuracy improved with EEG features and distillation.
Universal audio perturbations fool ASR systems.
problem Speech recognition systems are vulnerable to adversarial attacks.
method Developed an algorithm to create universal perturbations that fool ASR models.
result Universal perturbations can fool ASR systems across different models.