Enhances speech from reverberated and noisy signals using Wide Residual Networks.
problem Improving intelligibility of speech from reverberated and noisy signals.
method Wide Residual Network architecture with residual connections for processing contextually correlated representations.
result Enhanced speech from the proposed method succeeded for both enhancement and speech recognition tasks.
Study shows reverberant phase is not essential for weakly-supervised dereverberation.
problem Evaluating the role of reverberant phase in weakly-supervised dereverberation.
method Statistical Wave Field Theory and recent weak supervision framework.
result Wet phase carries limited useful information and is not essential for weakly supervised dereverberation.
This paper proposes RAS, a novel unsupervised loss function for speech separation.
problem Unsupervised speech separation without teacher models or synthesized mixtures.
method Train a neural network to separate sources given one channel, using Wiener filtering and SI-SDR.
result Achieves significant SDR improvement with minimal labeled data.
In this paper, we present a reverberation removal approach for speaker verification, utilizing dual-label deep neural networks (DNNs). The networks perform feature mapping between the spectral features of reverberant and clean speech. Long short term memory recurrent neural networks (LSTMs) are trained to map corrupted…
Unified deep learning framework improves SV in noisy, reverberant, and long non-speech segments.
problem Robust speaker verification in adverse environments, especially short speech segments.
method Feature Pyramid Module (FPM)-based Multi-scale Aggregation (MSA), Self-adaptive Soft VAD (SAS-VAD), Masking-based Speech Enhancement (SE).
result The proposed method outperforms baseline systems in challenging conditions.
Quaternion neural networks improve distant speech recognition.
problem Challenges in distant speech recognition due to noise and reverberation.
method Quaternion neural networks process multi-channel audio signals as quaternion entities, capturing internal and external dependencies.
result QLSTM outperforms real-valued LSTM on multi-channel distant speech recognition tasks.
We propose a spatial diffuseness feature for deep neural network (DNN)-based automatic speech recognition to improve recognition accuracy in reverberant and noisy environments. The feature is computed in real-time from multiple microphone signals without requiring knowledge or estimation of the direction of arrival, an…
This article addresses the modeling of reverberant recording environments in the context of under-determined convolutive blind source separation. We model the contribution of each source to all mixture channels in the time-frequency domain as a zero-mean Gaussian random variable whose covariance encodes the spatial cha…
Improved speech separation and enhancement using neural beamforming.
problem Challenging speech separation and enhancement in reverberant environments.
method Sequential neural beamforming combining spectral and spatial separation methods.
result Average improvement of 2.75 dB in scale-invariant signal-to-noise ratio and 14.2% absolute reduction in speech recognition metric.
Wavesplit separates speech from mixtures using clustering.
problem Permutation problem in speech separation.
method End-to-end system infers speaker representations and estimates signals.
result Robust separation of long recordings, new benchmarks set.
Large receptive field CNNs improve distant speech recognition.
problem Degrading performance of ASR systems in noisy environments.
method Investigated large receptive field CNN variants including recursive, dilated, and hourglass networks.
result Stacked hourglass networks show significant improvements in distant speech recognition.
RoyalFlush system improves multi-speaker ASR in M2MeT challenge.
problem Improving multi-speaker automatic speech recognition in noisy environments.
method Front-end processing with WPE and beamforming, data augmentation, and fusion of two ASR models.
result 12.22% absolute CER reduction on validation set and 12.11% on test set compared to baseline.
New unsupervised neural network for beamforming masks trained on real noisy speech.
problem Training neural beamforming masks without labeled data.
method Unsupervised training using a likelihood criterion from spatial mixture model.
result Performance comparable to supervised and teacher-trained systems.
Unified framework for speech separation using deep learning.
problem Extracting individual speech sources from mixed signals.
method Unified framework combining spectrogram and waveform separations.
result Unified framework achieves competitive performance.
Jointly reduces echo, reverberation, and noise using neural networks.
problem Simultaneous reduction of acoustic echo, reverberation, and noise.
method Multichannel Gaussian modeling and neural network for joint optimization of filters.
result Outperforms individual and joint non-spectral models.
We propose a method to generate audio adversarial examples that can attack a state-of-the-art speech recognition model in the physical world. Previous work assumes that generated adversarial examples are directly fed to the recognition model, and is not able to perform such a physical attack because of reverberation an…
Hybrid diarization framework handles overlapped speech and long recordings.
problem Challenges in clustering-based and end-to-end neural diarization approaches.
method Proposes a hybrid framework combining clustering and end-to-end neural diarization.
result Significantly better performance on long recordings with overlapped speech.
X-DC improves speech separation by making DNNs more interpretable.
problem Black-box nature of DNNs in speech separation tasks.
method Introduces X-DC, a DNN architecture that interprets as spectrogram template fitting followed by Wiener filtering.
result X-DC achieves comparable speech separation performance to DC but with enhanced interpretability.
Study neural networks learning from noisy examples via reverberation.
problem Learning from noisy examples in neural networks.
method Adapted Guerra's interpolation technique to provide statistical mechanics of supervised and unsupervised learning.
result Full phase diagrams and thresholds for learning are analytically obtained.
Neural codec for high-fidelity audio compression.
problem Efficiently compress audio while maintaining high quality.
method End-to-end neural network architecture with quantized latent space, single multiscale spectrogram adversary, loss balancer mechanism, and lightweight Transformer compression.
result 40% compression with no loss in quality, faster than real-time.
Proposes a DOA estimation method using IVs and DNNs for noise and reverberation reduction.
problem Accuracy of IV-based DOA estimation degrades due to noise and reverberation.
method Combines IV-based DOA estimation with DNNs for denoising and dereverberation.
result Average DOA error of 0.528 degrees, outperforming conventional methods.
Improved far-field speaker verification for short utterances in noisy conditions.
problem Challenges in speaker verification on short utterances in uncontrolled noisy environments.
method Used deep neural network architectures (TDNN and ResNet) and experimented with various embedding extractors and training procedures.
result ResNet architectures outperform x-vector approach in speaker verification quality for both long and short utterances.
End-to-end speech recognition using EEG without speech input.
problem Speech recognition without direct speech input.
method Implemented attention model and CTC-based ASR systems for EEG signals; fused EEG with noisy speech features.
result Demonstrated end-to-end speech recognition using EEG signals.
Paper shows EEG can improve ASR in noisy speech.
problem ASR performance drops in noisy conditions.
method Used EEG to train ASR models and improve performance.
result ASR accuracy improved with EEG features and distillation.
Paper shows continuous speech recognition with EEG features, no speech input.
problem Continuous speech recognition with limited vocabulary and noisy/no speech input.
method Connectionist temporal classification (CTC) model, EEG features, new deep learning architecture.
result Continuous speech recognition achieved on limited vocabulary with noisy/no speech input.
This paper presents sampling-based speech parameter generation using moment-matching networks for Deep Neural Network (DNN)-based speech synthesis. Although people never produce exactly the same speech even if we try to express the same linguistic and para-linguistic information, typical statistical speech synthesis pr…
RUSLAN is a large Russian speech corpus for text-to-speech.
problem Lack of high-quality annotated Russian speech data for text-to-speech.
method Developed a large annotated Russian speech corpus and trained a neural network for text-to-speech synthesis.
result Synthesized speech quality evaluated with MOS scores: 4.05 for naturalness, 3.78 for intelligibility.
AeGAN improves speech clarity in noisy environments.
problem Improving speech recognition in crowded noisy environments.
method Generative adversarial networks (GAN) with a novel architecture.
result The proposed framework outperforms traditional and learning-based methods.
Detects AI-synthesized speech using cepstral and bispectral analysis.
problem Validating the authenticity of speech from AI-generated content.
method Integrates cepstral and bispectral analysis for distinguishing human from AI-synthesized speech.
result Higher-order statistics show less correlation for human speech compared to AI-synthesis, and cepstral analysis reveals unique power components.
Improved visual speech synthesis using adapted ASR acoustic models.
problem Lack of synchronized audio, video, and depth data for speaker-independent speech-driven visual speech synthesis.
method Adapted an ASR acoustic model trained on audio-only data to the visual speech synthesis domain.
result Viewers significantly prefer animations generated from the adapted ASR acoustic model.
This paper presents a statistical method of single-channel speech enhancement that uses a variational autoencoder (VAE) as a prior distribution on clean speech. A standard approach to speech enhancement is to train a deep neural network (DNN) to take noisy speech as input and output clean speech. Although this supervis…
WaveCycleGAN2 improves speech synthesis quality by reducing aliasing.
problem Human ear can still distinguish synthesized speech from natural speech.
method WaveCycleGAN2 uses generators without down/up-sampling modules and combines discriminators from waveform and acoustic parameter domains.
result WaveCycleGAN2 achieves high-quality speech synthesis with comparable mean opinion scores to natural speech.
AudioPaLM combines text and speech models to improve speech processing and translation.
problem Improving speech understanding and generation models.
method Unified multimodal architecture combining text and speech models.
result AudioPaLM significantly outperforms existing speech translation systems.
Enhances spoken speech quality using EEG signals.
problem Improves speech clarity in noisy environments.
method Generative adversarial network (GAN), gated recurrent unit (GRU), temporal convolutional network (TCN) regression models.
result Significant improvement in speech enhancement quality compared to traditional methods.
Improved speech enhancement using diffusion models with MSE loss.
problem Efficient incorporation of noisy speech in generative speech enhancement.
method Augmented diffusion-based generative model with a MSE loss for enhanced speech.
result Proposed method improves speech enhancement performance compared to original diffusion model.
This work disentangles speech and non-speech components from found data.
problem Building robust acoustic models from found data with non-standard variations.
method Latent Stochastic Models and Multinode Latent Space Variational Autoencoders (VAE).
result Speech and music can be separated in the latent space of a VAE, independent of the language.
Continuous speech recognition from brain activity without vocalization.
problem Recognizing silent speech from EEG signals.
method Implemented a CTC ASR model using EEG signals.
result Demonstrated feasibility of EEG for continuous silent speech recognition.
Speech emotion recognition system using features and text.
problem Improving accuracy in emotion recognition from speech.
method Used speech features (Spectrogram, MFCC) and text, trained Deep Neural Networks.
result Combined MFCC-Text CNN model achieved highest accuracy.
AV-CPL uses continuous pseudo-labels for AVSR combining labeled and unlabeled data.
problem Improving AVSR performance with labeled and unlabeled data.
method Semi-supervised method using continuous pseudo-labels generated by the same AVSR model.
result Significant improvements in VSR performance on LRS3 dataset.
Enhanced text-to-speech synthesizes expressive speech from a single example.
problem Creating a new expressive speech style from a single example of speech.
method Combines VAE and Normalizing Flows to improve disentanglement and naturalness.
result Reduces KL-divergence by 22% and improves perceptual metrics.
Diffusion models enhance speech without supervision.
problem Challenges in generalizing supervised speech enhancement methods to unseen conditions.
method Unsupervised speech enhancement using diffusion-based generative models.
result Demonstrates promising results compared to supervised and unsupervised baselines.
Improved speech recognition using EEG and video.
problem Enhancing continuous speech recognition systems.
method Implemented a CTC-based ASR model using EEG features.
result EEG features improve continuous visual speech recognition.
Speech synthesis from EEG features using RNN.
problem Speech synthesis from EEG data.
method Recurrent Neural Network (RNN) regression model to predict acoustic features from EEG features.
result Feasibility of synthesizing speech directly from EEG features demonstrated.
Survey on DNNs for speech processing, focusing on limited data challenges.
problem Challenges in training DNNs for speech tasks with limited data.
method Overview of techniques for few-shot speech processing.
result Promising few-shot techniques for speech processing.
Speech enhancement improved by adapting to unknown speakers without auxiliary signals.
problem Improving speech enhancement accuracy for unknown speakers.
method Adopting multi-task learning for speech enhancement and speaker identification, using multi-head self-attention.
result Achieved state-of-the-art performance and improved subjective quality.
Synthetic speech data improves keyword spotting models with fewer real examples.
problem Training models for recognizing spoken keywords with limited real data.
method Used a pre-trained speech embedding model to extract features for training a small keyword spotting model.
result A model trained on synthetic speech data can detect 10 keywords with the same accuracy as a model trained on over 500 real examples.
Paper proposes a CNN for speech emotion recognition using center loss and reconstruction.
problem Speech emotion recognition (SER) in audio signals.
method Convolutional Neural Network (CNN) with center loss and reconstruction as regularizers.
result Proposed method achieves highly discriminative features for SER.
Paper explores EEG-based speech recognition using transformers, showing faster training and better performance for smaller vocabularies.
problem Continuous speech recognition using EEG features.
method Transformer-based ASR model compared to RNN-based models.
result Transformer models perform better for smaller vocabularies but RNN models outperform them for larger vocabularies.