Paper proposes a new loss function for training neural speech models.
problem Training high-performance neural speech waveform models.
method Uses short-time Fourier transform (STFT) spectra and assumes Gaussian and von Mises distributions for amplitude and phase spectra.
result Synthesized high-quality speech waveforms.
WaveCycleGAN2 improves speech synthesis quality by reducing aliasing.
problem Human ear can still distinguish synthesized speech from natural speech.
method WaveCycleGAN2 uses generators without down/up-sampling modules and combines discriminators from waveform and acoustic parameter domains.
result WaveCycleGAN2 achieves high-quality speech synthesis with comparable mean opinion scores to natural speech.
A faster neural waveform model for speech synthesis.
problem Slow waveform generation in existing neural models.
method Proposes a non-autoregressive neural source-filter model.
result Generated waveforms 100 times faster than AR WaveNet.
This study proposes a fully convolutional network (FCN) model for raw waveform-based speech enhancement. The proposed system performs speech enhancement in an end-to-end (i.e., waveform-in and waveform-out) manner, which dif-fers from most existing denoising methods that process the magnitude spectrum (e.g., log power …
WaveCycleGAN converts synthetic speech to natural speech using cycle-consistent adversarial networks.
problem Over-smoothing effect in synthetic speech, leading to quality degradation.
method Cycle-consistent adversarial networks for waveform-level modification.
result Improves naturalness of generated speech sounds.
Recent speech technology research has seen a growing interest in using WaveNets as statistical vocoders, i.e., generating speech waveforms from acoustic features. These models have been shown to improve the generated speech quality over classical vocoders in many tasks, such as text-to-speech synthesis and voice conver…
This study improves text-to-speech synthesis using GANs for glottal excitation.
problem Slow inference and computational cost of WaveNet and difficulty in parallel training of GANs.
method Adopted GANs for parallel waveform generation in speech signal and glottal excitation.
result GAN-based glottal excitation model achieves quality and voice similarity on par with WaveNet.
Real-time speech enhancement model removes various noises and reverb.
problem Real-time speech enhancement in noisy environments.
method Causal speech enhancement model using encoder-decoder architecture with skip-connections, optimized in time and frequency domains.
result The model matches state-of-the-art performance while working directly on raw waveform.
NSF models generate speech waveforms faster and better than WaveNet.
problem Efficiently generating speech waveforms for statistical parametric synthesis.
method Neural source-filter (NSF) models that combine sine-based excitation, non-AR filter, and conditional preprocessing.
result NSF models generate waveforms 100 times faster than WaveNet and have better quality.
Deep neural network learns robust acoustic models from speech waveforms.
problem Robustness in speech recognition systems using standard feature extraction techniques.
method Deep convolutional neural network with stochastic variational inference and cosine modulated filters.
result Superior performance compared to baseline waveform-based models and deep CNNs with FBANK features.
This paper improves speech recognition by using raw waveform signals in multi-span CNN acoustic models.
problem Improving speech recognition accuracy using raw waveform signals.
method Proposes a novel multi-span structure for acoustic modelling based on raw waveform signals with multiple CNN input layers.
result Multi-span acoustic models yield a lower word error rate (WER) than traditional FBANK feature-based models.
This research improves neural synthesizers for music sounds from speech data.
problem Applying speech synthesis techniques to musical instrument sounds.
method Comparison of three neural synthesizers in three scenarios: training, zero-shot learning, and fine-tuning.
result Neural synthesizers trained on speech data and fine-tuned on music data perform better.
Recent advances in speech synthesis suggest that limitations such as the lossy nature of the amplitude spectrum with minimum phase approximation and the over-smoothing effect in acoustic modeling can be overcome by using advanced machine learning approaches. In this paper, we build a framework in which we can fairly co…
A new TTS method uses diffusion and VAE for better speech synthesis.
problem Improving text-to-speech synthesis for better speech quality and robustness.
method Combines diffusion probabilistic model and variational autoencoder for latent variable conversion.
result The method is robust to poor orthography and alignment errors.
The paper proposes using CWT and STFT for training neural speech models.
problem Training high-quality neural speech models.
method Proposes spectral amplitude and phase losses from STFT and CWT for training.
result Shows that CWT spectral loss can train a high-quality model as good as STFT-based loss.
RawNet synthesizes speech without relying on human-designed features.
problem Speech synthesis models rely heavily on human-designed spectral features.
method End-to-end neural vocoder using auto-encoder structure, learning features and audio recovery directly from raw waveform.
result RawNet achieves better speech quality with a simplified model architecture and faster inference speed.
DiffWave generates high-fidelity audio waveforms efficiently.
problem Conditional and unconditional audio waveform generation.
method Non-autoregressive diffusion model using Markov chain synthesis.
result DiffWave produces high-quality audios in various tasks.
Anonymizes speech data to protect privacy.
problem Protecting personal speech data from misuse.
method Extracts features, uses x-vectors, and neural models to synthesize anonymized speech.
result Effective in concealing speaker identities without compromising speech quality.
Unified framework for speech separation using deep learning.
problem Extracting individual speech sources from mixed signals.
method Unified framework combining spectrogram and waveform separations.
result Unified framework achieves competitive performance.
This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoreg…
Recent neural networks such as WaveNet and sampleRNN that learn directly from speech waveform samples have achieved very high-quality synthetic speech in terms of both naturalness and speaker similarity even in multi-speaker text-to-speech synthesis systems. Such neural networks are being used as an alternative to voco…
Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech. We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform. Our best models …
Improves speech recognition in noisy environments using robust acoustic models.
problem Adverse environments with significant mismatch between training and test conditions.
method Theoretical analysis of data augmentation as vicinal risk minimization, using mixture of Gaussians to incorporate robust inductive bias.
result Waveform-based approach shows 150% relative improvement in out-of-distribution generalization.
Paper presents a novel waveform-to-waveform model for music source separation.
problem Isolating individual instruments from mixed music recordings.
method Adapted Conv-Tasnet to waveform domain and developed Demucs with U-Net and bidirectional LSTM.
result Demucs achieves superior performance on music source separation tasks, surpassing existing state-of-the-art.
VPFD uses vocoder features for adversarial training in VC.
problem Adversarial training on waveform data is time-consuming and memory-intensive.
method VPFD employs vocoder features for adversarial training.
result VPFD achieves VC performance comparable to waveform discriminators with reduced training time and memory.
The fundamental frequency (F0) represents pitch in speech that determines prosodic characteristics of speech and is needed in various tasks for speech analysis and synthesis. Despite decades of research on this topic, F0 estimation at low signal-to-noise ratios (SNRs) in unexpected noise conditions remains difficult. T…
Convolutional network converts speaker voices without text.
problem Speaker conversion without text-based methods.
method Fully convolutional wav-to-wav network with ASR pre-training.
result Successfully converts TTS robot's voice to narrated audiobook voices.
FRAE compresses speech data efficiently with low bitrate.
problem Online compression of sequential data with temporal dependency.
method Feedback Recurrent AutoEncoder architecture with learned prior and entropy coder.
result Achieves high-quality speech waveforms at low fixed bitrate and variable bitrate.
iSTFTNet speeds up mel-spectrogram vocoders without sacrificing quality.
problem Efficiently converting mel-spectrograms to speech with minimal computation.
method Replaces convolutional layers with iSTFT after frequency dimension reduction.
result Significant reduction in computational cost with comparable quality.
Speech enhancement model is used to map a noisy speech to a clean speech. In the training stage, an objective function is often adopted to optimize the model parameters. However, in most studies, there is an inconsistency between the model optimization criterion and the evaluation criterion on the enhanced speech. For …
Inspired by recent work on neural network image generation which rely on backpropagation towards the network inputs, we present a proof-of-concept system for speech texture synthesis and voice conversion based on two mechanisms: approximate inversion of the representation learned by a speech recognition neural network,…
Thanks to the growing availability of spoofing databases and rapid advances in using them, systems for detecting voice spoofing attacks are becoming more and more capable, and error rates close to zero are being reached for the ASVspoof2015 database. However, speech synthesis and voice conversion paradigms that are not…
Computational paralinguistic analysis is increasingly being used in a wide range of cyber applications, including security-sensitive applications such as speaker verification, deceptive speech detection, and medical diagnostics. While state-of-the-art machine learning techniques, such as deep neural networks, can provi…
Joint training model for TTS and VC tasks using Tacotron and WaveNet.
problem Training a shared model for text-to-speech and voice conversion.
method Extended Tacotron model with dual attention mechanism for shared tasks, WaveNet for waveform generation.
result Joint training of a shared model achieves both TTS and VC tasks efficiently.
Wavesplit separates speech from mixtures using clustering.
problem Permutation problem in speech separation.
method End-to-end system infers speaker representations and estimates signals.
result Robust separation of long recordings, new benchmarks set.
Review of deep learning techniques for audio signal processing.
problem Improving audio signal processing using deep learning.
method Analysis of various deep learning models and techniques.
result Advancements in speech, music, and environmental sound processing.
SpecGrad improves neural vocoder sound quality by adapting diffusion noise to log-mel spectrogram.
problem Improving neural vocoder sound quality, especially in high-frequency bands.
method Adapting the diffusion noise distribution to the conditioning log-mel spectrogram through time-varying filtering.
result SpecGrad generates higher-fidelity speech waveform than conventional DDPM-based neural vocoders.
SING generates musical notes from instruments in real-time.
problem Efficiently generating high-quality audio from MIDI data.
method Frame-by-frame waveform generation with a single decoder, using a new loss function.
result SING produces significantly improved audio quality compared to state-of-the-art models, with 32x faster training and 2,500x faster inference.
Realistic music generation is a challenging task. When building generative models of music that are learnt from data, typically high-level representations such as scores or MIDI are used that abstract away the idiosyncrasies of a particular performance. But these nuances are very important for our perception of musical…
New method improves speech synthesis quality.
problem Efficiently train parallel speech synthesis models.
method Spectral energy distance for implicit generative models.
result State-of-the-art generation quality achieved.
Improves speech separation by integrating time and frequency domains.
problem Speech separation using deep learning techniques.
method Proposes a framework that combines time and frequency domain features, using an embedding network and clustering.
result Obtained state-of-the-art results on WSJ0-2mix dataset.
MelNet generates high-fidelity audio with long-range structure.
problem Capturing long-range dependencies in audio waveforms.
method Generative model in frequency domain, leveraging spectrograms.
result Improves audio generation in various tasks.
This paper aims to address two issues existing in the current speech enhancement methods: 1) the difficulty of phase estimations; 2) a single objective function cannot consider multiple metrics simultaneously. To solve the first problem, we propose a novel convolutional neural network (CNN) model for complex spectrogra…
The paper uses learned prototypes to explain deep learning models for time-series data.
problem Lack of explainable AI in deep learning models for high-risk decisions.
method Learned prototypes in latent space of deep learning models.
result Prototypes improve classification decisions and provide explainable insights.
Unsupervised learning of speech representations using WaveNet autoencoders.
problem Extract meaningful latent representations of speech signals.
method Applying autoencoding neural networks to speech waveforms, using a high capacity WaveNet decoder, and comparing three variants of latent representations.
result Comparable performance with top entries in the ZeroSpeech 2017 unsupervised acoustic unit discovery task.
Paper optimizes KWS models using NAS and quantization for limited resources.
problem Developing efficient keyword spotting models in resource-constrained environments.
method Neural Architecture Search (NAS) for model structure optimization and quantization of weights and activations.
result Achieved high accuracy (95.55%) with minimal parameters and operations using NAS and quantization.
Improves TTS accuracy by correcting context-dependent units.
problem Improves text-to-speech accuracy through speaker adaptation.
method Statistical model predicting context-dependent phonetic unit classes and their mean error values.
result Corrected boundaries of units improve TTS accuracy compared to HMM segmentation.
User-specific KWS system learns new keywords on-device.
problem Out-of-vocabulary problem in traditional KWS systems.
method Query-by-example enrollment and testing, using phonetic posteriors and FST.
result Promising performance on two keywords, preserving simplicity.