Enhanced text-to-speech synthesizes expressive speech from a single example.
problem Creating a new expressive speech style from a single example of speech.
method Combines VAE and Normalizing Flows to improve disentanglement and naturalness.
result Reduces KL-divergence by 22% and improves perceptual metrics.
RUSLAN is a large Russian speech corpus for text-to-speech.
problem Lack of high-quality annotated Russian speech data for text-to-speech.
method Developed a large annotated Russian speech corpus and trained a neural network for text-to-speech synthesis.
result Synthesized speech quality evaluated with MOS scores: 4.05 for naturalness, 3.78 for intelligibility.
Improves text-to-speech speed by interleaving character reading and audio synthesis.
problem Latency in text-to-speech models limits their use in time-sensitive tasks.
method Reinforcement learning to train an agent to choose the order of character reading and audio synthesis.
result The proposed method successfully balances latency and audio quality.
Grad-TTS models speech from text using diffusion probabilistic techniques.
problem Creating high-quality speech from text input.
method Score-based decoder with stochastic differential equations for noise-to-speech transformation.
result Grad-TTS produces mel-spectrograms from text input with competitive quality.
Guided Flows enhance sample quality in conditional image generation and text-to-speech.
problem Improving sample quality in conditional generative models.
method Integrating classifier-free guidance into Flow Matching (FM) models for Continuous Normalizing Flows (CNFs).
result Guided Flows significantly improve sample quality in conditional image generation and text-to-speech synthesis.
A new TTS method uses diffusion and VAE for better speech synthesis.
problem Improving text-to-speech synthesis for better speech quality and robustness.
method Combines diffusion probabilistic model and variational autoencoder for latent variable conversion.
result The method is robust to poor orthography and alignment errors.
Meta-learning approach discovers task-specific modules for few-shot tasks.
problem Meta-learning large models with limited task-specific data.
method Bayesian shrinkage for automatic discovery of task-specific and general modules.
result Outperforms existing meta-learning methods in domains with little task data and long adaptation horizons.
Improves naturalness in TTS samples using quantized VAE and auto-regressive prosody.
problem Discontinuous and unnatural speech from standard VAE priors.
method Discretized latent features using vector quantization (VQ), and separately trained autoregressive (AR) prior model.
result Significantly improves naturalness in random sample generation.
The state-of-the-art in text-to-speech synthesis has recently improved considerably due to novel neural waveform generation methods, such as WaveNet. However, these methods suffer from their slow sequential inference process, while their parallel versions are difficult to train and even more expensive computationally. …
We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and independent learned embeddings for each speaker. The aim of training is not to produce a neural network with fixed weights, which is then deplo…
BOFFIN TTS optimizes hyper-parameters for new speaker adaptation.
problem Fine-tuning a pre-trained TTS model for a new speaker with limited data.
method Bayesian optimization to efficiently find optimal hyper-parameters.
result Average 30% improvement in speaker similarity over standard techniques.
Improves TTS accuracy by correcting context-dependent units.
problem Improves text-to-speech accuracy through speaker adaptation.
method Statistical model predicting context-dependent phonetic unit classes and their mean error values.
result Corrected boundaries of units improve TTS accuracy compared to HMM segmentation.
When the available data of a target speaker is insufficient to train a high quality speaker-dependent neural text-to-speech (TTS) system, we can combine data from multiple speakers and train a multi-speaker TTS model instead. Many studies have shown that neural multi-speaker TTS model trained with a small amount data f…
We investigated the training of a shared model for both text-to-speech (TTS) and voice conversion (VC) tasks. We propose using an extended model architecture of Tacotron, that is a multi-source sequence-to-sequence model with a dual attention mechanism as the shared model for both the TTS and VC tasks. This model can a…
Investigates neural TTS systems for Japanese and English.
problem Improving neural TTS systems for high-quality speech synthesis.
method Comparative study of neural sequence-to-sequence TTS vs. DNN pipeline TTS, varying model architecture, parameter size, and language.
result A neural sequence-to-sequence TTS system requires sufficient model parameters and a powerful encoder for high-quality speech synthesis.
Paper develops neural network for Mandarin polyphone disambiguation.
problem Homograph problem in Mandarin Chinese text-to-speech.
method Bidirectional RNN for context, prediction network for mapping embeddings to pronunciations.
result Achieves 94.69% accuracy on polyphonic character dataset.
This research explores various sampling methods and probability distributions for hard alignment in sequence-to-sequence TTS synthesis.
problem Improving alignment accuracy in sequence-to-sequence text-to-speech synthesis.
method Investigated various sampling methods (greedy, beam, random) and probability distributions (Bernoulli, Concrete) for hard alignment.
result Deterministic search is more preferable than stochastic search for natural alignment transition.
End-to-end Text-to-speech (TTS) system can greatly improve the quality of synthesised speech. But it usually suffers form high time latency due to its auto-regressive structure. And the synthesised speech may also suffer from some error modes, e.g. repeated words, mispronunciations, and skipped words. In this paper, we…
Grapheme ASR improves with G2G model that corrects spelling errors.
problem Rare long-tail words in non-phonemic languages like English.
method Train G2G model on text-to-speech data to rewrite character sequences into phonetically consistent forms.
result Reduces Word Error Rate by 3% to 11% over a strong graphemic baseline.
Improved TTS style transfer across disjoint datasets with adversarial cycle consistency.
problem Suboptimal TTS style transfer on disjoint datasets with underrepresented styles.
method Adversarial cycle consistency training with paired and unpaired triplets.
result 78% improvement in style transfer with minimal reduction in fidelity and naturalness.
End-to-end speech synthesis is a promising approach that directly converts raw text to speech. Although it was shown that Tacotron2 outperforms classical pipeline systems with regards to naturalness in English, its applicability to other languages is still unknown. Japanese could be one of the most difficult languages …
Telephonetic improves robustness of neural language models to ASR errors.
problem Handling ASR errors and semantic noise in neural language models trained on written text.
method Data augmentation framework using character-level and embedding space perturbations.
result Achieves state-of-the-art perplexity of 37.49 on Penn Treebank corpus.
Improves E2E ASR performance on numeric sequences with additional training data and denormalization.
problem Challenges in recognizing numeric sequences out-of-vocabulary in ASR systems.
method Uses text-to-speech for additional numeric training data and a small-footprint neural network for denormalization.
result Reduction of WER by up to a factor of 8 in the longest numeric sequences.
Recent neural networks such as WaveNet and sampleRNN that learn directly from speech waveform samples have achieved very high-quality synthetic speech in terms of both naturalness and speaker similarity even in multi-speaker text-to-speech synthesis systems. Such neural networks are being used as an alternative to voco…
We present a TTS neural network that is able to produce speech in multiple languages. The proposed network is able to transfer a voice, which was presented as a sample in a source language, into one of several target languages. Training is done without using matching or parallel data, i.e., without samples of the same …
Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech. We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform. Our best models …
Many supervised learning tasks are emerged in dual forms, e.g., English-to-French translation vs. French-to-English translation, speech recognition vs. text to speech, and image classification vs. image generation. Two dual tasks have intrinsic connections with each other due to the probabilistic correlation between th…
Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps. While long-range dependencies are difficult to model directly in the time domain, we show that they can be more tractably modelled in two-dimensional time-frequency representations suc…
End-to-end model detects articulatory features from speech data.
problem Detecting articulatory features from speech data for various applications.
method Apply Listen, Attend and Spell (LAS) architecture and attention models.
result End-to-end training of manners and places of articulation detectors.
Paper proposes a new model for speech synthesis with better interpretability.
problem Improving interpretability in speech synthesis models.
method Hierarchical, fine-grained latent variable model with conditional variational auto-encoder (VAE).
result The new model allows better interpretability without degrading performance.
VPFD uses vocoder features for adversarial training in VC.
problem Adversarial training on waveform data is time-consuming and memory-intensive.
method VPFD employs vocoder features for adversarial training.
result VPFD achieves VC performance comparable to waveform discriminators with reduced training time and memory.
Recent speech technology research has seen a growing interest in using WaveNets as statistical vocoders, i.e., generating speech waveforms from acoustic features. These models have been shown to improve the generated speech quality over classical vocoders in many tasks, such as text-to-speech synthesis and voice conver…
Many machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few. One of the key challenges in sequence transduction is learning to …
End-to-end Sanskrit TTS developed with limited data, achieving good quality.
problem Developing natural-sounding speech for Sanskrit with scarce data.
method Fine-tuning Tacotron2 model with WaveGlow and transfer learning.
result Achieved an overall MOS of 3.38 from 37 evaluators.
Global Style Tokens (GSTs) are a recently-proposed method to learn latent disentangled representations of high-dimensional data. GSTs can be used within Tacotron, a state-of-the-art end-to-end text-to-speech synthesis system, to uncover expressive factors of variation in speaking style. In this work, we introduce the T…
This paper describes a method based on a sequence-to-sequence learning (Seq2Seq) with attention and context preservation mechanism for voice conversion (VC) tasks. Seq2Seq has been outstanding at numerous tasks involving sequence modeling such as speech synthesis and recognition, machine translation, and image captioni…
Unified approach to DP problems using Gumbel distribution and variational Bayesian inference.
problem Solving classical optimal path problems in a probabilistic framework.
method Gumbel distribution and variational Bayesian inference for latent optimal paths.
result Unified approach transforms DP problems into directed acyclic graphs with Gibbs distribution.
Neural waveform models such as the WaveNet are used in many recent text-to-speech systems, but the original WaveNet is quite slow in waveform generation because of its autoregressive (AR) structure. Although faster non-AR models were recently reported, they may be prohibitively complicated due to the use of a distillin…
Proposes Textual Echo Cancellation to improve speech recognition.
problem Improving speech recognition performance and user experience for smart devices.
method A novel sequence-to-sequence model with multi-source attention that processes both the microphone mixture signal and source text of TTS playback.
result Demonstrates enhanced speech recognition performance and reduced latency.
Solution for voice conversion with limited data using hierarchical seq2seq and attention models.
problem Voice conversion between speakers with limited parallel audio pairs.
method Hierarchical sequence to sequence model with attention-based decoder, trained on single speaker dataset.
result Improved voice conversion quality using mel spectrograms and wavenet vocoder.
Unsupervised model generates distinct intonation codes for speech synthesis.
problem Lack of understanding of what prosodic variations are controlled in speech synthesis.
method Phrase-level variational autoencoder with multi-modal prior, using mode centres as intonation codes.
result Generated intonation codes are perceptually distinct and carry various affect-related styles.
Thanks to the growing availability of spoofing databases and rapid advances in using them, systems for detecting voice spoofing attacks are becoming more and more capable, and error rates close to zero are being reached for the ASVspoof2015 database. However, speech synthesis and voice conversion paradigms that are not…
Generative models produce varied intonation in speech synthesis.
problem Typical TTS systems lack the ability to produce multiple distinct renditions of a sentence.
method Use variational autoencoders (VAEs) to capture a distribution over multiple renditions and produce varied intonation.
result Sampling from the tails of the VAE prior produces more varied intonation than traditional approaches, while maintaining naturalness.
ES for non-differentiable parameters scales to large models.
problem Learning non-differentiable parameters in large models.
method Hybrid approach combining ES for non-differentiable and gradient-based methods for differentiable parameters.
result Hybrid approach is competitive and allows training sparse models from the start.
iSTFTNet speeds up mel-spectrogram vocoders without sacrificing quality.
problem Efficiently converting mel-spectrograms to speech with minimal computation.
method Replaces convolutional layers with iSTFT after frequency dimension reduction.
result Significant reduction in computational cost with comparable quality.
SemiNAS reduces NAS cost by predicting accuracy of unlabeled architectures.
problem Costly evaluation of architectures limits NAS efficiency.
method SemiNAS uses unlabeled architectures to train an accuracy predictor.
result SemiNAS achieves comparable accuracy with less data.
End-to-end TTS framework uses hard alignment to improve accuracy.
problem End-to-end TTS systems struggle with accurate alignment between input text and output acoustic features.
method Proposes a constrained alignment scheme with hard monotonic alignments, marginalized during training.
result Improves alignment learning and prediction in end-to-end TTS systems.
This paper compares log-likelihood and BLEU scores for sequence generation tasks.
problem The discrepancy between density estimation and sequence generation performance.
method Comparing several density estimators on five machine translation tasks.
result The correlation between log-likelihood and BLEU varies depending on model families.