Deep neural network translates speech directly into text without transcription.
problem Directly translating speech between languages without manual transcription.
method Recurrent encoder-decoder architecture with attention mechanism.
result End-to-end model achieves state-of-the-art performance on speech translation task.
Method converts speech with attention and context preservation.
problem Voice conversion with improved stability and efficiency.
method Sequence-to-Sequence learning with attention and context preservation.
result Synthesized speech quality comparable to advanced methods.
Online sequence-to-sequence model for noisy speech recognition.
problem Practical limitation of online sequence-to-sequence models for instantaneous speech recognition.
method Improvement to online sequence-to-sequence model training and application to noisy settings.
result Online sequence-to-sequence model can produce partial transcripts as data arrives.
End-to-end system improves multi-speaker speech recognition.
problem Efficiently recognizing speech from multiple speakers without additional training data.
method End-to-end sequence-to-sequence framework with unified source separation and recognition.
result 83.1% relative improvement in multi-speaker speech recognition.
Improves text-to-speech speed by interleaving character reading and audio synthesis.
problem Latency in text-to-speech models limits their use in time-sensitive tasks.
method Reinforcement learning to train an agent to choose the order of character reading and audio synthesis.
result The proposed method successfully balances latency and audio quality.
Improves seq2seq speech recognition by addressing overconfidence and incomplete transcriptions.
problem Overconfidence and incomplete transcriptions in seq2seq speech recognition.
method Proposed practical solutions to address overconfidence and incomplete transcriptions using a trigram language model.
result Achieved competitive speaker independent word error rates (6.7%) with a trigram language model.
Improved speech recognition with audio-visual fusion.
problem Enhance speech recognition accuracy in noisy conditions.
method Proposes an attention-based audio-visual fusion strategy to align and learn from acoustic and lip motion data.
result Significant improvements in recognition accuracy (7-30%) on TCD-TIMIT dataset.
Paper explores EEG-based speech recognition using transformers, showing faster training and better performance for smaller vocabularies.
problem Continuous speech recognition using EEG features.
method Transformer-based ASR model compared to RNN-based models.
result Transformer models perform better for smaller vocabularies but RNN models outperform them for larger vocabularies.
TristouNet improves speaker comparison using neural networks and triplet loss.
problem Speaker comparison and change detection in short speech turns.
method Triplet loss for training neural network to project speech sequences into fixed-dimensional space.
result Significant improvements over state-of-the-art techniques for speaker comparison and change detection.
Investigates neural TTS systems for Japanese and English.
problem Improving neural TTS systems for high-quality speech synthesis.
method Comparative study of neural sequence-to-sequence TTS vs. DNN pipeline TTS, varying model architecture, parameter size, and language.
result A neural sequence-to-sequence TTS system requires sufficient model parameters and a powerful encoder for high-quality speech synthesis.
End-to-end ASR system uses context n-grams for better speech recognition.
problem Contextual information impacts speech recognition accuracy.
method Jointly optimizes ASR components with context embeddings during inference.
result Proposed CLAS system outperforms traditional methods by 68% relative WER.
Enhanced text-to-speech synthesizes expressive speech from a single example.
problem Creating a new expressive speech style from a single example of speech.
method Combines VAE and Normalizing Flows to improve disentanglement and naturalness.
result Reduces KL-divergence by 22% and improves perceptual metrics.
Proposes Textual Echo Cancellation to improve speech recognition.
problem Improving speech recognition performance and user experience for smart devices.
method A novel sequence-to-sequence model with multi-source attention that processes both the microphone mixture signal and source text of TTS playback.
result Demonstrates enhanced speech recognition performance and reduced latency.
Many machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few. One of the key challenges in sequence transduction is learning to …
Optimal Completion Distillation improves speech recognition models efficiently.
problem Improving sequence to sequence models for speech recognition.
method Optimal suffix selection using dynamic programming and target distribution.
result Achieves state-of-the-art performance on speech datasets.
Joint training model for TTS and VC tasks using Tacotron and WaveNet.
problem Training a shared model for text-to-speech and voice conversion.
method Extended Tacotron model with dual attention mechanism for shared tasks, WaveNet for waveform generation.
result Joint training of a shared model achieves both TTS and VC tasks efficiently.
Attention forcing improves sequence-to-sequence model training stability.
problem Training auto-regressive sequence-to-sequence models with attention mechanism is challenging.
method Attention forcing guides the model with generated output history and reference attention.
result Attention forcing trains models to recover from mistakes without requiring a schedule or classifier.
Enhanced transformer converts whispered speech to natural speech.
problem Machine recognition of whispered speech is challenging.
method Proposes an enhanced transformer architecture trained end-to-end using supervised learning.
result Similar formant distributions of converted speech to groundtruth.
Stochastic models fail to outperform standard recurrent networks in sequence modeling.
problem Discrepancy in performance between stochastic and standard recurrent models in sequence modeling.
method Re-examine roles of latent variables, remove restriction on fully factorized output distribution, compare auto-regressive models.
result Standard recurrent models consistently outperform stochastic models in sequence modeling.
Long Short-Term Memory (LSTM) is a recurrent neural network (RNN) architecture that has been designed to address the vanishing and exploding gradient problems of conventional RNNs. Unlike feedforward neural networks, RNNs have cyclic connections making them powerful for modeling sequences. They have been successfully u…
Compressive Transformer learns long-range sequences by compressing past memories.
problem Learning long-range sequences in language and speech models.
method Compressive Transformer compresses past memories for efficient long-range sequence learning.
result State-of-the-art performance on language and speech benchmarks.
End-to-end speech recognition using CNNs and CTC.
problem Efficiently train end-to-end speech recognition systems.
method Combining hierarchical CNNs with CTC for sequence labeling.
result Proposed model is computationally efficient and competitive.
This work improves ASR noise robustness using parallel data and T/S learning.
problem Noise robustness in automatic speech recognition.
method Teacher-student learning with parallel clean and noisy data, logits selection.
result Best student model yields significant WER reductions in noisy conditions.
QLSTM improves speech recognition by considering internal quaternion dependencies.
problem Weak internal dependencies in traditional RNNs for speech recognition.
method Proposes QLSTM, a quaternion-based LSTM that considers both external and internal dependencies.
result QLSTM achieves better performance with up to 2.8 times fewer parameters.
Streamable model improves speech recognition performance.
problem Improving speech recognition in long-form audio.
method Chunk-based attention model for streaming speech recognition.
result Streamable model maintains competitive performance and generalizes well to long-form speech.
Model sequences through their segmentations using dynamic programming.
problem Modeling sequences with unknown segmentations.
method Probabilistic model based on segmentations, using existing tools for segments, summing over all valid segmentations, dynamic programming for efficient computation.
result Demonstrated on text and speech recognition, discovering meaningful segments.
Neural network framework for language recognition considers sequence information and improves accuracy.
problem Challenging task of automatic language identification in noisy conditions.
method Proposes a neural network framework with bidirectional LSTM and attention modeling for relevance weighting.
result Significant improvements over conventional methods in noisy conditions and multi-speaker speech.
Model learns accent patterns from small datasets.
problem Automatic speech recognition struggles with rare accents.
method Automatically retrieves phonological generalizations from a small dataset.
result Model generates a million phonological variations of words.
Improved speech recognition with language model integration in sequence-to-sequence models.
problem Improving word error rate in speech recognition models.
method Log-linear combination of acoustic and language models with per-token renormalization.
result The proposed method shows good improvements over standard model combination on Librispeech system.
MoChA splits sequences into chunks for efficient, online attention.
problem Efficiently decoding sequence-to-sequence models with soft attention.
method Adaptive chunking of input sequences for soft attention computation.
result MoChA models achieve state-of-the-art performance in online speech recognition and document summarization.
Model learns association between text and speech without paired data.
problem Learning association between text and speech with limited paired data.
method Noisy channel generative model with variational inference and KL encoder loss.
result Tiny paired data (5 minutes) sufficient to learn association.
Generative adversarial networks improve speech synthesis from MFCCs.
problem Synthesizing speech from MFCCs, which are typically unusable for synthesis.
method Predict fundamental frequency and voicing from MFCCs, convert spectral envelope to filters, train excitation model, add noise.
result High quality speech can be reconstructed from MFCCs alone.
The paper improves part-of-speech tagging with multi-task learning and character-level word representations.
problem Improving part-of-speech tagging accuracy.
method Developed a new character-level word representation using feedforward neural network, pretraining with existing word vectors, and an additional prediction of neighbour labels as an auxiliary loss.
result The methods significantly improved POS tagging performance on English and Russian languages.
Improves sequence modeling with a flow-based recurrent mixture density network.
problem Sequence modeling and sequence-to-sequence mapping applications.
method Generalized recurrent mixture density networks using normalized flow transformations.
result Significantly improved fit to image sequences measured by log-likelihood.
We have recently shown that deep Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as acoustic models for speech recognition. More recently, we have shown that the performance of sequence trained context dependent (CD) hidden Markov model (HMM) acoustic m…
Novel fusion of autoencoders predicts sleepiness from speech.
problem Predicting sleepiness from speech recordings.
method Attention-based and recurrent sequence to sequence autoencoders for unsupervised representation learning.
result Fusion of autoencoders' representations achieves higher correlation with sleepiness scales.
This research explores various sampling methods and probability distributions for hard alignment in sequence-to-sequence TTS synthesis.
problem Improving alignment accuracy in sequence-to-sequence text-to-speech synthesis.
method Investigated various sampling methods (greedy, beam, random) and probability distributions (Bernoulli, Concrete) for hard alignment.
result Deterministic search is more preferable than stochastic search for natural alignment transition.
Improved speech recognition model reduces error rate from 9.2% to 5.6%.
problem Improving speech recognition accuracy for voice search tasks.
method Structural and optimization improvements to Listen, Attend, and Spell (LAS) model.
result Significant reduction in word error rate (WER) from 9.2% to 5.6% on voice search task.
Improved speech recognition with end-to-end attention models.
problem End-to-end speech recognition with open-vocabulary.
method Sequence-to-sequence attention-based models on subword units, new pretraining scheme, CTC loss function, LSTM language models.
result State-of-the-art word error rates (3.54% and 3.82%) on LibriSpeech test-clean.
Improves sequence generation by training a backward network.
problem Generating long-term dependencies in sequence models.
method Train a backward recurrent network to predict states of a forward model.
result Achieves 9% relative improvement in speech recognition and significant improvement in caption generation.
Model learns disentangled, interpretable representations from sequential data without supervision.
problem Learning disentangled and interpretable representations from sequential data without supervision.
method Factorized hierarchical variational autoencoder with multi-scale priors.
result Model outperforms i-vector baseline in speaker verification and reduces word error rate by 35% in mismatched scenarios.
Enhances speech quality in noisy environments using symbolic sequential modeling.
problem Improving speech quality in noisy conditions.
method Incorporates symbolic sequential modeling into speech enhancement framework.
result Significant improvement in speech quality metrics (PESQ, STOI) on TIMIT dataset.
QRNN and QLSTM improve speech recognition with fewer parameters.
problem Speech recognition with multi-dimensional input features.
method Quaternion algebra for modeling internal and external dependencies.
result QRNN and QLSTM achieve better performance with fewer parameters.
Improves E2E ASR performance on numeric sequences with additional training data and denormalization.
problem Challenges in recognizing numeric sequences out-of-vocabulary in ASR systems.
method Uses text-to-speech for additional numeric training data and a small-footprint neural network for denormalization.
result Reduction of WER by up to a factor of 8 in the longest numeric sequences.
Improved speech segment clustering using FTDTW.
problem Clustering speech segments of varying lengths and features.
method Feature Trajectory Dynamic Time Warping (FTDTW) for pairwise alignment of feature trajectories.
result Consistent and statistically significant improvements in cluster quality (F-measure and NMI).
Enhances speech from reverberated and noisy signals using Wide Residual Networks.
problem Improving intelligibility of speech from reverberated and noisy signals.
method Wide Residual Network architecture with residual connections for processing contextually correlated representations.
result Enhanced speech from the proposed method succeeded for both enhancement and speech recognition tasks.
Streaming ASR with transformer achieves low WER.
problem Real-time ASR with speech recognition.
method Time-restricted self-attention and triggered attention mechanisms.
result 2.8% and 7.2% WER for LibriSpeech test data.
Quaternion CNNs improve speech recognition with fewer parameters.
problem Efficient end-to-end speech recognition with minimal parameters.
method Integrating quaternion algebra into CNNs for speech feature processing.
result Quaternion CNNs achieve lower phoneme error rates with fewer parameters.