Enhances speech recognition in new environments by embedding noise and scaling training data.
problem Improving speech recognition in unseen noisy environments.
method Embedding noise from unseen environments and scaling training data to 16,784 environments.
result Reduced word error rate from 34.04% to 15.46% on enhanced speech.
Sequence-to-sequence models, such as attention-based models in automatic speech recognition (ASR), are typically trained to optimize the cross-entropy criterion which corresponds to improving the log-likelihood of the data. However, system performance is usually measured in terms of word error rate (WER), not log-likel…
Large speech dataset for commercial use with 9.98% word error rate.
problem Creating a diverse speech recognition dataset for commercial purposes.
method Internet search for licensed audio data with transcriptions, training model on the dataset.
result Model trained on dataset achieves 9.98% word error rate on Librispeech's test-clean test set.
A hybrid ASR system using conformer architecture improves word-error-rate and training speed.
problem Improving word-error-rate and training efficiency for hybrid ASR systems.
method Used conformer architecture, applied time downsampling, and transposed convolutions.
result Conformer-based hybrid model achieves competitive results and significantly outperforms BLSTM-based hybrid model.
Improved speech recognition with cumulative adaptation methods.
problem Robust speech recognition in varying environments and speakers.
method Used a bidirectional LSTM neural network and i-vectors for adaptation.
result Achieved 13% relative improvement in word error rate.
Paper introduces a new optimisation method combining NG and Hessian Free for sequence training.
problem Overfitting and inefficiency in training DNNs with sequence criteria.
method Combines Natural Gradient and Hessian Free methods for better path traversal on parameter manifold.
result Achieves larger reductions in WER and lower WER compared to NG and HF methods.
FMP improves personalized ASR models on private devices.
problem Limited accuracy and privacy in federated fine-tuning of ASR models.
method FMP estimates global and personalized marginal distributions and adjusts NNLM probabilities.
result FMP achieves modest WER reductions on ASR rescoring tasks.
Algorithm extracts deterministic PDFA from probabilistic models with improved performance.
problem Learning deterministic models from probabilistic ones with noise.
method Adapted L* algorithm for probabilistic settings, using conditional probabilities and local tolerance.
result Achieves better performance on WER and NDCG than spectral extraction of WFAs.
A deep clustering model learns to separate audio sources without supervision.
problem Training deep clustering models requires supervision, limiting their applicability.
method Proposes an unsupervised spatial clustering approach to train a deep clustering system.
result The deep clustering model achieves similar performance to a multi-channel teacher without supervision.
New method improves ASR word confidence for diverse applications.
problem Mitigating ASR errors and improving word error rate.
method Heterogeneous Word Confusion Network (HWCN) with score calibration.
result Word sequence with best overall confidence is more accurate than 1-best result.
Graphical lasso models ASR utterance dependencies for consistent WER estimation.
problem Modeling dependent structure among ASR utterances for accurate significance analysis.
method Graphical lasso for dependency modeling, followed by blockwise bootstrap resampling.
result Statistically consistent variance estimator of WER under mild conditions.
The paper introduces a model to measure ASR fairness, addressing key issues.
problem Measuring fairness in ASR systems for different subgroups.
method Mixed-effects Poisson regression to control nuisance factors and handle unobserved heterogeneity.
result The method effectively addresses WER gaps among subgroups and is flexible for practical analyses.
Improved speech recognition with faster training and inference.
problem Training very deep CNNs for speech recognition is difficult.
method Proposed SNDCNN using SELU activations instead of RELU and shortcut connections/BN.
result Achieved similar or lower WER with faster training and inference.
System builds Somali ASR for UN humanitarian efforts.
problem Developing ASR for under-resourced Somali language.
method Acoustic model training with annotated speech, neural architectures, language model data augmentation, acoustic data perturbation.
result Best system achieved 53.75% word error rate.
Improved multilingual speech recognition with low latency for nine Indic languages.
problem Imbalance in training data across languages and low latency in interactive applications.
method Conditioning on a language vector and training language-specific adapter layers.
result Lower word error rate than monolingual E2E models and conventional systems.
Paper proposes low-rank gradient approximation to save memory for deep neural network training.
problem Memory limitation on mobile devices for deep neural network training.
method Approximating gradient matrices using low-rank parameterization.
result Reduces training memory by about 33.0% for Adam optimization and 4.5% relative lower word error rate on ASR personalization task.
Blockwise bootstrap improves ASR performance testing for correlated data.
problem Testing reliability of WER improvements between ASR systems.
method Divide evaluation utterances into nonoverlapping blocks and resample these blocks.
result The variance estimator of absolute WER difference is consistent under mild conditions.
State-level minimum Bayes risk (sMBR) training has become the de facto standard for sequence-level training of speech recognition acoustic models. It has an elegant formulation using the expectation semiring, and gives large improvements in word error rate (WER) over models trained solely using cross-entropy (CE) or co…
End-to-end ASR model combines word and character representation for improved performance.
problem Difficulty in training with word-level supervision due to sparsity of examples.
method Multi-task learning framework combining word and character representations.
result Improved word-error rate (WER) by interpolating between word-level and character-level models.
Streaming ASR with transformer achieves low WER.
problem Real-time ASR with speech recognition.
method Time-restricted self-attention and triggered attention mechanisms.
result 2.8% and 7.2% WER for LibriSpeech test data.
Grapheme ASR improves with G2G model that corrects spelling errors.
problem Rare long-tail words in non-phonemic languages like English.
method Train G2G model on text-to-speech data to rewrite character sequences into phonetically consistent forms.
result Reduces Word Error Rate by 3% to 11% over a strong graphemic baseline.
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
This work compares and evaluates various sampling methods for neural language models.
problem Lack of systematic comparison and myths about sampling methods.
method Monte Carlo sampling, importance sampling, compensated partial summation, noise contrastive estimation.
result All sampling methods can perform equally well if posterior probabilities are corrected.
This study improves knowledge distillation for RNN-T models with noisy labels.
problem Challenges in distilling knowledge from RNN-T models with variable quality teachers.
method Full-sum distillation and sequence-level knowledge distillation.
result Full-sum distillation outperforms other methods for RNN-T models, especially for bad teachers.
This paper improves speech recognition by using raw waveform signals in multi-span CNN acoustic models.
problem Improving speech recognition accuracy using raw waveform signals.
method Proposes a novel multi-span structure for acoustic modelling based on raw waveform signals with multiple CNN input layers.
result Multi-span acoustic models yield a lower word error rate (WER) than traditional FBANK feature-based models.
Seq2seq ASR adapts to speakers, improving performance by 25%.
problem Speaker adaptation for seq2seq ASR systems to match conventional methods.
method Applied Kullback-Leibler divergence and Linear Hidden Network adaptation to seq2seq models.
result 25% relative word error rate improvement with seq2seq model adaptation.
Conditional T/S learning improves student model performance by selectively learning from teacher or ground truth.
problem Teacher's occasional wrong guidance leads to suboptimal student model performance.
method Proposes a conditional T/S learning scheme where the student selectively chooses between teacher and ground truth based on teacher correctness.
result The conditional learning achieves significant performance improvements over traditional T/S learning.
TeLeS improves ASR confidence estimation by considering temporal alignment and lexical errors.
problem Inaccurate confidence scores from E2E ASR models, especially for overconfident predictions.
method Proposes TeLeS, a novel confidence score that considers temporal alignment and lexical errors, and uses shrinkage loss to handle data imbalance.
result TeLeS generalizes well across different languages and ASR models, leading to significant WER reduction.
SapAugment learns adaptive augmentation policies for better model training.
problem Fixed data augmentation methods often apply the same augmentation to all samples, ignoring sample difficulty.
method SapAugment adapts augmentation parameters based on training loss, learning a sample-adaptive policy.
result SapAugment achieves up to 21% relative reduction in word error rate on LibriSpeech dataset.
We propose a spatial diffuseness feature for deep neural network (DNN)-based automatic speech recognition to improve recognition accuracy in reverberant and noisy environments. The feature is computed in real-time from multiple microphone signals without requiring knowledge or estimation of the direction of arrival, an…
This paper explores the use of adversarial examples in training speech recognition systems to increase robustness of deep neural network acoustic models. During training, the fast gradient sign method is used to generate adversarial examples augmenting the original training data. Different from conventional data augmen…
We present a factorized hierarchical variational autoencoder, which learns disentangled and interpretable representations from sequential data without supervision. Specifically, we exploit the multi-scale nature of information in sequential data by formulating it explicitly within a factorized hierarchical graphical mo…
The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a single neural network trained in an end-to-end fashion. In this contribution, we analyse an attention-based seq2seq speech recognition syste…
The performance of automatic speech recognition (ASR) systems can be significantly compromised by previously unseen conditions, which is typically due to a mismatch between training and testing distributions. In this paper, we address robustness by studying domain invariant features, such that domain information become…
Recurrent neural network (RNN) language models (LMs) and Long Short Term Memory (LSTM) LMs, a variant of RNN LMs, have been shown to outperform traditional N-gram LMs on speech recognition tasks. However, these models are computationally more expensive than N-gram LMs for decoding, and thus, challenging to integrate in…
Deep Neural Network (DNN) acoustic models often use discriminative sequence training that optimises an objective function that better approximates the word error rate (WER) than frame-based training. Sequence training is normally implemented using Stochastic Gradient Descent (SGD) or Hessian Free (HF) training. This pa…
Vanishing long-term gradients are a major issue in training standard recurrent neural networks (RNNs), which can be alleviated by long short-term memory (LSTM) models with memory cells. However, the extra parameters associated with the memory cells mean an LSTM layer has four times as many parameters as an RNN with the…
Efficient DSP features reduce speech recognition memory usage.
problem Limited memory on DSPs for speech recognition.
method Developed efficient bottleneck features (BNFs) for DSPs.
result Reduced speech recognition memory usage by 10x with minimal accuracy loss.
Iterative AutoML improves ASR model compression by 5x without WER degradation.
problem Challenges in achieving high compression levels without degrading ASR performance.
method Iterative AutoML-based Low Rank Factorization (LRF) approach.
result Achieved over 5x compression without WER degradation.
Connectionist temporal classification (CTC) is widely used for maximum likelihood learning in end-to-end speech recognition models. However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e.g., word error rate (WER). This results in a mismatch…
Sequence-to-sequence attention-based models on subword units allow simple open-vocabulary end-to-end speech recognition. In this work, we show that such models can achieve competitive results on the Switchboard 300h and LibriSpeech 1000h tasks. In particular, we report the state-of-the-art word error rates (WER) of 3.5…
We present Listen, Attend and Spell (LAS), a neural network that learns to transcribe speech utterances to characters. Unlike traditional DNN-HMM models, this model learns all the components of a speech recognizer jointly. Our system has two components: a listener and a speller. The listener is a pyramidal recurrent ne…
We propose to model the acoustic space of deep neural network (DNN) class-conditional posterior probabilities as a union of low-dimensional subspaces. To that end, the training posteriors are used for dictionary learning and sparse coding. Sparse representation of the test posteriors using this dictionary enables proje…
AV-ASR system improves speech recognition with visual context.
problem Improving speech recognition accuracy with visual information.
method Transformer-based architecture with multiresolution and multimodal training.
result Multiresolution training speeds up convergence and improves WER by 18%.
Three LF training criteria improve neural network acoustic models without cross-entropy pre-training.
problem Improving purely sequence-trained neural network acoustic models.
method Comparison of three lattice-free discriminative training criteria (MMI, bMMI, sMBR) on LVCSR tasks.
result LF-bMMI models outperform plain LF-MMI models by 5% WER on Switchboard datasets.
This work improves ASR noise robustness using parallel data and T/S learning.
problem Noise robustness in automatic speech recognition.
method Teacher-student learning with parallel clean and noisy data, logits selection.
result Best student model yields significant WER reductions in noisy conditions.
Improved online AED models with multi-stage training and multi-task learning.
problem Enhance performance of online attention-based encoder-decoder models.
method Three-stage training with character encoder, BPE encoder, and attention decoder; multi-task learning at character and BPE levels; transfer learning from bidirectional encoder.
result 35% and 10% relative improvement over baselines for smaller and bigger models, respectively.
Improved robustness in ASR systems with speaker adaptation.
problem Improving robustness in automatic speech recognition systems.
method Weighted-Simple-Add method for adding weighted speaker information vectors to the conformer-based acoustic model.
result Achieved 11% relative improvement in WER on Switchboard 300h Hub5'00 dataset.