Improves ASR accuracy for domain mismatch using machine translation.
problem Domain mismatch in ASR systems leads to suboptimal results.
method Machine translation to map out-of-domain ASR errors to in-domain terms.
result 7% absolute improvement in word error rate, 4 point BLEU score improvement.
Grapheme ASR improves with G2G model that corrects spelling errors.
problem Rare long-tail words in non-phonemic languages like English.
method Train G2G model on text-to-speech data to rewrite character sequences into phonetically consistent forms.
result Reduces Word Error Rate by 3% to 11% over a strong graphemic baseline.
End-to-end ASR error detection using audio-transcript entailment.
problem Detecting transcription errors in ASR systems to prevent error propagation.
method Proposes a novel end-to-end approach using audio-transcript entailment, with acoustic and linguistic encoders.
result Achieves CER of 26.2% on all transcription errors and 23% on medical errors specifically, improving by 12% and 15.4% respectively over a strong baseline.
Streaming ASR with transformer achieves low WER.
problem Real-time ASR with speech recognition.
method Time-restricted self-attention and triggered attention mechanisms.
result 2.8% and 7.2% WER for LibriSpeech test data.
TeLeS improves ASR confidence estimation by considering temporal alignment and lexical errors.
problem Inaccurate confidence scores from E2E ASR models, especially for overconfident predictions.
method Proposes TeLeS, a novel confidence score that considers temporal alignment and lexical errors, and uses shrinkage loss to handle data imbalance.
result TeLeS generalizes well across different languages and ASR models, leading to significant WER reduction.
A hybrid ASR system using conformer architecture improves word-error-rate and training speed.
problem Improving word-error-rate and training efficiency for hybrid ASR systems.
method Used conformer architecture, applied time downsampling, and transposed convolutions.
result Conformer-based hybrid model achieves competitive results and significantly outperforms BLSTM-based hybrid model.
Blockwise bootstrap improves ASR performance testing for correlated data.
problem Testing reliability of WER improvements between ASR systems.
method Divide evaluation utterances into nonoverlapping blocks and resample these blocks.
result The variance estimator of absolute WER difference is consistent under mild conditions.
Graphical lasso models ASR utterance dependencies for consistent WER estimation.
problem Modeling dependent structure among ASR utterances for accurate significance analysis.
method Graphical lasso for dependency modeling, followed by blockwise bootstrap resampling.
result Statistically consistent variance estimator of WER under mild conditions.
The paper introduces a model to measure ASR fairness, addressing key issues.
problem Measuring fairness in ASR systems for different subgroups.
method Mixed-effects Poisson regression to control nuisance factors and handle unobserved heterogeneity.
result The method effectively addresses WER gaps among subgroups and is flexible for practical analyses.
FMP improves personalized ASR models on private devices.
problem Limited accuracy and privacy in federated fine-tuning of ASR models.
method FMP estimates global and personalized marginal distributions and adjusts NNLM probabilities.
result FMP achieves modest WER reductions on ASR rescoring tasks.
Iterative AutoML improves ASR model compression by 5x without WER degradation.
problem Challenges in achieving high compression levels without degrading ASR performance.
method Iterative AutoML-based Low Rank Factorization (LRF) approach.
result Achieved over 5x compression without WER degradation.
RoyalFlush system improves multi-speaker ASR in M2MeT challenge.
problem Improving multi-speaker automatic speech recognition in noisy environments.
method Front-end processing with WPE and beamforming, data augmentation, and fusion of two ASR models.
result 12.22% absolute CER reduction on validation set and 12.11% on test set compared to baseline.
Paper proposes a method to maintain ASR performance on new tasks without forgetting old ones.
problem Mitigating forgetting in ASR models when learning new tasks.
method A novel explainability-based knowledge distillation combined with response-based knowledge distillation.
result Our method outperforms existing ones in mitigating forgetting on multi-stage sequential training tasks.
AV-ASR system improves speech recognition with visual context.
problem Improving speech recognition accuracy with visual information.
method Transformer-based architecture with multiresolution and multimodal training.
result Multiresolution training speeds up convergence and improves WER by 18%.
We present a new method for computing ASR word confidences that effectively mitigates the effect of ASR errors for diverse downstream applications, improves the word error rate of the 1-best result, and allows better comparison of scores across different models. We propose 1) a new method for modeling word confidence u…
Nowadays, machine learning based Automatic Speech Recognition (ASR) technique has widely spread in smartphones, home devices, and public facilities. As convenient as this technology can be, a considerable security issue also raises -- the users' speech content might be exposed to malicious ASR monitoring and cause seve…
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
This paper improves speech recognition by distilling knowledge from acoustic models.
problem Improving speech recognition accuracy using ensemble models.
method Proposes multi-teacher distillation strategies for joint CTC-attention end-to-end ASR systems, integrating error rate metric for optimization.
result Reports state-of-the-art error rates on various datasets and languages.
Sequence-to-sequence (seq2seq) based ASR systems have shown state-of-the-art performances while having clear advantages in terms of simplicity. However, comparisons are mostly done on speaker independent (SI) ASR systems, though speaker adapted conventional systems are commonly used in practice for improving robustness…
The performance of automatic speech recognition (ASR) systems can be significantly compromised by previously unseen conditions, which is typically due to a mismatch between training and testing distributions. In this paper, we address robustness by studying domain invariant features, such that domain information become…
This paper presents an end-to-end text-independent speaker verification framework by jointly considering the speaker embedding (SE) network and automatic speech recognition (ASR) network. The SE network learns to output an embedding vector which distinguishes the speaker characteristics of the input utterance, while th…
Improves TTS accuracy by correcting context-dependent units.
problem Improves text-to-speech accuracy through speaker adaptation.
method Statistical model predicting context-dependent phonetic unit classes and their mean error values.
result Corrected boundaries of units improve TTS accuracy compared to HMM segmentation.
We establish the first benchmark for federated learning with differential privacy in ASR, achieving competitive performance.
problem Challenges in training large transformer models for ASR in federated learning with differential privacy.
method Per-layer clipping and layer-wise gradient normalization to mitigate gradient heterogeneity.
result FL with DP is viable in ASR with strong privacy guarantees, achieving competitive performance.
In this paper, we analyzed how audio-visual speech enhancement can help to perform the ASR task in a cocktail party scenario. Therefore we considered two simple end-to-end LSTM-based models that perform single-channel audio-visual speech enhancement and phone recognition respectively. Then, we studied how the two model…
Sequence-to-sequence models, such as attention-based models in automatic speech recognition (ASR), are typically trained to optimize the cross-entropy criterion which corresponds to improving the log-likelihood of the data. However, system performance is usually measured in terms of word error rate (WER), not log-likel…
This paper evaluates ASR models on various co-processors, showing hardware acceleration benefits.
problem Lack of detailed training efficiency analysis for modern ASR systems.
method Developed a deep neural network ASR model and evaluated it on three co-processor platforms.
result Hardware acceleration yields good results even with less expensive equipment.
Improved neural speaker embeddings enhance ASR performance.
problem Few studies have explored neural speaker embeddings for ASR.
method Integrating improved neural speaker embeddings into a conformer-based hybrid HMM ASR system.
result Improved neural embeddings achieve on-par performance with i-vectors.
Defense against ASR attacks using dropout uncertainty.
problem Adversarial attacks on ASR systems.
method Dropout uncertainty in neural networks.
result High detection accuracy across various ASR systems and datasets.
We present the recent advances along with an error analysis of the IBM speaker recognition system for conversational speech. Some of the key advancements that contribute to our system include: a nearest-neighbor discriminant analysis (NDA) approach (as opposed to LDA) for intersession variability compensation in the i-…
Paper proposes low-rank gradient approximation to save memory for deep neural network training.
problem Memory limitation on mobile devices for deep neural network training.
method Approximating gradient matrices using low-rank parameterization.
result Reduces training memory by about 33.0% for Adam optimization and 4.5% relative lower word error rate on ASR personalization task.
Deep learning is at the core of recent spoken language understanding (SLU) related tasks. More precisely, deep neural networks (DNNs) drastically increased the performances of SLU systems, and numerous architectures have been proposed. In the real-life context of theme identification of telephone conversations, it is c…
A new framework improves ASR alignment accuracy via optimal transport.
problem Peaky behavior and alignment inaccuracies in E2E ASR models.
method Differentiable alignment framework based on 1D optimal transport.
result Significant improvement in ASR alignment performance compared to CTC.
Paper proposes an online speech recognition model using Transformer.
problem Challenges in deploying Transformer-based E2E ASR for online speech recognition.
method Chunk self-attention encoder (chunk-SAE) and monotonic truncated attention (MTA) based self-attention decoder (SAD).
result Achieved 23.66% CER with 320 ms latency, significant improvement over offline models.
End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wa…
Speech processing systems rely on robust feature extraction to handle phonetic and semantic variations found in natural language. While techniques exist for desensitizing features to common noise patterns produced by Speech-to-Text (STT) and Text-to-Speech (TTS) systems, the question remains how to best leverage state-…
End-to-end automatic speech recognition (ASR) commonly transcribes audio signals into sequences of characters while its performance is evaluated by measuring the word-error rate (WER). This suggests that predicting sequences of words directly may be helpful instead. However, training with word-level supervision can be …
New dataset for evaluating speech recognition fairness across demographics.
problem Lack of fairness metrics in speech recognition datasets.
method Developed Fair-Speech dataset with diverse demographic information.
result Helps evaluate ASR models for fairness across demographics.
Non-autoregressive transformer improves speech recognition speed and accuracy.
problem Reducing inference computation cost in speech recognition.
method Two non-autoregressive transformer structures (A-CMLM and A-FMLM) trained with masked tokens and iterative prediction during inference.
result Non-autoregressive transformer can match autoregressive transformer performance with 7x speedup.
Robust ASR model removes fast-changing features to resist attacks.
problem Vulnerability of ASR systems to adversarial attacks.
method Removing fast-changing features using slow feature analysis or low-pass filtering.
result Hybrid ASR models are more than four times more robust against targeted attacks.
Hybrid ASR systems can model graphemes effectively using chenones, outperforming traditional methods.
problem Traditional hybrid ASR systems struggle with English's poor grapheme-phoneme correspondence.
method Leveraging tied context-dependent graphemes (chenones) to model graphemes directly.
result Chenone-based systems significantly outperform senone baselines by 4.5% to 11.1% on English datasets.
Recognizing written domain numeric utterances (e.g. I need $1.25.) can be challenging for ASR systems, particularly when numeric sequences are not seen during training. This out-of-vocabulary (OOV) issue is addressed in conventional ASR systems by training part of the model on spoken domain utterances (e.g. I need one …
In this paper, we explore various approaches for semi supervised learning in an end to end automatic speech recognition (ASR) framework. The first step in our approach involves training a seed model on the limited amount of labelled data. Additional unlabelled speech data is employed through a data selection mechanism …
This paper improves ASR performance by aligning frames more accurately.
problem Disagreement between teacher-student models in frame-level alignment.
method Introduces self-knowledge distillation (SKD) to guide frame-level alignment.
result Improves both resource efficiency and performance in ASR.
In this work, we demonstrate the existence of universal adversarial audio perturbations that cause mis-transcription of audio signals by automatic speech recognition (ASR) systems. We propose an algorithm to find a single quasi-imperceptible perturbation, which when added to any arbitrary speech signal, will most likel…
ClovaCall introduces a new Korean call speech corpus for contact centers.
problem Lack of large-scale call-based speech corpora for Korean dialog scenarios.
method Development of a new large-scale Korean call-based speech corpus (ClovaCall) in a restaurant reservation domain.
result Validation of the dataset with ASR models shows its effectiveness.
In this paper we demonstrate end-to-end continuous speech recognition (CSR) using electroencephalography (EEG) signals with no speech signal as input. An attention model based automatic speech recognition (ASR) and connectionist temporal classification (CTC) based ASR systems were implemented for performing recognition…
CL methods improve monolingual ASR models across new tasks without forgetting past data.
problem Catastrophic Forgetting in monolingual ASR models when adapting to new domains or accents.
method Implement and compare various Continual Learning methods for monolingual ASR.
result Best CL method reduces performance gap by over 40% with minimal past data.
A model corrects Lithuanian grammatical errors.
problem Lack of language skills and typing errors in Lithuanian.
method Transformer architectures for subword and byte-level approaches.
result F0.5=0.92 for Lithuanian grammatical error correction.