End-to-end speech recognition using EEG without speech input.
problem Speech recognition without direct speech input.
method Implemented attention model and CTC-based ASR systems for EEG signals; fused EEG with noisy speech features.
result Demonstrated end-to-end speech recognition using EEG signals.
Improved speech recognition using EEG and video.
problem Enhancing continuous speech recognition systems.
method Implemented a CTC-based ASR model using EEG features.
result EEG features improve continuous visual speech recognition.
Improved speech recognition models with data augmentation and dropout.
problem Overfitting in end-to-end speech recognition models.
method Data augmentation and dropout applied to all layers of the network.
result Combination of data augmentation and dropout gives over 20% performance improvement.
End-to-end model detects articulatory features from speech data.
problem Detecting articulatory features from speech data for various applications.
method Apply Listen, Attend and Spell (LAS) architecture and attention models.
result End-to-end training of manners and places of articulation detectors.
End-to-end system improves multi-speaker speech recognition.
problem Efficiently recognizing speech from multiple speakers without additional training data.
method End-to-end sequence-to-sequence framework with unified source separation and recognition.
result 83.1% relative improvement in multi-speaker speech recognition.
End-to-end speech recognition system trained on GPUs and CPUs.
problem Building state-of-the-art speech recognition systems.
method Utilizes CPUs and GPUs for training, data augmentation, and neural network updates. Uses vocal tract length perturbation and acoustic simulator for data augmentation. Employed Horovod allreduce for training.
result Achieved 7.92% WER on proprietary English Bixby open domain test set using a Bidirectional Full Attention (BFA) model.
Jointly training with policy learning improves speech recognition performance.
problem Mismatch between CTC objective function and word error rate metric.
method Joint training with maximum likelihood and policy gradient.
result Joint training improves performance by 4% to 13%.
End-to-end ASR system uses context n-grams for better speech recognition.
problem Contextual information impacts speech recognition accuracy.
method Jointly optimizes ASR components with context embeddings during inference.
result Proposed CLAS system outperforms traditional methods by 68% relative WER.
Paper proposes an online speech recognition model using Transformer.
problem Challenges in deploying Transformer-based E2E ASR for online speech recognition.
method Chunk self-attention encoder (chunk-SAE) and monotonic truncated attention (MTA) based self-attention decoder (SAD).
result Achieved 23.66% CER with 320 ms latency, significant improvement over offline models.
Convolutional Neural Networks (CNNs) are effective models for reducing spectral variations and modeling spectral correlations in acoustic features for automatic speech recognition (ASR). Hybrid speech recognition systems incorporating CNNs with Hidden Markov Models/Gaussian Mixture Models (HMMs/GMMs) have achieved the …
This paper improves speech recognition by distilling knowledge from acoustic models.
problem Improving speech recognition accuracy using ensemble models.
method Proposes multi-teacher distillation strategies for joint CTC-attention end-to-end ASR systems, integrating error rate metric for optimization.
result Reports state-of-the-art error rates on various datasets and languages.
Improved speech recognition with end-to-end attention models.
problem End-to-end speech recognition with open-vocabulary.
method Sequence-to-sequence attention-based models on subword units, new pretraining scheme, CTC loss function, LSTM language models.
result State-of-the-art word error rates (3.54% and 3.82%) on LibriSpeech test-clean.
Paper explores EEG-based speech recognition using transformers, showing faster training and better performance for smaller vocabularies.
problem Continuous speech recognition using EEG features.
method Transformer-based ASR model compared to RNN-based models.
result Transformer models perform better for smaller vocabularies but RNN models outperform them for larger vocabularies.
Mobile training improves speech recognition for users with unique speech characteristics.
problem Limited generalization of speaker-independent speech recognition models for users with very different speech characteristics.
method Securely training personalized end-to-end speech recognition models on mobile devices, splitting gradient computation to reduce memory usage.
result On-device personalization achieved 58.1% relative word error rate reduction compared to 63.7% in a server environment, with 18.7% performance degradation.
CAT is a new ASR toolkit using CRF and CTC for state-of-the-art speech recognition.
problem Improving automatic speech recognition systems.
method CRF-based discriminative training with CTC-inspired state topology.
result CAT achieves state-of-the-art results with fewer parameters and is competitive with hybrid models.
Improved multilingual speech recognition with low latency for nine Indic languages.
problem Imbalance in training data across languages and low latency in interactive applications.
method Conditioning on a language vector and training language-specific adapter layers.
result Lower word error rate than monolingual E2E models and conventional systems.
Mobile app improves speech recognition of names with user feedback.
problem Improving speech recognition of proper names on mobile devices.
method Data synthesis and user feedback for keyword-dependent metrics.
result User feedback can significantly improve speech recognition of names.
Hybrid and end-to-end models compare in syllable recognition.
problem Comparing hybrid and end-to-end models for syllable recognition.
method Traditional hybrid system (kaldi) vs. end-to-end (TensorFlow) models.
result Hybrid models with explicit syllable knowledge outperform end-to-end models.
Quaternion CNNs improve speech recognition with fewer parameters.
problem Efficient end-to-end speech recognition with minimal parameters.
method Integrating quaternion algebra into CNNs for speech feature processing.
result Quaternion CNNs achieve lower phoneme error rates with fewer parameters.
Streaming ASR with transformer achieves low WER.
problem Real-time ASR with speech recognition.
method Time-restricted self-attention and triggered attention mechanisms.
result 2.8% and 7.2% WER for LibriSpeech test data.
End-to-end deep learning detects emotions in real-life emergency calls.
problem Recognizing emotions in real-life emergency call center recordings.
method Used an end-to-end deep learning architecture trained on IEMOCAP and CEMO datasets.
result Obtained 45.6% Unweighted Accuracy Recall on CEMO with 4 classes, 76.9% on 2 classes (Anger, Neutral).
The paper improves ASR accuracy using semi-supervised learning and dropout.
problem Improving ASR accuracy with limited labeled data.
method Training a seed model on limited labeled data, using dropout for uncertainty, and data selection for diversity.
result The approach significantly reduces ASR errors compared to baseline.
End-to-end voice conversion without vocoder.
problem Speech conversion without vocoder.
method Transformer network for raw spectrum conversion.
result Transformer model converts real voices efficiently.
SpecAugment improves speech recognition with simple feature augmentation.
problem Improving automatic speech recognition accuracy.
method Applying warping, frequency channel masking, and time step masking to feature inputs of neural networks.
result Achieved state-of-the-art performance on LibriSpeech and Switchboard tasks.
The paper explores modifications to filter banks for speech recognition.
problem Improving speech recognition accuracy using modified filter banks.
method The authors investigate replacing triangular filters with Gabor or Gammatone filters, and rearranging filter bank computations to integrate features over smaller time scales.
result No significant improvements in phone error rate were observed with the modifications.
WEEND uses a neural network to recognize speech and assign speakers to words.
problem End-to-end neural diarization without additional ASR and orchestration.
method Multi-task learning with an auxiliary network for ASR and speaker diarization.
result WEEND outperforms turn-based diarization and can handle 5-minute audio.
In training speech recognition systems, labeling audio clips can be expensive, and not all data is equally valuable. Active learning aims to label only the most informative samples to reduce cost. For speech recognition, confidence scores and other likelihood-based active learning methods have been shown to be effectiv…
End-to-end FCN framework optimizes speech enhancement metrics.
problem Inconsistency between model optimization and evaluation metrics.
method End-to-end utterance-based FCN for direct optimization of STOI.
result Enhanced speech has better STOI and improved intelligibility.
The paper analyzes how speech enhancement and recognition can be improved in noisy environments.
problem Improving speech recognition in multi-talker scenarios with limited resources.
method Developed and trained two LSTM-based models for speech enhancement and phone recognition, then studied their joint optimization.
result Joint optimization of speech enhancement and recognition leads to a significant reduction in Phone Error Rate (PER).
Hybrid ASR systems can model graphemes effectively using chenones, outperforming traditional methods.
problem Traditional hybrid ASR systems struggle with English's poor grapheme-phoneme correspondence.
method Leveraging tied context-dependent graphemes (chenones) to model graphemes directly.
result Chenone-based systems significantly outperform senone baselines by 4.5% to 11.1% on English datasets.
Enhanced transformer converts whispered speech to natural speech.
problem Machine recognition of whispered speech is challenging.
method Proposes an enhanced transformer architecture trained end-to-end using supervised learning.
result Similar formant distributions of converted speech to groundtruth.
CL methods improve monolingual ASR models across new tasks without forgetting past data.
problem Catastrophic Forgetting in monolingual ASR models when adapting to new domains or accents.
method Implement and compare various Continual Learning methods for monolingual ASR.
result Best CL method reduces performance gap by over 40% with minimal past data.
Iterative AutoML improves ASR model compression by 5x without WER degradation.
problem Challenges in achieving high compression levels without degrading ASR performance.
method Iterative AutoML-based Low Rank Factorization (LRF) approach.
result Achieved over 5x compression without WER degradation.
We present a recurrent encoder-decoder deep neural network architecture that directly translates speech in one language into text in another. The model does not explicitly transcribe the speech into text in the source language, nor does it require supervision from the ground truth source language transcription during t…
End-to-end ASR model combines word and character representation for improved performance.
problem Difficulty in training with word-level supervision due to sparsity of examples.
method Multi-task learning framework combining word and character representations.
result Improved word-error rate (WER) by interpolating between word-level and character-level models.
Optimal Completion Distillation improves speech recognition models efficiently.
problem Improving sequence to sequence models for speech recognition.
method Optimal suffix selection using dynamic programming and target distribution.
result Achieves state-of-the-art performance on speech datasets.
End-to-end speaker verification framework reduces text dependency.
problem Improving text-independent speaker verification.
method Jointly trains SE and ASR networks with triplet loss and adversarial gradient.
result Lower equal error rate and better text-independency compared to other approaches.
We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bi-directional recurrent neural network encoder coupled to a recurrent neural network decoder that directly emits a stream of phonemes. The alignment between the input and output sequences is established usin…
Neural network framework for language recognition considers sequence information and improves accuracy.
problem Challenging task of automatic language identification in noisy conditions.
method Proposes a neural network framework with bidirectional LSTM and attention modeling for relevance weighting.
result Significant improvements over conventional methods in noisy conditions and multi-speaker speech.
Improved neural network training for speech recognition using power-law nonlinearity and uniform distribution criterion.
problem Stability and uniformity of feature distribution in neural network training.
method Power-function based and histogram-based Maximum Uniformity of Distribution (MUD) algorithms.
result Power-function based MUD outperforms conventional MFCCs in speech recognition systems.
Paper proposes a method to maintain ASR performance on new tasks without forgetting old ones.
problem Mitigating forgetting in ASR models when learning new tasks.
method A novel explainability-based knowledge distillation combined with response-based knowledge distillation.
result Our method outperforms existing ones in mitigating forgetting on multi-stage sequential training tasks.
End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wa…
The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a single neural network trained in an end-to-end fashion. In this contribution, we analyse an attention-based seq2seq speech recognition syste…
ShrinkML uses RL to compress ASR models efficiently.
problem Large and complex ASR models reduce accuracy.
method Reinforcement Learning for compression of LSTM layers using SVD.
result RL-based compression achieves better accuracy than manual methods.
End-to-end ASR error detection using audio-transcript entailment.
problem Detecting transcription errors in ASR systems to prevent error propagation.
method Proposes a novel end-to-end approach using audio-transcript entailment, with acoustic and linguistic encoders.
result Achieves CER of 26.2% on all transcription errors and 23% on medical errors specifically, improving by 12% and 15.4% respectively over a strong baseline.
CAT toolkit combines hybrid and E2E approaches for efficient speech recognition.
problem Improving speech recognition efficiency and latency.
method CTC-CRF based framework with contextualized soft forgetting.
result CAT achieves state-of-the-art results with simpler training and streaming ASR.
Paper shows EEG can improve ASR in noisy speech.
problem ASR performance drops in noisy conditions.
method Used EEG to train ASR models and improve performance.
result ASR accuracy improved with EEG features and distillation.
Paper shows continuous speech recognition with EEG features, no speech input.
problem Continuous speech recognition with limited vocabulary and noisy/no speech input.
method Connectionist temporal classification (CTC) model, EEG features, new deep learning architecture.
result Continuous speech recognition achieved on limited vocabulary with noisy/no speech input.