Deep neural network translates speech directly into text without transcription.
problem Directly translating speech between languages without manual transcription.
method Recurrent encoder-decoder architecture with attention mechanism.
result End-to-end model achieves state-of-the-art performance on speech translation task.
AudioPaLM combines text and speech models to improve speech processing and translation.
problem Improving speech understanding and generation models.
method Unified multimodal architecture combining text and speech models.
result AudioPaLM significantly outperforms existing speech translation systems.
Paper surveys and introduces Acoustic Dialect Decoder for voice translation.
problem Machine understanding of natural language in speech translation.
method Recognition, Translation, and Synthesis units using HMMs, RNNs, and HTS.
result Initial successful translation of English to Tamil.
Spoken language translation (SLT) has become very important in an increasingly globalized world. Machine translation (MT) for automatic speech recognition (ASR) systems is a major challenge of great interest. This research investigates that automatic sentence segmentation of speech that is important for enriching speec…
This research investigates the Statistical Machine Translation approaches to translate speech in real time automatically. Such systems can be used in a pipeline with speech recognition and synthesis software in order to produce a real-time voice communication system between foreigners. We obtained three main data sets …
Improves ASR accuracy for domain mismatch using machine translation.
problem Domain mismatch in ASR systems leads to suboptimal results.
method Machine translation to map out-of-domain ASR errors to in-domain terms.
result 7% absolute improvement in word error rate, 4 point BLEU score improvement.
Improved speech translation model using cleaned data and ensemble decoding.
problem End-to-end speech translation from English to German.
method Fine-tuning on cleaned data, weight normalization, label smoothing, checkpoint averaging, ensemble decoding.
result Ensemble model achieved a BLEU score of 10.24 on test data.
Continuous speech recognition from brain activity without vocalization.
problem Recognizing silent speech from EEG signals.
method Implemented a CTC ASR model using EEG signals.
result Demonstrated feasibility of EEG for continuous silent speech recognition.
Models improve syntactic clustering by adding more translation and part-of-speech decoders.
problem Improving syntactic saliency in hidden sentence representations.
method Training multi-task autoencoders on linguistic tasks and analyzing the learned hidden representations.
result The representation space becomes less entangled with more decoders, leading to better syntactic clustering.
AeGAN improves speech clarity in noisy environments.
problem Improving speech recognition in crowded noisy environments.
method Generative adversarial networks (GAN) with a novel architecture.
result The proposed framework outperforms traditional and learning-based methods.
Dual supervised learning improves model performance for dual tasks.
problem Separate training of dual tasks misses probabilistic connections.
method Simultaneous training of dual tasks exploiting probabilistic correlations.
result Dual supervised learning improves practical performance across various applications.
This paper tackles multilingual speech processing by optimizing conflicting objectives hierarchically.
problem Training models for multilingual, multi-task speech processing is hampered by conflicting objectives.
method Investigates three multi-objective MSP formulations and introduces a lightweight layer-selection mechanism.
result A bi-level recipe outperforms standard flat optimization in state-of-the-art MSP models.
Method converts speech with attention and context preservation.
problem Voice conversion with improved stability and efficiency.
method Sequence-to-Sequence learning with attention and context preservation.
result Synthesized speech quality comparable to advanced methods.
Method converts emotions in nonparallel speech data.
problem Lack of parallel data for speech emotion conversion.
method Unsupervised style transfer technique for nonparallel training.
result Effectiveness demonstrated on nonparallel corpora with four emotions.
Many machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few. One of the key challenges in sequence transduction is learning to …
Study uses PPG signals for detecting speech events and speaker characteristics.
problem Detecting speech events and speaker characteristics from PPG signals.
method End-to-end convolutional neural network architectures for gender and person verification.
result Promising results showing potential of PPG for speech processing tasks.
This research explores the effects of various training settings from Polish to English Statistical Machine Translation system for spoken language. Various elements of the TED parallel text corpora for the IWSLT 2013 evaluation campaign were used as the basis for training of language models, and for development, tuning …
Paper explores deep learning features for complex emotion recognition.
problem Improving emotion recognition accuracy in complex emotions.
method Used pretrained networks (AudioSet Net, VoxCeleb Net, Deep Speech Net) and their deep layer features for emotion recognition.
result Achieved highest F1 score of 0.85 on EmoReact dataset.
Novel fusion of autoencoders predicts sleepiness from speech.
problem Predicting sleepiness from speech recordings.
method Attention-based and recurrent sequence to sequence autoencoders for unsupervised representation learning.
result Fusion of autoencoders' representations achieves higher correlation with sleepiness scales.
Attention forcing improves sequence-to-sequence model training stability.
problem Training auto-regressive sequence-to-sequence models with attention mechanism is challenging.
method Attention forcing guides the model with generated output history and reference attention.
result Attention forcing trains models to recover from mistakes without requiring a schedule or classifier.
Improved speech recognition with cumulative adaptation methods.
problem Robust speech recognition in varying environments and speakers.
method Used a bidirectional LSTM neural network and i-vectors for adaptation.
result Achieved 13% relative improvement in word error rate.
Compact neural networks for speech recognition with dropout training.
problem Training large neural networks for speech recognition tasks.
method Introducing a sparsity-inducing prior on dropout retention probability to prune hidden units during training.
result Achieved comparable accuracy with fewer than 50% of the hidden units, resulting in a 2.5x speedup.
Bardo Composer generates tabletop RPG music based on player speech.
problem Creating immersive background music for tabletop RPGs.
method Speech recognition, emotion classification, and music generation using a novel beam search algorithm.
result Generated music pieces can be accurately identified by human subjects as conveying the intended emotion.
Recurrent sequence generators conditioned on input data through an attention mechanism have recently shown very good performance on a range of tasks in- cluding machine translation, handwriting synthesis and image caption gen- eration. We extend the attention-mechanism with features needed for speech recognition. We sh…
Deep learning models generalize well despite their complexity.
problem Understanding the source of deep learning's generalization ability.
method Overview of existing theory and bounds for generalization error.
result Combining classical and recent results, deep neural networks generalize well.
Single model performs well across diverse tasks.
problem Improving deep learning across multiple domains.
method Concurrent training on diverse tasks, incorporating multiple domain-specific components.
result Single model yields good results on various domains.
Recent advances in conditional recurrent language modelling have mainly focused on network architectures (e.g., attention mechanism), learning algorithms (e.g., scheduled sampling and sequence-level training) and novel applications (e.g., image/video description generation, speech recognition, etc.) On the other hand, …
Syntax-enhanced models boost machine translation and NLP performance.
problem Limited training data and complex models struggle in NLP tasks.
method Syntax information was explicitly fed into Transformer and BERT models.
result Syntax-infused models achieved significant BLEU improvements.
Deep learning methods optimize complex models with large datasets.
problem Optimizing large and complex models with deep learning.
method Various gradient optimization techniques for practical machine learning.
result Effective optimization of wide, deep models with many parameters.
End-to-end speech recognition using EEG without speech input.
problem Speech recognition without direct speech input.
method Implemented attention model and CTC-based ASR systems for EEG signals; fused EEG with noisy speech features.
result Demonstrated end-to-end speech recognition using EEG signals.
Survey of automated neural architecture search methods.
problem Manual development of neural architectures is time-consuming and error-prone.
method Categorizes existing automated neural architecture search methods.
result Growing interest in automated methods due to manual development's limitations.
Paper introduces sampling-based speech synthesis with natural variation.
problem Synthetic speech lacks natural inter-utterance variation.
method Moment-matching networks trained to match moments of generated speech parameters to natural speech parameters.
result Sampling-based generation does not degrade synthetic speech quality.
Paper shows EEG can improve ASR in noisy speech.
problem ASR performance drops in noisy conditions.
method Used EEG to train ASR models and improve performance.
result ASR accuracy improved with EEG features and distillation.
New method prevents entropy collapse in Transformer training, leading to more stable and robust models.
problem Training instability in Transformers, especially in attention layers.
method Spectral normalization with a learned scalar to prevent entropy collapse.
result Prevents entropy collapse, leading to more stable training.
Paper shows continuous speech recognition with EEG features, no speech input.
problem Continuous speech recognition with limited vocabulary and noisy/no speech input.
method Connectionist temporal classification (CTC) model, EEG features, new deep learning architecture.
result Continuous speech recognition achieved on limited vocabulary with noisy/no speech input.
RUSLAN is a large Russian speech corpus for text-to-speech.
problem Lack of high-quality annotated Russian speech data for text-to-speech.
method Developed a large annotated Russian speech corpus and trained a neural network for text-to-speech synthesis.
result Synthesized speech quality evaluated with MOS scores: 4.05 for naturalness, 3.78 for intelligibility.
Unified model improves speech enhancement in unseen environments.
problem Robustness against unknown environments in speech enhancement.
method Probabilistic integration of VAE and NMF.
result Outperforms conventional DNN-based method in unseen environments.
NovoGrad improves deep learning training with adaptive moments and layer-wise normalization.
problem Training deep neural networks efficiently and effectively.
method Layer-wise adaptive moments with gradient normalization and decoupled weight decay.
result NovoGrad outperforms well-tuned SGD with momentum and Adam/AdamW in various tasks.
Detects AI-synthesized speech using cepstral and bispectral analysis.
problem Validating the authenticity of speech from AI-generated content.
method Integrates cepstral and bispectral analysis for distinguishing human from AI-synthesized speech.
result Higher-order statistics show less correlation for human speech compared to AI-synthesis, and cepstral analysis reveals unique power components.
Solution for voice conversion with limited data using hierarchical seq2seq and attention models.
problem Voice conversion between speakers with limited parallel audio pairs.
method Hierarchical sequence to sequence model with attention-based decoder, trained on single speaker dataset.
result Improved voice conversion quality using mel spectrograms and wavenet vocoder.
Combines LSTM with HMM to improve interpretability of RNNs.
problem Making deep neural networks more understandable and interpretable.
method Integrates LSTM and HMM, training them sequentially or jointly.
result A hybrid model outperforms standalone LSTM, especially on smaller datasets.
Improved visual speech synthesis using adapted ASR acoustic models.
problem Lack of synchronized audio, video, and depth data for speaker-independent speech-driven visual speech synthesis.
method Adapted an ASR acoustic model trained on audio-only data to the visual speech synthesis domain.
result Viewers significantly prefer animations generated from the adapted ASR acoustic model.
WaveCycleGAN2 improves speech synthesis quality by reducing aliasing.
problem Human ear can still distinguish synthesized speech from natural speech.
method WaveCycleGAN2 uses generators without down/up-sampling modules and combines discriminators from waveform and acoustic parameter domains.
result WaveCycleGAN2 achieves high-quality speech synthesis with comparable mean opinion scores to natural speech.
Enhances spoken speech quality using EEG signals.
problem Improves speech clarity in noisy environments.
method Generative adversarial network (GAN), gated recurrent unit (GRU), temporal convolutional network (TCN) regression models.
result Significant improvement in speech enhancement quality compared to traditional methods.
WaveCycleGAN converts synthetic speech to natural speech using cycle-consistent adversarial networks.
problem Over-smoothing effect in synthetic speech, leading to quality degradation.
method Cycle-consistent adversarial networks for waveform-level modification.
result Improves naturalness of generated speech sounds.
Improved speech enhancement using diffusion models with MSE loss.
problem Efficient incorporation of noisy speech in generative speech enhancement.
method Augmented diffusion-based generative model with a MSE loss for enhanced speech.
result Proposed method improves speech enhancement performance compared to original diffusion model.
End-to-end FCN framework optimizes speech enhancement metrics.
problem Inconsistency between model optimization and evaluation metrics.
method End-to-end utterance-based FCN for direct optimization of STOI.
result Enhanced speech has better STOI and improved intelligibility.
This work disentangles speech and non-speech components from found data.
problem Building robust acoustic models from found data with non-standard variations.
method Latent Stochastic Models and Multinode Latent Space Variational Autoencoders (VAE).
result Speech and music can be separated in the latent space of a VAE, independent of the language.