Improved CSKS with limited data using novel loss functions and transfer learning.
problem Spotting keywords in continuous speech with limited training data.
method Combination of Prototypical networks' loss and metric loss with transfer learning.
result Improves F1 score by over 10%
Synthetic speech data improves keyword spotting models with fewer real examples.
problem Training models for recognizing spoken keywords with limited real data.
method Used a pre-trained speech embedding model to extract features for training a small keyword spotting model.
result A model trained on synthetic speech data can detect 10 keywords with the same accuracy as a model trained on over 500 real examples.
User-specific KWS system learns new keywords on-device.
problem Out-of-vocabulary problem in traditional KWS systems.
method Query-by-example enrollment and testing, using phonetic posteriors and FST.
result Promising performance on two keywords, preserving simplicity.
A new multi-layer attention mechanism improves speech keyword recognition accuracy.
problem Inaccurate attention weights in LSTM networks for speech keyword recognition.
method Introducing information from layers prior to feature extraction into attention weights calculations.
result The proposed multi-layer attention mechanism leads to more accurate attention weights and improved keyword spotting performance.
SpeechYOLO detects and locates speech objects in audio signals.
problem Detecting and localizing speech in audio signals.
method Inspired by YOLO, SpeechYOLO uses a convolutional neural network with a least-mean-squares loss function.
result SpeechYOLO performs well on keyword spotting tasks, including read and spontaneous speech.
We consider multilingual bottleneck features (BNFs) for nearly zero-resource keyword spotting. This forms part of a United Nations effort using keyword spotting to support humanitarian relief programmes in parts of Africa where languages are severely under-resourced. We use 1920 isolated keywords (40 types, 34 minutes)…
Paper optimizes KWS models using NAS and quantization for limited resources.
problem Developing efficient keyword spotting models in resource-constrained environments.
method Neural Architecture Search (NAS) for model structure optimization and quantization of weights and activations.
result Achieved high accuracy (95.55%) with minimal parameters and operations using NAS and quantization.
Accurate on-device keyword spotting (KWS) with low false accept and false reject rate is crucial to customer experience for far-field voice control of conversational agents. It is particularly challenging to maintain low false reject rate in real world conditions where there is (a) ambient noise from external sources s…
Enhances KWS in vehicles with multi-source fusion.
problem Improving precision and recall rates in vehicle keyword spotting.
method Integrates vehicle information into a DNN for speech classification and selects optimal sensitivity parameters.
result Significantly improved performance metrics (precision, recall, MSE) compared to baseline.
We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mi…
The problem of keyword spotting i.e. identifying keywords in a real-time audio stream is mainly solved by applying a neural network over successive sliding windows. Due to the difficulty of the task, baseline models are usually large, resulting in a high computational cost and energy consumption level. We propose a new…
Channel pruning and weight binarization improve keyword spotting accuracy.
problem Improving accuracy of keyword spotting in neural networks.
method Group-wise splitting method using group Lasso penalty for channel sparsity, combined with 1-bit weight precision.
result Achieved over 50% channel sparsity with minimal accuracy loss.
Paper introduces regularization for multi-head attention to spot keywords.
problem Redundancy in multi-head attention leads to lack of rich information.
method Regularization technique to enforce orthogonality between attention heads.
result Significant improvement in keyword spotting performance.
Large size models are implemented in recently ASR system to deal with complex speech recognition problems. The num- ber of parameters in these models makes them hard to deploy, especially on some resource-short devices such as car tablet. Besides this, at most of time, ASR system is used to deal with real-time problem …
Self-training with noisy student-teacher boosts keyword spotting accuracy.
problem Robust keyword spotting in challenging conditions.
method Aggressive data augmentation and self-training with noisy student-teacher approach.
result Significant accuracy improvement in difficult conditions, up to 60%.
Keyword spotting--or wakeword detection--is an essential feature for hands-free operation of modern voice-controlled devices. With such devices becoming ubiquitous, users might want to choose a personalized custom wakeword. In this work, we present DONUT, a CTC-based algorithm for online query-by-example keyword spotti…
Loihi neuromorphic chip outperforms conventional hardware in keyword spotting efficiency.
problem Benchmarking keyword spotting efficiency on neuromorphic hardware.
method Comparative analysis of a two-layer neural network trained to recognize a single phrase on Intel's Loihi neuromorphic chip and conventional hardware devices.
result Loihi outperforms conventional hardware on energy cost per inference for this keyword spotting application.
Neural model detects phoneme boundaries from speech, outperforming baselines.
problem Phoneme boundary detection for speech processing applications.
method Learnable segmental features with a structured loss function.
result Model achieves state-of-the-art performance on TIMIT and Buckeye corpora.
We present our first efforts in building an automatic speech recognition system for Somali, an under-resourced language, using 1.57 hrs of annotated speech for acoustic model training. The system is part of an ongoing effort by the United Nations (UN) to implement keyword spotting systems supporting humanitarian relief…
The problem of identifying voice commands has always been a challenge due to the presence of noise and variability in speed, pitch, etc. We will compare the efficacies of several neural network architectures for the speech recognition problem. In particular, we will build a model to determine whether a one second audio…
New KWS neural networks improve accuracy and power efficiency.
problem Maximizing accuracy and power efficiency in KWS systems.
method Hardware Aware Training (HAT) for LMU-based neural networks.
result Achieved state-of-the-art accuracy and low parameter counts.
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
Detects adversarial examples in deep speech recognition.
problem Vulnerability of deep speech recognition systems to adversarial attacks.
method Formulated as a classification problem, generated adversarial and normal datasets, trained CNN.
result Accurately distinguishes between adversarial and normal examples for known attacks.
New eGRU unit improves keyword spotting on ultra-low-power devices.
problem Resource constraints of edge devices for neural network deployment.
method Optimized recurrent unit architecture for ultra-low power.
result eGRU is 60x faster and 10x smaller than GRU, maintaining accuracy.
We compare features for dynamic time warping (DTW) when used to bootstrap keyword spotting (KWS) in an almost zero-resource setting. Such quickly-deployable systems aim to support United Nations (UN) humanitarian relief efforts in parts of Africa with severely under-resourced languages. Our objective is to identify aco…
We explore the application of end-to-end stateless temporal modeling to small-footprint keyword spotting as opposed to recurrent networks that model long-term temporal dependencies using internal states. We propose a model inspired by the recent success of dilated convolutions in sequence modeling applications, allowin…
We consider the task of building compact deep learning pipelines suitable for deployment on storage and power constrained mobile devices. We propose a unified framework to learn a broad family of structured parameter matrices that are characterized by the notion of low displacement rank. Our structured transforms admit…
Mobile app improves speech recognition of names with user feedback.
problem Improving speech recognition of proper names on mobile devices.
method Data synthesis and user feedback for keyword-dependent metrics.
result User feedback can significantly improve speech recognition of names.
Research forecasts electricity spot prices using stochastic volatility models.
problem Forecasting day-ahead electricity prices in a spot market.
method Exploring and enriching a baseline stochastic volatility model with exogenous regressors.
result A better fitting model confirmed by out-of-sample forecasts.
Personal VAD detects target speaker voice activity efficiently.
problem Efficiently detect target speaker voice activity for reduced computational cost and battery usage.
method Trains a neural network conditioned on speaker embedding or verification score, outputs probabilities for three speech classes.
result Trained model with 130K parameters outperforms combined standard VAD and speaker recognition networks.
Paper shows continuous speech recognition with EEG features, no speech input.
problem Continuous speech recognition with limited vocabulary and noisy/no speech input.
method Connectionist temporal classification (CTC) model, EEG features, new deep learning architecture.
result Continuous speech recognition achieved on limited vocabulary with noisy/no speech input.
Continuous speech recognition from brain activity without vocalization.
problem Recognizing silent speech from EEG signals.
method Implemented a CTC ASR model using EEG signals.
result Demonstrated feasibility of EEG for continuous silent speech recognition.
Improved speech recognition using EEG and video.
problem Enhancing continuous speech recognition systems.
method Implemented a CTC-based ASR model using EEG features.
result EEG features improve continuous visual speech recognition.
AV-CPL uses continuous pseudo-labels for AVSR combining labeled and unlabeled data.
problem Improving AVSR performance with labeled and unlabeled data.
method Semi-supervised method using continuous pseudo-labels generated by the same AVSR model.
result Significant improvements in VSR performance on LRS3 dataset.
Paper explores EEG-based speech recognition using transformers, showing faster training and better performance for smaller vocabularies.
problem Continuous speech recognition using EEG features.
method Transformer-based ASR model compared to RNN-based models.
result Transformer models perform better for smaller vocabularies but RNN models outperform them for larger vocabularies.
End-to-end speech recognition using EEG without speech input.
problem Speech recognition without direct speech input.
method Implemented attention model and CTC-based ASR systems for EEG signals; fused EEG with noisy speech features.
result Demonstrated end-to-end speech recognition using EEG signals.
Paper improves EEG-based speech recognition using CTC and beam search.
problem Improving continuous speech recognition from EEG signals.
method Implemented CTC-based ASR system, initialized weights, used external language model, studied articulatory feature prediction.
result Enhanced performance of EEG-based speech recognition systems.
Study Fourier estimator for spot volatility with unbounded coefficients and jumps.
problem Estimating spot volatility with unbounded coefficients and jumps in price process.
method Fourier estimator for spot volatility, convergence analysis for unbounded coefficients and jumps.
result Convergence of trigonometric polynomial to volatility's path, almost sure convergence of reconstructed volatility.
Improved speech recognition for voice assistants by analyzing speech data.
problem Reducing false triggers in speech-enabled assistants.
method Post-processing LVCSR hypothesis lattice with a Bidirectional Lattice Recurrent Neural Network (LatticeRNN).
result LatticeRNN significantly improves detection accuracy over traditional methods.
The paper proposes using CWT and STFT for training neural speech models.
problem Training high-quality neural speech models.
method Proposes spectral amplitude and phase losses from STFT and CWT for training.
result Shows that CWT spectral loss can train a high-quality model as good as STFT-based loss.
Derives pricing formulas for perpetual futures contracts.
problem Ensuring fair pricing of perpetual futures contracts without expiration.
method Explicit expressions derived for various types of perpetual contracts, including linear, inverse, and quantos futures.
result Futures price is the risk-neutral expectation of the spot price sampled at a random time reflecting funding payments.
Human infants can discover words directly from unsegmented speech signals without any explicitly labeled data. In this paper, we develop a novel machine learning method called nonparametric Bayesian double articulation analyzer (NPB-DAA) that can directly acquire language and acoustic models from observed continuous sp…
A dynamic keyword selection model for topic modeling of tweets.
problem Adjusting keywords dynamically to mimic past topics with novelty.
method Generative process selects keywords and documents, trained with variational lower bound and stochastic gradient optimization.
result Keyword-based topic model outperforms a sophisticated baseline model by 67%.
This paper proposes a new loss using short-time Fourier transform (STFT) spectra for the aim of training a high-performance neural speech waveform model that predicts raw continuous speech waveform samples directly. Not only amplitude spectra but also phase spectra obtained from generated speech waveforms are used to c…
CL methods improve monolingual ASR models across new tasks without forgetting past data.
problem Catastrophic Forgetting in monolingual ASR models when adapting to new domains or accents.
method Implement and compare various Continual Learning methods for monolingual ASR.
result Best CL method reduces performance gap by over 40% with minimal past data.
This paper examines the speaker identification potential of breath sounds in continuous speech. Speech is largely produced during exhalation. In order to replenish air in the lungs, speakers must periodically inhale. When inhalation occurs in the midst of continuous speech, it is generally through the mouth. Intra-spee…
Automatic continuous speech recognition (CSR) is sufficiently mature that a variety of real world applications are now possible including large vocabulary transcription and interactive spoken dialogues. This paper reviews the evolution of the statistical modelling techniques which underlie current-day systems, specific…
Paper defends LSTM-based text classification models from backdoor attacks.
problem Backdoor attacks in LSTM models cause misclassification of spam or malicious speech.
method Backdoor Keyword Identification (BKI) to identify and exclude poisoned samples.
result BKI method effectively mitigates backdoor attacks in various text classification datasets.