New task AQA tackles acoustic reasoning from sound scenes.
problem Promote research in acoustic reasoning.
method Generate acoustic scenes from elementary sounds and formulate questions.
result Preliminary results with models FiLM and MAC show promise.
We introduce the task of acoustic question answering (AQA) in the area of acoustic reasoning. In this task an agent learns to answer questions on the basis of acoustic context. In order to promote research in this area, we propose a data generation paradigm adapted from CLEVR (Johnson et al. 2017). We generate acoustic…
Paper predicts EEG features from acoustic features using RNN and GAN.
problem Predicting EEG features from acoustic features.
method Recurrent Neural Network (RNN) and Generative Adversarial Network (GAN).
result Lower RMSE and normalized RMSE values compared to generating acoustic features from EEG features.
Paper aims to find joint representation between vocal tract geometry and speech sound acoustics.
problem Finding a joint latent representation between articulatory and acoustic domains for vowel sounds.
method Invertible neural network models, convolutional autoencoder, normalizing flows, semi-supervised learning.
result Satisfactory performance in articulatory-to-acoustic and acoustic-to-articulatory mapping.
Study shows integrating acoustic features in financial forecasting models can degrade performance.
problem Predicting stock market volatility from corporate earnings calls using speech features.
method Empirical investigation of acoustic feature extraction in teleconference environments using a two-stream late-fusion architecture.
result Integrating acoustic features via late fusion significantly degraded performance, reducing recall to 47.08%.
Bayesian SHMM discovers acoustic units from unlabeled speech.
problem Discovering language-specific acoustic units from unlabeled speech.
method Bayesian Subspace Hidden Markov Model (SHMM) trained on labeled data to find new acoustic units on target language.
result Significantly outperforms previous HMM-based systems and compares favorably with Variational Auto Encoder-HMM.
Improved deep CNNs for ASC by optimizing receptive field size.
problem Deep CNNs perform poorly in ASC compared to simpler models.
method Analyzed and adapted the receptive field of ResNet and DenseNet.
result State-of-the-art performance achieved with optimized receptive field.
Recent advances in speech synthesis suggest that limitations such as the lossy nature of the amplitude spectrum with minimum phase approximation and the over-smoothing effect in acoustic modeling can be overcome by using advanced machine learning approaches. In this paper, we build a framework in which we can fairly co…
We present a supervised neural network model for polyphonic piano music transcription. The architecture of the proposed model is analogous to speech recognition systems and comprises an acoustic model and a music language model. The acoustic model is a neural network used for estimating the probabilities of pitches in …
ConvNet classifies whale vocalizations and ambient noise in acoustic recordings.
problem Automated detection and classification of marine mammal vocalizations in acoustic recordings.
method Convolutional Neural Network with a novel acoustic representation.
result Classifier accurately detects and classifies whale vocalizations and ambient noise.
Improved acoustic word embeddings using shared decoder in multi-view encoders.
problem Learning discriminative acoustic word embeddings from text labels.
method Combining Siamese multi-view encoders with a shared decoder network to maximize the relationship between acoustic and text embeddings.
result 11.1% relative improvement in average precision on acoustic word discrimination task with WSJ dataset.
Self-supervised method detects replay spoofing using acoustic configurations.
problem Challenges in collecting large-scale datasets for replay spoofing detection.
method Self-supervised pretraining of acoustic configurations using existing datasets.
result The method outperforms baseline by 30% on ASVspoof 2019 physical access dataset.
Recent neural networks such as WaveNet and sampleRNN that learn directly from speech waveform samples have achieved very high-quality synthetic speech in terms of both naturalness and speaker similarity even in multi-speaker text-to-speech synthesis systems. Such neural networks are being used as an alternative to voco…
An audiovisual speaker conversion method is presented for simultaneously transforming the facial expressions and voice of a source speaker into those of a target speaker. Transforming the facial and acoustic features together makes it possible for the converted voice and facial expressions to be highly correlated and f…
This paper introduces a model of environmental acoustic scenes which adopts a morphological approach by ab-stracting temporal structures of acoustic scenes. To demonstrate its potential, this model is employed to evaluate the performance of a large set of acoustic events detection systems. This model allows us to expli…
Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.
problem Mapping speech and text to fixed dimensions for phonetic confusability.
method Adapting SNE to sequential inputs, training two encoder neural networks.
result More accurate results with low-dimensional embeddings in word recognition tasks.
This work learns shared word embeddings for acoustic and phonetic sequences.
problem Mapping variable-length acoustic and phonetic sequences to fixed-dimensional vectors.
method Weak supervision and binary classification task to predict word similarity.
result Best model achieves an F1 score of 0.95 for binary classification.
We present a novel deep Recurrent Neural Network (RNN) model for acoustic modelling in Automatic Speech Recognition (ASR). We term our contribution as a TC-DNN-BLSTM-DNN model, the model combines a Deep Neural Network (DNN) with Time Convolution (TC), followed by a Bidirectional Long Short-Term Memory (BLSTM), and a fi…
Improved visual speech synthesis using adapted ASR acoustic models.
problem Lack of synchronized audio, video, and depth data for speaker-independent speech-driven visual speech synthesis.
method Adapted an ASR acoustic model trained on audio-only data to the visual speech synthesis domain.
result Viewers significantly prefer animations generated from the adapted ASR acoustic model.
This paper improves speech recognition by using raw waveform signals in multi-span CNN acoustic models.
problem Improving speech recognition accuracy using raw waveform signals.
method Proposes a novel multi-span structure for acoustic modelling based on raw waveform signals with multiple CNN input layers.
result Multi-span acoustic models yield a lower word error rate (WER) than traditional FBANK feature-based models.
Improved hybrid acoustic model using interleaved self-attention and convolution.
problem Limited application of transformer in hybrid acoustic models.
method Proposed a model structure with interleaved self-attention and 1D convolution.
result Competitive recognition results on Librispeech dataset.
Environmental acoustic sensing involves the retrieval and processing of audio signals to better understand our surroundings. While large-scale acoustic data make manual analysis infeasible, they provide a suitable playground for machine learning approaches. Most existing machine learning techniques developed for enviro…
Geometric model explains music perception combining neuroscience and acoustics.
problem Rationalize and predict psycho-acoustic phenomena in music perception.
method Combining neuroscientific theories with acoustic observations, a geometric model of the space of all chords is created.
result The geometric model allows for rigorous studies of psychoacoustic quantities like roughness and harmonicity.
We have recently shown that deep Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as acoustic models for speech recognition. More recently, we have shown that the performance of sequence trained context dependent (CD) hidden Markov model (HMM) acoustic m…
We study large-scale kernel methods for acoustic modeling and compare to DNNs on performance metrics related to both acoustic modeling and recognition. Measuring perplexity and frame-level classification accuracy, kernel-based acoustic models are as effective as their DNN counterparts. However, on token-error-rates DNN…
Improved acoustic modeling with attentive adversarial learning.
problem Domain variability in acoustic modeling.
method Proposes an attentive ADIT method with an attention mechanism to improve domain-invariance of deep features.
result Improves deep feature domain-invariance and senone-discriminativity over ADIT.
U-Net trained to recover acoustic interference striations from distorted data.
problem Recovering acoustic interference striations from distorted signals.
method Training a U-Net using a random mode-coupling matrix model to generate training data.
result U-Net successfully recovers AISs under various conditions.
Meta-learning improves few-shot acoustic event detection.
problem Detecting new audio events with limited labeled data.
method Formulated few-shot AED problem; explored supervised and meta-learning approaches.
result Meta-learning achieves superior performance in few-shot AED.
In this paper, we propose a classification based glottal closure instants (GCI) detection from pathological acoustic speech signal, which finds many applications in vocal disorder analysis. Till date, GCI for pathological disorder is extracted from laryngeal (glottal source) signal recorded from Electroglottograph, a d…
Improved ASR for English-isiZulu code-switched speech with semi-supervised training.
problem Improving ASR for code-switched speech between English and isiZulu.
method Semi-supervised training using automatic transcription of multilingual speech data.
result Semi-supervised training achieved significant WER reduction in ASR performance.
Improved multi-modal emotion recognition using deep learning.
problem Combining acoustic and text modalities for emotion recognition.
method Proposes a deep learning-based approach to fuse text and acoustic data using SincNet for acoustic features and parallel DCNN and Bi-RNN branches for text processing with cross attention.
result Achieves 3.5% improvement in weighted accuracy compared to existing methods.
Biodiversity monitoring using audio recordings is achievable at a truly global scale via large-scale deployment of inexpensive, unattended recording stations or by large-scale crowdsourcing using recording and species recognition on mobile devices. The ability, however, to reliably identify vocalising animal species is…
Deep learning outperforms classic machine learning in DAS event detection.
problem Event detection in Distributed Acoustic Sensing (DAS).
method Comparison of classic machine learning and image-based deep learning approaches.
result Image-based deep learning offers significantly faster event detection and execution times.
Jointly reduces echo, reverberation, and noise using neural networks.
problem Simultaneous reduction of acoustic echo, reverberation, and noise.
method Multichannel Gaussian modeling and neural network for joint optimization of filters.
result Outperforms individual and joint non-spectral models.
Paper tackles invariance of demodulation in shallow water acoustic communications.
problem Frequency-selective signal distortion (Doppler effect) in shallow water environments.
method Developed ML-based demodulation methods using DBN-NN and DBN-CNN.
result Demonstrated invariance of the proposed method to Doppler effect with 2dB error margin.
Report on acoustic model building from 1M hours of unlabeled speech.
problem Building acoustic models from large amounts of unlabeled speech data.
method Student/teacher training, scheduled learning, BMUF, gradient threshold compression SGD.
result Extremely large amounts of unlabeled data lead to significant relative WER improvements.
A new neural network learns from acoustic scenes by suppressing irrelevant patterns.
problem Acoustic scenes are rich and redundant, making classification challenging.
method Spatio-temporal attention pooling layer coupled with a convolutional recurrent neural network.
result The method outperforms a strong convolutional neural network baseline and sets new state-of-the-art performance.
Acoustic scene classification is the task of identifying the scene from which the audio signal is recorded. Convolutional neural network (CNN) models are widely adopted with proven successes in acoustic scene classification. However, there is little insight on how an audio scene is perceived in CNN, as what have been d…
Improved language identification accuracy through signal combination methods.
problem Enhancing speech recognition accuracy across multiple languages.
method Combining low-level acoustic signals with language-specific recognizer signals using lattice-based ensemble models and deep neural networks.
result Deep neural network model outperforms lattice-based ensemble model, reducing error rate from 5.5% to 4.3%.
Improves speech recognition in noisy environments using robust acoustic models.
problem Adverse environments with significant mismatch between training and test conditions.
method Theoretical analysis of data augmentation as vicinal risk minimization, using mixture of Gaussians to incorporate robust inductive bias.
result Waveform-based approach shows 150% relative improvement in out-of-distribution generalization.
CNNs improve generalization to unseen audio devices with increased width, not depth.
problem CNNs are sensitive to specific audio recording devices in acoustic scene classification.
method Investigated the relationship between over-parameterization and generalization in CNNs for audio classification.
result Increasing width improves generalization to unseen devices without increasing the number of parameters.
In this work, three lattice-free (LF) discriminative training criteria for purely sequence-trained neural network acoustic models are compared on LVCSR tasks, namely maximum mutual information (MMI), boosted maximum mutual information (bMMI) and state-level minimum Bayes risk (sMBR). We demonstrate that, analogous to L…
New method improves anomaly detection in acoustic signals.
problem Poor anomaly detection performance in existing acoustic signal-based unsupervised methods.
method Deep autoencoding Gaussian mixture model with hyper-parameter optimization.
result Significantly improved anomaly detection performance compared to previous methods.
New method estimates animal density using acoustic data, accounting for unknown call identities.
problem Estimating animal density or call density from acoustic data with unknown call identities.
method Monte Carlo Expectation-Maximization (MCEM) method to resolve unknown call identities.
result Estimates are within 15% of expert-constructed estimates and incorporate uncertainty about call identities.
Enhances speech quality in noisy environments using symbolic sequential modeling.
problem Improving speech quality in noisy conditions.
method Incorporates symbolic sequential modeling into speech enhancement framework.
result Significant improvement in speech quality metrics (PESQ, STOI) on TIMIT dataset.
VPFD uses vocoder features for adversarial training in VC.
problem Adversarial training on waveform data is time-consuming and memory-intensive.
method VPFD employs vocoder features for adversarial training.
result VPFD achieves VC performance comparable to waveform discriminators with reduced training time and memory.
A scalable system learns acoustic models from 1 Million hours of untranscribed audio.
problem Learning acoustic models from large, untranscribed audio datasets.
method Semi-supervised learning with a student/teacher learning paradigm, focusing on the data and model pipelines.
result Relative accuracy improvements of 10-20% in noisy conditions, with no extensive hyper-parameter tuning.
In acoustic scene classification researches, audio segment is usually split into multiple samples. Majority voting is then utilized to ensemble the results of the samples. In this paper, we propose a punishment voting algorithm based on the super categories construction method for acoustic scene classification. Specifi…