Generative model improves audio synthesis from TF features.
problem Challenges in generating high-quality audio from TF features.
method Used a GAN to generate invertible TF features from short-time Fourier transforms.
result Generated TF features from GAN outperformed direct waveform generation.
Zero-shot audio classification using class label embeddings.
problem Classifying audio without labeled data.
method Bilinear model with audio feature embeddings and class label embeddings.
result Achieved accuracy up to 39.7% for natural audio categories.
This research aims to develop robust audio spoofing detection methods that work across various spoofing techniques.
problem Detecting audio spoofing attacks in speaker verification systems.
method Examined traditional and machine learned audio features for robust spoofing detection.
result Fused models based on both known and machine learned features achieve comparable performance with an EER of 12.
Paper proposes ICCN to learn correlations between text, audio, and video for multimodal sentiment analysis.
problem Improving multimodal sentiment analysis by learning hidden correlations between text and audio/video features.
method Interaction Canonical Correlation Network (ICCN) using deep canonical correlation analysis (DCCA).
result Empirical results confirm the effectiveness of ICCN in capturing useful information from all three views.
CRNN improves artist classification with temporal audio features.
problem Artist classification using deep learning with temporal structure.
method Convolutional Recurrent Neural Network (CRNN) applied to music artist identification dataset.
result Best model achieves an average F1 score of 0.937.
Audio fingerprinting, also named as audio hashing, has been well-known as a powerful technique to perform audio identification and synchronization. It basically involves two major steps: fingerprint (voice pattern) design and matching search. While the first step concerns the derivation of a robust and compact audio si…
Online audio advertising is a particular form of advertising used abundantly in online music streaming services. In these platforms, which tend to host tens of thousands of unique audio advertisements (ads), providing high quality ads ensures a better user experience and results in longer user engagement. Therefore, th…
Deep Fusion improves audio-video emotion recognition accuracy.
problem Challenges in automatic emotion recognition due to abstract concept and multiple expressions of emotion.
method Introduces factorized bilinear pooling (FBP) with embedded attention mechanism to integrate audio and video features.
result Achieves an accuracy of 62.48% on AFEW database, outperforming state-of-the-art results.
This research converts visual information into audio for users to perceive.
problem Brevity in conveying visual information through spoken language.
method Pretrained image embedding network, GAN for metric space mapping, human subject testing.
result Users can accurately classify audio sonifications of faces.
Proposes COALA method for learning audio representations aligned with tags.
problem Lack of annotated data for high-performance audio representation learning.
method Aligns latent representations of audio and tags using a contrastive loss.
result Audio embedding model captures both acoustic and semantic characteristics.
A model classifies music genres from MP3 files using metric learning and feature extraction.
problem Classifying music genres from MP3 files efficiently and accurately.
method Metric learning and feature extraction using MFCC and PCA.
result Promising results in classification accuracy compared to baseline algorithms.
Study improves radio show segmentation using audio embeddings.
problem Automated segmentation of radio shows.
method Created audio embeddings from multi-class classification tasks on different datasets, evaluated performance against text-only baseline.
result Audio embeddings from non-speech sound event classification significantly outperformed text-only baseline by 32.3% in F1-measure.
Speaker clustering is the task of differentiating speakers in a recording. In a way, the aim is to answer "who spoke when" in audio recordings. A common method used in industry is feature extraction directly from the recording thanks to MFCC features, and by using well-known techniques such as Gaussian Mixture Models (…
RawNet synthesizes speech without relying on human-designed features.
problem Speech synthesis models rely heavily on human-designed spectral features.
method End-to-end neural vocoder using auto-encoder structure, learning features and audio recovery directly from raw waveform.
result RawNet achieves better speech quality with a simplified model architecture and faster inference speed.
Feature learning and deep learning have drawn great attention in recent years as a way of transforming input data into more effective representations using learning algorithms. Such interest has grown in the area of music information retrieval (MIR) as well, particularly in music audio classification tasks such as auto…
GACELA fills long gaps in musical audio with a GAN and context conditioning.
problem Restoring long gaps in musical audio with varying complexity and duration.
method Generative adversarial network (GAN) with five parallel discriminators and context conditioning.
result Reduced artifacts in inpaintings from unacceptable to mildly disturbing.
Deep learning predicts mental disorders from audio and text samples.
problem Predicting mental disorders from speech samples.
method Multimodal deep learning structure using various pre-trained models for audio and text embeddings, transfer learning, and auxiliary corpora.
result Acceptable accuracy in predicting mental disorders through multimodal analysis.
Bird sounds possess distinctive spectral structure which may exhibit small shifts in spectrum depending on the bird species and environmental conditions. In this paper, we propose using convolutional recurrent neural networks on the task of automated bird audio detection in real-life environments. In the proposed metho…
Paper proposes a robust audio classification method against adversarial attacks.
problem Adversarial attacks can fool machine learning models into making incorrect predictions.
method Proposes a novel SVM-based approach using DWT and SURF features.
result The proposed method provides a good balance between accuracy and resilience against adversarial attacks.
Level assessment for foreign language students is necessary for putting them in the right level group, furthermore, interviewing students is a very time-consuming task, so we propose to automate the evaluation of speaker fluency level by implementing machine learning techniques. This work presents an audio processing s…
Review of deep learning techniques for audio signal processing.
problem Improving audio signal processing using deep learning.
method Analysis of various deep learning models and techniques.
result Advancements in speech, music, and environmental sound processing.
Machine learning detects frog calls in audio recordings with high accuracy.
problem Estimating frog presence in audio recordings from tropical environments.
method Gaussian mixture model classification of frog calls with 20 cepstral features.
result The algorithm achieved an average weighted error rate of 0.9% for nine species classification.
Efficient neural network for audio source separation.
problem End-to-end general purpose audio source separation.
method SuDoRMRF structure with simple one-dimensional convolutions for feature aggregation.
result SuDoRMRF achieves high quality audio source separation with minimal computational resources.
Generates music with video emotion using deep neural networks.
problem Generating music with video emotion.
method Hybrid deep neural network combining ANFIS and LSTM.
result Low mean absolute errors and similar global features in spectrograms.
Unified model for audio control and style transfer.
problem Explicit control and style transfer in music generation.
method Diffusion autoencoders for semantic feature extraction, disentanglement using adversarial criterion.
result Model generates audio matching timbre targets with specified structure.
Improves audio source separation using dilated convolutions and dense connections.
problem Optimizing feature extraction in audio source separation models.
method Adaptive dilated convolutions and dense connections in U-Net architecture.
result Improved performance on MUSDB test dataset.
Wavelet Attribution Method (WAM) improves feature attribution for deep models.
problem Inability of pixel-based heatmaps to capture data structure and variability in feature attribution.
method Wavelet domain for feature attribution, leveraging spatial and scale-localized properties of wavelet coefficients.
result WAM provides quantitatively superior explanations across audio, image, and volume modalities.
Existing automatic music generation approaches that feature deep learning can be broadly classified into two types: raw audio models and symbolic models. Symbolic models, which train and generate at the note level, are currently the more prevalent approach; these models can capture long-range dependencies of melodic st…
SVM algorithm extracts digits from audio CAPTCHAs.
problem Recognizing audio CAPTCHAs from computer programs.
method Used RastaPLP features and SVM algorithm.
result Successfully extracted digits from audio CAPTCHAs.
Convolutional Neural Networks (CNNs) have proven very effective in image classification and show promise for audio. We use various CNN architectures to classify the soundtracks of a dataset of 70M training videos (5.24 million hours) with 30,871 video-level labels. We examine fully connected Deep Neural Networks (DNNs)…
Enhances sound texture in CNN for better acoustic scene classification.
problem Limited understanding of how CNNs perceive audio scenes.
method Used Class Activation Mapping (CAM) to analyze log-Mel features and proposed edge enhancement using DoG and Sobel operators.
result Edge-enhanced log-Mel features improve CNN performance in acoustic scene classification.
Being able to predict whether a song can be a hit has impor- tant applications in the music industry. Although it is true that the popularity of a song can be greatly affected by exter- nal factors such as social and commercial influences, to which degree audio features computed from musical signals (whom we regard as …
End-to-end probabilistic inference improves audio signal processing.
problem Efficiently processing large audio signals with varying characteristics.
method Formulated a spectral mixture Gaussian process model with nonstationary priors, enabling infinite-horizon Gaussian process regression.
result The method outperforms standard techniques in processing audio signals with hundreds of thousands of data points.
FSD50K provides an open dataset of over 51k audio clips for sound event recognition.
problem Small and domain-specific sound event recognition datasets.
method Creation of an open dataset with over 51k audio clips manually labeled using 200 classes.
result FSD50K is a new open benchmark for sound event recognition research.
A fusion approach combines audio and video features for emotion recognition.
problem Continuous emotion recognition using both visual and auditory modalities.
method Pre-trained CNN features from video frames and minimalistic auditory descriptors. Fusion at feature or prediction level. SVR for prediction.
result Improves CCCs of 0.749 and 0.565 for arousal and valence respectively.
Improved audio classification with limited labels using multitask and self-supervised learning.
problem Limited labeled data for audio classification.
method Multitask learning and self-supervised learning on unlabeled data.
result Significant improvement in performance (up to 6%) through multitask and self-supervised learning.
Enhances ASC using time- and frequency-liked CNNs and bilinear pooling.
problem Improving acoustic scene classification accuracy.
method Harmonic and percussive source separation, two-stream CNN architecture, bilinear pooling.
result Improved accuracy on DCASE 2019 sub task 1a dataset.
AV-ASR system improves speech recognition with visual context.
problem Improving speech recognition accuracy with visual information.
method Transformer-based architecture with multiresolution and multimodal training.
result Multiresolution training speeds up convergence and improves WER by 18%.
This paper addresses the problem of localizing audio sources using binaural measurements. We propose a supervised formulation that simultaneously localizes multiple sources at different locations. The approach is intrinsically efficient because, contrary to prior work, it relies neither on source separation, nor on mon…
Improved music source separation using spectrogram feature loss.
problem Music source separation quality improvement.
method Added a high-level feature loss term from spectrograms using a VGG net to a deep learning model.
result Improvement in separation quality of drums and vocals from songs.
We propose the product-of-filters (PoF) model, a generative model that decomposes audio spectra as sparse linear combinations of "filters" in the log-spectral domain. PoF makes similar assumptions to those used in the classic homomorphic filtering approach to signal processing, but replaces hand-designed decompositions…
AaSP improves audio self-supervised learning by addressing aliasing issues.
problem Alias issues in audio spectrogram transformers.
method AaSP combines aliasing-aware patch representation, teacher-student masked modeling, cross-attention predictor, and contrastive regularization.
result AaSP learns more stable representations that integrate high-frequency cues.
Crowd-sourced mosquito audio dataset for malaria research.
problem Understanding mosquito locations for malaria reduction.
method Release of a large mosquito audio dataset with labels from contributors.
result Demonstrated the feasibility of training a CNN on mosquito audio data.
Deep learning models generate music with arbitrary control strategies.
problem Lack of efficient methods for generating music with arbitrary control.
method Deep generative models learn to navigate arbitrary sound spaces.
result Deep learning enables high-quality, arbitrary sound synthesis.
Develops deep NMF models using β-divergences for feature extraction.
problem Inadequate evaluation metrics for deep NMF on diverse datasets.
method Introduces new deep NMF models using Kullback-Leibler divergence.
result Improves feature extraction quality across different types of data.
We propose a multi-label multi-task framework based on a convolutional recurrent neural network to unify detection of isolated and overlapping audio events. The framework leverages the power of convolutional recurrent neural network architectures; convolutional layers learn effective features over which higher recurren…
A scalable system learns acoustic models from 1 Million hours of untranscribed audio.
problem Learning acoustic models from large, untranscribed audio datasets.
method Semi-supervised learning with a student/teacher learning paradigm, focusing on the data and model pipelines.
result Relative accuracy improvements of 10-20% in noisy conditions, with no extensive hyper-parameter tuning.
TFiLM expands convolutional models' receptive field with minimal overhead.
problem Capturing long-range dependencies in sequential data.
method A novel architectural component using a recurrent neural network to modulate convolutional model activations.
result TFiLM significantly improves learning speed and accuracy on various tasks.