New method clusters unknown music artists using audio metrics.
problem Disambiguating large catalogs of unknown artists.
method Metric learning from audio data with negative sampling.
result Our method outperforms a classifier-based approach when audio data is available.
This research converts visual information into audio for users to perceive.
problem Brevity in conveying visual information through spoken language.
method Pretrained image embedding network, GAN for metric space mapping, human subject testing.
result Users can accurately classify audio sonifications of faces.
A model classifies music genres from MP3 files using metric learning and feature extraction.
problem Classifying music genres from MP3 files efficiently and accurately.
method Metric learning and feature extraction using MFCC and PCA.
result Promising results in classification accuracy compared to baseline algorithms.
Study shows current metrics for audio adversarial examples are unreliable for human perception.
problem The reliability of metrics for evaluating audio adversarial examples.
method Analytical framework and human evaluation experiment.
result Current metrics for audio adversarial examples are not reliable for human perception.
Optimizes audio codec selection with statistical guarantees.
problem Selecting the best audio encoding scheme for various data types.
method Supervised learning with uniform convergence theory.
result Rigorous statistical guarantees for codec selection.
Paper tackles GAN instability in audio and speech signals using a new similarity metric.
problem Improving stability of LS-GANs for audio and speech signals.
method Proposes a new similarity metric in unitary space of Schur decomposition for 2D audio and speech representations.
result Enhanced stability in training with less mode collapse compared to baseline GANs.
Online audio advertising is a particular form of advertising used abundantly in online music streaming services. In these platforms, which tend to host tens of thousands of unique audio advertisements (ads), providing high quality ads ensures a better user experience and results in longer user engagement. Therefore, th…
GANSynth uses GANs to efficiently synthesize high-fidelity audio.
problem Efficient and high-fidelity audio synthesis is challenging.
method Model log magnitudes and instantaneous frequencies with GANs.
result GANSynth outperforms WaveNet on automated and human evaluation metrics.
Model for directed synthesis of audio textures using multi-scale RNNs.
problem Challenges in modeling complex audio textures with traditional methods.
method Combining multi-scale RNNs with a conditioning strategy for user-directed synthesis.
result Demonstrated improved performance on various audio texture datasets.
Improved emotion prediction using autoencoder codebooks.
problem Continuous emotion prediction from audio data.
method Bag-of-Words model based on autoencoder codebook.
result Improved CCC scores for emotion dimensions.
Proposes a self-supervised method for generating spatial audio from monaural audio and video.
problem Generating spatial audio from monaural audio and video recordings is challenging and expensive.
method Uses a self-supervised network with an auxiliary classifier to classify video channels and generate spatial audio.
result The proposed method effectively generates spatial audio from monaural audio and video.
Deep audio prior uses neural networks to solve audio problems without data.
problem Challenging audio problems like source separation, editing, and synthesis.
method Randomly-initialized neural network with carefully designed audio prior.
result Superior audio results on Universal-150 benchmark dataset.
Zero-shot audio classification using class label embeddings.
problem Classifying audio without labeled data.
method Bilinear model with audio feature embeddings and class label embeddings.
result Achieved accuracy up to 39.7% for natural audio categories.
A new autoencoder speeds up audio style transfer.
problem Efficiently generating stylized audio from content audio.
method Autoencoder architecture for real-time audio style transfer.
result Single forward pass for stylized audio generation.
Method learns audio embeddings with contextualized tags.
problem Align audio and tags for cross-modal tasks.
method Audio autoencoder, word embeddings, multi-head self-attention, contrastive loss.
result Multi-head self-attention improves audio representations.
Study improves radio show segmentation using audio embeddings.
problem Automated segmentation of radio shows.
method Created audio embeddings from multi-class classification tasks on different datasets, evaluated performance against text-only baseline.
result Audio embeddings from non-speech sound event classification significantly outperformed text-only baseline by 32.3% in F1-measure.
Automatic conflict detection has grown in relevance with the advent of body-worn technology, but existing metrics such as turn-taking and overlap are poor indicators of conflict in police-public interactions. Moreover, standard techniques to compute them fall short when applied to such diversified and noisy contexts. W…
Generative model improves audio synthesis from TF features.
problem Challenges in generating high-quality audio from TF features.
method Used a GAN to generate invertible TF features from short-time Fourier transforms.
result Generated TF features from GAN outperformed direct waveform generation.
Adversarial attacks on spectrograms can fool audio classifiers trained on waveforms.
problem Susceptibility of audio classifiers to adversarial attacks on spectrograms.
method Applying adversarial attacks to spectrograms and reconstructing audio waveforms.
result Perturbed spectrograms can fool 2D CNNs and 1D CNNs trained on audio waveforms.
Study improves animal audio classification using data augmentation.
problem Improving automated animal audio classification accuracy.
method Exploits different data augmentation techniques for training CNNs.
result Best recognition rates on animal audio classification datasets.
MicAugment transfers audio style from few seconds of input to match target conditions.
problem Audio model robustness to diverse acquisition conditions.
method Identifies and applies transformations learned from target audio to input audio.
result MicAugment significantly improves model robustness in downstream tasks.
Detects audio adversarial examples using anomalous pattern detection.
problem Identifies adversarial audio attacks in deep neural networks.
method Applies anomalous pattern detection in activation space of audio models.
result Can detect adversarial examples with up to 0.98 AUC, no degradation on benign samples.
Speech enhancement (SE) aims to reduce noise in speech signals. Most SE techniques focus only on addressing audio information. In this work, inspired by multimodal learning, which utilizes data from different modalities, and the recent success of convolutional neural networks (CNNs) in SE, we propose an audio-visual de…
Speech enhancement (SE) aims to reduce noise in speech signals. Most SE techniques focus only on addressing audio information. In this work, inspired by multimodal learning, which utilizes data from different modalities, and the recent success of convolutional neural networks (CNNs) in SE, we propose an audio-visual de…
Unified model for audio control and style transfer.
problem Explicit control and style transfer in music generation.
method Diffusion autoencoders for semantic feature extraction, disentanglement using adversarial criterion.
result Model generates audio matching timbre targets with specified structure.
Audio fingerprinting, also named as audio hashing, has been well-known as a powerful technique to perform audio identification and synchronization. It basically involves two major steps: fingerprint (voice pattern) design and matching search. While the first step concerns the derivation of a robust and compact audio si…
Transformer model estimates keywords for better audio captioning.
problem Indeterminacy in word selection for audio events/scenes.
method Transformer-based model with keyword estimation.
result Achieved state-of-the-art performance in AAC.
As part of the 2016 public evaluation challenge on Detection and Classification of Acoustic Scenes and Events (DCASE 2016), the second task focused on evaluating sound event detection systems using synthetic mixtures of office sounds. This task, which follows the `Event Detection - Office Synthetic' task of DCASE 2013,…
LumièreNet creates lecture videos from audio narration.
problem Creating high-quality lecture videos from audio narration.
method Modular deep-learning architecture that learns from audio to video.
result Synthesizes high-quality lecture videos from any length of audio.
Review of deep learning techniques for audio signal processing.
problem Improving audio signal processing using deep learning.
method Analysis of various deep learning models and techniques.
result Advancements in speech, music, and environmental sound processing.
This paper describes Task 2 of the DCASE 2018 Challenge, titled "General-purpose audio tagging of Freesound content with AudioSet labels". This task was hosted on the Kaggle platform as "Freesound General-Purpose Audio Tagging Challenge". The goal of the task is to build an audio tagging system that can recognize the c…
AV-CPL uses continuous pseudo-labels for AVSR combining labeled and unlabeled data.
problem Improving AVSR performance with labeled and unlabeled data.
method Semi-supervised method using continuous pseudo-labels generated by the same AVSR model.
result Significant improvements in VSR performance on LRS3 dataset.
MelNet generates high-fidelity audio with long-range structure.
problem Capturing long-range dependencies in audio waveforms.
method Generative model in frequency domain, leveraging spectrograms.
result Improves audio generation in various tasks.
HEAR benchmark evaluates audio representations for diverse tasks.
problem Developing a general-purpose audio representation for various tasks.
method Evaluated 29 models across 19 tasks using 16 datasets.
result No single audio representation performs holistically.
New method improves speech synthesis quality.
problem Efficiently train parallel speech synthesis models.
method Spectral energy distance for implicit generative models.
result State-of-the-art generation quality achieved.
Proposes COALA method for learning audio representations aligned with tags.
problem Lack of annotated data for high-performance audio representation learning.
method Aligns latent representations of audio and tags using a contrastive loss.
result Audio embedding model captures both acoustic and semantic characteristics.
Paper introduces a noisy-labeled audio tagging challenge.
problem Acoustic mismatch and noisy labels in audio tagging.
method Large dataset with minimal supervision, convolutional neural network baseline.
result Demonstrates effectiveness of minimal supervision in noisy conditions.
Efficient sparse GP model improves audio source separation.
problem Sparse Gaussian Process (GP) inference is computationally expensive for long audio frames.
method Used GP regression, spectral mixture kernels, and variational sparse GPs.
result Proposed method outperforms LD-PSDTF, KL-NMF, and IS-NMF.
Machine learning identifies species by voice in remote areas.
problem Continuous monitoring of endangered species in remote areas.
method Training machine learning models on audio data to recognize species.
result Machine learning can accurately classify and recognize various species sounds.
A new method for audio denoising using deep neural networks.
problem Improving audio quality by removing background noise.
method Combines time and time-frequency domain processing; trains a deep neural network to fit the signal.
result The method effectively disentangles clean audio from noisy signals.
Capsule networks improve audio classification performance.
problem Improving audio classification accuracy.
method Dynamic routing-by-agreement mechanism for capsule networks.
result Significant improvements in performance compared to baseline models.
Mic2Mic reduces microphone variability for speech systems.
problem Microphone variability impacts speech system accuracy.
method Cycle-consistent GANs for learning microphone variability reduction.
result Mic2Mic recovers 66% to 89% of lost accuracy due to microphone variability.
FSD50K provides an open dataset of over 51k audio clips for sound event recognition.
problem Small and domain-specific sound event recognition datasets.
method Creation of an open dataset with over 51k audio clips manually labeled using 200 classes.
result FSD50K is a new open benchmark for sound event recognition research.
Paper proposes a robust audio classification method against adversarial attacks.
problem Adversarial attacks can fool machine learning models into making incorrect predictions.
method Proposes a novel SVM-based approach using DWT and SURF features.
result The proposed method provides a good balance between accuracy and resilience against adversarial attacks.
Nonnegative matrix factorization (NMF) is now a common tool for audio source separation. When learning NMF on large audio databases, one major drawback is that the complexity in time is O(FKN) when updating the dictionary (where (F;N) is the dimension of the input power spectrograms, and K the number of basis spectra),…
Self-supervised attention model improves weakly labeled audio event classification.
problem Efficiently classify audio events with minimal labeled data.
method Develops a self-supervised attention model for weakly labeled audio clips.
result Self-supervised attention model performs comparably to strongly supervised model trained with strong labels.
Deep Fusion improves audio-video emotion recognition accuracy.
problem Challenges in automatic emotion recognition due to abstract concept and multiple expressions of emotion.
method Introduces factorized bilinear pooling (FBP) with embedded attention mechanism to integrate audio and video features.
result Achieves an accuracy of 62.48% on AFEW database, outperforming state-of-the-art results.
This research tackles backdoor attacks on audio data using a stochastic investment approach.
problem The threat of backdoor attacks on audio data, especially in voice-activated systems.
method A Stochastic investment-based backdoor attack (MarketBack) approach.
result MarketBack can achieve an average attack success rate close to 100% with less than 1% of poisoned data.