Improved music source separation using spectrogram feature loss.
problem Music source separation quality improvement.
method Added a high-level feature loss term from spectrograms using a VGG net to a deep learning model.
result Improvement in separation quality of drums and vocals from songs.
ResNet with Focal Loss improves speech emotion recognition.
problem Speech emotion recognition using plain text features is insufficient.
method Residual Convolutional Neural Network (ResNet) trained with Focal Loss.
result Focal Loss enhances model's focus on hard examples.
Study compares new audio representation methods for limited data music retrieval.
problem Improving machine learning for audio data with limited training data.
method Investigated mel-spectrogram and Mel scattering representations, and augmented target loss function.
result All proposed methods outperform standard mel-spectrogram when using limited data.
Sound source separation has attracted attention from Music Information Retrieval(MIR) researchers, since it is related to many MIR tasks such as automatic lyric transcription, singer identification, and voice conversion. In this paper, we propose an intuitive spectrogram-based model for source separation by adapting U-…
Paper proposes a CNN for speech emotion recognition using center loss and reconstruction.
problem Speech emotion recognition (SER) in audio signals.
method Convolutional Neural Network (CNN) with center loss and reconstruction as regularizers.
result Proposed method achieves highly discriminative features for SER.
CycleGAN-VC3 improves CycleGAN-VCs for mel-spectrogram conversion.
problem Ambiguity in CycleGAN-VC/VC2 effectiveness for mel-spectrogram conversion.
method Proposes CycleGAN-VC3 with time-frequency adaptive normalization (TFAN).
result CycleGAN-VC3 outperforms or matches CycleGAN-VC2 for mel-spectrogram conversion.
Deep learning detects atrial fibrillation with high accuracy.
problem Detecting atrial fibrillation in ECG signals.
method Extracted deep features from spectrograms using convolutional networks.
result Convolutional network achieved 93.16% classification accuracy.
Optimal transport as a loss for machine learning optimization problems has recently gained a lot of attention. Building upon recent advances in computational optimal transport, we develop an optimal transport non-negative matrix factorization (NMF) algorithm for supervised speech blind source separation (BSS). Optimal …
Deep model generates high-quality speech from spectrograms.
problem Speech reconstruction from spectrograms.
method Deep generative model with Gaussian and von Mises distributions for magnitude and phase, variational autoencoder framework.
result Generated speech has high perceptual quality and intelligibility.
Paper presents a deep learning framework for classifying respiratory anomalies and lung diseases from sound recordings.
problem Classifying respiratory anomalies and lung diseases from respiratory sound recordings.
method The framework uses front-end feature extraction to transform sound into spectrograms, and a deep learning network to classify these features.
result The proposed deep learning system outperforms current state-of-the-art methods on the ICBHI benchmark dataset.
Paper proposes a novel unsupervised feature learning approach for environmental sound classification.
problem Classifying environmental sounds without labeled data.
method Cycle-consistent GAN for high-level data augmentation, unsupervised feature learning, codebook construction.
result Improves classification rate by 3.51% to 14.34% compared to state-of-the-art classifiers.
SRMD uses random features for efficient time-frequency analysis.
problem Efficiently analyzing time-series data with low computational cost.
method Sparse Random Mode Decomposition (SRMD) constructs a sparse approximation to the spectrogram.
result SRMD outperforms other methods in signal representation, outlier removal, and mode decomposition.
iSTFTNet speeds up mel-spectrogram vocoders without sacrificing quality.
problem Efficiently converting mel-spectrograms to speech with minimal computation.
method Replaces convolutional layers with iSTFT after frequency dimension reduction.
result Significant reduction in computational cost with comparable quality.
Adversarial attacks on spectrograms can fool audio classifiers trained on waveforms.
problem Susceptibility of audio classifiers to adversarial attacks on spectrograms.
method Applying adversarial attacks to spectrograms and reconstructing audio waveforms.
result Perturbed spectrograms can fool 2D CNNs and 1D CNNs trained on audio waveforms.
A deep learning framework learns wavelet packet transforms for efficient feature extraction.
problem Efficiently extracting meaningful time-frequency features from high-frequency signals.
method Learnable wavelet packet transforms using deep learning.
result Improved spectral leakage and enhanced anomaly detection performance.
AaSP improves audio self-supervised learning by addressing aliasing issues.
problem Alias issues in audio spectrogram transformers.
method AaSP combines aliasing-aware patch representation, teacher-student masked modeling, cross-attention predictor, and contrastive regularization.
result AaSP learns more stable representations that integrate high-frequency cues.
SpecGrad improves neural vocoder sound quality by adapting diffusion noise to log-mel spectrogram.
problem Improving neural vocoder sound quality, especially in high-frequency bands.
method Adapting the diffusion noise distribution to the conditioning log-mel spectrogram through time-varying filtering.
result SpecGrad generates higher-fidelity speech waveform than conventional DDPM-based neural vocoders.
This paper aims to address two issues existing in the current speech enhancement methods: 1) the difficulty of phase estimations; 2) a single objective function cannot consider multiple metrics simultaneously. To solve the first problem, we propose a novel convolutional neural network (CNN) model for complex spectrogra…
Improves speech separation by integrating time and frequency domains.
problem Speech separation using deep learning techniques.
method Proposes a framework that combines time and frequency domain features, using an embedding network and clustering.
result Obtained state-of-the-art results on WSJ0-2mix dataset.
End-to-end neural network based approaches to audio modelling are generally outperformed by models trained on high-level data representations. In this paper we present preliminary work that shows the feasibility of training the first layers of a deep convolutional neural network (CNN) model to learn the commonly-used l…
Improved speech emotion recognition using pitch-synchronous single frequency filtering spectrogram.
problem Uncertainty principles in STFT spectrogram limit time and frequency resolutions.
method Modified SFF spectrogram by averaging amplitudes between GCI locations, named pitch-synchronous SFF spectrogram.
result Improved SER accuracy (63.95% to 70.4%) on IEMOCAP dataset.
Neural codec for high-fidelity audio compression.
problem Efficiently compress audio while maintaining high quality.
method End-to-end neural network architecture with quantized latent space, single multiscale spectrogram adversary, loss balancer mechanism, and lightweight Transformer compression.
result 40% compression with no loss in quality, faster than real-time.
Study improves voice conversion model with Mel-spectrogram augmentation.
problem Insufficient speech pairs data for training sequence-to-sequence voice conversion models.
method Experimented with Mel-spectrogram augmentation using SpecAugment policies and proposed new augmentation policies.
result Time axis warping policies showed better performance in training the voice conversion model.
Advanced deep learning model improves speech enhancement by estimating phase accurately.
problem Difficulty in estimating the phase of clean speech in speech enhancement.
method Proposes Deep Complex U-Net, polar coordinate-wise complex-valued masking, and wSDR loss function.
result Achieves state-of-the-art performance in all metrics, outperforming previous approaches.
In this paper, we present a reverberation removal approach for speaker verification, utilizing dual-label deep neural networks (DNNs). The networks perform feature mapping between the spectral features of reverberant and clean speech. Long short term memory recurrent neural networks (LSTMs) are trained to map corrupted…
Improved U-Nets with various intermediate blocks enhance singing voice separation.
problem Improving singing voice separation accuracy using U-Net architectures.
method Implemented and compared U-Nets with different intermediate spectrogram transformation blocks.
result A specific block type achieves state-of-the-art SDR by 0.9 dB.
This paper proposes a multichannel source separation technique called the multichannel variational autoencoder (MVAE) method, which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class l…
Novel CSK kernel improves GP model generalization for non-stationary patterns.
problem Improving generalization of Gaussian process models for non-stationary data.
method Introduced convolutional spectral kernel (CSK) derived from convolution of imaginary radial basis functions, using Fourier transform for interpretation.
result CSK improves GP model generalization on spatiotemporal datasets.
In this paper, we present a novel system that separates the voice of a target speaker from multi-speaker signals, by making use of a reference signal from the target speaker. We achieve this by training two separate neural networks: (1) A speaker recognition network that produces speaker-discriminative embeddings; (2) …
In this paper, we address the problem of reconstructing a time-domain signal (or a phase spectrogram) solely from a magnitude spectrogram. Since magnitude spectrograms do not contain phase information, we must restore or infer phase information to reconstruct a time-domain signal. One widely used approach for dealing w…
A new neural network separates singing voices more effectively.
problem Separating singing voices from mixed signals with high accuracy.
method MBR-FCN that processes different frequency bands with varying resolutions and filters.
result The MBR-FCN achieves better performance with fewer parameters.
A new RBM model handles both linear and log-amplitude spectrograms.
problem Handling amplitude spectra with existing models.
method Proposed gamma-Bernoulli RBM that uses gamma distribution.
result The model can naturally handle positive numbers and log-amplitude spectrograms.
Novel fusion of autoencoders predicts sleepiness from speech.
problem Predicting sleepiness from speech recordings.
method Attention-based and recurrent sequence to sequence autoencoders for unsupervised representation learning.
result Fusion of autoencoders' representations achieves higher correlation with sleepiness scales.
Speech emotion recognition system using features and text.
problem Improving accuracy in emotion recognition from speech.
method Used speech features (Spectrogram, MFCC) and text, trained Deep Neural Networks.
result Combined MFCC-Text CNN model achieved highest accuracy.
VPFD uses vocoder features for adversarial training in VC.
problem Adversarial training on waveform data is time-consuming and memory-intensive.
method VPFD employs vocoder features for adversarial training.
result VPFD achieves VC performance comparable to waveform discriminators with reduced training time and memory.
Paper proposes a time-frequency analysis method for blind modulation classification in MIMO systems.
problem Blind modulation classification in MIMO systems with overlapping signals and unknown channel parameters.
method Time-frequency analysis using windowed short-time Fourier transform, conversion to RGB spectrogram images, convolutional neural network for classification, decision fusion.
result Proposed scheme achieves high classification accuracy at different SNRs, outperforming existing methods.
Separating a singing voice from its music accompaniment remains an important challenge in the field of music information retrieval. We present a unique neural network approach inspired by a technique that has revolutionized the field of vision: pixel-wise image classification, which we combine with cross entropy loss a…
Improving speech system performance in noisy environments remains a challenging task, and speech enhancement (SE) is one of the effective techniques to solve the problem. Motivated by the promising results of generative adversarial networks (GANs) in a variety of image processing tasks, we explore the potential of cond…
Graph neural networks improve music genre classification on audio datasets.
problem Difficulty in applying deep learning on spectrograms due to lack of quality data and augmentation.
method Combination of CNN and Graph Neural Networks (GNN) with Siamese Neural Networks.
result Achieved state-of-the-art results on GTZAN and AudioSet datasets.
Solution for voice conversion with limited data using hierarchical seq2seq and attention models.
problem Voice conversion between speakers with limited parallel audio pairs.
method Hierarchical sequence to sequence model with attention-based decoder, trained on single speaker dataset.
result Improved voice conversion quality using mel spectrograms and wavenet vocoder.
X-DC improves speech separation by making DNNs more interpretable.
problem Black-box nature of DNNs in speech separation tasks.
method Introduces X-DC, a DNN architecture that interprets as spectrogram template fitting followed by Wiener filtering.
result X-DC achieves comparable speech separation performance to DC but with enhanced interpretability.
Automatically detecting sound units of humpback whales in complex time-varying background noises is a current challenge for scientists. In this paper, we explore the applicability of Convolution Neural Network (CNN) method for this task. In the evaluation stage, we present 6 bi-class classification experimentations of …
MaskCycleGAN-VC improves voice conversion without parallel data.
problem Limited ability to convert mel-spectrogram data without parallel data.
method Integrates a novel auxiliary task called filling in frames (FIF) to learn time-frequency structures.
result MaskCycleGAN-VC outperforms existing methods with similar model size.
We present a monophonic source separation system that is trained by only observing mixtures with no ground truth separation information. We use a deep clustering approach which trains on multi-channel mixtures and learns to project spectrogram bins to source clusters that correlate with various spatial features. We sho…
CLCNet improves noise reduction in hearing aids with deep learning.
problem Noise reduction in hearing aids is challenging due to real-time and frequency resolution constraints.
method Proposes CLCNet, a deep learning framework based on complex linear coding.
result CLCNet outperforms traditional methods in noisy environments.
To address Task 5 in the Detection and Classification of Acoustic Scenes and Events (DCASE) 2018 challenge, in this paper, we propose an ensemble learning system. The proposed system consists of three different models, based on convolutional neural network and long short memory recurrent neural network. With extracted …
iSTFTNet2 improves iSTFTNet's speed and lightness with 1D-2D CNN.
problem Efficiently synthesizing high-fidelity speech.
method Improved iSTFTNet using 1D-2D CNNs for temporal and spectrogram structures.
result iSTFTNet2 is faster and more lightweight with comparable speech quality.
Even in the absence of any explicit semantic annotation, vast collections of audio recordings provide valuable information for learning the categorical structure of sounds. We consider several class-agnostic semantic constraints that apply to unlabeled nonspeech audio: (i) noise and translations in time do not change t…