Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.
Stabilizes training of DNN for speech enhancement using PESQ scores.
problem Stability issues in training DNNs using non-differentiable OSQA scores.
method Approximate OSQA scores with a differentiable auxiliary DNN and stabilize training with reinforcement learning techniques.
result Stable training of DNN to achieve state-of-the-art PESQ scores and better sound quality.
Quaternion Conformer GAN (QC-GAN) is a parameter-efficient speech enhancement framework that combines a Quaternion Conformer generator with MetricGAN-based training.
problem Speech Enhancement
method Quaternion Conformer GAN
result Achieved a PESQ score of 3.48 with 0.89M parameters, comparable to state-of-the-art models at less than half their size.
Thanks to the growing availability of spoofing databases and rapid advances in using them, systems for detecting voice spoofing attacks are becoming more and more capable, and error rates close to zero are being reached for the ASVspoof2015 database. However, speech synthesis and voice conversion paradigms that are not…
PriorGrad improves speech synthesis models by using data-dependent adaptive priors.
problem Inefficiency in denoising diffusion models due to mismatch between prior and data distributions.
method Proposes PriorGrad, an adaptive prior derived from data statistics based on conditional information.
result PriorGrad achieves faster convergence and superior performance in speech synthesis models.
Improved speech enhancement with larger neural networks using novel embeddings and biases.
problem Decreased robustness of speech enhancement models to real-world use cases.
method Frequency-positional embeddings, semi-supervised training, biased loss function.
result Better performance on real recordings with improved large neural network architecture.
Speech enhancement improved using variational autoencoders and alpha-stable noise models.
problem Improving speech quality in noisy environments.
method Deep generative model using variational autoencoders with alpha-stable noise model.
result The proposed method outperforms conventional approaches in speech intelligibility and quality.
Enhanced text-to-speech synthesizes expressive speech from a single example.
problem Creating a new expressive speech style from a single example of speech.
method Combines VAE and Normalizing Flows to improve disentanglement and naturalness.
result Reduces KL-divergence by 22% and improves perceptual metrics.
Enhances speech quality in noisy environments using symbolic sequential modeling.
problem Improving speech quality in noisy conditions.
method Incorporates symbolic sequential modeling into speech enhancement framework.
result Significant improvement in speech quality metrics (PESQ, STOI) on TIMIT dataset.
Deep model generates high-quality speech from spectrograms.
problem Speech reconstruction from spectrograms.
method Deep generative model with Gaussian and von Mises distributions for magnitude and phase, variational autoencoder framework.
result Generated speech has high perceptual quality and intelligibility.
Human cochlear models improve DNN noise suppression systems.
problem DNN-based noise suppression systems lack robustness to unseen noise conditions.
method Coupled cochlear models with DNNs to improve noise suppression.
result Cochlear models enhance DNN generalizability to various noise conditions.
New SSIM measure improves adversarial attacks' perceptual quality.
problem Lack of perceptual quality in traditional adversarial attacks.
method Replaced Lp norms with SSIM measure for better quality. result SSIM-constrained attacks achieve better quality and similar success rate.
Perceptual Kalman filters maintain human-perceptual quality while processing data.
problem Maintaining human-perceptual quality in signal processing under temporal constraints.
method An optimal causal filtering approach under a perfect perceptual-quality constraint.
result Adding perceptual quality constraints introduces a dilemma that requires sacrificing MSE for temporal consistency.
This research improves neural synthesizers for music sounds from speech data.
problem Applying speech synthesis techniques to musical instrument sounds.
method Comparison of three neural synthesizers in three scenarios: training, zero-shot learning, and fine-tuning.
result Neural synthesizers trained on speech data and fine-tuned on music data perform better.
This study proposes a fully convolutional network (FCN) model for raw waveform-based speech enhancement. The proposed system performs speech enhancement in an end-to-end (i.e., waveform-in and waveform-out) manner, which dif-fers from most existing denoising methods that process the magnitude spectrum (e.g., log power …
Improving speech system performance in noisy environments remains a challenging task, and speech enhancement (SE) is one of the effective techniques to solve the problem. Motivated by the promising results of generative adversarial networks (GANs) in a variety of image processing tasks, we explore the potential of cond…
New study shows tradeoffs between compression quality, distortion, and perception.
problem Optimizing compression for low distortion often sacrifices perceptual quality.
method Adopted Blau & Michaeli's perceptual quality definition and studied the rate-distortion-perception tradeoff.
result Restricting perceptual quality to high generally requires a trade-off between rate and distortion.
TinyLSTMs reduces speech enhancement model size and latency for hearing aids.
problem Large RNNs limit practical deployment in hearing aid hardware.
method Model compression techniques (pruning, integer quantization, state update skipping) for RNN speech enhancement.
result Reduction in model size and operations by 11.9imes and 2.9imes, respectively, without perceptual degradation. PerceptNet mimics human vision to estimate image quality.
problem Estimating perceptual distance between images and their perturbations.
method Inspired by human visual system, PerceptNet uses convolutional neural network architecture.
result PerceptNet outperforms traditional image quality metrics and deep learning methods.
Stochastic encoders outperform deterministic ones in 'perfect perceptual quality'.
problem Understanding when stochastic encoders outperform deterministic ones.
method Provided a toy example to illustrate performance.
result Stochastic encoders can significantly outperform deterministic ones in 'perfect perceptual quality'.
End-to-end denoising framework improves SDR and PESQ metrics.
problem Spectrum and metric mismatches in speech enhancement networks.
method Optimizes network on time-domain signals after ISTFT and uses improved loss functions.
result Significantly improved SDR and PESQ performance.
Paper uses CNN to create realistic LiDAR data.
problem Creating high-resolution LiDAR data with missing points.
method Modified per-point loss and perceptual loss applied to generate realistic LiDAR data.
result Significant improvement in geometric and semantic accuracy over traditional methods.
Convex optimization with sparsity-promoting convex regularization is a standard approach for estimating sparse signals in noise. In order to promote sparsity more strongly than convex regularization, it is also standard practice to employ non-convex optimization. In this paper, we take a third approach. We utilize a no…
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
Paper presents a new video generation model using diffusion probabilistic methods.
problem Generating high-quality video sequences.
method Denoising diffusion probabilistic models, autoregressive, end-to-end optimization.
result Significant improvements in perceptual quality and probabilistic frame forecasting.
This paper presents sampling-based speech parameter generation using moment-matching networks for Deep Neural Network (DNN)-based speech synthesis. Although people never produce exactly the same speech even if we try to express the same linguistic and para-linguistic information, typical statistical speech synthesis pr…
RUSLAN is a large Russian speech corpus for text-to-speech.
problem Lack of high-quality annotated Russian speech data for text-to-speech.
method Developed a large annotated Russian speech corpus and trained a neural network for text-to-speech synthesis.
result Synthesized speech quality evaluated with MOS scores: 4.05 for naturalness, 3.78 for intelligibility.
WaveCycleGAN2 improves speech synthesis quality by reducing aliasing.
problem Human ear can still distinguish synthesized speech from natural speech.
method WaveCycleGAN2 uses generators without down/up-sampling modules and combines discriminators from waveform and acoustic parameter domains.
result WaveCycleGAN2 achieves high-quality speech synthesis with comparable mean opinion scores to natural speech.
Generative networks improve fluid simulation quality by focusing on high frequencies.
problem Low-frequency details missing in fluid simulation reconstructions.
method Frequency-aware loss function for generative networks.
result Improved perceptual quality of fluid simulation results in mid-frequency bands.
Enhances spoken speech quality using EEG signals.
problem Improves speech clarity in noisy environments.
method Generative adversarial network (GAN), gated recurrent unit (GRU), temporal convolutional network (TCN) regression models.
result Significant improvement in speech enhancement quality compared to traditional methods.
A new loss function for VAEs improves image quality and efficiency.
problem Training VAEs to generate realistic images requires a loss function that reflects human perception.
method Based on Watson's perceptual model, the loss function computes a weighted distance in frequency space, accounts for luminance and contrast masking, and is extended to color images.
result VAEs trained with the new loss function generated high-quality, less blurry images with fewer artifacts and less computational resources.
WaveGlow generates high-quality speech from spectrograms.
problem Speech synthesis quality and efficiency.
method WaveGlow combines Glow and WaveNet insights, using a single network and cost function for efficient, high-quality audio synthesis.
result WaveGlow produces audio samples at over 500 kHz, matching WaveNet quality.
New hierarchical VQ-VAE scheme improves image compression quality and features at low bitrates.
problem Low bitrate image compression maintaining quality and features.
method Hierarchical VQ-VAE with stochastic quantization and Markovian latent variables.
result High perceptual quality and semantic features at low bitrates.
Proposes DNN for enhancing sound quality scores.
problem Improving sound quality assessment scores.
method Develops a DNN optimization scheme based on black-box optimization and policy gradient method.
result Significant increase in OSQA scores without minimizing MSE.
Combining data from multiple speakers improves neural TTS quality, especially with imbalanced data.
problem Training high-quality TTS systems with imbalanced speaker data.
method Combine data from multiple speakers, train multi-speaker models, and use ensemble methods.
result Ensemble multi-speaker models improve synthetic speech quality for underrepresented speakers.
Bayesian SHMM models speech units from unannotated speech.
problem Discover latent speech units from unannotated data.
method Bayesian Subspace Hidden Markov Model (SHMM).
result System performs well on synthesis quality, lower bitrate.
Study shows current metrics for audio adversarial examples are unreliable for human perception.
problem The reliability of metrics for evaluating audio adversarial examples.
method Analytical framework and human evaluation experiment.
result Current metrics for audio adversarial examples are not reliable for human perception.
SteganoGAN hides data in images with high capacity and high quality.
problem Hiding secret messages in images without detection.
method Generative adversarial networks (GANs) to optimize image quality and payload size.
result Achieves 4.4 bits per pixel payload, evades steganalysis, works on multiple datasets.
AIQN improves generative models with quantile regression.
problem Improving perceptual quality and diversity in generative models.
method AIQN uses autoregressive implicit quantile networks for generative modeling.
result AIQN achieves superior perceptual quality and diversity without sacrificing diversity.
Neural codec for high-fidelity audio compression.
problem Efficiently compress audio while maintaining high quality.
method End-to-end neural network architecture with quantized latent space, single multiscale spectrogram adversary, loss balancer mechanism, and lightweight Transformer compression.
result 40% compression with no loss in quality, faster than real-time.
Improved image compression with diffusion models outperforming state-of-the-art methods.
problem Difficulties in replicating text-to-image success in image compression.
method Two-stage approach combining autoencoder targeting MSE followed by score-based decoder.
result Significantly improved perceptual quality at a given bit-rate, measured by FID score.
Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech. We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform. Our best models …
PeL separates sensory interface optimization from decision learning.
problem Optimizing sensory interfaces without task-specific information.
method Formal separation of perception and decision learning, using metrics for stability, informativeness, and geometry.
result Updates preserving invariants are orthogonal to decision gradients.
This paper introduces a database to assess the perceptual similarity of adversarial images.
problem The lack of reliable metrics to assess the perceptual similarity of adversarial images generated by Lp norms. method Creation of a database and evaluation of fifteen FR image fidelity assessment metrics.
result The database and metrics can help in designing new metrics for adversarial examples.
In this work we explored building automatic speech recognition models for transcribing doctor patient conversation. We collected a large scale dataset of clinical conversations (14,000 hr), designed the task to represent the real word scenario, and explored several alignment approaches to iteratively improve data qua…
Polynomial fusion layer improves speech-driven facial animation.
problem Recent facial synthesis relies on low-dimensional representations and concatenation, ignoring higher-order interactions.
method Proposes a polynomial fusion layer to model higher-order interactions of facial encodings.
result Demonstrates improved video quality, audiovisual synchronisation, and blink generation.
End-to-end Sanskrit TTS developed with limited data, achieving good quality.
problem Developing natural-sounding speech for Sanskrit with scarce data.
method Fine-tuning Tacotron2 model with WaveGlow and transfer learning.
result Achieved an overall MOS of 3.38 from 37 evaluators.
Proposes new terms for neural image compression to improve quality and efficiency.
problem Improving the quality and efficiency of neural image compression.
method Introduces a compression objective and a cycle loss term, applied to autoencoder encoder outputs, combined with reconstruction losses.
result Different autoencoders trained with varying losses produce images with distinct perceptual qualities and image-domain distortions.