SpecGrad improves neural vocoder sound quality by adapting diffusion noise to log-mel spectrogram.
problem Improving neural vocoder sound quality, especially in high-frequency bands.
method Adapting the diffusion noise distribution to the conditioning log-mel spectrogram through time-varying filtering.
result SpecGrad generates higher-fidelity speech waveform than conventional DDPM-based neural vocoders.
WaveFit uses fixed-point iteration to create high-quality neural vocoders.
problem Creating high-quality neural vocoders with fast inference.
method Integrates GANs' adversarial training into a DDPM-like iterative framework based on fixed-point iteration.
result WaveFit synthesizes speech with naturalness comparable to human speech, and is significantly faster than existing methods.
Recent advances in speech synthesis suggest that limitations such as the lossy nature of the amplitude spectrum with minimum phase approximation and the over-smoothing effect in acoustic modeling can be overcome by using advanced machine learning approaches. In this paper, we build a framework in which we can fairly co…
Recent neural networks such as WaveNet and sampleRNN that learn directly from speech waveform samples have achieved very high-quality synthetic speech in terms of both naturalness and speaker similarity even in multi-speaker text-to-speech synthesis systems. Such neural networks are being used as an alternative to voco…
iSTFTNet speeds up mel-spectrogram vocoders without sacrificing quality.
problem Efficiently converting mel-spectrograms to speech with minimal computation.
method Replaces convolutional layers with iSTFT after frequency dimension reduction.
result Significant reduction in computational cost with comparable quality.
Neural network-based vocoders have recently demonstrated the powerful ability to synthesize high-quality speech. These models usually generate samples by conditioning on spectral features, such as Mel-spectrogram and fundamental frequency, which is crucial to speech synthesis. However, the feature extraction procession…
VPFD uses vocoder features for adversarial training in VC.
problem Adversarial training on waveform data is time-consuming and memory-intensive.
method VPFD employs vocoder features for adversarial training.
result VPFD achieves VC performance comparable to waveform discriminators with reduced training time and memory.
DiffWave generates high-fidelity audio waveforms efficiently.
problem Conditional and unconditional audio waveform generation.
method Non-autoregressive diffusion model using Markov chain synthesis.
result DiffWave produces high-quality audios in various tasks.
Recent speech technology research has seen a growing interest in using WaveNets as statistical vocoders, i.e., generating speech waveforms from acoustic features. These models have been shown to improve the generated speech quality over classical vocoders in many tasks, such as text-to-speech synthesis and voice conver…
Mel-frequency filter bank (MFB) based approaches have the advantage of learning speech compared to raw spectrum since MFB has less feature size. However, speech generator with MFB approaches require additional vocoder that needs a huge amount of computation expense for training process. The additional pre/post processi…
iSTFTNet2 improves iSTFTNet's speed and lightness with 1D-2D CNN.
problem Efficiently synthesizing high-fidelity speech.
method Improved iSTFTNet using 1D-2D CNNs for temporal and spectrogram structures.
result iSTFTNet2 is faster and more lightweight with comparable speech quality.
The recent success of raw audio waveform synthesis models like WaveNet motivates a new approach for music synthesis, in which the entire process --- creating audio samples from a score and instrument information --- is modeled using generative neural networks. This paper describes a neural music synthesis model with fl…
We propose a learning-based filter that allows us to directly modify a synthetic speech waveform into a natural speech waveform. Speech-processing systems using a vocoder framework such as statistical parametric speech synthesis and voice conversion are convenient especially for a limited number of data because it is p…
In this work, we propose a new recurrent autoencoder architecture, termed Feedback Recurrent AutoEncoder (FRAE), for online compression of sequential data with temporal dependency. The recurrent structure of FRAE is designed to efficiently extract the redundancy along the time dimension and allows a compact discrete re…
Deep neural networks (DNN) are able to successfully process and classify speech utterances. However, understanding the reason behind a classification by DNN is difficult. One such debugging method used with image classification DNNs is activation maximization, which generates example-images that are classified as one o…
We investigated the impact of noisy linguistic features on the performance of a Japanese speech synthesis system based on neural network that uses WaveNet vocoder. We compared an ideal system that uses manually corrected linguistic features including phoneme and prosodic information in training and test sets against a …
The state-of-the-art in text-to-speech synthesis has recently improved considerably due to novel neural waveform generation methods, such as WaveNet. However, these methods suffer from their slow sequential inference process, while their parallel versions are difficult to train and even more expensive computationally. …
WeSinger improves singing voice synthesis with data augmentation and specialized modules.
problem Improving the accuracy and naturalness of synthesized singing voices.
method Developed a multi-singer Chinese neural singing voice synthesis system with deep bi-directional LSTM, Transformer, LPCNet, and data augmentation.
result WeSinger achieves state-of-the-art performance on the Opencpop corpus.
Singing voice conversion is a task to convert a song sang by a source singer to the voice of a target singer. In this paper, we propose using a parallel data free, many-to-one voice conversion technique on singing voices. A phonetic posterior feature is first generated by decoding singing voices through a robust Automa…
Recent progress in deep learning for audio synthesis opens the way to models that directly produce the waveform, shifting away from the traditional paradigm of relying on vocoders or MIDI synthesizers for speech or music generation. Despite their successes, current state-of-the-art neural audio synthesizers such as Wav…
We present a voice conversion solution using recurrent sequence to sequence modeling for DNNs. Our solution takes advantage of recent advances in attention based modeling in the fields of Neural Machine Translation (NMT), Text-to-Speech (TTS) and Automatic Speech Recognition (ASR). The problem consists of converting be…
Investigates neural TTS systems for Japanese and English.
problem Improving neural TTS systems for high-quality speech synthesis.
method Comparative study of neural sequence-to-sequence TTS vs. DNN pipeline TTS, varying model architecture, parameter size, and language.
result A neural sequence-to-sequence TTS system requires sufficient model parameters and a powerful encoder for high-quality speech synthesis.
Neural waveform models such as WaveNet have demonstrated better performance than conventional vocoders for statistical parametric speech synthesis. As an autoregressive (AR) model, WaveNet is limited by a slow sequential waveform generation process. Some new models that use the inverse-autoregressive flow (IAF) can gen…
Most generative models of audio directly generate samples in one of two domains: time or frequency. While sufficient to express any signal, these representations are inefficient, as they do not utilize existing knowledge of how sound is generated and perceived. A third approach (vocoders/synthesizers) successfully inco…
End-to-end speech synthesis is a promising approach that directly converts raw text to speech. Although it was shown that Tacotron2 outperforms classical pipeline systems with regards to naturalness in English, its applicability to other languages is still unknown. Japanese could be one of the most difficult languages …
PriorGrad improves speech synthesis models by using data-dependent adaptive priors.
problem Inefficiency in denoising diffusion models due to mismatch between prior and data distributions.
method Proposes PriorGrad, an adaptive prior derived from data statistics based on conditional information.
result PriorGrad achieves faster convergence and superior performance in speech synthesis models.
MaskCycleGAN-VC improves voice conversion without parallel data.
problem Limited ability to convert mel-spectrogram data without parallel data.
method Integrates a novel auxiliary task called filling in frames (FIF) to learn time-frequency structures.
result MaskCycleGAN-VC outperforms existing methods with similar model size.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
Convolutional Neural Processes improve data efficiency in neural processes.
problem Improving data efficiency in neural processes for small datasets.
method Convolutional Neural Processes (ConvNPs) improve data efficiency by leveraging translation equivariance and convolutional neural networks.
result ConvNPs enhance the performance of neural processes in small-data problems.
Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discrete neural networks (e.g. dropout, Gaussian noise) are missing in current Neural ODE networks. In this paper, we propose a new continuous ne…
Investigates how neural network graph structure impacts predictive performance.
problem Lack of understanding between neural network graph structure and predictive performance.
method Developed relational graph representation to analyze neural networks, identifying a 'sweet spot' for improved performance.
result Identified a 'sweet spot' in relational graph structure that significantly improves neural network predictive performance.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
Investigates neural codes and their embeddings, proving conjectures and introducing new code types.
problem Analyzing neural codes and their embedding dimensions.
method Combinatorial, topological, and algebraic analysis; proving conjectures; introducing new neural code types.
result Proves conjectures about neural codes and their embeddings, introduces new code types.
Neural dynamical systems are dynamical systems that are described at least in part by neural networks. The class of continuous-time neural dynamical systems must, however, be numerically integrated for simulation and learning. Here, we present a compact neural circuit for two common numerical integrators: the explicit …
Quadratic models explain neural network behavior during training.
problem Understanding neural network dynamics during training with large learning rates.
method Developed and tested Neural Quadratic Models.
result Neural Quadratic Models exhibit the 'catapult phase' similar to neural networks.
Graph Metanetworks process diverse neural architectures efficiently.
problem Processing diverse neural architectures efficiently.
method Builds metanetworks using graph neural networks to process graphs representing input neural networks.
result Proves GMNs are expressive and equivariant to parameter permutation symmetries.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
New metric compares noisy neural trajectories using optimal transport.
problem Existing metrics fail to capture differences in noisy, dynamic neural responses.
method Proposed an optimal transport distance metric for Gaussian processes.
result Metric effectively compares neural dynamics in different systems.
The neural tangent kernel equivalence theorem fails in practice.
problem Does the neural tangent kernel (NTK) equivalence theorem hold in practical neural network training?
method Rigorously derived NTK and conducted numerical experiments to evaluate the equivalence theorem.
result Adding a layer to a neural network and the corresponding updated NTK do not yield matching changes in predictor error.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
Use simplified layerwise linear models to understand neural dynamics.
problem Complex neural network dynamics are hard to grasp.
method Apply simplified layerwise linear models to explain neural phenomena.
result Simplified models explain neural collapse, emergence, etc.
Analysis of over-parameterized neural networks has drawn significant attention in recentyears. It was shown that such systems behave like convex systems under various restrictedsettings, such as for two-level neural networks, and when learning is only restricted locally inthe so-called neural tangent kernel space aroun…
The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
We introduce Graph Neural Processes (GNP), inspired by the recent work in conditional and latent neural processes. A Graph Neural Process is defined as a Conditional Neural Process that operates on arbitrary graph data. It takes features of sparsely observed context points as input, and outputs a distribution over targ…
Paper benchmarks quantum neural networks against classical ones for binary classification tasks.
problem Comparing quantum neural networks with classical ones for binary classification.
method Evaluated with two toy examples, focusing on model complexity and training data size.
result EQNN and QNN outperform ENN and DNN for smaller parameter sets and training data samples.