Convert the original text into characters/phonemes, through textfrontend module.
Convert characters/phonemes into acoustic features , such as linear spectrogram, mel spectrogram, LPC features, etc. through Acousticmodels.
Convert acoustic features into waveforms through Vocoders.
When training Tacotron2、TransformerTTS and WaveFlow, we use English single speaker TTS dataset LJSpeech by default. However, when training SpeedySpeech, FastSpeech2 and ParallelWaveGAN, we use Chinese single speaker dataset CSMSC by default.
In the future, PaddleSpeechTTS will mainly use Chinese TTS datasets for default examples.
Here, we will display three types of audio samples:
Audio samples generated from ground-truth spectrograms with a vocoder.
LJSpeech(English)
Text
GT
WaveFlow
Printing, in the only sense with which we are at present concerned, differs from most if not from all the arts and crafts represented in the Exhibition
in being comparatively modern.
For although the Chinese took impressions from wood blocks engraved in relief for centuries before the woodcutters of the Netherlands, by a similar process
produced the block books, which were the immediate predecessors of the true printed book
the invention of movable metal letters in the middle of the fifteenth century may justly be considered as the invention of the art of printing.
Audio samples generated by a TTS system. Text is first transformed into spectrogram by a text-to-spectrogram model, then the spectrogram is converted into raw audio by a vocoder.
LJSpeech(English)
Text
TransformerTTS + WaveFlow
Tacotron2 + WaveFlow
Life was like a box of chocolates, you never know what you're gonna get.
With great power there must come great responsibility.
To be or not to be, that’s a question.
A man can be destroyed but not defeated.
Do not, for one repulse, give up the purpose that you resolved to effort.
Death is just a part of life, something we're all destined to do.
I think it's hard winning a war with words.
Don’t argue with the people of strong determination, because they may change the fact!
PaddleSpeech also support Multi-Speaker TTS, we provide the audio demos generated by FastSpeech2 + ParallelWaveGAN, we use AISHELL-3 Multi-Speaker TTS dataset. Each line is a different person.
In our FastSpeech2, we can control duration, pitch and energy.
We provide the audio demos of duration control here. duration means the duration of phonemes, when we reduce duration, the speed of audios will increase, and when we incerase duration, the speed of audios will reduce.
The duration of different phonemes in a sentence can have different scale ratios (when you want to slow down one word and keep the other words' speed in a sentence). Here we use a fixed scale ratio for different phonemes to control the speed of audios.
The duration control in FastSpeech2 can control the speed of audios will keep the pitch. (in some speech tool, increase the speed will increase the pitch, and vice versa.)
Speed(0.8x)
Speed(1x)
Speed(1.2x)
We provide the audio demos of pitch control here.
When we set pitch of one sentence to a mean value and set tones of phones to 1, we will get a robot-style timbre.
When we raise the pitch of an adult female (with a fixed scale ratio), we will get a child-style timbre.
The pitch of different phonemes in a sentence can also have different scale ratios.
The nomal audios are in the second column of the previous table.
We provide a complete Chinese text frontend module in PaddleSpeechTTS. TextNormalization and G2P are the most important modules in text frontend, We assume that the texts are normalized already, and mainly compare G2P module here.