PyTorch implementation of Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions.
This implementation includes distributed and automatic mixed precision support and uses the LJSpeech dataset.
Distributed and Automatic Mixed Precision support relies on NVIDIA's Apex and AMP.
Visit our website for audio samples using our published Tacotron 2 and WaveGlow models.
- NVIDIA GPU + CUDA cuDNN
- このリポジトリをクローン
git clone https://ko-ma-ki/NVIDIA/tacotron2.git - ダウンロードしたフォルダ内に移動
cd tacotron2 - イニシャライズ(WaveGlowフォルダ内にWaveGlowがあるか確認すること)
git submodule init; git submodule update - condaで適当な仮想環境を作る
- anacondaリポジトリからいろいろインストール
conda install python cmake cython unidecode inflect - PyTorchをインストールする
最新版が使えないなら過去のバージョンからできるだけ新しいものをインストール
1.12.0以降なら多分大丈夫 - Conda-ForgeリポジトリからLibrosaをインストール
conda install -c conda-forge librosa -y - PyOpenJTalkをインストール
pip install git+https://github.com/r9y9/pyopenjtalk.git --no-build-isolation
python train.py --output_directory=outdir --log_directory=logdir
- (OPTIONAL)
tensorboard --logdir=outdir/logdir
Training using a pre-trained model can lead to faster convergence
By default, the dataset dependent text embedding layers are ignored
- Download our published Tacotron 2 model
python train.py --output_directory=outdir --log_directory=logdir -c tacotron2_statedict.pt --warm_start
python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True
- Download our published Tacotron 2 model
- Download our published WaveGlow model
jupyter notebook --ip=127.0.0.1 --port=31337- Load inference.ipynb
N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2 and the Mel decoder were trained on the same mel-spectrogram representation.
WaveGlow Faster than real time Flow-based Generative Network for Speech Synthesis
nv-wavenet Faster than real time WaveNet.
This implementation uses code from the following repos: Keith Ito, Prem Seetharaman as described in our code.
We are inspired by Ryuchi Yamamoto's Tacotron PyTorch implementation.
We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan Wang and Zongheng Yang.