This is the official Pytorch implementation of the paper:
Yizhi Wang and Zhouhui Lian. DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning. SIGGRAPH Asia. 2021.
Paper: arxiv Supplementary: Link Homepage: DeepVecFont
- DeepVecFont can now read data from single files when training, which significantly reduces memory consuming.
- DeepVecFont now better supports training images with resolution of 128 * 128, which should give better results. (the default image size now is 128 * 128, switch back to 64 by setting
image_sizeinoptions.pyto64.) - [To Do] adding Chinese vector font support (will be available soon)
Given a few vector glyphs of a font as reference, our model generates the full vector font:
Input glyphs:
Input glyphs:
Input glyphs:
Rendered:
Rendered:
(Please read notes.txt for more info on this fork. Some are steps omitted for this build specifically. To see full set of instructions, visit the original repository)
- python 3.9
- pip 20.3.4 (or whatever works with python3 .9)
- Pytorch 1.9 (it may work on some lower versions, but not tested; the contextual loss can't work in Pytorch 1.10)
Install pip if you haven't already:
python3.9 -m ensurepip --upgradeInstall pytorch via the instructions:
pip3 install torch torchvision torchaudioInstall other dependencies
pip install tensorboardX scikit-image multiprocess matplotlib(This dataset is a subset from SVG-VAE, ICCV 2019.) There are two modes to access the dataset:
- pkl files: (Download links: Google Drive)
Using pkl files (that combine all fonts) consumes a lot of memory, but you don't need to generate a large number of single files.
Please download the
vecfont_dataset_pklsdir and put it under./data/. If you use this mode, setread_mode=pkl, inoptions.py. If you use the uploaded data, setimage_sizeto64. - dirs:
Generate directories for each data entry, which consumes much less memory.
You need to first download the ttf/otf files from Google Drive and extract it to
data_utils. Then follow the instructions inCustomize your own datasetto process these ttf files. The dataset will be saved with the name ofvecfont_dataset_dirs. If you use this mode, setread_mode=dirs, inoptions.py.
This dataset is too huge so I suggest to create it by yourself (see the details below about creating your own dataset).
Download link: image size 64x64, image size 128x128. Please download the dvf_neural_raster dir and put it under ./experiments/.
Download link: image size 64x64 -> 256x256, image size 128x128 -> 256x256. Please download the image_sr dir and put it under ./experiments/.
Download link: image size 64x64, image size 128x128. Please download the dvf_main_model dir and put it under ./experiments/.
Note that recently we switched from Tensorflow to Pytorch, we may update the models that have better performances.
To train our main model, run
python3.9 main.py --mode train --experiment_name dvf --model_name main_model
The configurations can be found in options.py.
To test our main model, run
python3.9 test_sf.py --mode test --experiment_name dvf --model_name main_model --test_epoch 1200 --batch_size 1 --mix_temperature 0.0001 --gauss_temperature 0.01
This will output the synthesized fonts without refinements. Note that batch_size must be set to 1. The results will be written in ./experiments/dvf_main_model/results/.
To refinement the vector glyphs, run
python3.9 refinement_mp.py --experiment_name dvf --fontid 14 --candidate_nums 20 --num_processes 4
where the fontid denotes the index of testing font. The results will be written in ./experiments/dvf_main_model/results/0014/svgs_refined/. Set num_processes according to your GPU's computation capacity. Setting init_svgbbox_align2img to True could give better results when the initial svg and raster image don't align well.
We have pretrained the neural rasterizer and image super-resolution model. If you want to train them yourself:
To train the neural rasterizer:
python3.9 train_nr.py --mode train --experiment_name dvf --model_name neural_raster
To train the image super-resolution model:
python3.9 train_sr.py --mode train --name image_sr
- Prepare ttf/otf files
Put the ttf/otf files in ./data_utils/font_ttfs/train and ./data_utils/font_ttfs/test, and organize them as 0000.ttf, 0001.ttf, 0002.ttf...
The ttf/otf files in our dataset can be found in Google Drive.
- Install Fontforge
for python > 3.0:
apt-get install python3-fontforge
- Get SFD files via Fontforge
cd data_utils
python3.9 convert_ttf_to_sfd_mp.py --split train
python3.9 convert_ttf_to_sfd_mp.py --split test
- Generate glyph images
python3.9 write_glyph_imgs.py --split train
python3.9 write_glyph_imgs.py --split test
- package them to dirs or pkl
dirs (recommended):
python3.9 write_data_to_dirs.py --split train
python3.9 write_data_to_dirs.py --split test
pkl
python3.9 write_data_to_pkl.py --split train
python3.9 write_data_to_pkl.py --split test
Note:
(1) If you use the mean and stddev files calculated from your own data, you need to retrain the neural rasterizer. For English fonts, just use the mean and stddev files we provided.
If you use this code or find our work is helpful, please consider citing our work:
@article{wang2021deepvecfont,
author = {Wang, Yizhi and Lian, Zhouhui},
title = {DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning},
journal = {ACM Transactions on Graphics},
numpages = {15},
volume={40},
number={6},
month = December,
year = {2021},
doi={10.1145/3478513.3480488}
}