[Project page] [Paper] [Arxiv]
Official pytorch implementation for "GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splats"
- 🎆 (24/09/26) We are excited to announce that our paper has been accepted at NeurIPS 2024!
- ⭐ (24/08/29) We just uploaded the pretrained checkpoints (https://drive.google.com/open?id=1S7ct8GGQebBOyjmRvBZW0KkwXBxLsqe6)!
We build our code on EG3D (https://github.com/NVlabs/eg3d) and 3D-GS (https://github.com/graphdeco-inria/gaussian-splatting).
Please dependencies of those two repository to run the codes.
One different thing is that we use different version for "diff-gaussian-rasterization" that support depth and alpha rendering in the link "https://github.com/ashawkey/diff-gaussian-rasterization".
Or you can run below code snippet for installing dependencies.
conda env create -f environment.yml
conda activate gsgan
git clone --recursive https://github.com/ashawkey/diff-gaussian-rasterization
pip install ./diff-gaussian-rasterization
Pretrained checkpoints are available in Google Drive link (https://drive.google.com/open?id=1S7ct8GGQebBOyjmRvBZW0KkwXBxLsqe6).
We exactly follow the data-preprocessing of EG3D, so please refer to them (https://github.com/NVlabs/eg3d).
We provide the training script in "scripts/train{dataset}.sh" file. Only need to change is the "dataset_path" variable. Training results are saved at "outdir" directory.
sh _script/train_afhq512.sh # training on AFHQ512
sh _script/train_ffhq512.sh # training on FFHQ512
We provide the evaluation of FID.
Set the path of "network_pkl" in "_scripts/eval.sh" file properly, and run it.
Or you can simply run below code for evaluation.
python calc_metrics.py --metrics=fid50k_full --network=${network_pkl}
Set the path of "network_pkl" in "_scripts/generate_videos.sh" file properly, and run it.
Or you can simply run below code for generating videos.
python gen_videos.py --outdir=out --trunc=0.7 --seeds=0-3 --grid=2x2 \
--network=${network_pkl}
and we provide some visualization parameters in 338-349 lines in "gen_videos_gsparams.py" as belows.
# Configuration for visualization
gs_params = {
"rendering_scale": 0, # reduce the scale of Gaussians at rendering
"res_visualize": res_visualize, # visualizing specific blocks only (e.g. [0, 1] for two coarsest blocks)
"disable_background": False, # True, for disabling background generator
"opacity_ones": opacity_ones, # True, for setting opacity of Gaussians to 1
"point_index": point_index, # index of the initial point and its children to visualize
"num_init_near_point": num_init_near_point, # number of points to visualize near the initial point (use with "point_index")
"visualize_anchor": visualize_anchor, # visualinzg anchors, not actual Gaussians
'visualize_index': False,
'camera_cond': [[-4.7208e-08, 4.7208e-08, - 1.0000e+00]], # camera direction condition of color layer for visualization (fix for consistent representation)
}
Our generation script outputs the gaussians of the first seed in ply format (.ply), which can be visualized by viewers for gaussian splatting.
So if you want to visualize gaussians with an interactive viewer, you can use this ply file.
Below is an example of visualization with web viewer provided in (https://github.com/antimatter15/splat).
Our code is based on the following repositories.
EG3D (https://github.com/NVlabs/eg3d)
Gaussian-Splatting (https://github.com/graphdeco-inria/gaussian-splatting)
dreamgaussian (https://github.com/dreamgaussian/dreamgaussian)
For any further questions, please contact hse1032@gmail.com
If our paper or codebase is useful in your research, please cite our paper!
@inproceedings{hyungsgan,
title={GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splats},
author={Hyun, Sangeek and Heo, Jae-Pil},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems}
}