This is the official codebase for CityCraft: A Diffusion and LLM Powered Framework for Automatic Comprehensive Virtual City Generation.
Development environment:
- OS: Ubuntu 22.04
- CUDA: 11.7
- PyTorch: 2.0.1
Create a new environment and install the requirements:
conda create -n citycraft python=3.10
conda activate citycraft
pip install -r requirements.txtStep 1: Download osm data from "https://huggingface.co/datasets/GearlessJ333/CityCraft/tree/main" and put it in the folder "CityCraft/dataset/CityCraft_OSM/"
cd dataset
unzip layouts.zipAfter this step, you will the the following structure:
CityCraft/
dataset/
CityCraft_OSM/
layouts/
1.png
2.png
3.png
...
Step 2: prepare training features
cd dataset/extract_features
python extract_ratio_features.pyThis will exract latent image features and ratio features for training. After this step, you will have the following structure:
CityCraft_OSM/
layouts/
features/
image_ratio_features/
Step 1: Download assets data from "https://huggingface.co/datasets/GearlessJ333/CityCraft/tree/main" and put it in the folder "CityCraft/dataset/city_render/"
Step 2: Unzip data and features
cd dataset/CityCraft_Buildings
unzip assets.zipThis will extract the assets data and features. After this step, you will have the following structure:
CityCraft_Buildings/
assets/
assets_features/
assets_filtered/
assets_renders/
Step 1: Download pretrained model from "https://huggingface.co/GearlessJ333/CityCraft/tree/main" and put it in the folder "CityCraft/pretrained_models/"
Step 2: Sample from pretrained models
cd layout_generation
python sample_uncondition.pyStep 3: Post-process the generated layouts
cd layout_generation
python utils/post_process.pystep 1 (optional): put your selected layout images in the folder "urban_planning/layouts/", and modify llm_planning.sh to set planning target and requirements.
step 2: set API key in llm_planning.sh
api_key = 'your_api_key'
You need an API key for OpenAI models, you can get one from here.
step 3: start planning
cd urban_planning
sh llm_planning.shAfter step, the plan will be generated in the folder "urban_planning/planning_result/" at the output directory you set in llm_planning.sh.
step 4: select assets
In "select_buildings.sh", set the path to the planning result you want to select assets from, then run:
sh select_buildings.shAfter this step, .pkl file containing the selected assets will be generated in the folder, for example: CityCraft/urban_planning/planning_result/layout_1/selected_buildings.pkl.
step 1: Download Blender Installer
Go back to the root directory CityCraft/:
mkdir BlenderDownload Blender from here, please select the version that matches your operating system, and put it in the folder "CityCraft/Blender/".
(For example, we used here to install Blender 4.0.0 for our Linux x86_64 system; You can put it anywhere but just remember to set the path later when you run the rendering script.)
step 2: Install Blender
cd Blender
tar -xJvf blender-4.0.2-linux-x64.tar.xzstep 3: Download extra assets and textures
Download extra assets and textures from "https://huggingface.co/GearlessJ333/CityCraft/tree/main" and put it in the folder "CityCraft/render_scene/"
cd render_scene
unzip extra_assets.zip
unzip textures.zipAfter this step, you will have the following structure:
render_scene/
extra_assets/
textures/
step 4: Start Rendering
Note: You need to set the path to the layout image and the selected assets pkl file in the render_scene.sh file. You also need to set the path to the Blender executable file to where you installed it.
cd render_scene
sh render_scene.shAfter this step, you will have the rendered scenes in the folder "render_scene/rendered_scenes/".
If you feel our work is helpful, please consider citing our work:
@article{deng2024citycraft,
title={CityCraft: A Real Crafter for 3D City Generation},
author={Deng, Jie and Chai, Wenhao and Huang, Junsheng and Zhao, Zhonghan and Huang, Qixuan and Gao, Mingyan and Guo, Jianshu and Hao, Shengyu and Hu, Wenhao and Hwang, Jenq-Neng and others},
journal={arXiv preprint arXiv:2406.04983},
year={2024}
}