TL;DR:
Introducing RewardSDS, a novel approach that weights noise samples based on alignment scores from a reward model, producing a weighted SDS loss that prioritizes gradients from noise samples that yield aligned high-reward output.
This project has been tested with Python 3.8, CUDA 11.8, and an L40S GPU.
We provide our code for text-based NeRF optimization (with MVDream as the 2D prior) as an extension for Threestudio. To use it, please install threestudio first and then follow the following steps:
# Update path according to your threestudio installation
cp -r threestudio-reward-sds ../threestudio/custom/
cd ../threestudio/custom/threestudio-reward-sds
# First install xformers (https://github.com/facebookresearch/xformers#installing-xformers)
# cuda 11.8 version
pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu118
# cuda 12.1 version
# pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121
# Then install other dependencies
pip install -r requirements.txtIn the threestudio repo:
python launch.py --config custom/threestudio-reward-sds/configs/reward-mvdream-sd21.yaml --train --gpu 0 system.prompt_processor.prompt="A penguin with a brown bag in the snow"Checkout the README for configuration details. In addition, other optimization details (shading, resume from checkpoints, etc.) can be found in the MVDream repo.
We offer a simpler installation than Threestudio with minimal dependencies if you want to run experiments in 2D.
# Create a new conda environment
conda create --name reward-sds -y python=3.8
conda activate reward-sds
pip install --upgrade pip
# Install dependencies
pip install torch==2.1.2+cu118 torchvision==0.16.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118
cd 2D_experiments
pip install -r requirements.txtIn the 2D_experiments directory:
python generate.py --prompt "A white car and a red sheep"See generate.py for more options, including but not limited to:
--reward_strategy,reward_model,n_noises- reward related fields, details in README.--prompt- text prompt for the generated image.--mode- choose between SDS-like loss functions SDS, VSD, sds-bridge.
You may find the evaluation scripts useful for reproducing the results in the paper. They are available in the evaluation directory (follow the TODOs in each script).
This project is based on the following repositories:
Found RewardSDS useful? Please consider citing our work:
@misc{chachy2025rewardsdsaligningscoredistillation,
title={RewardSDS: Aligning Score Distillation via Reward-Weighted Sampling},
author={Itay Chachy and Guy Yariv and Sagie Benaim},
year={2025},
eprint={2503.09601},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2503.09601},
}