Code for CVPR 2019 paper: Zoom to Learn, Learn to Zoom
This paper shows that when applying machine learning to digital zoom for photography, it is beneficial to use real, RAW sensor data for training. This code is based on tensorflow (tested on V 1.13.1). It has been tested on Ubuntu 16.04 LTS.
SR-RAW training and testing now available here.
(If you want to try out without downloading the full train/test dataset, please see the section of quick inference)
To download testing dataset (7 GB), run:
bash ./scripts/download.sh 19zlN1fqRRm7E_6i5J3B1OskJocVeuvzG test.zip
unzip test.zip
rm test.zip
We used 35 mm images (mostly named '00006' in the sequences) for test.
To download training dataset (58 GB), run:
bash ./scripts/download.sh 1qp6z3F4Ru9srwq1lNZr3pQ4kcVN-AOlM train.zip
unzip train.zip
rm train.zip
Our model is trained on raw data in Sony Digital Camera Raw. If you use other types of raw data formats, like DNG used by iPhone (you can use the app Halide to store raw from iPhone), it is necessary to fine tune the model with raw data in that format.
We will download the pre-trained model and example raw data.
git clone https://github.com/ceciliavision/zoom-learn-zoom.git
cd zoom-learn-zoom
bash ./scripts/download.sh 1iForbFhhWqrq22FA1xIusfUpdi8td4Kq model.zip
unzip model.zip
bash ./scripts/download.sh 1WVSGaKIJVHwphTKhcr9ajolEnBh3aUkR quick_inference.zip
unzip quick_inference.zip
rm *.zip
python3 inference.py
Notes about config/inference.yaml
- To do inference on a folder, set
modetoinferenceand setinference_root(e.g../quick_inference/) - To do inference on a single image, set
modetoinference_singleand setinference_path(e.g../quick_inference/00134.ARW) - Set
task_folder(e.g../restore_4x) - Results are saved in
./[task_folder]/[mode]
Coming soon
If you find this work useful for your research, please cite:
@inproceedings{zhang2019zoom
title={Zoom to Learn, Learn to Zoom},
author={Zhang, Xuaner and Chen, Qifeng and Ng, Ren and Koltun, Vladlen},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2019}
}
Please contact me if there is any question (Cecilia Zhang cecilia77@berkeley.edu).