Misaligned Image Integration With Local Linear Model
Pages 2035 - 2044
Abstract
We present a new image integration technique for a flash and long-exposure image pair to capture a dark scene without incurring blurring or noisy artifacts. Most existing methods require well-aligned images for the integration, which is often a burdensome restriction in practical use. We address this issue by locally transferring the colors of the flash images using a small fraction of the corresponding pixels in the long-exposure images. We formulate the image integration as a convex optimization problem with the local linear model. The proposed method makes it possible to integrate the color of the long-exposure image with the detail of the flash image without causing any harmful effects to its contrast, where we do not need perfect alignment between the images by virtue of our new integration principle. We show that our method successfully outperforms the state of the art in the image integration and reference-based color transfer for challenging misaligned data sets.
References
[1]
A. Buades, B. Coll, and J.-M. Morel, “A review of image denoising algorithms, with a new one,” Multisc. Model. Simul., vol. 4, no. 2, pp. 490–530, 2005.
[2]
A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” in Proc. IEEE Conf. CVPR, vol. 2. Jun. 2005, pp. 60–65.
[3]
M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736–3745, Dec. 2006.
[4]
P. C. Hansen, J. G. Nagy, and D. P. O’Leary, Deblurring Images: Matrices, Spectra, and Filtering, vol. 3. Philadelphia, PA, USA: SIAM, 2006.
[5]
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080–2095, Aug. 2007.
[6]
A. Beck and M. Teboulle, “Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems,” IEEE Trans. Image Process., vol. 18, no. 11, pp. 2419–2434, Nov. 2009.
[7]
Q. Shan, J. Jia, and A. Agarwala, “High-quality motion deblurring from a single image,” ACM Trans. Graph., vol. 27, no. 3, pp. 73:1–73:10, 2008.
[8]
S. Ono and I. Yamada, “A convex regularizer for reducing color artifact in color image recovery,” in Proc. IEEE Conf. CVPR, Jun. 2013, pp. 1775–1781.
[9]
G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital photography with flash and no-flash image pairs,” ACM Trans. Graph., vol. 23, no. 3, pp. 664–672, Aug. 2004.
[10]
T. Baba, R. Matsuoka, S. Ono, K. Shirai, and M. Okuda, “Flash/no-flash image integration using convex optimization,” in Proc. IEEE ICASSP, May 2014, pp. 1185–1189.
[11]
K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1397–1409, Jun. 2013.
[12]
K. Shirai, M. Ikehara, and M. Okamoto, “Noiseless no-flash photo creation by color transform of flash image,” in Proc. IEEE ICIP, Sep. 2011, pp. 3437–3440.
[13]
S. Zhuo, D. Guo, and T. Sim, “Robust flash deblurring,” in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 2440–2447.
[14]
H.-J. Seo and P. Milanfar, “Robust flash denoising/deblurring by iterative guided filtering,” EURASIP J. Adv. Sig. Process., vol. 2012(1), no. 3, pp. 1–19, 2012.
[15]
R. Matsuoka, T. Baba, M. Okuda, and K. Shirai, “High dynamic range image acquisition using flash image,” in Proc. IEEE ICASSP, May 2013, pp. 1612–1616.
[16]
R. Matsuoka, T. Yamauchi, T. Baba, and M. Okuda, “Weight optimization for multiple image integration,” in Proc. IEEE ICIP, Sep. 2013, pp. 795–799.
[17]
Y. HaCohen, E. Shechtman, D. B. Goldman, and D. Lischinski, “Non-rigid dense correspondence with applications for image enhancement,” ACM Trans. Graph., vol. 30, no. 4, pp. 70:1–70:10, 2011.
[18]
Y. Hwang, J.-Y. Lee, I. S. Kweon, and S. J. Kim, “Color transfer using probabilistic moving least squares,” in Proc. IEEE Conf. CVPR, Jun. 2014, pp. 3342–3349.
[19]
S. Cho and S. Lee, “Fast motion deblurring,” ACM Trans. Graph., vol. 28, no. 5, pp. 145:1–145:8, 2009.
[20]
L. Mai and F. Liu, “Kernel fusion for better image deblurring,” in Proc. IEEE Conf. CVPR, Jun. 2015, pp. 371–380.
[21]
S. Cho, Y. Matsushita, and S. Lee, “Removing non-uniform motion blur from images,” in Proc. IEEE ICCV, Oct. 2007, pp. 1–8.
[22]
L. Yuan, J. Sun, L. Quan, and H.-Y. Shum, “Image deblurring with blurred/noisy image pairs,” ACM Trans. Graph., vol. 26, no. 3, pp. 1:1–1:10, 2007.
[23]
J. Chen, L. Yuan, C.-K. Tang, and L. Quan, “Robust dual motion deblurring,” in Proc. IEEE Conf. CVPR, Jun. 2008, pp. 1–8.
[24]
S. Ono and I. Yamada, “Decorrelated vectorial total variation,” in Proc. IEEE Conf. CVPR, Jun. 2014, pp. 4090–4097.
[25]
E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl., vol. 21, no. 5, pp. 34–41, Sep./Oct. 2001.
[26]
F. Pitié, A. C. Kokaram, and R. Dahyot, “N-dimensional probability density function transfer and its application to color transfer,” in Proc. IEEE ICCV, Oct. 2005, pp. 1434–1439.
[27]
F. Pitié, A. C. Kokaram, and R. Dahyot, “Automated colour grading using colour distribution transfer,” J. Comput. Vis. Image Understand., vol. 107, nos. 1–2, pp. 123–137, Jul./Aug. 2007.
[28]
R. M. H. Nguyen, S. Kim, and M. S. Brown, “Illuminant aware gamut-based color transfer,” Comput. Graph. Forum, vol. 33, no. 7, pp. 319–328, Oct. 2014.
[29]
A. Levin, D. Lischinski, and Y. Weiss, “Colorization using optimization,” ACM Trans. Graph., vol. 23, no. 3, pp. 689–694, 2004.
[30]
R. Irony, D. Cohen-Or, and D. Lischinski, “Colorization by example,” in Proc. Eurograph. Conf. Rendering Techn., 2005, pp. 201–210.
[31]
R. K. Gupta, A. Y.-S. Chia, D. Rajan, E. S. Ng, and H. Zhiyong, “Image colorization using similar images,” in Proc. ACM Int. Conf. Multimedia, 2012, pp. 369–378.
[32]
A. Criminisi, P. Pérez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200–1212, Sep. 2004.
[33]
A. Bugeau, M. Bertalmío, V. Caselles, and G. Sapiro, “A comprehensive framework for image inpainting,” IEEE Trans. Image Process., vol. 19, no. 10, pp. 2634–2645, Oct. 2010.
[34]
C. Liu, J. Yuen, A. Torralba, J. Sivic, and W. T. Freeman, “SIFT flow: Dense correspondence across different scenes,” in Proc. ECCV, 2008, pp. 28–42.
[35]
S. Gould and Y. Zhang, “PatchMatchGraph: Building a graph of dense patch correspondences for label transfer,” in Proc. ECCV, 2012, pp. 439–452.
[36]
C. Barnes, E. Shechtman, D. B. Goldman, and A. Finkelstein, “The generalized PatchMatch correspondence algorithm,” in Proc. ECCV, 2010, pp. 29–43.
[37]
X. Shen, L. Xu, Q. Zhang, and J. Jia, “Multi-modal and multi-spectral registration for natural images,” in Proc. ECCV, 2014, pp. 309–324.
[38]
H.-R. Su and S.-H. Lai, “Non-rigid registration of images with geometric and photometric deformation by using local affine Fourier-moment matching,” in Proc. IEEE Conf. CVPR, Jun. 2015, pp. 2874–2882.
[39]
D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004.
[40]
W.-Y. Lin, L. Liu, Y. Matsushita, K.-L. Low, and S. Liu, “Aligning images in the wild,” in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 1–8.
[41]
A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 2, pp. 228–242, Feb. 2008.
[42]
L. Condat, “A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms,” J. Optim. Theory Appl., vol. 158, no. 2, pp. 460–479, Aug. 2013.
Index Terms
- Misaligned Image Integration With Local Linear Model
Index terms have been assigned to the content through auto-classification.
Recommendations
Learning single image defocus deblurring with misaligned training pairs
AAAI'23/IAAI'23/EAAI'23: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial IntelligenceBy adopting popular pixel-wise loss, existing methods for de-focus deblurring heavily rely on well aligned training image pairs. Although training pairs of ground-truth and blurry images are carefully collected, e.g., DPDD dataset, misalignment is ...
Image Restoration for Linear Local Motion-Blur Based on Cepstrum
ICGEC '12: Proceedings of the 2012 Sixth International Conference on Genetic and Evolutionary ComputingThis paper presents a real-time restoration method for linear local motion-blur image. for an image in which only the fast moving-object is blurred but the background is clear, the proposed basic strategy is to divide such an image into many sub images ...
An image fusion method for misaligned panchromatic and multispectral data
Misalignment between multispectral (MS) and panchromatic (PAN) images blurs fused images and reduces the spectral quality of the latter. When a PAN pixel is misaligned with the corresponding MS pixel that is previously upsampled to match the pixel size ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2016.
Publisher
IEEE Press
Publication History
Published: 01 May 2016
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Jan 2025