Abstract
Improper functioning, or lack, of human cone cells leads to vision defects, making it impossible for affected persons to distinguish certain colors. Colorblind persons have color perception, but their ability to capture color information differs from that of normal people: colorblind and normal people perceive the same image differently. It is necessary to devise solutions to help persons with color blindness understand images and distinguish different colors. Most research on this subject is aimed at adjusting insensitive colors, enabling colorblind persons to better capture color information, but ignores the attention paid by colorblind persons to the salient areas of images. The areas of the image seen as salient by normal people generally differ from those seen by the colorblind. To provide the same saliency for colorblind persons and normal people, we propose a saliency-based image correction algorithm for color blindness. Adjusted colors in the adjusted image are harmonious and realistic, and the method is practical. Our experimental results show that this method effectively improves images, enabling the colorblind to see the same salient areas as normal people.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Ohata, F.; Kudo, H.; Matsumoto, T.; Takeuchi, Y.; Ohnishi, N. Image transform based on the distribution of representative colors for color deficient. IEEJ Transactions on Electronics, Information and Systems Vol. 130, No. 12, 2176–2177, 2010.
Meguro, M.; Taguchi, A. A color conversion method for realizing barrier free of color defective vision. IEEJ Transactions on Electronics, Information and Systems Vol. 131, No. 2, 482–483, 2011.
Yanagida, T.; Okajima, K.; Mimura, H. Color scheme adjustment by fuzzy constraint satisfaction for color vision deficiencies. Color Research & Application Vol. 40, No. 5, 446–464, 2015.
Shen, W. Y.; Mao, X. Y.; Hu, X. H.; Wong, T. T. Seamless visual sharing with color vision deficiencies. ACM Transactions on Graphics Vol. 35, No. 4, Article No. 70, 2016.
Tanuwidjaja, E.; Huynh, D.; Koa, K.; Nguyen, C.; Shao, C.; Torbett, P.; Emmenegger, C.; Weibel, N. Chroma: A wearable augmented-reality solution for color blindness. In: Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing, 799–810, 2014.
Melillo, P.; Riccio, D.; di Perna, L.; Sanniti di Baja, G.; de Nino, M.; Rossi, S.; Testa, F.; Simonelli, F.; Frucci, M. Wearable improved vision system for color vision deficiency correction. IEEE Journal of Translational Engineering in Health and Medicine Vol. 5, 1–7, 2017.
Weale, R. Defective colour vision: Fundamentals, diagnosis and management. British Journal of Ophthalmology Vol. 70, No. 2, 159, 1986.
Rosenstock, H. B.; Swick, D. A. Color discrimination for the color blind. Aerospace Medicine Vol. 45, No. 10, 1194, 1974.
Kessler, J. What can be done for the color blind? Annals of Ophthalmology Vol. 9, No. 4, 431–433, 1977.
Subbian, V.; Ratcliff, J.; Meunier, J.; Korfhagen, J.; Beyette, F.; Shaw, G. Integration of new technology for research in the emergency department: Feasibility of deploying a robotic assessment tool for mild traumatic brain injury evaluation. IEEE Journal of Translational Engineering in Health and Medicine Vol. 3, Article No. 3200109, 2015.
Nakayama, K. Assist device in color discrimination using Heilmeier type guest-host liquid crystal for red-green color vision defect. Electronics and Communications in Japan Vol. 102, No. 8, 17–24, 2019.
Hunt, R. W. G. Colour standards and calculations. In: The Reproduction of Colour. Kriss, M. A.; Hunt, R. John Wiley & Sons, Ltd, 92–125, 2005.
Nathans, J.; Thomas, D.; Hogness, D. Molecular genetics of human color vision: The genes encoding blue, green, and red pigments. Science Vol. 232, No. 4747, 193–202, 1986.
Wong, B. Points of view: Color blindness. Nature Methods Vol. 8, No. 6, 441, 2011.
Scoles, D.; Sulai, Y. N.; Dubra, A. In vivo dark-field imaging of the retinal pigment epithelium cell mosaic. Biomedical Optics Express Vol. 4, No. 9, 1710, 2013.
Brettel, H.; Vi´enot, F.; Mollon, J. D. Computerized simulation of color appearance for dichromats. Journal of the Optical Society of America A Vol. 14, No. 10, 2647, 1997.
Meyer, G. W.; Greenberg, D. P. Color-defective vision and computer graphics displays. IEEE Computer Graphics and Applications Vol. 8, No. 5, 28–40, 1988.
MacHado, G. M.; Oliveira, M. M.; Fernandes, L. A physiologically-based model for simulation of color vision deficiency. IEEE Transactions on Visualization and Computer Graphics Vol. 15, No. 6, 1291–1298, 2009.
Chen, C. S.; Wu, S. Y.; Huang, J. B. Enhancing color representation for the color vision impaired. In: Proceedings of ECCV Workshop on Computer Vision Applications for the Visually Impaired, 2008.
Okajima, K.; Kanbe, S. A real-time color simulation of dichromats. Technical Report of the IEICE, 107: 107110, 2007.
Flatla, D. R.; Gutwin, C. Individual models of color differentiation to improve interpretability of information visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2563–2572, 2010.
Kuhn, G. R.; Oliveira, M. M.; Fernandes, L. An efficient naturalness-preserving image-recoloring method for dichromats. IEEE Transactions on Visualization and Computer Graphics Vol. 14, No. 6, 1747–1754, 2008.
Jiang, H. Z.; Wang, J. D.; Yuan, Z. J.; Wu, Y.; Zheng, N. N.; Li, S. P. Salient object detection: A discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2083–2090, 2013.
Peng, H. W.; Li, B.; Ling, H. B.; Hu, W. M.; Xiong, W. H.; Maybank, S. J. Salient object detection via structured matrix decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 4, 818–832, 2017.
Cong, R. M.; Lei, J. J.; Fu, H. Z.; Huang, Q. M.; Cao, X. C.; Hou, C. P. Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation. IEEE Transactions on Image Processing Vol. 27, No. 2, 568–579, 2018.
Cheng, M. M.; Mitra, N. J.; Huang, X. L.; Torr, P. H. S.; Hu, S. M. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 3, 569–582, 2015.
Wang, W. G.; Shen, J. B.; Shao, L.; Porikli, F. Correspondence driven saliency transfer. IEEE Transactions on Image Processing Vol. 25, No. 11, 5025–5034, 2016.
Borji, A.; Cheng, M. M.; Jiang, H. Z.; Li, J. Salient object detection: A benchmark. IEEE Transactions on Image Processing Vol. 24, No. 12, 5706–5722, 2015.
Wang, L. J.; Lu, H. C.; Ruan, X.; Yang, M. H. Deep networks for saliency detection via local estimation and global search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3183–3192, 2015.
Zhao, R.; Ouyang, W. L.; Li, H. S.; Wang, X. G. Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1265–1274, 2015.
Han, J. W.; Zhang, D. W.; Hu, X. T.; Guo, L.; Ren, J. C.; Wu, F. Background prior-based salient object detection via deep reconstruction residual. IEEE Transactions on Circuits and Systems for Video Technology Vol. 25, No. 8, 1309–1321, 2015.
Huang, X.; Shen, C. Y.; Boix, X.; Zhao, Q. SALICON: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, 262–270, 2015.
Chen, T. S.; Lin, L.; Liu, L. B.; Luo, X. N.; Li, X. L. DISC: Deep image saliency computing via progressive representation learning. IEEE Transactions on Neural Networks and Learning Systems Vol. 27, No. 6, 1135–1149, 2016.
Zhang, J. M.; Sclaroff, S.; Lin, Z.; Shen, X. H.; Price, B.; Mech, R. Unconstrained salient object detection via proposal subset optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5733–5742, 2016.
Li, G. B.; Xie, Y.; Lin, L.; Yu, Y. Z. Instance-level salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2386–2395, 2017.
Wang, W. G.; Shen, J. B.; Xie, J. W.; Cheng, M. M.; Ling, H. B.; Borji, A. Revisiting video saliency prediction in the deep learning era. IEEE Transactions on Pattern Analysis and Machine Intelligence DOI: 10.1109/TPAMI.2019.2924417, 2019.
Wang, W. G.; Shen, J. B.; Yang, R. G.; Porikli, F. Saliency-aware video object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 40, No. 1, 20–33, 2018.
Wei, Y. C.; Feng, J. S.; Liang, X. D.; Cheng, M. M.; Zhao, Y.; Yan, S. C. Object region mining with adversarial erasing: A simple classification to semantic segmentation approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1568–1576, 2017.
Wei, Y. C.; Liang, X. D.; Chen, Y. P.; Shen, X. H.; Cheng, M. M.; Feng, J. S.; Zhao, Y.; Yan, S. STC: A simple to complex framework for weakly-supervised semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 11, 2314–2320, 2017.
Wang, W. G.; Shen, J. B.; Ling, H. B. A deep network solution for attention and aesthetics aware photo cropping. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 41, No. 7, 1531–1544, 2019.
Sun, J.; Ling, H. B. Scale and object aware image retargeting for thumbnail browsing. In: Proceedings of the International Conference on Computer Vision, 1511–1518, 2011.
Zhou, L.; Yang, Z. H.; Zhou, Z. T.; Hu, D. W. Salient region detection using diffusion process on a two-layer sparse graph. IEEE Transactions on Image Processing Vol. 26, No. 12, 5882–5894, 2017.
Mateescu, V. A.; Bajic, I. V. Visual attention retargeting. IEEE MultiMedia Vol. 23, No. 1, 82–91, 2016.
Nguyen, T. V.; Ni, B.; Liu, H.; Xia, W.; Luo, J.; Kankanhalli, M.; Yan, S. Image re-attentionizing. IEEE Transactions on Multimedia Vol. 15, No. 8, 1910–1919, 2013.
Mateescu, V. A.; Baji´c, I. V. Attention retargeting by color manipulation in images. In: Proceedings of the 1st International Workshop on Perception Inspired Video Processing, 15–20, 2014.
Mendez, E.; Feiner, S.; Schmalstieg, D. Focus and context in mixed reality by modulating first order salient features. In: Smart Graphics. Lecture Notes in Computer Science, Vol. 6133. Taylor, R.; Boulanger, P.; Kru¨ger, A.; Olivier, P. Eds. Springer Berlin Heidelberg, 232–243, 2010.
Lu, S. P.; Dauphin, G.; Lafruit, G.; Munteanu, A. Color retargeting: Interactive time-varying color image composition from time-lapse sequences. Computational Visual Media Vol. 1, No. 4, 321–330, 2015.
Fried, O.; Shechtman, E.; Goldman, D. B.; Finkelstein, A. Finding distractors in images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1703–1712, 2015.
Su, S. L.; Durand, F.; Agrawala, M. De-emphasis of distracting image regions using texture power maps. In: Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, 164, 2005.
Li, J. J.; Li, G. H.; Fan, H. Image dehazing using residual-based deep CNN. IEEE Access Vol. 6, 26831–26842, 2018.
Mechrez, R.; Shechtman, E.; Zelnik-Manor, L. Saliency driven image manipulation. Machine Vision and Applications Vol. 30, No. 2, 189–202, 2019.
Huang, J. B.; Chen, C. S.; Jen, T. C.; Wang, S. J. Image recolorization for the colorblind. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1161–1164, 2009.
Lin, H. Y.; Chen, L. Q.; Wang, M. L. Improving discrimination in color vision deficiency by image re-coloring. Sensors Vol. 19, No. 10, 2250, 2019.
Sundaram, N.; Brox, T.; Keutzer, K. Dense point trajectories by GPU-accelerated large displacement optical flow. In: Computer Vision–ECCV 2010. Lecture Notes in Computer Science, Vol. 6311. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 438–451, 2010.
Yang, C.; Zhang, L. H.; Lu, H. C.; Ruan, X.; Yang, M. H. Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3166–3173, 2013.
Zhang, S. Y.; Liang, R. Z.; Wang, M. ShadowGAN: Shadow synthesis for virtual objects with conditional adversarial networks. Computational Visual Media Vol. 5, No. 1, 105–115, 2019.
Acknowledgements
The authors acknowledge the National Natural Science Foundation of China (Grant Nos. 61772319, 61976125, 61873177, and 61773244), and Shandong Natural Science Foundation of China (Grant No. ZR2017MF049). We thank the editors and anonymous reviewers for their comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Jinjiang Li received his B.S. and M.S. degrees in computer science from Taiyuan University of Technology, Taiyuan, China, in 2001 and 2004, respectively, his Ph.D. degree in computer science from Shandong University, Jinan, China, in 2010. From 2004 to 2006, he was an assistant research fellow at the Institute of Computer Science and Technology of Peking University, Beijing, China. From 2012 to 2014, he was a post-doctoral fellow at Tsinghua University, Beijing, China. He is currently a professor at the School of Computer Science and Technology, Shandong Technology and Business University. His research interests include image processing, computer graphics, computer vision, and machine learning.
Xiaomei Feng received her B.S. degree in School of Computer Science and Technology from Qilu Normal University, Jinan, China in 2018. Currently, she is an M.S. degree candidate in the School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai, China. Her research interests include computer graphics, computer vision, and image processing.
Hui Fan received his B.S. degree in computer science from Shandong University, Jinan, China, in 1984. He received his Ph.D. degree in computer science from Taiyuan University of Technology, Taiyuan, China, in 2007. From 1984 to 2001, he was a professor at the Computer Department of Taiyuan University Technology. He is currently a professor at Shandong Technology and Business University. His research interests include computer aided geometric design, computer graphics, information visualization, virtual reality, and image processing.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Li, J., Feng, X. & Fan, H. Saliency-based image correction for colorblind patients. Comp. Visual Media 6, 169–189 (2020). https://doi.org/10.1007/s41095-020-0172-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-020-0172-x