[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content
Springer Nature Link
Log in

PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15059))

Included in the following conference series:

  • 393 Accesses

Abstract

Due to the diversity of assessment requirements in various application scenarios for the IQA task, existing IQA methods struggle to directly adapt to these varied requirements after training. Thus, when facing new requirements, a typical approach is fine-tuning these models on datasets specifically created for those requirements. However, it is time-consuming to establish IQA datasets. In this work, we propose a Prompt-based IQA (PromptIQA) that can fast adapt to new requirements without fine-tuning after training. On one hand, it utilizes a short sequence of Image-Score Pairs (ISP) as prompts for targeted predictions, which significantly reduces the dependency on the data requirements. On the other hand, PromptIQA is trained on a mixed dataset with two proposed data augmentation strategies to learn diverse requirements, thus enabling it to fast adapt to new requirements. Experiments indicate that the PromptIQA outperforms SOTA methods with higher performance and better generalization. The code is available at the link.

Z. Chen and H. Qin—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 89.99
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 109.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Most datasets do not provide the standard deviation values for each MOS as required by the UNIQUE, resulting in incomplete experiment results.

References

  1. Chen, C., et al.: Topiq: a top-down approach from semantics to distortions for image quality assessment. arXiv preprint arXiv:2308.03060 (2023)

  2. Chen, Z., et al.: Gmc-iqa: Exploiting global-correlation and mean-opinion consistency for no-reference image quality assessment. arXiv preprint arXiv:2401.10511 (2024)

  3. Chen, Z., et al.: Teacher-guided learning for blind image quality assessment. In: Proceedings of the Asian Conference on Computer Vision, pp. 2457–2474 (2022)

    Google Scholar 

  4. Ciancio, A., da Silva, E.A., Said, A., Samadani, R., Obrador, P., et al.: No-reference blur assessment of digital pictures based on multifeature classifiers. IEEE Trans. Image Process. 20(1), 64–75 (2010)

    Article  MathSciNet  Google Scholar 

  5. Dosovitskiy, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  6. Fang, Y., Zhu, H., Zeng, Y., Ma, K., Wang, Z.: Perceptual quality assessment of smartphone photography. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3677–3686 (2020)

    Google Scholar 

  7. Ghadiyaram, D., Bovik, A.C.: Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 25(1), 372–387 (2015)

    Article  MathSciNet  Google Scholar 

  8. Guo, C., et al.: Underwater ranker: learn which is better and how to be better. In: Proceedings of the AAAI Conference on Artificial Intelligence (2023)

    Google Scholar 

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  10. Hosu, V., Lin, H., Sziranyi, T., Saupe, D.: Koniq-10k: an ecologically valid database for deep learning of blind image quality assessment. IEEE Trans. Image Process. 29, 4041–4056 (2020)

    Article  Google Scholar 

  11. Jo, B., Cho, D., Park, I.K., Hong, S.: Ifqa: interpretable face quality assessment. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3444–3453 (2023)

    Google Scholar 

  12. Kang, L., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1733–1740 (2014)

    Google Scholar 

  13. Ke, J., Wang, Q., Wang, Y., Milanfar, P., Yang, F.: Musiq: multi-scale image quality transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5148–5157 (2021)

    Google Scholar 

  14. Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging 19(1), 011006 (2010)

    Article  Google Scholar 

  15. Li, C., et al.: Agiqa-3k: An open database for ai-generated image quality assessment. arXiv preprint arXiv:2306.04717 (2023)

  16. Li, D., Jiang, T., Jiang, M.: Unified quality assessment of in-the-wild videos with mixed datasets training. Int. J. Comput. Vision 129, 1238–1257 (2021)

    Article  Google Scholar 

  17. Lin, H., Hosu, V., Saupe, D.: Kadid-10k: A large-scale artificially distorted iqa database. In: 2019 Tenth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–3. IEEE (2019)

    Google Scholar 

  18. Lin, K.Y., Wang, G.: Hallucinated-iqa: no-reference image quality assessment via adversarial learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 732–741 (2018)

    Google Scholar 

  19. Liu, X., Van De Weijer, J., Bagdanov, A.D.: Rankiqa: learning from rankings for no-reference image quality assessment. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1040–1049 (2017)

    Google Scholar 

  20. Liu, Y., et al.: Uiqi: a comprehensive quality evaluation index for underwater images. IEEE Trans. Multimedia

    Google Scholar 

  21. Ma, J., et al.: Blind image quality assessment with active inference. IEEE Trans. Image Process. 30, 3650–3663 (2021)

    Article  Google Scholar 

  22. Ma, K., Liu, W., Liu, T., Wang, Z., Tao, D.: dipiq: blind image quality assessment by learning-to-rank discriminable image pairs. IEEE Trans. Image Process. 26(8), 3951–3964 (2017)

    Article  MathSciNet  Google Scholar 

  23. Madhusudana, P.C., Birkbeck, N., Wang, Y., Adsumilli, B., Bovik, A.C.: Image quality assessment using contrastive learning. IEEE Trans. Image Process. 31, 4149–4161 (2022)

    Article  Google Scholar 

  24. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2012)

    Google Scholar 

  25. Ou, F.Z., et al.: Sdd-fiqa: unsupervised face image quality assessment with similarity distribution distance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7670–7679 (2021)

    Google Scholar 

  26. Ponomarenko, N., et al.: Color image database tid2013: peculiarities and preliminary results. In: European Workshop on Visual Information Processing (EUVIP), pp. 106–111. IEEE (2013)

    Google Scholar 

  27. Qin, G., Hu, R., Liu, Y., Zheng, X., Liu, H., Li, X., Zhang, Y.: Data-efficient image quality assessment with attention-panel decoder (2023)

    Google Scholar 

  28. Saha, A., Mishra, S., Bovik, A.C.: Re-iqa: unsupervised learning for image quality assessment in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5846–5855 (2023)

    Google Scholar 

  29. Salimans, T., et al.: Improved techniques for training gans. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 29. Curran Associates, Inc. (2016). https://proceedings.neurips.cc/paper_files/paper/2016/file/8a3363abe792db2d8761d6403605aeb7-Paper.pdf

  30. Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)

    Article  Google Scholar 

  31. Shi, J., Gao, P., Qin, J.: Transformer-based no-reference image quality assessment via supervised contrastive learning. arXiv preprint arXiv:2312.06995 (2023)

  32. Su, S., et al.: Going the extra mile in face image quality assessment: a novel database and model. IEEE Trans. Multimedia (2023)

    Google Scholar 

  33. Su, S., et al.: Blindly assess image quality in the wild guided by a self-adaptive hyper network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3667–3676 (2020)

    Google Scholar 

  34. Sun, W., Min, X., Tu, D., Ma, S., Zhai, G.: Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training. IEEE J, Selected Topics Signal Process. (2023)

    Google Scholar 

  35. Wang, J., Chan, K.C., Loy, C.C.: Exploring clip for assessing the look and feel of images. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 2555–2563 (2023)

    Google Scholar 

  36. Wang, J., Chan, K.C., Loy, C.C.: Exploring clip for assessing the look and feel of images. In: AAAI (2023)

    Google Scholar 

  37. Wang, J., Duan, H., Liu, J., Chen, S., Min, X., Zhai, G.: Aigciqa2023: a large-scale image quality assessment database for ai generated images: from the perspectives of quality, authenticity and correspondence. arXiv preprint arXiv:2307.00211 (2023)

  38. Wang, J., Chen, Z., Yuan, C., Li, B., Ma, W., Hu, W.: Hierarchical curriculum learning for no-reference image quality assessment. Inter. J. Comput. Vis. 1–20 (2023)

    Google Scholar 

  39. Wang, Z., Ma, K.: Active fine-tuning from gmad examples improves blind image quality assessment. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 4577–4590 (2021)

    MathSciNet  Google Scholar 

  40. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4) (2004)

    Google Scholar 

  41. Wu, H., et al.: Q-bench: a benchmark for general-purpose foundation models on low-level vision. In: ICLR (2024)

    Google Scholar 

  42. Wu, H., et al.: Q-instruct: improving low-level visual abilities for multi-modality foundation models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 25490–25500 (2024)

    Google Scholar 

  43. Wu, H., et al.: Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090 (2023)

  44. Yang, M., Sowmya, A.: An underwater color image quality evaluation metric. IEEE Trans. Image Process. 24(12), 6062–6071 (2015)

    Article  MathSciNet  Google Scholar 

  45. Yang, N., Zhong, Q., Li, K., Cong, R., Zhao, Y., Kwong, S.: A reference-free underwater image quality assessment metric in frequency domain. Signal Process. Image Commun. 94, 116218 (2021)

    Article  Google Scholar 

  46. Yang, S., et al.: Maniqa: multi-dimension attention network for no-reference image quality assessment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1191–1200 (2022)

    Google Scholar 

  47. You, Z., Li, Z., Gu, J., Yin, Z., Xue, T., Dong, C.: Depicting beyond scores: Advancing image quality assessment through multi-modal language models. arXiv preprint arXiv:2312.08962 (2023)

  48. Yuan, J., Cao, X., Cao, L., Lin, J., Cao, X.: Pscr: Patches sampling-based contrastive regression for aigc image quality assessment. arXiv preprint arXiv:2312.05897 (2023)

  49. Zhang, L., Zhang, L., Mou, X., Zhang, D.: Fsim: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    Article  MathSciNet  Google Scholar 

  50. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. IEEE (2018)

    Google Scholar 

  51. Zhang, W., Li, D., Ma, C., Zhai, G., Yang, X., Ma, K.: Continual learning for blind image quality assessment. IEEE Trans. Pattern Anal. Mach. Intell. 45(3), 2864–2878 (2022)

    Google Scholar 

  52. Zhang, W., Ma, K., Yan, J., Deng, D., Wang, Z.: Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans. Circuits Syst. Video Technol. 30(1), 36–47 (2018)

    Article  Google Scholar 

  53. Zhang, W., Ma, K., Zhai, G., Yang, X.: Uncertainty-aware blind image quality assessment in the laboratory and wild. IEEE Trans. Image Process. 30, 3474–3486 (2021)

    Article  Google Scholar 

  54. Zhang, Z., et al.: Q-boost: On visual quality assessment ability of low-level multi-modality foundation models. arXiv preprint arXiv:2312.15300 (2023)

  55. Zhao, K., Yuan, K., Sun, M., Li, M., Wen, X.: Quality-aware pre-trained models for blind image quality assessment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22302–22313 (2023)

    Google Scholar 

  56. Zhu, H., Li, L., Wu, J., Dong, W., Shi, G.: Metaiqa: deep meta-learning for no-reference image quality assessment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14143–14152 (2020)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Science and Technology Major Project (2020AAA0105802), Beijing Natural Science Foundation (JQ21017, L243015, L223003), the Natural Science Foundation of China (Grant No. 62202470, 62122086, 62192782, 62036011, U2033210), the Project of Beijing Science and technology Committee (Project No. Z231100005923046), the Key Research and Development Program of Xinjiang Urumgi Autonomous Region under Grant No.2023B01005, and Bing Li is also supported by Youth Innovation Promotion Association, CAS.

Author information

Authors and Affiliations

  1. State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA, Beijing, China

    Zewen Chen, Haina Qin, Juan Wang, Chunfeng Yuan, Bing Li & Weiming Hu

  2. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

    Zewen Chen, Haina Qin & Weiming Hu

  3. PeopleAl Inc., Beijing, China

    Bing Li

  4. School of Information Science and Technology, ShanghaiTech University, Shanghai, China

    Weiming Hu

  5. OPPO Hardware System, Beijing, China

    Liang Wang

Authors
  1. Zewen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haina Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunfeng Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weiming Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Liang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bing Li .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Germany

    Stefan Roth

  4. Princeton University, Princeton, NJ, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, Z. et al. (2025). PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15059. Springer, Cham. https://doi.org/10.1007/978-3-031-73232-4_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-73232-4_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-73231-7

  • Online ISBN: 978-3-031-73232-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation