Abstract
Crowdsourced testing is an emerging testing method in the field of software testing and industrial practice. Crowdsourced testing can provide a more realistic user experience. But crowdsourced workers are independent of each other, they may submit test reports for the same issue, resulting in highly redundant test reports submitted. In addition, crowdsourced test reports with multi-source heterogeneous information tend to have short text descriptions, but the screenshots are rich, and using only text information can lead to information bias in test reports. In view of this, this paper attempts to use the screenshot information in the crowdsourced test report to assist the text information to cluster the crowdsourced test report. Firstly, the text similarity and screenshot similarity of crowdsourced test reports are calculated respectively, then the similarity between crowdsourced test reports is weighted. Finally, test reports are grouped by clustering algorithm based on similarity measure. Testers only need to audit the test report as the representative, which greatly reduces the pressure of the tester’s report audit. The final experimental results show that the effective use of the screenshot information in the test report can achieve higher clustering accuracy.
Supported by National Natural Science Foundation of China (No. 61573362, 61773384, and 61502212), and National Key Research and Development Program of China (No. 2018YFB1003802-01).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yao, X.J., Gong, D.W., Zhang, G.J.: Constrained multi-objective test data generation based on set evolution. IET Softw. 9(4), 103–108 (2015)
Yao, X.J., Gong, D.W., Wang, W.L.: Test data generation for multiple paths based on local evolution. Chin. J. Electron. 24, 46–51 (2015)
Feng, Y., Jones, J.A., Chen, Z.Y., Fang, C.R.: Multi-objective test report prioritization using image understanding. In: ACM International Conference on Automated Software Engineering, pp. 202–213. ACM, Singapore (2016)
Jiang, H., Chen, X., He, T., Chen, Z., Li, X.: Fuzzy clustering of crowdsourced test reports for apps. ACM Trans. Internet Technol. 18(2), 1–28 (2018)
Gong, D.W., Zhang, W.Q., Yao, X.J.: Evolutionary generation of test data for many paths coverage based on grouping. J. Syst. Softw. 84(12), 2222–2233 (2018)
Yao, X.J., Harman, M., Jia, Y.: A study of equivalent and stubborn mutation operators using human analysis of equivalence. In: Proceedings of 36th International Conference on Software Engineering, ICSE, Hyderabad, pp. 919–930 (2014)
Gong, D.W., Yao, X.J.: Testability transformation based on equivalence of target statements. Neural Comput. Appl. 21, 1871–1882 (2012)
Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)
Liu, D., Bias, R.G., Lease, M., Kuipers, R.: Crowdsourcing for usability testing. Proc. Am. Soc. Inf. Sci. Technol. 49(1), 1–10 (2013)
Pastore, F., Mariani, L., Fraser, G.: CrowdOracles: can the crowd solve the oracle problem? In: IEEE 6th International Conference on Software Testing. Verification and Validation, pp. 342–351. IEEE Computer Society, Luxembourg (2013)
Dolstra, E., Vliegendhart, R., Pouwelse, J.: Crowdsourcing gui tests. In: 2013 IEEE 6th International Conference on Software Testing. Verification and Validation, pp. 332–341. IEEE Computer Society, Luxembourg (2013)
Musson, R., Richards, J., Fisher, D., Bird, C., Bussone, B., Ganguly, S.: Leveraging the crowd: how 48,000 users helped improve lync performance. IEEE Softw. 30(4), 38–45 (2013)
Nebeling, M., Speicher, M., Grossniklaus, M., Norrie, M.C.: Crowdsourced web site evaluation with crowdstudy. In: Brambilla, M., Tokuda, T., Tolksdorf, R. (eds.) ICWE 2012. LNCS, vol. 7387, pp. 494–497. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31753-8_52
Runeson, P., Alexandersson, M., Nyholm, O.: Detection of duplicate defect reports using natural language processing. In: International Conference on Software Engineering, pp. 499–510. IEEE Computer Society, Minneapolis (2007)
Wang, X., Zhang, L., Xie, T., Anvik, J., Sun, J.: An approach to detecting duplicate bug reports using natural language and execution information. In: International Conference on Software Engineering, pp. 461–470. IEEE Computer Society, Leipzig (2008)
Jalbert, N., Weimer, W.: Automated duplicate detection for bug tracking systems. In: IEEE International Conference on Dependable Systems and Networks with Ftcs and DCC, pp. 52–61. IEEE, Anchorage (2008)
Tian, Y., Sun, C., Lo, D.: Improved duplicate bug report identification. Softw. Maint. Reengineering 94(3), 385–390 (2012)
Sun, C., Lo, D., Wang, X., Jiang, J., Khoo, S.C.: A discriminative model approach for accurate duplicate bug report retrieval. In: Proceedings of the IEEE International Conference on Software Engineering, pp. 45–54. IEEE Computer Society, Cape Town (2010)
Sun, C., Lo, D., Khoo, S.C., Jiang, J.: Towards more accurate retrieval of duplicate bug reports. In: International Conference on Automated Software Engineering, ASE, USA, pp. 253–262 (2011)
Wang, J., Cui, Q., Wang, Q, Wang, S.: Towards effectively test report classification to assist crowdsourced testing. In: ACM International Symposium on Empirical Software Engineering and Measurement. ACM, Spain (2016)
Wang, J., Wang, S., Cui, Q., Wang, Q.: Local-based active classification of test report to assist crowdsourced testing. In: ACM International Conference on Automated Software Engineering, pp. 190–201. ACM, Singapore (2016)
Feng, Y., Chen, Z., Jones, J.A., Fang, C.R., Xu, B.: Test report prioritization to assist crowdsourced testing. In: Proceeding of the 2015 10th Joint Meeting on Foundations of Software Engineering, pp. 225–236. ACM, Bergamo (2015)
Che, W., Li, Z., Liu, T.: LTP: a Chinese language technology platform. In: International Conference on Computational Linguistics. Demonstrations, pp. 23–27. Chinese Information Processing Society of China, Beijing (2010)
Bhattacharyya, A.: On a measure of divergence between two statistical populations defined by their probability distributions. Bull. Calcutta Math. Soc. 35, 99–109 (1943)
Gopalan, R.P., Krishna, A.: Duplicate bug report detection using clustering. In: International Conference on Software Engineering, pp. 104–109. IEEE Computer Society, Hyderabad (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yang, Y., Yao, X., Gong, D. (2019). Clustering Study of Crowdsourced Test Report with Multi-source Heterogeneous Information. In: Tan, Y., Shi, Y. (eds) Data Mining and Big Data. DMBD 2019. Communications in Computer and Information Science, vol 1071. Springer, Singapore. https://doi.org/10.1007/978-981-32-9563-6_14
Download citation
DOI: https://doi.org/10.1007/978-981-32-9563-6_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-32-9562-9
Online ISBN: 978-981-32-9563-6
eBook Packages: Computer ScienceComputer Science (R0)