More Web Proxy on the site http://driver.im/

research-article

Some lessons learned in conducting software engineering surveys in china

Authors:

Reidar Conradi,

Weibing ChenAuthors Info & Claims

ESEM '08: Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement

Pages 168 - 177

https://doi.org/10.1145/1414004.1414033

Published: 09 October 2008 Publication History

Abstract

Component-Based Software Engineering (CBSE) with Open Source Software and Commercial-Off-the-Shelf (COTS) components, Open Source Software (OSS) based development, and Software Outsourcing (SO) are becoming increasingly important for the Chinese software industry. It is therefore necessary to establish pragmatic and possibly nation-specific guidelines for Chinese software companies regarding the use of CBSE, OSS, and SO. Such guidelines should be based on insights from actual practice, which are in our case, obtained through surveys. A European state-of-the-practice survey on COTS- and OSS-oriented CBSE was conducted in Germany, Italy, and Norway in 2004-2005. We repeated similar surveys in China, with an extended survey on OSS and SO. We encountered many difficulties in conducting the surveys, but in most cases managed to find working solutions. We report on the lessons learned while conducting these surveys. In particular, we address issues relating to sampling, contacting respondents, data collection, and data validation. The main lessons are: 1) it was necessary to cooperate with a third-party organization with close relations to Chinese software companies; 2) it was necessary to assign researchers to this third-party organization to facilitate data collection and to control the quality of the data collected; and 3) an email survey, after an initial telephone call to establish contact, was the best method for getting questionnaires completed by Chinese respondents.

References

[1]

Annual Report of China Software Industry. 2005. Ministry of Information of the People's Republic of China & Chinese Software Industry Association, pp. 78--92.

[2]

Blackburn J. D., Scudder G. D., and Van Wassenhove L. N. 1996. Improving Speed and Productivity of Software Development: A Global Survey of Software Developers. IEEE Transactions on Software Engineering, 22(12), pp. 875--885.

Digital Library

[3]

Chen W., Li J., Ma J., Conradi R., Ji J., and Liu C. 2008. A Survey of Software Development with Open Source Components in Chinese Software Industry. Journal of Software Process: Improvement and Practice. 13(1), pp. 89--100.

Digital Library

[4]

Ciolkowski M., Laitenberger O., and Biffl S. 2003. Software Reviews: The State of the Practice. IEEE Software, 20(6), pp. 46--51.

Digital Library

[5]

Conradi R., Li J., Slyngstad O. P. N., Kampenes V. B., Bunse C., Morisio M., and Torchiano M. 2005. Reflections on Conducting an International Survey of Software Engineering. Proc. of the 4th International Symposium on Empirical Software Engineering (Noosa, Australia, November 17-18). IEEE CS Press, pp. 214--223.

[6]

Corra M. and Willer D. 2002. The Gatekeeper. Sociological Theory, 20 (2), pp. 180--207.

[7]

Cusumano M., MacCormack A., Kemerer C. F., and Crandall B. 2003. Software Development Worldwide: The State of the Practice. IEEE Software, 20 (6), pp. 2--8.

Digital Library

[8]

Distribution, Questions and Measures of the Software and Integrate Circuit Industry in China (in Chinese). In: the Journal of Chinese Academy of Social Sciences. Available online at: http://www.edu.cn at 2005-08-17.

[9]

Dybå T. 2003. Factors of Software Process Improvement Success in Small and Large Organizations: An Empirical Study in the Scandinavian Context. Proc. of 11th ACM SIGSOFT international symposium on Foundations of software engineering (Helsinki, Finland, Sept. 2003). ACM Press, pp. 148--157.

Digital Library

[10]

Fowler, F. 2001. Survey Research Methods. 3rd edition, Sage Publications.

[11]

Harris Interactive. 2001. A survey of consumer privacy attitudes and behaviour. Available at http://www.bbbonline.org/UnderstandingPrivacy/library/harrissummary.pdf.

[12]

Hofstede G. 1980. Culture's Consequences: International Differences in Work Related Values. Beverly Hills, CA: Sage.

[13]

Holt J. 1997. Current Practice in Software Engineering: A Survey. Computing and Control Engineering Journal, 8(4), pp. 167--172.

[14]

Jiang S., Lambert E. G., and Wang J. 2006. Issues in Cross-cultural Criminal Justice Survey Research: A Case of U.S. and China Collaboration. Internet Journal of Criminology. Available at: http://www.internetjournalofcriminology.com/ijcprimaryresearch.html.

[15]

Kasunic M. 2006. The State of Software Measurement Practice: Results of 2006 Survey. Technical report, CMU/SEI-2006-TR-009.

[16]

Kshetri N. 2005. Structural Shifts in the Chinese Software Industry. IEEE Software, 22(4), pp. 86--93.

Digital Library

[17]

Lee N. Y. and Litecky C. R. 2003. Generalizing Generalizability in Information Systems Research. Information Systems Research, 14(3), pp. 221--243.

Digital Library

[18]

Lee C. Y. 2004. TQM in Small Manufacturers: An Exploratory Study in China. International Journal of Quality and Reliability Management, 21(2), pp.175---197.

[19]

Lee D. Y. and Dawes P. L. 2005. Guanxi, Trust and Long-term Orientation in Chinese Business Markets. Journal of International Marketing, 13(2), pp. 28--56.

[20]

Li J., Conradi R., Slyngstad O. P. N., Torchiano M., Morisio M., and Bunse C. 2008. A State-of-the-Practice Study on Risk Management in Development with Off-the-Shelf Software Component. IEEE Transactions on Software Engineering, 34(2), pp. 271--286.

Digital Library

[21]

Ma J., Li J., Chen W., Conradi R., Ji J., and Liu C. 2007. An Industrial Survey of Software Outsourcing in China. Proc. of 8th International Conference on Product Focused Software Development and Process Improvement (Riga, Latvia, June, 2007) Springer LNCS, vol. 4589, pp.5--19.

Digital Library

[22]

Manion M. 1994. Survey Research in the Study of Contemporary Culture China: Learning from Local Sampled. The China Quarterly, 131, pp. 741--765.

[23]

Mei H., Chen F., Feng Y., and Yang J. 2003. ABC: An Architecture Based Component Oriented Approach to Software Development. Journal of Software, 14(4). pp. 721--732.

[24]

Neill C. J. and Laplante P. A. 2003. Requirements Engineering: The State of the Practice. IEEE Software, 20 (6), 40--45.

Digital Library

[25]

Pfleeger S. and Kitchenham B. 2001. Principles of Survey Research. 6 parts in ACM SIGSOFT Engineering Notes, Nov. 2001 - March 2003.

Digital Library

[26]

Punter T., Ciolkowski M., Freimut B., and John I. 2003. Conducting On-line Surveys in Software Engineering. Proc. of ACM IEEE Int. Symposium on Empirical Software Engineering (Rome, Italy, September 2003), IEEE CS Press, pp. 80--88.

Digital Library

[27]

Ramaseshan B., Yip L. S. C., and Pae J. A. 2006. Power, Satisfaction and Relationship Commitment in Chinese Store-tenant Relationship and Their Impact on Performance. Journal of Retailing, 82(1), pp. 63--70.

[28]

Reiche B. S. 2006. Key Issues in International Survey Research. Available at: http://www.harzing.com/resources.htm#/intresearch_keyissues.htm.

[29]

Ropponen J. and Lyytinen K. 2000. Components of Software Development Risk: How to Address Them? A Project Manager Survey. IEEE Transactions on Software Engineering, 26(2), pp. 98--111.

Digital Library

[30]

Ross C. E. and Mirowsky J. 1984. Socially-Desirable Response and Acquiescence in a Cross-Culture Survey of Mental Health. Journal of Health and Social Behavior, 25(2), pp. 189--197.

[31]

Roy A., Walters P. G. P., and Luk S. T. K. 2001. Chinese Puzzles and Paradoxes: Conducting Business Research in China. Journal of Business Research, 52(2), pp. 203--210.

[32]

Sampling Terminology. 2007. Available at: www.socialresearchmethods.net/kb/sampterm.htm.

[33]

Shenkar, O. and Glinow M. A. V., 1994. Paradoxes of Organizational Theory and Research: Using the Case of China to Illustrate National Contingency. Management Science, 40(1), pp. 56--71.

Digital Library

[34]

Shin S. K., Ishman M., and Sanders G. L. An Empirical Investigation of Socio-cultural Factors of Information Sharing in China. Journal of Information and Management, 44(2), pp. 165--174.

Digital Library

[35]

The Stand to Define Chinese Software Enterprises (in Chinese). Ministry of Information Industry of the People's Republic of China. Available at: http://www.mii.gov.cn/art/2005/12/22/art_80_3124.html. 2008.

[36]

The average salary of residences in Beijing. 2006. Available at: http://finance.ce.cn/macro/gdxw/200703/28/t20070328_10845636.shtml.

[37]

The resume of Chinese software Industry development in 2000. Available at: http://www.cia.org.cn/h/h29.htm, 2007.

[38]

Triandis H. C., Bontempo R., Villareal M. J., Asai M., and Lucca N. 1988. Individualism and Collectivism: Cross-Cultural Perspectives on Self-Ingroup Relationships, Journal of Personality and Social Psychology, 54 (2), pp. 323--338.

[39]

Wong, J. 2006. China's Economy in 2005: At a New Turning Point and Need to Fix Its Development Problems. China and World Economy, 14(2), pp. 1.

[40]

Xie E., Teo H. H., and Wan W. 2006. Volunteering Personal Information on the Internet: Effects of Reputation, Privacy Notices, and Rewards on Online Consumer Behavior. Marketing Letters, 17(1), pp. 61--74.

[41]

Yang F. 2005. Thinking on the Development of Software Engineering Technology, Journal of software, 16(1), pp. 1--7.

[42]

Yang F., Mei H., Lu J., and Jing Z. 2002. Some Discussion on the Development of Software Technology. Acta Electronica Sinica, 30(12A), pp. 1901--1906.

[43]

Yeung I. Y. M., and Tung R. L. 1996. Achieving Business Success in Confucian Societies: the Importance of Guanxi (connections), Organizational Dynamics, 25 (2), pp. 54--65.

[44]

Zhao X., Flynn B. B., and Roth A. 2006. Decision Sciences Research in China: A Critical Review and Research Agenda - Foundations and Overview. Journal of Decision Science, 37(4), pp. 451--496.

[45]

Zhang Z., Waszink A., and Wijngaard J. 2000. An Instrument for Measuring TQM Implementation for Chinese Manufacturing Companies. International Journal of Quality and Reliability Management, 17(7), pp. 730--755.

Cited By

Ghazi APetersen KReddy SNekkanti H(2019)Survey Research in Software Engineering: Problems and Mitigation StrategiesIEEE Access10.1109/ACCESS.2018.28810417(24703-24718)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2018.2881041
de Mello RTravassos GGenero MJedlitschka AJørgensen MScanniello GSampath SCaivano DPort D(2016)Surveys in Software EngineeringProceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2961111.2962632(1-6)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2961111.2962632
Molléri JPetersen KMendes EGenero MJedlitschka AJørgensen MScanniello GSampath SCaivano DPort D(2016)Survey Guidelines in Software EngineeringProceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2961111.2962619(1-6)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2961111.2962619
Show More Cited By

Index Terms

Some lessons learned in conducting software engineering surveys in china
1. Software and its engineering
  1. Software notations and tools
    1. Software libraries and repositories

Recommendations

Establishment of a National Benchmark of Software Engineering Practices
ISESS '99: Proceedings of the 4th IEEE International Symposium and Forum on Software Engineering Standards

Software engineering process benchmarking is one of the important methodologies in software process assessment and improvement. In collaboration with IBM, IVF has conducted a Swedish national benchmarking survey to derive a national benchmark of ...
Lessons learned from conducting industry surveys in software testing
CESI '14: Proceedings of the 1st International Workshop on Conducting Empirical Studies in Industry

We have conducted a number of industry studies looking into aspects of software testing. These include a survey of practitioner view on software tester performance indicators and team formation issues; detailed worklog analysis of representative tasks ...
Surveys in Software Engineering: Identifying Representative Samples
ESEM '16: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Context: The representativeness of samples in Software Engineering primary studies is still a great challenge, mainly when identifying available sources for establishing adequate sampling frames, characterizing subjects and stimulating their ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEM '08: Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement

October 2008

374 pages

ISBN:9781595939715

DOI:10.1145/1414004

General Chair:
Dieter Rombach
University of Kaiserslautern and Fraunhofer Institute for Experimental Software Engineering (Germany)
,
Program Chairs:
Sebastian Elbaum
University of Nebraska-Lincoln (USA)
,
Jürgen Münch
Fraunhofer Institute for Experimental Software Engineering (Germany)

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEM '08

Sponsor:

ESEM '08: 2008 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement

October 9 - 10, 2008

Kaiserslautern, Germany

Acceptance Rates

Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
630
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ghazi APetersen KReddy SNekkanti H(2019)Survey Research in Software Engineering: Problems and Mitigation StrategiesIEEE Access10.1109/ACCESS.2018.28810417(24703-24718)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2018.2881041
de Mello RTravassos GGenero MJedlitschka AJørgensen MScanniello GSampath SCaivano DPort D(2016)Surveys in Software EngineeringProceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2961111.2962632(1-6)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2961111.2962632
Molléri JPetersen KMendes EGenero MJedlitschka AJørgensen MScanniello GSampath SCaivano DPort D(2016)Survey Guidelines in Software EngineeringProceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2961111.2962619(1-6)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2961111.2962619
de Mello Rda Silva PTravassos G(2015)Investigating probabilistic sampling approaches for large-scale surveys in software engineeringJournal of Software Engineering Research and Development10.1186/s40411-015-0023-03:1Online publication date: 10-Jun-2015
https://doi.org/10.1186/s40411-015-0023-0
Wang YShi H(2009)Software Outsourcing Subcontracting and Its ImpactsProceedings of the 2009 33rd Annual IEEE International Computer Software and Applications Conference - Volume 0110.1109/COMPSAC.2009.42(263-270)Online publication date: 20-Jul-2009
https://dl.acm.org/doi/10.1109/COMPSAC.2009.42

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents