More Web Proxy on the site http://driver.im/

research-article

Open access

A study on the prevalence of human values in software engineering publications, 2015 -- 2018

Authors:

Arif Nurwidyantoro,

Davoud Mougouei,

Rifat Ara Shams,

Gillian OliverAuthors Info & Claims

ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

Pages 409 - 420

https://doi.org/10.1145/3377811.3380393

Published: 01 October 2020 Publication History

Abstract

Failure to account for human values in software (e.g., equality and fairness) can result in user dissatisfaction and negative socio-economic impact. Engineering these values in software, however, requires technical and methodological support throughout the development life cycle. This paper investigates to what extent top Software Engineering (SE) conferences and journals have included research on human values in SE. We investigate the prevalence of human values in recent (2015 -- 2018) publications in these top venues. We classify these publications, based on their relevance to different values, against a widely used value structure adopted from the social sciences. Our results show that: (a) only a small proportion of the publications directly consider values, classified as directly relevant publications; (b) for the majority of the values, very few or no directly relevant publications were found; and (c) the prevalence of directly relevant publications was higher in SE conferences compared to SE journals. This paper shares these and other insights that may motivate future research on human values in software engineering.

References

[1]

Huib Aldewereld, Virginia Dignum, and Yao-hua Tan. 2015. Design for Values in Software Development. Handbook of Ethics, Values, and Technological Design: Sources, Theory, Values and Application Domains (2015), 831--845.

[2]

Julia Angwin, Jeff Larson, Lauren Kirchner, and Surya Mattu. 2016. Machine Bias. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

[3]

Mojtaba Bagherzadeh, Nafiseh Kahani, Cor-Paul Bezemer, Ahmed E Hassan, Juergen Dingel, and James R Cordy. 2018. Analyzing a decade of Linux system calls. Empirical Software Engineering 23, 3 (2018), 1519--1551.

Digital Library

[4]

Nick Baker. 2019. Molly Russell: Instagram Bans Graphic Self-Harm Images After Suicide of UK Teen. https://www.sbs.com.au/news/molly-russell-instagram-bans-graphic-self-harm-images-after-suicide-of-uk-teen

[5]

Juan A Barceló, Florencia Del Castillo Bernal, Ricardo Del Olmo, Laura Mameli, FJ Miguel Quesada, David Poza, and Xavier Vilà. 2014. Social Interaction in Hunter-Gatherer Societies: Simulating the Consequences of Cooperation and Social Aggregation. Social Science Computer Review 32, 3 (2014), 417--436.

Digital Library

[6]

Antonia Bertolino, Antonello Calabrò, Francesca Lonetti, Eda Marchetti, and Breno Miranda. 2018. A Categorization Scheme for Software Engineering Conference Papers and its Application. Journal of Systems and Software 137 (2018), 114--129.

[7]

Corinne Cath, Sandra Wachter, Brent Mittelstadt, Mariarosaria Taddeo, and Luciano Floridi. 2018. Artificial Intelligence and the 'Good Society': the US, EU, and UK Approach. Science and Engineering Ethics 24, 2 (2018), 505--528.

[8]

Common Cause. 2011. The Common Cause Handbook-A Guide to Values and Frames for Campaigners, Community Organisers, Civil Servants, Fundraisers, Educators, Social Entrepreneurs, Activists, Funders, Politicians, and everyone in between. Public Interest Research Centre, Machynlleth, Wales (2011).

[9]

An-Shou Cheng and Kenneth R. Fleischmann. 2010. Developing a Meta-Inventory of Human Values. Proceedings of the American Society for Information Science and Technology 47, 1, 1--10.

[10]

Paresh Dave. 2018. Google Bars Uses of its Artificial Intelligence Tech in Weapons. https://www.reuters.com/article/us-alphabet-ai/google-bars-uses-of-its-artificial-intelligence-tech-in-weapons-idUSKCN1J32M7

[11]

Roel van Dijk, Christophe Creeten, Jeroen van der Ham, and Jeroen van den Bos. 2017. Model-driven Software Engineering in Practice: Privacy-Enhanced Filtering of Network Traffic. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 860--865.

[12]

Amitai Etzioni and Oren Etzioni. 2017. Incorporating Ethics into Artificial Intelligence. The Journal of Ethics 21, 4 (2017), 403--418.

[13]

Robert Feldt. 2016. ISI SE Journals (Ranked). http://www.robertfeldt.net/advice/sevenues/

[14]

Maria Angela Ferrario, Will Simm, Stephen Forshaw, Adrian Gradinar, Marcia Tavares Smith, and Ian Smith. 2016. Values-first SE: Research Principles in Practice. In Companion Proceedings of the 38th International Conference on Software Engineering (ICSE Companion 2016). ACM, 553--562.

Digital Library

[15]

Maria Angela Ferrario, Will Simm, Peter Newman, Stephen Forshaw, and Jon Whittle. 2014. Software Engineering for 'Social Good': Integrating Action Research, Participatory Design, and Agile Development. In Companion Proceedings of the 36th International Conference on Software Engineering (ICSE Companion 2014). ACM, New York, NY, USA, 520--523.

Digital Library

[16]

Mary Flanagan, Daniel C. Howe, and Helen Nissenbaum. 2005. Values at Play: Design Tradeoffs in Socially-Oriented Game Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 751--760.

Digital Library

[17]

Batya Friedman. 1996. Value-sensitive Design. Interactions 3, 6 (1996), 16--23.

Digital Library

[18]

Batya Friedman and Peter H. Kahn Jr. 2007. Human Values, Ethics, and Design. In The Human-computer Interaction Handbook. CRC Press, 1223--1248.

[19]

Robert L. Glass, Iris Vessey, and Venkataraman Ramesh. 2002. Research in Software Engineering: An Analysis of the Literature. Information and Software technology 44, 8 (2002), 491--506.

[20]

Preston Gralla. 2016. Amazon Prime and the Racist Algorithms. https://www.computerworld.com.au/article/599661/amazon-prime-racist-algorithms

[21]

ICSE. 2018. ICSE - Software Engineering in Society. https://www.icse2018.org/track/icse-2018-Software-Engineering-in-Society

[22]

John P.A. Ioannidis, Daniele Fanelli, Debbie Drake Dunne, and Steven N. Goodman. 2015. Meta-Research: Evaluation and Improvement of Research Methods and Practices. PLoS Biology 13, 10 (2015).

[23]

J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics (1977), 159--174.

[24]

Michela Montesi and Patricia Lago. 2008. Software Engineering Article Types: An Analysis of the Literature. Journal of Systems and Software 81, 10 (2008), 1694--1714.

Digital Library

[25]

Rodrigo Morales, Rubén Saborido, Foutse Khomh, Francisco Chicano, and Giuliano Antoniol. 2018. Earmo: An Energy-Aware Refactoring Approach for Mobile Apps. IEEE Transactions on Software Engineering 44, 12 (2018), 1176--1206.

Digital Library

[26]

Davoud Mougouei, Harsha Perera, Waqar Hussain, Rifat Shams, and Jon Whittle. 2018. Operationalizing Human Values in Software: A Research Roadmap. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering - ESEC/FSE 2018. 780--784.

Digital Library

[27]

Md. Saddam Hossain Mukta, Euna Mehnaz Khan, Mohammed Eunus Ali, and Jalal Mahmud. 2017. Predicting Movie Genre Preferences from Personality and Values of Social Media Users. In Eleventh International AAAI Conference on Web and Social Media.

[28]

Mark O. Riedl and Brent Harrison. 2016. Using Stories to Teach Human Values to Artificial Agents. In Workshops at the Thirtieth AAAI Conference on Artificial Intelligence.

[29]

Milton Rokeach. 1973. The Nature of Human Values. Free Press.

[30]

Justin Sablich. 2017. 'Price Gouging' and Hurricane Irma: What Happened and What to Do. https://www.nytimes.com/2017/09/17/travel/price-gouging-hurricane-irma-airlines.html

[31]

Shalom H. Schwartz. 1992. Universals in the Content and Structure of Values: Theoretical Advances and Empirical Tests in 20 Countries. In Advances in Experimental Social Psychology. Vol. 25. Elsevier, 1--65.

[32]

Shalom H. Schwartz. 2005. Basic Human Values: Their Content and Structure Across Countries. Valores e Comportamento nas Organizações (2005), 21--55.

[33]

Shalom H Schwartz. 2006. Basic Human Values: An Overview. Recuperado de http://www.yourmorals.org/schwartz (2006).

[34]

Shalom H. Schwartz. 2012. An Overview of the Schwartz Theory of Basic Values. Online Readings in Psychology and Culture 2, 1 (2012).

[35]

Mary Shaw. 2003. Writing Good Software Engineering Research Papers. In Proceedings of the 25th International Conference on Software Engineering. IEEE, 726--736.

[36]

Dag I.K. Sjøberg, Jo Erskine Hannay, Ove Hansen, Vigdis By Kampenes, Amela Karahasanovic, N.-K. Liborg, and Anette C. Rekdal. 2005. A Survey of Controlled Experiments in Software Engineering. IEEE Transactions on Software Engineering 31, 9 (2005), 733--753.

Digital Library

[37]

David Smith. 2018. Zuckerberg Put on Back Foot as House Grills Facebook CEO over User Tracking. https://www.theguardian.com/technology/2018/apr/11/zuckerberg-hearing-facebook-tracking-questions-house-back-foot

[38]

Igor Steinmacher, Tayana Uchoa Conte, Christoph Treude, and Marco Aurélio Gerosa. 2016. Overcoming Open Source Project Entry Barriers with a Portal for Newcomers. In Proceedings of the 38th International Conference on Software Engineering (ICSE '16). ACM, New York, NY, USA, 273--284.

Digital Library

[39]

Klaas-Jan Stol and Brian Fitzgerald. 2015. A Holistic Overview of Software Engineering Research Strategies. In Proceedings of the Third International Workshop on Conducting Empirical Studies in Industry. IEEE Press, 47--54.

Digital Library

[40]

Tarja Systä, Maarit Harsu, and Kai Koskimies. 2012. Inbreeding in Software Engineering Conferences. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.361.7040

[41]

Christopher Theisen, Marcel Dunaiski, Laurie Williams, and Willem Visser. 2017. Writing Good Software Engineering Research Papers: Revisited. In Proceedings of the 39th International Conference on Software Engineering Companion. IEEE Press, 402--402.

Digital Library

[42]

Sarah Thew and Alistair Sutcliffe. 2018. Value-based Requirements Engineering: Method and Experience. Requirements Engineering 23, 4 (2018), 443--464.

Digital Library

[43]

Bogdan Vasilescu, Alexander Serebrenik, Tom Mens, Mark G.J. van den Brand, and Ekaterina Pek. 2014. How Healthy are Software Engineering Conferences? Science of Computer Programming 89 (2014), 251--272.

[44]

Iris Vessey, Venkataraman Ramesh, and Robert L Glass. 2002. Research in Information Systems: An Empirical Study of Diversity in the Discipline and its Journals. Journal of Management Information Systems 19, 2 (2002), 129--174.

Digital Library

[45]

Roel Wieringa, Neil Maiden, Nancy Mead, and Colette Rolland. 2006. Requirements Engineering Paper Classification and Evaluation Criteria: A Proposal and a Discussion. Requirements Engineering 11, 1 (2006), 102--107.

Digital Library

[46]

Courtney Williams, Margaret-Anne Storey, Neil A Ernst, Alexey Zagalsky, and Eirini Kalliamvakou. 2019. Methodology Matters: How We Study Socio-Technical Aspects in Software Engineering. arXiv preprint arXiv:1905.12841 (2019).

[47]

Emily Winter, Steve Forshaw, and Maria Angela Ferrario. 2018. Measuring Human Values in Software Engineering. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. ACM, 48.

Digital Library

[48]

Carmen Zannier, Grigori Melnik, and Frank Maurer. 2006. On the Success of Empirical Studies in the International Conference on Software Engineering. In Proceedings of the 28th International Conference on Software Engineering. ACM, 341--350.

Digital Library

[49]

Marvin V. Zelkowitz and Dolores Wallace. 1997. Experimental Validation in Software Engineering. Information and Software Technology 39, 11 (1997), 735--743.

Digital Library

Cited By

Gao HZahedi MTreude CRosenstock SCheong M(2024)Documenting Ethical Considerations in Open Source AI ModelsProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686679(177-188)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686679
Jamieson JYamashita NFoong ERoychoudhury APaiva AAbreu RStorey M(2024)Predicting open source contributor turnover from value-related discussions: An analysis of GitHub issuesProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623340(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3623340
Zulkifli SGharib MTaveter K(2024)First Steps Towards Building a Dictionary of Emotional Requirements in Healthcare and Well-Being2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00059(376-384)Online publication date: 24-Jun-2024
https://doi.org/10.1109/REW61692.2024.00059
Show More Cited By

Recommendations

Measuring human values in software engineering
ESEM '18: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Background: Human values, such as prestige, social justice, and financial success, influence software production decision-making processes. While their subjectivity makes some values difficult to measure, their impact on software motivates our research. ...
Towards a Human Values Dashboard for Software Development: An Exploratory Study
ESEM '21: Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

Background: There is a growing awareness of the importance of human values (e.g., inclusiveness, privacy) in software systems. However, there are no practical tools to support the integration of human values during software development. We argue that a ...
Operationalizing human values in software: a research roadmap
ESEC/FSE 2018: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Software influences several aspects of people's lives and therefore needs to reflect their values. However, existing software engineering methods fail to account for human values, which may result in breaching those values in software and, therefore, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

June 2020

1640 pages

ISBN:9781450371216

DOI:10.1145/3377811

General Chairs:
Gregg Rothermel
North Carolina State University
,
Doo-Hwan Bae
KAIST, South Korea

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

KIISE: Korean Institute of Information Scientists and Engineers
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2020

Check for updates

Badges

Artifacts Available / v1.1

Author Tags

Qualifiers

Research-article

Conference

ICSE '20

Sponsor:

SIGSOFT

ICSE '20: 42nd International Conference on Software Engineering

June 27 - July 19, 2020

Seoul, South Korea

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
1,009
Total Downloads

Downloads (Last 12 months)254
Downloads (Last 6 weeks)21

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gao HZahedi MTreude CRosenstock SCheong M(2024)Documenting Ethical Considerations in Open Source AI ModelsProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686679(177-188)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686679
Jamieson JYamashita NFoong ERoychoudhury APaiva AAbreu RStorey M(2024)Predicting open source contributor turnover from value-related discussions: An analysis of GitHub issuesProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623340(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3623340
Zulkifli SGharib MTaveter K(2024)First Steps Towards Building a Dictionary of Emotional Requirements in Healthcare and Well-Being2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00059(376-384)Online publication date: 24-Jun-2024
https://doi.org/10.1109/REW61692.2024.00059
Pant AHoda RTantithamthavorn CTurhan B(2024)Ethics in AI through the practitioner’s view: a grounded theory literature reviewEmpirical Software Engineering10.1007/s10664-024-10465-529:3Online publication date: 6-May-2024
https://dl.acm.org/doi/10.1007/s10664-024-10465-5
Hoda RHoda R(2024)Research Design CanvasQualitative Research with Socio-Technical Grounded Theory10.1007/978-3-031-60533-8_4(61-92)Online publication date: 10-Jun-2024
https://doi.org/10.1007/978-3-031-60533-8_4
Ferrario MWinter E(2023)Applying Human Values Theory to Software Engineering Practice: Lessons and ImplicationsIEEE Transactions on Software Engineering10.1109/TSE.2022.317008749:3(973-990)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TSE.2022.3170087
Kaur JMussbacher G(2023)Based on Past Experience: Highlighting Potential Human Value Issues in Domain Modelling2023 IEEE 31st International Requirements Engineering Conference Workshops (REW)10.1109/REW57809.2023.00054(277-286)Online publication date: Sep-2023
https://doi.org/10.1109/REW57809.2023.00054
Shams RBano MZowghi DLu QWhittle J(2023)Human Value Requirements in AI Systems: Empirical Analysis of Amazon Alexa2023 IEEE 31st International Requirements Engineering Conference Workshops (REW)10.1109/REW57809.2023.00030(138-145)Online publication date: Sep-2023
https://doi.org/10.1109/REW57809.2023.00030
Marczak-Czajka ACleland-Huang J(2023)Using ChatGPT to Generate Human-Value User Stories as Inspirational Triggers2023 IEEE 31st International Requirements Engineering Conference Workshops (REW)10.1109/REW57809.2023.00016(52-61)Online publication date: Sep-2023
https://doi.org/10.1109/REW57809.2023.00016
Sangaroonsilp PDam HGhose AGrundy JPollock LPenta M(2023)On Privacy Weaknesses and Vulnerabilities in Software SystemsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00097(1071-1083)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00097
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents