More Web Proxy on the site http://driver.im/

research-article

Evaluating Quality in Use of Corporate Web Sites: An Empirical Investigation

Authors:

Giovanni GuidaAuthors Info & Claims

ACM Transactions on the Web (TWEB), Volume 12, Issue 3

Article No.: 15, Pages 1 - 35

https://doi.org/10.1145/3184646

Published: 17 July 2018 Publication History

Abstract

In our prior work, we presented a novel approach to the evaluation of quality in use of corporate web sites based on an original quality model (QM-U) and a related methodology (EQ-EVAL). This article focuses on two research questions. The first one aims at investigating whether expected quality obtained through the application of EQ-EVAL methodology by employing a small panel of evaluators is a good approximation of actual quality obtained through experimentation with real users. To answer this research question, a comparative study has been carried out involving 5 evaluators and 50 real users. The second research question aims at demonstrating that the adoption of the EQ-EVAL methodology can provide useful information for web site improvement. Three original indicators, namely coherence, coverage and ranking have been defined to answer this question, and an additional study comparing the assessments of two panels of 5 and 10 evaluators, respectively, has been carried out. The results obtained in both studies are largely positive and provide a rational support for the adoption of the EQ-EVAL methodology.

References

[1]

M. Q. Abbasi, J. Weng, Y. Wang, I. Wang, I. Rafique, X. Wang, and P. Lew. 2012. Modeling and evaluating user interface aesthetics: Employing ISO 25010 quality standard. In Proceedings 8th International Conference on Quality of Information and Communications Technology (QUATIC’12). 303--306.

Digital Library

[2]

G. A. Adams and G. R. Frost. 2004. The Development of Corporate Web-sites and Implications for Ethical, Social and Environmental Reporting through these Media. Institute of Chartered Accountants of Scotland, Edinburgh, Scotland, UK.

[3]

A. M. Aladwani and P. C. Palvia. 2002. Developing and validating and instrument for measuring user-perceived web quality. Info. Manage. 39, 6, 467--476.

Digital Library

[4]

A. Albunquerque and A. D. Belchior. 2002. E-commerce websites: A qualitative evaluation. In Proceedings of the International World Wide Web Conference (WWW’02).

[5]

S. J. Barnes and R. T. Vidgen. 2002. An integrative approach to the assessment of e-commerce quality. J. Electron. Commerce Res. 3, 3, 114--127.

[6]

P. Becker, P. Lew, and L. Olsina. 2012. Specifying process views for a measurement, evaluation, and improvement strategy. Adv. Softw. Eng. 2012, Art. 2.

Digital Library

[7]

K. S. Bordens and B. B. Abbott. 2011. Research Design and Methods: A Process Approach. McGraw-Hill, New York, NY.

[8]

G. Büyüközkan, J. Arsenyan, and G. Ertek. 2010. Evaluation of E-Learning Web Sites Using Fuzzy Axiomatic Design Based Approach. Int. J. Comput. Intell. Syst. 3, 1, 24--42.

[9]

D. Campbell and A. C. Beck. 2004. Answering allegations: the use of the corporate website for restorative ethical and social disclosure. Business Ethics: Euro. Rev. 13, 2/3, 100--116.

[10]

A. Fernandez, E. Insfran, and S. Abrahão. 2011. Usability evaluation methods for the web: A systematic mapping study. Info. Softw. Technol. 53, 789--817.

Digital Library

[11]

D. Fogli and G. Guida. 2015. A practical approach to the assessment of quality in use of corporate web sites. J. Syst. Softw. 99, 52--65.

Digital Library

[12]

A. M. Graziano and M. L. Raulin. 2012. Research Methods: A Process of Inquiry (8th ed.). Pearson.

[13]

M. Herrera, M. A. Moraga, L. I. Caballero, and C. Calero. 2010. Quality in use model for web portals (QiUWeP). In Current Trends in Web Engineering, F. Daniel and F. M Facca (Eds.). LNCS 6385 (91--101). Springer, Berlin.

Digital Library

[14]

M. Hertzum and N. E. Jacobsen. 1999. The evaluator effect during first-time use of the cognitive walkthrough technique. In Proceedings Human-Computer Interaction International Conference (HCI’99).

Digital Library

[15]

M. Hertzum and N. E. Jacobsen. 2001. The evaluator effect: A chilling fact about usability evaluation methods. Int. J. Human-Comput. Interact. 13, 421--443.

[16]

M. Hertzum, N. E. Jacobsen, and R. Molich. 2002. Usability inspection by groups of specialists: Perceived agreement in spite of disparate observations. In Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI’02). ACM Press, New York.

Digital Library

[17]

A. Holzinger. 2005. Usability engineering methods for software developers. Commun. ACM 48, 1, 71--74.

Digital Library

[18]

K. Hornbæk and E. Frøkjær. 2008. A Study of the Evaluator Effect in Usability Testing. Human-Comput. Interact. 23, 251--277.

[19]

Y. C. Hu. 2014. Fuzzy multiple-criteria decision making in the determination of critical criteria for assessing service quality of travel websites. Expert Syst. Appl. 36, 3, Part 2, 6439--6445.

Digital Library

[20]

ISO (International Organization for Standardization). 2011. ISO/IEC 25010:2011--System and Software Engineering--Systems and Software Quality Requirements and Evaluation (SQuaRE)--System and Software Quality Models.

[21]

Y. Lee and K. A. Kozar. 2006. Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach. Decis. Support Syst. 43, 1383--1401.

Digital Library

[22]

P. Lew, L. Olsina, and L. Zhang. 2010. Integrating quality, quality in use, actual usability and user experience. In Proceedings of the 6th Central and Eastern European Software Engineering Conference (CEE-SECR’10). 978, 117--123.

[23]

C. T. Liu, T. C. Du, and H. H. Tsai. 2009. A study of the service quality of general portals. Info. Manage. 46, 52--56.

Digital Library

[24]

H. F. Lin. 2010. An application of fuzzy AHP for evaluating course website quality. Comput. Educat. 54, 4, 877--888.

Digital Library

[25]

I. S. MacKenzie. 2013. Human-Computer Interaction: An Empirical Research Perspective. Morgan Kaufmann Publishers Inc., San Francisco, CA.

Digital Library

[26]

G. Malak and H. Sahraoui. 2010. Modeling web quality using a probabilistic approach: An empirical validation. ACM Trans. Web 4, 3, 9:1--9:31.

Digital Library

[27]

N. Manouselis and D. Sampson. 2004. Multiple dimensions of user satisfaction as quality criteria for web portals. In Proceedings of the IADIS WWW/Internet Conference. 535--542.

[28]

L. Mich, M. Franch, and L. Gaio. 2003. Evaluating and designing web site quality. IEEE Multimedia, January--March, 34--43.

Digital Library

[29]

Jacob Nielsen. 1994. Usability Engineering. Academic Press, San Diego.

[30]

J. Nielsen and R. Molich. 1990. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, J. Carrasco Chew and J. Whiteside (Eds.). ACM Press, New York, 249--256.

Digital Library

[31]

N. Nwasra, N. Basir, and M. F. Marhusin. 2015. A framework for evaluating QinU based on ISO/IEC 25010 and 25012 Standards. In Proceedings 2015 9th Malaysian Software Engineering Conference. 70--75.

[32]

J. Offutt. 2002. Quality attributes of web software applications. IEEE Software, March--April, 25--32.

Digital Library

[33]

L. Olsina, F. Papa, and H. Molina. 2008. How to measure and evaluate web applications in a consistent way. In Web Engineering: Modeling and Implementing Web Applications, G. Rossi, O. Pastor, D. Schwabe, and L. Olsina (Eds.). Springer, Berlin, 385--420.

[34]

L. Olsina and G. Rossi. 2002. Measuring web application quality with WebQEM. IEEE Multimedia, October--December, 20--29.

Digital Library

[35]

L. Olsina, P. Lew, A. Dieser, and B. Rivera. 2011. Using web quality models and a strategy for purpose-oriented evaluations. J. Web Eng. 10, 4, 316--352.

Digital Library

[36]

T. Orehovački, A. Graníc, and D. Kermek. 2012. Exploring the quality in use of web 2.0 applications: The case of mind mapping services. In Proceedings of the Conference on Current Trends in Web Engineering (ICWE’12). 266--277. Springer, Berlin.

Digital Library

[37]

T. Orehovački, A. Graníc, and D. Kermek. 2013. Evaluating the perceived and estimated quality in use of Web 2.0 applications. J. Syst. Softw. 86, 3039--3059.

Digital Library

[38]

R. Polillo. 2012. A core quality model for web applications. J. Web Eng. 11, 3, 181--208.

Digital Library

[39]

R. Rekik and I. Kallel. 2013. Fuzz-Web: A methodology based on fuzzy logic for assessing web sites. Int. J. Comput. Info. Syst. Industr. Manage. Appl. 5, 126--136.

[40]

R. Rekik, I. Kallel, J. Casillas, and A. M. Alimi. 2016. Using multiple criteria decision making approaches to assess the quality of web sites. Int. J. Comput. Sci. Info. Secur. 14, 747--761.

[41]

D. Sheskin. 2011. Handbook of Parametric and Nonparametric Statistical Procedures (5th ed.). CRC Press, Boca Raton, FL.

[42]

A. Stefani and M. Xenos. 2008. E-commerce system quality assessment using a model based on ISO 9126 and Belief Networks. Softw. Qual. J. 16, 107--129.

Digital Library

[43]

A. Stefani and M. Xenos. 2011. Wight-modeling of B2C system quality. Comput. Stand. Interf. 33, 4, 411--421.

Digital Library

[44]

D. Straub, M.-C. Boudreau, and D. Gefen. 2004. Validation guidelines for IS positivist research. Commun. Assoc. Info. Syst. 13, 24, 380--427.

[45]

C.-C. Sun and G. T. R. Lin. 2009. Using fuzzy TOPSIS method for evaluating the competitive advantages of shopping websites. Expert Syst. Appl. 36, 9, 11764--11771.

Digital Library

[46]

WAI (Web Accessibility Initiative). 2008. Web Content Accessibility Guidelines 2.0. Retrieved from http://www.w3.org/TR/WCAG/.

Cited By

Salomón SDuque RBringas Sde Oliveira K(2025)Quality‐in‐Use in PracticeJournal of Software: Evolution and Process10.1002/smr.276437:1Online publication date: 22-Jan-2025
https://dl.acm.org/doi/10.1002/smr.2764
Salomón SDuque RMontaña JTenés L(2022)Towards automatic evaluation of the Quality-in-Use in context-aware software systemsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-021-03693-w14:8(10321-10346)Online publication date: 5-Feb-2022
https://doi.org/10.1007/s12652-021-03693-w
Sanabre CPedraza-Jiménez RVinyals-Mirabent S(2020)Double-entry analysis system (DEAS) for comprehensive quality evaluation of websites: case study in the tourism sectorEl profesional de la información10.3145/epi.2020.jul.32Online publication date: 6-Oct-2020
https://doi.org/10.3145/epi.2020.jul.32
Show More Cited By

Index Terms

Evaluating Quality in Use of Corporate Web Sites: An Empirical Investigation
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
2. Information systems
  1. World Wide Web
    1. Web interfaces

Recommendations

A practical approach to the assessment of quality in use of corporate web sites

A novel perspective on web site quality based on stakeholders.A comprehensive and structured model for quality in use of corporate web sites.A practical methodology for the assessment of expected quality in use.An evaluation exercise with seven ...
Quality in use model for web portals (QiUWeP)
ICWE'10: Proceedings of the 10th international conference on Current trends in web engineering

Web Portals are increasingly being used for important tasks, at a work, personal or leisure related issues. Since Web Portals provide with applications, services and information, their levels of quality are an important prerequisite for their success, ...
Assessing Corporate Web Sites: Quality Model and Methodology
Revised Selected Papers of the ICWE 2013 International Workshops on Current Trends in Web Engineering - Volume 8295

The paper presents a new quality model for corporate web sites based on three fundamental concepts: ultimate quality, external quality, and internal quality. External quality is defined in detail through a suitable set of characteristics and sub-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on the Web

ACM Transactions on the Web Volume 12, Issue 3

August 2018

207 pages

ISSN:1559-1131

EISSN:1559-114X

DOI:10.1145/3240924

Editor:
Brian D. Davison
Lehigh University, USA

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2018

Accepted: 01 January 2018

Revised: 01 January 2018

Received: 01 August 2017

Published in TWEB Volume 12, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
202
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)3

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Salomón SDuque RBringas Sde Oliveira K(2025)Quality‐in‐Use in PracticeJournal of Software: Evolution and Process10.1002/smr.276437:1Online publication date: 22-Jan-2025
https://dl.acm.org/doi/10.1002/smr.2764
Salomón SDuque RMontaña JTenés L(2022)Towards automatic evaluation of the Quality-in-Use in context-aware software systemsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-021-03693-w14:8(10321-10346)Online publication date: 5-Feb-2022
https://doi.org/10.1007/s12652-021-03693-w
Sanabre CPedraza-Jiménez RVinyals-Mirabent S(2020)Double-entry analysis system (DEAS) for comprehensive quality evaluation of websites: case study in the tourism sectorEl profesional de la información10.3145/epi.2020.jul.32Online publication date: 6-Oct-2020
https://doi.org/10.3145/epi.2020.jul.32
Salomón SDuque RMontaña JTenés L(2019)A Method for Analyzing the Quality-in-Use in Collaborative ContextsProceedings of the XX International Conference on Human Computer Interaction10.1145/3335595.3335633(1-8)Online publication date: 25-Jun-2019
https://dl.acm.org/doi/10.1145/3335595.3335633

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents