Cited By
View all- Akhigbe B(2021)UcEF for Semantic IRAdvanced Concepts, Methods, and Applications in Semantic Computing10.4018/978-1-7998-6697-8.ch010(190-217)Online publication date: 2021
A Toolbox for Information System Evaluation
Pages 283 - 290
Abstract
An information system (IS) measures toolbox is proposed for evaluating implemented IS. The toolbox supports the evaluator in his action from the definition stage to the results one. The software is based on the content, context, process (CCP) framework with adaptations and the IS systemic view examination. An IS evaluation models and measures library is integrated to the toolbox in order to give material to evaluators about previous studies and their results when addressing the main evaluation questions what is being evaluated, with what measures, why evaluation is being done, who is interested by the evaluation and how to conduct evaluation. The toolbox makes possible the cumulative tradition to be carried forward within the discipline.
References
[1]
Bailey E.J. and Pearson S.W. 1983. Development of a Tool for Measuring and Analyzing Computer User Satisfaction. Management Science, vol.29, n. 5, p. 530--545. https://www.jstor.org/stable/2631354
[2]
Baroudi J.J. and Orlikowski W.J., 1988. A Short Form Measure of User Information Satisfaction: a Psychometric Evaluation and Notes on Use. Journal of Management Information Systems, vol.4, n. 4, p. 44--59. https://www.jstor.org/stable/41432887
[3]
Brynjolfsson E. and Hitt L. M. 1996. Paradox Lost? Firmlevel Evidence on the Returns to Information Systems Spending. Management Science, vol.42, p. 541--558. https://pdfs.semanticscholar.org/a89f/4018cc37c1473646dc8cafa106adb9bbe463.pdf
[4]
Brynjolfsson E. and Hitt L. M., 2000. Beyond Computation: Information Technology, Organizational Transformation and Business Performance. The Journal of Economics Perspectives, vol.14, n. 4, p. 23--48. https://www.jstor.org/stable/2647074
[5]
Davis F.D., 1989 Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, vol.13, n. 3, p. 319--340. https://www.jstor.org/stable/249008
[6]
Davis G.B. 2000 Information Systems Conceptual Foundations: Looking Backward and Forward. In: Baskerville R., Stage J., DeGross J.I. (eds) Organizational and Social Perspectives on Information Technology. IFIP --- The International Federation for Information Processing, vol 41. Springer, Boston, MA perspectives on information technology. Springer US, 61--82. org/10.1007/978-0-387-35505-4_5
[7]
Davis, G.B. and Olson, M.H., 1985, Management information systems: Conceptual foundations, structure and development, (2nd edn), McGraw-Hill, New York, N.Y.
[8]
Davis F.D., Venkatesh V., 1996. A critical Assessment of Potential Measurement Biases in the Technology Acceptance Model: Three Experiments. International Journal of Human-Computer Studies, vol.45, n. 1, p. 19--45. https://doi.org/10.1006/ijhc.1996.0040
[9]
Delone W.H. McLean E.R., 1992.Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, vol.3, n. 1, p. 60--95. https://doi.org/10.1287/isre.3.1.60
[10]
Delone W.H., McLean E.R., 2003. Information Systems Success Revisited. The 35th Hawaii International Conference on System Sciences.
[11]
DeLone, W. H., & McLean, E. R. 2016. Information systems success measurement. Foundations and Trends® in Information Systems, 2(1), 1--116. http://dx.doi.org/10.1561/2900000005
[12]
Doll W.J., and Torkzadeh G., 1988. The Measurement of End User Satisfaction. MIS Quarterly, vol.12, n. 2, p. 259--274. https://www.jstor.org/stable/248851
[13]
Doll W.J., and Torkzadeh G., 1998. Developing a Multidimensional Measure of System-Use in an Organizational Context. Information & Management, vol.33, n. 4, p. 171--185. https://doi.org/10.1016/S0378-7206(98)00028-7
[14]
Farbey, B., Land, F., and Targett, D. 1993. IT investment: A study of methods and practices. Management Today. Butterworth-Heinemann Ltd., UK.
[15]
Farbey, B., Land, F. and Targett, D. 1999. Moving IS evaluation forward: learning themes and research issues. The Journal of Strategic Information Systems8.2: 189--207. https://doi.org/10.1016/S0963-8687(99)00021-9
[16]
Goodhue D.L., 1995 Understanding User Evaluation of Information Systems. Management Science, vol.41, n. 12, p. 18--27. https://doi.org/10.1287/mnsc.41.12.1827
[17]
Goodhue,D.L. and Thompson, R. L., 1995 Task-Technology Fit and Individual Performance. MIS Quarterly, vol.19, n. 2, p. 213--236 (1995). https://www.jstor.org/stable/249689
[18]
Grover, V., Jeong S.R. and Segars, A.H. 1996, Information Systems Effectiveness: the Construct Space and Patterns of Application. Information & Management, vol.31, n. 4, p. 177--191. https://doi.org/10.1016/S0378-7206(96)01079-8
[19]
Henderson J.C. and Venkatraman N. 1993, Strategic Alignment: Leveraging Information Technology for Transforming Organizations. IBM Systems Journal, vol.32, n. 1, p. 415.
[20]
Hirschheim R. and Smithson S., 1998. Analyzing Information Systems Evaluation: Another Look at an Old Problem. European Journal of Information Systems, vol.7, n. 3, p. 158--174. https://doi.org/10.1057/palgrave.ejis.3000304
[21]
Irani, Z., and Love, P. E., 2002. Developing a frame of reference for ex-ante IT/IS investment evaluation. European Journal of Information Systems, 11(1), 74--82.
[22]
Ives B., Olson M.H. and Baroudi J. J., 1983. The Measurement of User Information Satisfaction. Communications of the ACM, vol.26, n. 10, p. 785--793.
[23]
Jurison J., 1996. The Temporal Nature of IS Benefits: a Longitudinal Study. Information & Management, vol.30, n. 2, p. 75--79. https://doi.org/10.1016/0378-7206(95)00050-X
[24]
Kettinger W. J. and Lee C.C. 1997, Pragmatic Perspectives on the Measurement of Information Systems Service Quality. MIS Quarterly, vol.21, n. 2, p. 223--240. https://www.jstor.org/stable/249421
[25]
Kohli R. and Grover V. 2008. Business Value of IT: an Essay on Expanding Research Directions to Keep up with the Times. Journal of the Association for Information Systems, vol.9, n. 1, p. 23--39. https://aisel.aisnet.org/jais/vol9/iss1/1
[26]
Markus M.L. and Robey D. 1988. Information Technology and Organizational Change: Casual Structure in Theory and Research. Management Science, vol.34, n. 5, p. 583598. https://doi.org/10.1287/mnsc.34.5.583
[27]
Martinsons M. and Davison R. 1999. The Balanced Score Card: a Foundation for the Strategic Management of Information Systems. Decision Support Systems, vol.25, n. 1, p. 71--87. https://doi.org/10.1016/S0167-9236(98)00086-4
[28]
Mirani R. and Lederer A., 1998. An Instrument for Assessing the Organizational Benefits of IS Projects. Decision Sciences, vol.29, n. 4, p. 803--838 (1998). https://doi.org/10.1111/j.1540-5915.1998.tb00878.x
[29]
Myers M. 1997. Qualitative Research in Information Systems.MISQuarterly, vol.21, n. 2, p. 241--242. http://www.misq.org/misq/downloads/download/editorial/353
[30]
Nelson R.R., Todd P.A. and Wixom B.H., 2005. Antecedents of Information and System Quality: an Empirical Examination Within the Context of Data Warehousing. Journal of Management Information Systems, vol.21, n. 4, p 199--235 (2005). https://doi.org/10.1080/07421222.2005.11045823
[31]
Patton M.Q. 2003. Utilization-Focused Evaluation. In: Kellaghan T., Stufflebeam D. L. (eds) International Handbook of Educational Evaluation. Kluwer International Handbooks of Education, vol 9. Springer, Dordrecht.
[32]
Petter S., Delone W.H. and McLean E.R., 2008. Measuring Information Systems Success: Models, Dimensions, Measures, and Interrelationships. European Journal of Information Systems, vol.17, n. 3, p. 236--263. https://doi.org/10.1057/ejis.2008.15
[33]
Petter, S., DeLone, W. and McLean, E.R., 2012.The past, present, and future of" IS Success". Journal of the Association for Information Systems, 13(5), p.341. https://aisel.aisnet.org/jais/vol13/iss5/2
[34]
Rivard, S., Poirier, G., Raymond, L., and Bergeron, F. 1997.Development of a measure to assess the quality of user-developed applications. ACM SIGMIS Database, 28(3), 44--58.
[35]
Rossi, P. H., Lipsey, M. W., and Howard, E. Freeman, 2004. Evaluation: A Systematic Approach. Sage Publications. ISBN: 0761908943
[36]
Seddon P.B. 1997. A Respecification and Extension of the Delone and McLean Model of IS Success. Information Systems Research, vol.8, n. 3, p. 240--254. https://doi.org/10.1287/isre.8.3.240
[37]
Sethi V. and King W.R., 1994. Development of Measures to Assess the Extent to Which an Information Technology Application Provides Competitive Advantage. Management Science, vol.40, n. 12, p. 1601--1627 (1994). https://doi.org/10.1287/mnsc.40.12.1601
[38]
Soh C. and Markus M.L., 1995. How IT Creates Business Value: a Process Theory Synthesis. Proceedings of the Sixteenth International Conference on Information Systems, Amsterdam, p. 29--41. Doi=10.1.1.88.8687
[39]
Stockdale, R. and Standing, C., 2006. An interpretive approach to evaluating information systems: A content, context, process framework. European journal of operational research, 173(3), pp. 1090--1102. https://doi.org/10.1016/j.ejor.2005.07.006
[40]
Symons V.J. 1991. A Review of Information Systems Evaluation: Content Context and Process. European Journal of Information Systems, vol.1, p. 205--212. https://doi.org/10.1057/ejis.1991.35
[41]
Tate, M., Sedera, D., McLean, E. and Burton-Jones, A., 2011. Information systems success research: the "20-year update?" panel report from PACIS, Communications of the Association for Information Systems, 34, pp. 1235--1246. https://aisel.aisnet.org/cgi/viewcontent.cgi?article=3788&context=cais
[42]
Torkzadeh G. and Doll W.J., 1999. The Development of a Tool for Measuring the Perceived Impact of Information Technology on Work. Omega - The International Journal of Management Science, vol.27, n. 3, p. 327--339 https://doi.org/10.1016/S0305-0483(98)00049-8
[43]
Urbach, N., Smolnik, S., and Riempp, G. 2009.The state of research on information systems success. Business & Information Systems Engineering, 1(4), 315--325.
[44]
Venkatesh V. 2000. Determinants of Perceived Ease of Use: Integrating Perceived Behavioral Control, Computer Anxiety and Enjoyment into the Technology Acceptance Model. Information Systems Research, vol.11, p. 342--365.
[45]
Venkatesh V. and Bala H. 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences, vol.39, n. 2, p. 273--315. https://doi.org/10.1111/j.1540-5915.2008.00192.x
[46]
Venkatesh, V., and Davis, F. D. 1996.A model of the antecedents of perceived ease of use: Development and test. Decision sciences, 27(3), 451--481. https://doi.org/10.1111/j.1540-5915.1996.tb00860.x
[47]
Venkatesh V. and Davis F.D. 2000.Theorical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science, vol.46, n. 2, p. 186--204. https://doi.org/10.1287/mnsc.46.2.186.11926
[48]
Willcocks, L. 1992, Evaluating information technology investments: research findings and reappraisal. Information Systems Journal 2, no. 4: 243--268. https://doi.org/10.1111/j.1365-2575.1992.tb00081.x
[49]
Wixom B.H. and Todd P.A. 2005. A Theorical Integration of User Satisfaction and Technology Acceptance. Information Systems Research, vol.16, n. 1, p. 85--102.
Index Terms
- A Toolbox for Information System Evaluation
Recommendations
Providing a holistic perspective to evaluating e-service quality in web-based systems: a multi-stakeholder approach
SAICSIT '07: Proceedings of the 2007 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countriesOver a period spanning approximately twenty five years IS researchers have been plagued by the problem of how to evaluate IS effectiveness or success. After the advent of the WWW in the 1990s, questions have arisen regarding the relevance of previously ...
Measuring organizational IS effectiveness: an overview and update of senior management perspectives
This paper reports the views of 80 senior IT managers about IT evaluation approaches, and the benefits that IT provides for their organizations. Their views were obtained through a survey mailed to medium to large organizations in both Europe and the ...
IT service management evaluation method based on content, context, and process approach: A literature review
AbstractThe main purpose of implementing Information Technology Service Management (ITSM) is to accomplish high service quality. ITSM evaluation is not an easy matter because of its scale and complexity. Choosing the right evaluation method is essential, ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2019
382 pages
ISBN:9781450371926
DOI:10.1145/3358528
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- Shandong Univ.: Shandong University
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 August 2019
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICBDT2019
ICBDT2019: 2019 2nd International Conference on Big Data Technologies
August 28 - 30, 2019
Jinan, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 138Total Downloads
- Downloads (Last 12 months)13
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
Cited By
View all- Akhigbe B(2021)UcEF for Semantic IRAdvanced Concepts, Methods, and Applications in Semantic Computing10.4018/978-1-7998-6697-8.ch010(190-217)Online publication date: 2021
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in