Conducting Systematic Literature Reviews and Systematic Mapping Studies

Context: An essential part of conducting software engineering (SE) research is the ability to identify extant research on tools, technologies, concepts and methods in order to evaluate and make rational and scientific decisions. The domain from which such knowledge is extracted is typically existing research literature found in journals, conference proceedings, books and gray literature. Empirical approaches that include various systematic review (SR) methodologies such as systematic literature review (SLR) and systematic mapping study (SMS) are found to be effective in this context. They adopt rigorous planning, follow repeatable and well-defined processes, and produce unbiased and evidence-based outcomes. Despite these significant benefits, the general trend on using these systematic review (SR) methodologies is not encouraging in SE research. The primary reasons emerging are twofold - a) SR methodologies are largely cited as time-consuming activities and b) lack of guidance to conduct systematic reviews. This tutorial discusses these concerns and describes an effective way of using SR methodologies for SE research.

Objectives: Attendees will be introduced to the key concepts, methods and processes for conducting systematic literature reviews (SLR) and systematic mapping studies (SMS). The benefits, limitations, guidelines for using SR methodologies in an effective manner will discussed in the session. Attendees will be guided on the appropriate formulation of a research question and sub questions; the development of a review protocol such as inclusion criteria, exclusion criteria, quality criteria and classification structures; and execution of review protocol using digital libraries and syntheses of review data. A web based software tool1, for supporting the systematic literature review process will be demonstrated and attendees will get the opportunity to use the tool to conduct the review to help in identification of relevant research and extraction and synthesis of data.

Method: We will use a blend of information presentation, interactive hands-on session and knowledge sharing session. The presentation will introduce the key concepts, benefits, limitations and how to overcome the limitations; hands on session will illustrate a review process with a case study, and finally the knowledge sharing session will discuss the experiences, best practices and the lesson learnt.