Survey: Refinements in Protocol Creation of Systematic Literature Review in Software Engineering
Systematic Literature Reviews are means of finding, critically evaluating and interpreting available research papers (primary studies) on a particular set of research question, topic area, or phenomenon of interest. The methodology is intended to ensure that the literature review is unbiased, rigorous and auditable. The Review Process comprises three main phases: (i) Planning: refers to the pre-review activities, and aims at establishing a review protocol; (ii) Conducting: aims at searching and selecting the studies, in order to extract and synthesize data from them; (iii Reporting: is the final phase, and aims at writing up the results and circulating them to potentially interested parties (Kitchenham et al., 2007). This process may appear to be sequential, but it is important to recognise that many of the stages involve iteration. In particular, many activities are initiated during the protocol development stage, and refined when the review proper takes place (Kitchenham et al., 2007).

The protocol is a critical element of any secondary study (Kitchenham et al., 2007). In general, the protocol goes through many reviews and iterations aiming to improve it. Consequently, items of the protocol can be refined during the various phases and activities of the review process. The review protocol typically include several items, among which the following stand out:

1.   Motivation for conducting the review: describes the main goal and the motivation for the secondary study.
2.   Research Questions (RQs): refer to the research questions that the secondary study is intended to answer.
3.   Search Strategy: refers to the strategy to be used to search for primary studies including search terms and resources to be searched.
4.   Strategy for identifying the primary studies: includes the study selection criteria that are to be used to determine which studies are included in (inclusion criteria), or excluded from (exclusion criteria) the secondary study, as well as how the selection criteria will be applied (e.g. how many assessors will evaluate each prospective primary study, and how disagreements among assessors will be resolved).
5.   Quality assessment criteria: refer to the analyses to define  whether  included studies are robust enough to answer the RQs.
6.   Data extraction form: defines how the information required from each primary study will be obtained and registered.

The aim of this survey is to evaluate which of these items are subject to more refinements throughout the review process. Thus, considering your experience in conducting secondary studies, assign a refinement degree (repetitions for improvements) that it deems necessary to achieve the improvement of each of the aforementioned item of the protocol. Refinement degree scales are presented below.

1. No Refinement
2. Low Degree of Refinement
3. Moderate Degree of Refinement
4. High Degree of Refinement
5. Very High Degree of Refinement

In Figure 1 is presented a hypothetical example of refinement to the "Motivation for conducting the review". For each activity of the review process, a degree of refinement is assigned considering the range of values shown above.

Reference: B.A. Kitchenham and S. Charters. Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE 2007-001, Keele University and Durham University, UK, 2007.
登入 Google 即可儲存進度。瞭解詳情
Figure 1. Example of response about refinements considering the motivation item of the protocol
1. What is your role? *
2. How many secondary studies have you been conducted? *
3. How many of the secondary studies mentioned by you in the previous question have been published in a conference or a journal? *
4. Considering the "Motivation for conducting the review" item, what is the refinement degree of this item in the following activities?
No refinement
Low Degree of Refinement
Moderate Degree of Refinement
High Degree of Refinement
Very High Degree of Refinement
Protocol definition
Pilot test
Study searching
Study selection
Study quality assessment
Data extraction
清除選取的項目
Comments on the question 4?
5. Considering the "Research Question (s)" item, what is the refinement degree of this item in the following activities?
No refinement
Low Degree of Refinement
Moderate Degree of Refinement
High Degree of Refinement
Very High Degree of Refinement
Protocol definition
Pilot test
Study searching
Study selection
Study quality assessment
Data extraction
清除選取的項目
Comments on the question 5?
6. Considering the "Search Strategy" item, what is the refinement degree of this item in the following activities?
No refinement
Low Degree of Refinement
Moderate Degree of Refinement
High Degree of Refinement
Very High Degree of Refinement
Protocol definition
Pilot test
Study searching
Study selection
Study quality assessment
Data extraction
清除選取的項目
Comments on the question 6?
7. Considering the "Strategy for identifying the primary studies" item, what is the refinement degree of this item in the following activities?
No refinement
Low Degree of Refinement
Moderate Degree of Refinement
High Degree of Refinement
Very High Degree of Refinement
Protocol definition
Pilot test
Study searching
Study selection
Study quality assessment
Data extraction
清除選取的項目
Comments on the question 7?
8. Considering the "Quality Assessment" item, what is the refinement degree of this item in the following activities?
No refinement
Low Degree of Refinement
Moderate Degree of Refinement
High Degree of Refinement
Very High Degree of Refinement
Protocol definition
Pilot test
Study searching
Study selection
Study quality assessment
Data extraction
清除選取的項目
Comments on the question 8?
9. Considering the "Data extraction form" item, what is the refinement degree of this item in the following activities?
No refinement
Low Degree of Refinement
Moderate Degree of Refinement
High Degree of Refinement
Very High Degree of Refinement
Protocol definition
Pilot test
Study searching
Study selection
Study quality assessment
Data extraction
清除選取的項目
Comments on the question 9?
提交
清除表單
請勿利用 Google 表單送出密碼。
Google 並未認可或建立這項內容。 檢舉濫用情形 - 服務條款 - 隱私權政策