Evaluating the content validity of two versions of an instrument used in measuring pediatric pain knowledge and attitudes in the Ghanaian context.

In this article, we compared the content validity of two instruments used in measuring pediatric pain knowledge and attitudes. This was considered necessary due to the universal differences in culture, semantics and healthcare resources in different parts of the globe. Thirteen (13) pediatric experts in Ghana assessed the content validity of two instruments: the 42-item Pediatric Nurses' Knowledge and Attitudes Survey Regarding Pain (PNKAS) and the 41-item Pediatric Healthcare Providers' Knowledge and Attitudes Survey Regarding Pain (PHPKASRP). The relevance and clarity of each item on these instruments were rated on a four-point likert scaled options from 1 (not relevant/ not clear) to 4 (very relevant/ very clear). The item-level content validity index (I-CVI) was calculated by dividing the number of experts who rated an item with 3 or 4 by the total number of experts. The average scale-level content validity index (S-CVI/Ave) was also estimated by summing up the I-CVIs of all items and dividing them by the total number of items. The I-CVIs on the PNKAS ranged from 0.62 to 1.00 for the relevance component and 0.69 to 1.00 for the clarity component. The I-CVIs on the PHPKASRP ranged from 0.62 to 1.00 for both the relevance and clarity components. The S-CVI/Ave were 0.87 and 0.89 for the relevance and clarity aspects on the PNKAS respectively. The S-CVI/Ave for the PHPKASRP instrument were 0.86 and 0.89 for the relevance and clarity aspects correspondingly. At the end of the validation process, 5 items were revised on both instruments whilst 37 and 36 items were maintained on the PNKAS and PHPKASRP instruments respectively. The PNKAS and PHPKASRP have an acceptable level of content validity in the Ghanaian context and recommended for educational and research purposes. Other forms of validity and reliability of these instruments should also be examined in future studies.

Introduction

Unrelieved pediatric pain remains a significant health problem globally despite decades of research on pain and advanced technologies [1-3]. Inadequately treated pain does not only affect the child-in-pain but also their family and society at large. Earlier studies have demonstrated the untoward consequences of untreated or inadequately treated pain on children to include impaired physical function, emotional disturbances [4], social isolation [5], delayed recovery, prolonged hospitalization, increased cost of healthcare, and development of chronic pain which decreases their quality of life [6-8]. The socio-economic burden of this menace on families and societies has also been documented in the pain literature [9, 10].

Nurses and other healthcare providers (HCPs) play a significant role in the assessment and management of hospitalized children’s pain [11]. The assessment of HCPs’ knowledge and attitudes regarding children’s pain is important as it serves as the bedrock of their pain assessment and management practices in this vulnerable population [12]. Considering the importance of assessing HCPs’ knowledge and attitudes regarding children’s pain, a number of instruments have been developed to measure this construct [13, 14]. Key among them is the Pediatric Nurses’ Knowledge and Attitudes Survey regarding pain (PNKAS) [13]. According to the developer (Manworren, R. C. B. via email communication), the PNKAS was revised in 2014 to reflect the diverse roles performed by the multidisciplinary healthcare team involved in providing pain care for pediatric patients; this occasioned the development of the Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (PHPKASRP).

Following the development of the PNKAS instrument, its validity and reliability has been assessed [13]. Face and content validity of the instrument was established by five pain management experts in the United States of America. Internal consistency of the PNKAS instrument has also been assessed by two distinct groups with a reported acceptable Cronbach’s alpha value of 0.72 among 247 pediatric nurses and 0.77 among 88 members of a children’s nursing organization. Test-retest reliability analysis among 12 clinicians (6 nurses and 6 child life specialists) recorded a correlation coefficient of 0.67, signifying an acceptable level of instrument stability. The PNKAS instrument has been modified to suit nurses taking care of children who do not have any form of malignancies; this version has been termed as the Modified Mongolian Pediatric Nurses’ Knowledge and Attitudes Survey-Shriner’s version (MMPNKAS-S) [15]. The instrument has also been translated into the Norwegian language and has demonstrated an acceptable linguistic validity [16].

In lieu of diverse roles performed by the multidisciplinary pediatric pain team, the revised version (PHPKASRP) became necessary. Presently, there are no published findings on the psychometric properties of the PHPKASRP. In spite of this, the instrument has been used to assess changes in healthcare providers’ knowledge and attitudes following a multidisciplinary educational intervention program [17].

Validity is not a property of an instrument but dependent on the interpretation or purpose of an instrument with particular context and participants [18] due to universal differences in culture, semantics and resources in different parts of the globe [19, 20]. It appears from the review of relevant literature that, the content validity of these two instruments have not been assessed from a low-middle income country’s context. As part of plans to use a pediatric pain knowledge and attitude instrument as a tool in assessing the effectiveness of a short-course educational program for nursing students and nurses, the current study sought to evaluate and compare the content validity of both the original PNKAS instrument and its revised version, PHPKASRP from a Ghanaian perspective.

Development of the PNKAS and PHPKASRP instruments

The Pediatric Nurses' Knowledge and Attitudes Survey Regarding Pain (PNKAS) is a modification of Nurses' Knowledge and Attitudes Survey Regarding Pain (NKASRP) instrument developed by Ferrell and McCaffery in 1987 [13]. As the name implies, it was originally developed to measure nurses’ knowledge and attitudes toward patients in pain [21]. The content of the NKASRP tool was derived from the prevailing standards of pain management from organizational bodies such as the American Pain Society, the World Health Organization, and the National Comprehensive Cancer Network Pain Guidelines. The tool since its development has undergone revisions to reflect changes in pain management. The NKASRP consists of 39 items and take about 25–30 minutes to be completed. The instrument has been proven to be valid and reliable in different settings [22, 23].

In order to develop a tool which would be more sensitive to pediatric patients, Manworren developed a new survey in 1998 called the “Pediatric Nurses’ Knowledge and Attitudes Survey Regarding Pain” (PNKAS) based on the original work of McCaffery and Ferrell [24]. Three major changes were executed by Manworren in order to change the focus of the NKASRP tool from adults to infants, children and adolescents. The first amendment was adding three procedural pain items (questions 8, 14, and 21) to the original survey. Secondly, she modified questions that were related to meperidine and aspirin to other recommended analgesics due to the effects of these analgesics on the pediatric population. Aspirin increases the risk of Reye’s syndrome [25], whereas meperidine has toxic metabolic effects [26], hence, they are contraindicated in children’s pain management [13]. Thirdly, the dosage of analgesics in the original questionnaire was adjusted to suit paediatric patients. For instance, morphine dosages (in question number 26), were adjusted to fit the paediatric population. The 42-item PNKAS is a self-administered instrument that assesses nurses’ knowledge and attitudes regarding pain assessment and management in the pediatric population [24]. It comprises of 25 binary response-type questions (True/ False), 13 multiple choice questions (MCQs), and two case studies extended into four MCQs.

The Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (PHPKASRP) is a revised instrument developed from the Pediatric Nurses' Knowledge and Attitudes Survey Regarding Pain (PNKAS). In 2014, the 41-item self-administered revised version (PHPKASRP instrument) was developed to reflect the different roles of the multidisciplinary healthcare team involved in providing pain care for pediatric patients. The revised instrument which takes about 25–30 minutes to complete consists of 25 binary response-type questions (True/ False), 12 multiple choice questions (MCQs), and two case studies extended into four MCQs.

Materials and methods

Study design, sample and setting

This descriptive methodological study is part of a larger project that sought to examine the educational needs on pediatric pain management in Ghana. Based on the recommended minimum number of eight to 12 experts [27], the researchers physically contacted 15 pediatric experts to assess the content validity of both the PNKAS and PHPKASRP instruments from a Ghanaian perspective. The experts were given hard copies of these instruments and had to complete them within two to four weeks before returning them to the researchers. The pediatric experts were from eight hospitals and four nursing educational institutions located in the Ashanti region of Ghana; and were contacted between October 2018 and February 2019. The experts were chosen based on their level of training, clinical and/ or teaching experience in pediatrics.

The pediatric care settings in Ghanaian healthcare facilities take care of sick children with medical and surgical conditions on both out-patient and in-patient basis. The in-patient pediatric care settings consist of incubators, cots and beds which accommodate children from birth up to 12 or 13 years old depending on the facility’s protocol. The pediatric settings are sub-divided into various sections based on the procedures performed, children’s age or severity of their condition (medical or surgical). The units are colourfully painted with child-friendly designs and have television sets and toys for entertainment purposes; there are also designated areas with resources for playing purposes. Sick children are accommodated with at least one family caregiver or guardian during hospital admissions. Healthcare is mainly provided to sick children and their families by physicians, nurses, pharmacists, dietitians, physiotherapists, psychologists, healthcare assistants among others. Vital signs monitoring (including pain as the fifth vital sign) form an integral part of the role of healthcare providers towards hospitalised children and their families.

Data collection instrument, procedures and analysis

The pediatric experts rated the relevance and clarity of both instruments (the 42-item PNKAS instrument and the 41-item PHPKASRP instrument) on a four-point rating scale with 1 (not relevant or not clear), 2 (somewhat relevant or somewhat clear), 3 (quite relevant or quite clear) or 4 (very relevant or very clear) as done in earlier studies [28, 29]. In addition, they were required to make comments on each of the items regarding their grammatical construction, simplicity, representativeness, comprehension, ambiguity, modification (deletion or addition) among others as they deemed appropriate [30].

Both individual item-level and scale level content validity indices were estimated for both relevance and clarity aspects of the two instruments (PNKAS and PHPKASRP). Item-level content validity index (I-CVI) was calculated by dividing the number of experts who rated an item with a score of 3 (quite relevant/ quite clear) or 4 (very relevant/ very clear) over the total number of experts [31, 32]. As a general criterion, I-CVI should be ≥ 0.70 [33] to be regarded as measuring an appropriate sample of the instrument items for a particular construct. In line with the recommendations of Delgado-Rico and colleagues [34], decisions on items (i.e., elimination, modification or conservation) were made on the basis of the content validity indices, feedback given by experts, inputs from the instrument developer and the contribution of the items to the overall construct under investigation. The scale-level content validity index (S-CVI) or average scale-level content validity index (S-CVI/Ave) focuses on the average item quality and is estimated by summing up the I-CVIs of all items and dividing them by the total number of items [29, 32, 35]. The minimum acceptable value of S-CVI/Ave should be 0.80 [31, 32, 36]; values greater than or equal to 0.90 are considered as excellent average [37].

The pediatric experts’ comments on the individual items of the instruments were first reviewed by the research team. On the basis of the nature of the comment, the research team made decisions regarding the elimination, modification or conservation of the items involved. The pediatric experts’ comments and the research team’s decisions were then sent to the instrument developer for her review and feedback. Final decisions on the items were made through dialogue between the instrument developer and the research team. These decisions were also underpinned by the prevailing research evidence and the contribution of each item to the overall goals of the instruments.

The data were initially entered and cleaned in Microsoft Excel before being exported into Statistical Packages for the Social Sciences (SPSS) version 25.0 (SPSS, Chicago, IL, USA) for further analysis. Frequencies of items which were rated as relevant/ irrelevant and clear/ unclear were calculated. The proportion or percentage of items which were rated as relevant and clear by the pediatric experts were also estimated. Additionally, the mean I-CVIs of both the 42-item PNKAS instrument and the 41-item PHPKASRP instrument were determined.

Ethical considerations

Ethical approval for the study with reference number CHRPE/AP/574/18 was provided by the Committee on Human Research, Publications and Ethics (CHRPE), School of Medical Sciences (SMS), Kwame Nkrumah University of Science and Technology (KNUST), Ghana. Participants in the present study signed a written informed consent form and submitted the data collection instrument after completion. They were assured of anonymity, confidentiality, and their right to voluntary participation in the study. Authorization for the use of the PNKAS and PHPKASRP instruments in the current study was granted by the instrument developer, Manworren on August 16, 2018 (via email communication).

Results

Demographic characteristics of the pediatric experts

Thirteen (13) out of the 15 pediatric experts completed and returned the data collection instrument, yielding a response rate of 87%. The experts comprised of four pediatric nursing educators, eight pediatric nurses and one pediatrician. Their median (range) age was 38 (32–51) years (refer to Table 1). Majority of them were female (69%) and had a postgraduate degree (61.5%). The experts had worked in the healthcare profession for a median duration of 13 years and in pediatrics for 6 years.

Table 1

Demographic characteristics of pediatric experts (n = 13).

Variables	Frequency (%)	Median (range)
Age (years)		38 (32–51)
Gender
Male	4 (30.8)
Female	9 (69.2)
Working years in the health profession		13 (7–24)
Working years in pediatrics		6 (3–14)
Educational level
Bachelor’s degree	5 (38.5)
Postgraduate degree	8 (61.5)

Content validity assessment of the PNKAS instrument

The number of items considered relevant (with a rating of 3 or 4) by all 13 pediatric experts was five (refer to Table 2). The I-CVI for the 42 items ranged from 0.62 to 1.00 for the relevance aspect of the instrument. Four (4) out of the 42 items fell below the recommended I-CVI of 0.70; thus, the proportion of items considered relevant on this basis was 90.5%. The average relevance CVI for the scale (S-CVI/Ave) was 0.87, indicating an acceptable level of content validity and slightly below the 0.90 accepted excellent value.

Table 2

Content validity assessments of PNKAS and PHPKASRP instruments by pediatric experts (n = 13).

Items on the PNKAS 1999 version (Answer)	PNKAS		Items on the PHPKASRP 2014 revised version (Answer)	PHPKASRP		Pediatric Experts’ Comments (Number of Experts)	Action Taken; Revised Form
	Relevance; I-CVIs	Clarity; I-CVIs		Relevance; I-CVIs	Clarity; I-CVIs
Q1_Observable changes in vital signs must be relied upon to verify a child’s/ adolescent’s statement that he/ she has severe pain. (False)	13; 1.00	13; 1.00	Q1_Observable changes in vital signs must be relied upon to verify a child’s/ adolescent’s self-report of severe pain. (False)	13; 1.00	13; 1.00	This is quite broad, can question be directed at specific vital sign or signs e.g. heart rate (n = 1); Can do away with the / and use one of them (n = 1)	Kept; –
Q2_Because of an underdeveloped neurological system, children under 2 years of age have decreased pain sensitivity and limited memory of painful experiences. (False)	12; 0.92	13; 1.00	Q2_Because their nervous system is underdeveloped, children under 2 years of age have decreased pain sensitivity and limited memory of painful experiences. (False)	12; 0.92	13; 1.00		Kept; –
Q3_If the infant/ child/ adolescent can be distracted from his/ her pain, this usually means that he is not experiencing a high level of pain. (False)	10; 0.77	11; 0.85	Q3_Pediatric patients (infants, children, adolescents) who can be distracted from pain usually do not have severe pain. (False)	10; 0.77	11; 0.85	Pediatric patients cover all so no need to put them all in brackets (n = 1)	Kept; –
Q4_Infants/ children/ adolescents may sleep in spite of severe pain. (True)	12; 0.92	12; 0.92	Q8_Infants/ children/ adolescents may sleep in spite of severe pain. (True)	12; 0.92	12; 0.92		Kept; –
Q5_Comparable stimuli in different people produce the same intensity of pain. (False)	11; 0.85	11; 0.85	Q5_Comparable stimuli in different people produce the same intensity of pain. (False)	11;0.85	11; 0.85		Kept; –
Q6_Ibuprofen and other nonsteroidal anti-inflammatory agents are NOT effective analgesics for bone pain caused by metastases. (False)	11; 0.85	10; 0.77	Q9_Ibuprofen and other nonsteroidal anti-inflammatory agents are NOT effective analgesics for pain from bone metastases. (False)	11; 0.85	10; 0.77
Q7_Non-drug interventions (e.g. heat, music, imagery etc.) are very effective for mild-moderate pain control but are rarely helpful for more severe pain. (False)	12; 0.92	12; 0.92	Q10_Non-drug interventions (e.g. guided imagery, biofeedback, transcutaneous electrical nerve stimulation (TENS) etc.) are very effective for mild-moderate pain control but are rarely helpful for more severe pain. (False)	12; 0.92	12; 0.92	Examples of the non-drug interventions are not familiar to practitioners in the Ghanaian context (n = 1); Examples should be contextualized or removed to generalize the question (n = 1).	Amended; Evidence-based non-drug interventions are very effective for mild-moderate pain control but are rarely helpful for more severe pain. (False)
Q8_Children who will require repeated painful procedures (e.g., daily blood draws), should receive maximum treatment for the pain and anxiety of the first procedure to minimize the development of anticipatory anxiety before subsequent procedures. (True)	13; 1.00	11; 0.85	Q6_Children who will require repeated painful procedures (e.g., daily blood draws), should receive maximum treatment for the pain and anxiety of the first procedure to minimize the development of anticipatory anxiety before subsequent procedures. (True)	13; 1.00	11; 0.85	Question should be simplified (n = 1)	Kept; –
Q9_Respiratory depression rarely occurs in children/ adolescents who have been receiving opioids over a period of months. (True)	9; 0.69	11; 0.85	Q7_Respiratory depression rarely occurs in children/ adolescents who have been receiving stable doses of opioids over a period of months. (True)	9; 0.69	11; 0.85	Not the common practice in Ghana (n = 1).	Kept; –
Q10_Acetaminophen 650 mg PO is approximately equal in analgesic effect to codeine 32 mg PO. (True)	11; 0.85	11; 0.85
Q11_The World Health Organization (WHO) pain ladder suggests using single analgesic agents rather than combining classes of drugs (i.e. combining an opioid with a non-steroidal agent). (False)	11; 0.85	10; 0.77	Q11_Combining analgesics and non-drug therapies that work by different mechanisms (e.g. using acetaminophen, topical anesthetics, sucrose, and non-nutritive sucking) may result in better pain control with fewer side effects than using a single analgesic agent. (True)	11; 0.85	10; 0.77	Examples should be removed or positioned beside each intervention (n = 1).	Amended; Combining analgesics (e.g. using acetaminophen, topical anesthetics) and non-drug therapies (e.g. sucrose, and non-nutritive sucking) that work by different mechanisms may result in better pain control with fewer side effects than using a single analgesic agent.
Q12_The usual duration of analgesia of morphine IV is 4–5 hours. (False)	12; 0.92	13; 1.00	Q4_The usual duration of analgesia of morphine IV is 4–5 hours. (False)	12; 0.92	13; 1.00	This requires specific knowledge of morphine pharmacology (something I consider too detailed for basic nursing) (n = 1).	Kept; –
Q13_Research shows that promethazine (Phenergan®) is a reliable potentiator of opioid analgesics. (False)	10; 0.77	13; 1.00	Q12_Benzodiazepines do not reliably potentiate the analgesia of opioids’ unless the pain is related to muscle spasms (False)	10; 0.77	13; 1.00	Another question that requires detailed knowledge (n = 1).	Kept; –
Q14_Parents should not be present during painful procedures (False)	11; 0.85	12; 0.92	Q13_Parents should not be present during painful procedures. (False)	11; 0.85	12; 0.92		Kept; –
Q15_Adolescents with a history of substance abuse should not be given opioids for pain because they are at high risk for repeated addiction. (False)	13; 1.00	13; 1.00	Q14_Adolescents with a history of substance abuse should not be given opioids for pain because they are at high risk for repeated addiction. (False)	13; 1.00	13; 1.00		Kept; –
Q16_Beyond a certain dosage of morphine, increases in dosage will NOT provide increased pain relief. (False)	12; 0.92	13; 1.00	Q15_Beyond a certain dosage of morphine, increases in dosage will NOT provide increased pain relief. (False)	12; 0.92	13; 1.00		Kept; –
Q17_Young infants, less than 6 months of age, cannot tolerate opioids for pain relief. (False)	10; 0.77	10; 0.77	Q16_Young infants, less than 6 months of age, cannot tolerate opioids for pain relief. (False)	10; 0.77	10; 0.77		Kept; –
Q18_The child/ adolescent with pain should be encouraged to endure as much pain as possible before resorting to a pain relief measure. (False)	8; 0.62	11; 0.85	Q18_The child/ adolescent with pain should be encouraged to endure as much pain as possible before resorting to an opioid for pain relief. (False)	8; 0.62	11; 0.85		Kept; –
Q19_Children less than 8 years cannot reliably report pain intensity and, therefore, the nurse should rely on the parent’s assessment of the child’s pain intensity. (False)	10; 0.77	13; 1.00	Q19_Children less than 8 years cannot reliably report pain intensity and therefore, the healthcare provider should rely on the parent’s assessment of the child’s pain intensity. (False)	10; 0.77	13; 1.00	It does not apply to all children who are less than 8 years. (n = 1).	Amended; Most children as young as 4 years of age can reliably report pain intensity using a developmentally appropriate self-report tool. (True)
Q20_ Based on one’s religious beliefs, a child/ adolescent may think that pain and suffering is necessary. (True)	12; 0.92	13; 1.00	Q17_Spiritual beliefs may lead a child /adolescent to think that pain and suffering are necessary. (True)	12; 0.92	13; 1.00		Kept; –
Q21_Anxiolytics, sedatives and barbiturates are appropriate medications for the relief of pain during painful procedures. (False)	9; 0.69	11; 0.85	Q20_Anxiolytics, sedatives and barbiturates are appropriate medications for the relief of pain during painful procedures. (False)	9; 0.69	11; 0.85		Kept; –
Q22_After the initial recommended dose of opioid analgesic, subsequent doses should be adjusted in accordance with the individual patient’s response. (True)	11; 0.85	13; 1.00	Q21_After the initial dose of opioid analgesic is given, subsequent doses should be adjusted based on the individual patient’s response. (True)	11; 0.85	13; 1.00	This may also require detailed knowledge beyond basic nursing (n = 1).	Kept; –
Q23_The child/ adolescent should be advised to use non-drug techniques alone rather than concurrently with pain medications. (False)	10; 0.77	13; 1.00	Q22_The child/ adolescent should be advised to use non-drug techniques alone rather than concurrently with pain medications. (False)	10; 0.77	13; 1.00		Kept; –
Q24_Giving children/ adolescents sterile water by injection (placebo) is often a useful test to determine if the pain is real. (False)	12; 0.92	12; 0.92	Q23_Giving children/ adolescents sterile water by injection (placebo) is often a useful test to determine if the pain is real. (False)	12; 0.92	12; 0.92		Kept; –
Q25_In order to be effective, heat and cold should be applied directly to the painful area. (False)	11; 0.85	11; 0.85
			Q24_Sedation always precedes opioid related respiratory depression. (True)	9; 0.69	8; 0.62	I don’t get this (n = 1); The use of “always” gives out the answer and makes it a leading question (n = 1).	Kept; –
Q26_The recommended route of administration of opioid analgesics to children with prolonged cancer-related pain is: (oral)	12; 0.92	12; 0.92	Q26_The recommended route of administration of opioid analgesics to children with prolonged cancer-related pain is: (oral)	12; 0.92	12; 0.92		Kept; –
			Q27_The usual time to peak effects for traditional analgesics (acetaminophen, non-steroidal anti-inflammatory drugs, and opioids given orally is: (60 minutes)	11; 0.85	11; 0.85		Kept; –
Q27_The recommended route of administration of opioid analgesics to children with brief, severe pain of sudden onset, e.g., trauma or postoperative pain is: (intravenous)	12; 0.92	12; 0.92	Q28_ The recommended route administration of opioid analgesics to children with brief, severe pain of sudden onset, e.g., trauma or postoperative pain, is: (intravenous)	12; 0.92	12; 0.92		Kept; –
Q28_ Which of the following analgesic medications is considered the drug of choice for the treatment of prolonged moderate to severe pain for children with cancer? (morphine)	12; 0.92	12; 0.92	Q29_ Which of the following analgesic medications is considered the drug of choice for the treatment of prolonged moderate to severe pain for children with cancer? (morphine)	12; 0.92	12; 0.92		Kept; –
Q29_ Which of the following IV doses of morphine administered would be equivalent to 15 mg of oral morphine? (morphine 5mg IV)	11; 0.85	10; 0.77	Q30_ Which of the following IV morphine doses is approximately equivalent to 15 mg of oral morphine? (morphine 5mg IV)	11; 0.85	10; 0.77	Requires knowledge of pharmacology (n = 1).	Kept; –
Q30_Analgesics for post-operative pain should initially be given: (around the clock on a fixed schedule)	11; 0.85	13; 1.00	Q31_Analgesics for post-operative pain should initially be given: (around the clock on a fixed schedule)	11; 0.85	13; 1.00		Kept; –
Q31_A child with chronic cancer pain has been receiving daily opioid analgesics for 2 months. The doses increased during this time period. Yesterday the child was receiving morphine 20 mg/hour intravenously. Today he has been receiving 25 mg/hour intravenously for 3 hours. The likelihood of the child developing clinically significant respiratory depression is: (<1%)	11; 0.85	11; 0.85
Q32_Analgesia for chronic cancer pain should be given: (around the clock on a fixed schedule)	13; 1.00	13; 1.00	Q32_ Analgesia for chronic cancer pain should be given: (around the clock on a fixed schedule)	13; 1.00	13; 1.00		Kept; –
Q33_The most likely explanation for why a child/ adolescent with pain would request increased doses of pain medication is: (the child/ adolescent is experiencing increased pain)	12; 0.92	12; 0.92	Q33_The most likely reason a child/ adolescent with pain would request increased doses of pain medication is: (the child/ adolescent is experiencing increased pain)	12; 0.92	12; 0.92	Can use child alone (n = 1).	Kept; –
Q34_ Which of the following drugs are useful for treatment of cancer pain? (all of the above)	12; 0.92	11; 0.85	Q34_Which of the following drugs are potentially useful for treatment of children’s cancer pain? (all of the above)	12; 0.92	11; 0.85		Kept; –
Q35_The most accurate judge of the intensity of the child’s/adolescent’s pain is: (the child/ adolescent)	13; 1.00	11; 0.85	Q35_The most accurate judge of the intensity of the child’s/ adolescent’s pain is the: (child/ adolescent)	13; 1.00	11; 0.85		Kept; –
Q36_Which of the following describes the best approach for cultural considerations in caring for a child/ adolescent in pain? a. Because of the diverse and mixed cultures in the United States, there are no longer cultural influences on the pain experience. b. Nurses should use knowledge that has defined clearly the influence of pain on culture (e.g., Asians are generally stoic, Hispanics are expressive and exaggerate their pain, etc.). c. Children/ adolescents should be individually assessed to determine cultural influences on pain.	11; 0.85	12; 0.92	Q36_Which of the following describes the best approach for cultural considerations in caring for a child/ adolescent in pain? a. There are no longer cultural influences on the pain experience in the United States due to the diversity of the population. b. Healthcare providers should use knowledge that has defined clearly the influence of pain on culture (e.g. Asians are generally stoic, Hispanics are expressive and exaggerate their pain, etc.) c. Children/ adolescents should be individually assessed to determine cultural influences on pain.	11; 0.85	12; 0.92	Examples given should be modified to suit the Ghanaian context or deleted to make it more generalized (n = 1).	Amended; Which of the following describes the best approach for cultural considerations in caring for child/adolescent in pain? a. There are no longer cultural influences on the pain experience due to the diversity of the population. b. Nurses/ healthcare providers should use knowledge that has defined clearly the influence of pain on culture c. Children/ adolescents should be individually assessed to determine cultural influences on pain.
Q37_What do you think is the percentage of patients who over report the amount of pain they have? Circle the correct answer. (0 or 10%)	8; 0.62	9; 0.69	Q37_What do you think is the percentage of patients who over report the amount of pain they have? (0 and 10%)	8; 0.62	9; 0.69	On what basis are respondents expected to guess this percentage? (n = 1).	Amended; Children generally over report their pain. (True/ False)
Q38_Narcotic/ opioid addiction is defined as psychological dependence accompanied by overwhelming concern with obtaining and using narcotics for psychic effect, not for medical reasons. It may occur with or without the physiological changes of tolerance to analgesia and physical dependence (withdrawal). Using this definition, how likely is it that opioid addiction will occur as a result of treating pain with opioid analgesics? Circle the number closest to what you consider the correct answer. (<1%)	12; 0.92	9; 0.69	Q25_Opioid/ narcotic addiction is defined as a chronic neurobiological disease, characterized by impaired control over drug use, compulsive use, continued use despite harm, and craving. It may occur with or without the physiological changes of tolerance to analgesia and physical dependence (withdrawal). Given this information, all children /adolescents whose pain have been treated with opioids for longer than a month are addicted to opioids. (False)	12; 0.92	9; 0.69	Question should be straight forward (n = 1).	Kept; –
Q39_On the patient’s record you must mark his pain on the scale below. Circle the number that represents your assessment of Andrew’s pain. (8)	12; 0.92	10; 0.77	Q38_On the patient’s record you must mark his pain on the scale below. Choose the number that represents your assessment of Andrew’s pain. (8)	12; 0.92	10; 0.77		Kept; –
Q40_Your assessment, above, is made two hours after he received morphine 2 mg IV. After he received the morphine, his pain ratings every half-hour ranged from 6 to 8 and he had no clinically significant respiratory depression, sedation, or other untoward side effects. He has identified 2 as an acceptable level of pain relief. His physician’s order for analgesia is “morphine IV 1–3 mg q1h PRN pain relief.” Check the action you will take at this time. (administer morphine 3 mg IV now)	12; 0.92	13; 1.00	Q39_Your assessment, above, is made two hours after he received morphine 2 mg IV. After he received the morphine, his pain ratings every half-hour ranged from 6 to 8 and he had no clinically significant respiratory depression, sedation, or other untoward side effects. He has identified 2 as an acceptable level of pain relief. His physician’s order for analgesia is “morphine IV 1–3 mg q1h PRN pain relief.” Check the action you will take at this time. (administer morphine 3 mg IV now)	12; 0.92	13; 1.00		Kept; –
Q41_On the patient’s record you must mark his pain on the scale below. Circle the number that represents your assessment of Robert’s pain: (8)	12; 0.92	11; 0.85	Q40_Select the number that represents your assessment of Robert’s pain: (8)	12; 0.92	11; 0.85		Kept; –
Q42_Your assessment, above, is made two hours after he received morphine 2 mg IV. After he received the morphine, his pain ratings every half-hour ranged from 6 to 8 and he had no clinically significant respiratory depression, sedation, or other untoward side effects. He has identified 2 as an acceptable level of pain relief. His physician’s order for analgesia is “morphine IV 1–3 mg q1h PRN pain relief.” Check the action you will take at this time: (administer morphine 3 mg IV now)	12; 0.92	11; 0.85	Q41_Your assessment, above, is made two hours after he received morphine 2 mg IV. After he received the morphine, his pain ratings every half-hour ranged from 6 to 8 and he had no clinically significant respiratory depression, sedation, or other untoward side effects. He has identified 2 as an acceptable level of pain relief. His order for analgesia is “morphine IV 1–3 mg q1h PRN pain relief.” Check the action you will take at this time: (administer morphine 3 mg IV now)	12; 0.92	11; 0.85		Kept; –

NB: PNKAS–Pediatric Nurses' Knowledge and Attitudes Survey Regarding Pain, PHPKASRP–Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain, PO–Per os (by mouth), mg–Milligram, IV–Intravenous, q1h –Hourly, PRN–When necessary.

The number of items considered clear (with a rating of 3 or 4) by all the 13 experts was 13. The I-CVI for the 42 items ranged from 0.69 to 1.00 for the clarity component of the PNKAS instrument. Two (2) out of the 42 items fell below the recommended I-CVI of 0.70; the proportion of items considered clear on this foundation was 95.2%. The average clarity CVI for the scale (S-CVI/Ave) was 0.89, indicating an acceptable level of content validity which is almost at the 0.90 acceptable excellent value.

Content validity assessment of the PHPKASRP instrument

The number of items considered relevant (with a rating of 3 or 4) by all 13 pediatric experts was five. The individual-item content validity index (I-CVI) for the 41 items ranged from 0.62 to 1.00 for the relevance component of the PHPKASRP instrument. Five (5) out of the 41 items fell below the recommended I-CVI of 0.70; thus, the proportion of items considered relevant on this basis was 87.8%. The average CVI relevance for the scale (S-CVI/Ave) was 0.86, indicating an acceptable level of content validity and slightly below the 0.90 accepted excellent value.

The number of items considered clear (with a rating of 3 or 4) by all 13 experts was 13. The individual-item content validity index (I-CVI) for the 41 items ranged from 0.62 to 1.00 for the clarity component of the PHPKASRP instrument. Three (3) out of the 41 items fell below the recommended I-CVI of 0.70; the resultant proportion of items considered clear on this basis was 92.7%. The average CVI clarity for the scale (S-CVI/Ave) was 0.89, indicating an acceptable level of content validity and almost at the 0.90 accepted excellent value.

Comparison of the content validity of PNKAS and PHPKASRP instruments

The results revealed that both instruments have an acceptable level of content validity (refer to Table 3). Nevertheless, the PNKAS instrument performed slightly better than the PHPKASRP instrument in four areas. These were related to the following: the number of items considered relevant with an I-CVI ≥ 0.70 (90.5% versus 87.8%) and those considered clear with an I-CVI ≥ 0.70 (95.2% versus 92.7%); the average scale-level content validity index for the relevance component of the items (0.87 versus 0.86) and the range of items considered clear (0.69–1.00 versus 0.62–1.00).

Table 3

Comparison of the content validity of PNKAS and PHPKASRP instruments (n = 13).

Variable	PNKAS (42 items)	PHPKASRP (41 items)
Relevance Components
Universal Agreement	5	5
Number of Items with I-CVI ≥ 0.70	38 (90.5%)	36 (87.8%)
Number of Items with I-CVI < 0.70	4 (9.5%)	5 (12.2%)
Minimum–Maximum I-CVI	0.62–1.00	0.62–1.00
SCI/Ave	0.87	0.86
Clarity Components
Universal Agreement	13	13
Number of Items with I-CVI ≥ 0.70	40 (95.2%)	38 (92.7%)
Number of Items with I-CVI 0.70	2 (4.8%)	3 (7.3%)
Minimum–Maximum I-CVI	0.69–1.00	0.62–1.00
SCI/Ave	0.89	0.89

NB: I-CVI–Individual item level content validity index, SCI/Ave–Average scale-level content validity index

Experts’ comments on the PNKAS and PHPKASRP instruments

The pediatric experts made general comments on both instruments (PNKAS and PHPKASRP) which included: simplifying the sentences, separating knowledge questions from those of attitude for clarity purposes, and restructuring questions according to the different pediatric pain topics. They additional made specific comments on the individual items which have been presented in Table 2. In consultation with the instrument developer (Manworren, R.C.B.), the researchers addressed all comments from the experts through a review process which resulted in the maintenance of 37 items and revision of 5 items on the PNKAS instrument and the maintenance of 36 items and revision of 5 items on the PHPKASRP instrument.

Discussion

The current study aimed at comparing the content validity of two versions of an instrument used in measuring pediatric pain knowledge and attitudes (PNKAS and PHPKASRP). Our results showed that both instruments have an acceptable level of content validity, signifying that the instruments sufficiently represent the content of “pediatric pain knowledge and attitudes” for which they intend to measure [32]. Nevertheless, the PNKAS instrument performed slightly better than the PHPKASRP instrument with regards to some aspects of its content validity properties. For instance, the number of items with I-CVI ≥ 0.70 for both relevance and clarity aspects of PNKAS was 90.5% and 95.2% as against 87.8% and 92.7% for PHPKASRP respectively. On the basis of the current study findings, preference may be given for the use of the PNKAS due to its slightly higher level of content validity and extensive content. However, both instruments are comparable in the length of time they take to be completed. Thus, both instruments are promising in being used in clinical practice and for research purposes due to their acceptable level of content validity.

A critical review of both instruments (PNKAS and PHPKASRP) seems to be missing some important aspects of pediatric pain management, especially on the role of family caregivers in children’s pain management. The role of family caregivers has been shown to be critical in pediatric pain management as they serve as the mouthpiece for vulnerable children [38, 39], especially in the Ghanaian society where children are expected to be “seen” but not “heard” [40]. The pediatric experts also brought to the fore their unfamiliarities with given examples of nonpharmacological pain management interventions (such as guided imagery, biofeedback among others) and the need to contextualize pediatric pain care considerations to the Ghanaian setting instead of referring to other countries such as United States and foreign ethnic origins such as Hispanics, Asians and so on. This underscores the importance of validity assessment as there exists universal differences in culture, semantics and resources in different parts of the world [19, 20]. It further supports the assessment of content validity as a precondition for other forms of validity such as construct and criterion validity [41].

The production of high-quality data in quantitative research requires thorough evaluation of an instrument to build sufficient evidence for its validity [27]. Content validity testing is thus, concerned with determining the inferences that can be made about an instrument’s construction. The processes involved in the content validity assessment has led to an improvement in clarity and relevance of the items contained in these instruments. This process has also provided empirical data supporting the adaptation process [42], which will also facilitate subsequent testing of the instrument for other types of validity and reliability [43]. In line with the recommendations of Delgado-Rico and colleagues [34], decisions on items (i.e., elimination, modification or conservation) were made on the basis of the content validity indices, feedback given by experts, inputs from the instrument developer and the contribution of the items to the overall construct under investigation. At the end of the validation processes, 37 and 36 items were respectively kept on the PNKAS and PHPKASRP instruments whereas 5 items were modified on each of the two instruments.

Feasibility of the instruments was reflected in the high response rate and the absence of missing values as all the experts completely filled the data collection instruments. On the basis of the current study findings, we recommend the use of the either of these instruments (PNKAS or PHPKASRP) as one that sufficiently covers the construct of pediatric pain knowledge and attitudes. Nonetheless, preference may be given to the PNKAS instrument for use in clinical practice and research due to its slightly higher level of content validity. On the basis of the current study findings, we recommend the use of either one of two instruments (PNKAS and PHPKASRP) in the proposed pediatric pain education for nursing students and nurses in Ghana due to their acceptable level of content validity.

One of the short-comings of the current study was our inability to conclude on the validity of these revised instruments as they were not tested; future studies should examine the content validity of these revised instruments. Other forms of validity (construct and criterion) and reliability (internal consistency, test-retest, intrarater) should also be evaluated in the future to enhance our understanding on their psychometric properties. It is also worth mentioning that our method of content validity assessment did not cater for the possibility of chance agreement among the experts which is achieved using Kappa statistic coefficient [32, 44].

Conclusions

The PNKAS and PHPKASRP instruments have demonstrated an acceptable level of content validity in the Ghanaian context. Both instruments sufficiently cover the construct of “pediatric pain knowledge and attitudes”. We recommend the use of either of these two instruments for use in clinical practice and research purposes. The modifications made on these two instruments should be assessed for content validity in the future. Other forms of validity (construct, criterion) and reliability (internal consistency, test-retest, item analysis) should also be examined in future studies.

Revised Pediatric Nurses’ Knowledge and Attitudes Survey regarding pain (r-PNKAS).

(DOCX)

Click here for additional data file.

Revised Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (r-PHPKASRP).

(DOCX)

Click here for additional data file.

(XLSX)

Click here for additional data file.

22 Apr 2020

PONE-D-20-05637

Comparison of the content validity of two instruments used in measuring pediatric pain knowledge and attitudes

PLOS ONE

Dear Ms. Kusi Amponsah,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

ACADEMIC EDITOR:

Editor Decision - Major Revision

Please, follow all reviewers commentaries.

We would appreciate receiving your revised manuscript by Jun 06 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Matias Noll, Ph.D

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: 1 Summary of the research and your overall impression

1.1 Reviewer comment:

The manuscript compared the content validity of two instruments used in measuring nurse pediatric pain knowledge and attitudes.

As strong points of the article, it should be noted that they have had a wide participation of experts. Furthermore, it appears that the author of the questionnaire, Manworren, has been involved in the counseling process.

As weaknesses, the authors acknowledge both in the limitations section and in the conclusions that, as future lines, a validation of the criterion and the construct is required.

From my point of view, we must differentiate between (Lobiondo-Wood G, Haber J.; 2013) :

• The content validity. Content validity evaluates qualitatively whether the questionnaire covers all the dimensions of the phenomenon that wants to measure, since it is considered that an instrument is valid in its content if contemplates all related aspects with the concept that measures. The researcher begins by defining the concept and identifying the attributes or dimensions of the concept. The items that reflect the concept and its domain are developed.

• The apparent validity. It is a subtype of content validity. It is a face validity, which is a rudimentary type of validity that basically verifies that the instrument fives the appearance of measuring the concept. It is an intuitive type of validity in which colleagues or subjects are asked to read the instrument and evaluate the content in terms of whether it appear to reflect the concept the researcher intends to measure. Or if the elements included in an instrument are relevant.

• Criterion validity. It is the degree of correlation between an instrument and another measure of the variable under study that serves as criterion or reference.

• Construct validity. It is understood as the degree to which an instrument measures the bipolar evaluative dimension for which it was designed.

Actually, in the present article, what they do is to measure apparent validity. It is important because the acceptance of a scale by several people gives consistency when using it. However, apparent and content validity is a relevant method especially when designing an instrument. It is not so important when the instrument has been previously validated and used in different areas.

On the other hand, and as the authors say, these tools have already been validated and used in various studies. Knowing that its content validity has already been reviewed as they describe and explain. In that case, why have they revised the apparent validity again? Why haven't they gone a step further? Why have the criteria not been applied and reviewed?

For these reasons, I think the article does not reach the level required for a journal like PLOS ONE, and should be rejected.

From my point of view, the approach and content is well developed, but due to the characteristics of the tools used, these require a criterion validity, a construct validity, or a cross-cultural validation.

The authors summarize the main research question and key findings. Even, the authors identify other literature on the topic and explain how the study relates to this previously published research.

However, I would like to make some specific suggestions in the next point.

2 Discussion of specific areas for improvement

2.1 Major issues

2.1.1 Reviewer comment:

Suggestions for improvement do not refer to major issues

2.2 Minor issues

2.2.1 Reviewer comment:

2.2.1.1 Title

As a suggestion, and to contextualize more the article from the first moment, the title could refer to Pediatric Healthcare Providers' (HCPs) or pediatric nurses as well as keywords.

In fact, HCPs are referenced every time in the page 3, line 61 paragraph.

Another more clarifying explanation is found in page 3 and line 68: “These two instruments were developed to measure healthcare professionals and students’ knowledge and attitudes regarding children’s pain [13].” But it is still a personal appreciation.

2.2.1.2 Abstract and introduction

Page 2; line 24. The introduction does not set the stage adequately. As in the title, it is required to specify who the study is aimed at. To miss this information may decontextualize and imply the study population.

In the abstract, the authors explain why the study matters and put the research in context properly. However:

In page 2; line 25; the authors clarify that “This was considered necessary due to the universal differences in culture, semantics and healthcare resources in different parts of the globe”, but in the following paragraph:

In page 2; line 31; they specify that the experts only will check the relevance and clarity of the items will be reviewed without mention culture, semantics and healthcare resources. If it is so important, because you mention it at the beginning of the abstract, could the experts have been asked about the changes due to the cultural factor? Has any change been made due to semantic and cultural changes?

Content validity is a relevant method especially when designing an instrument. It is not so important when the instrument has been previously validated and used in different areas. However, when an instrument is translated into another language, if the explored concepts are supposed to change significantly from one culture to another, it may be useful to recheck the face validity.

Moreover, in page 4, line 92 It is said that validity is not the property of an instrument, but depends on the interpretation related to the context and participants. However, knowledge of health is based on science, evidence, principles, theories, and is universal. Therefore, they do not depend on the interpretation of people and cultures.

Content validity evaluates qualitatively whether the questionnaire covers all the dimensions of the phenomenon to be measured, since an instrument is considered to be valid in its content if it considers all aspects related to the concept it measures. For this, it is necessary to have a clear idea of the conceptual aspects to be measured. And in this case, since the instrument had previously been validated and used in different areas, the content was already available.

Page 3; line 67. This is what I cannot understand. In this section, the questions that arise are: When talking about the revised version (PHPKASRP). Is the article review being done for the first time in this article? Or has this version already been created before? Why do you call short version if they have almost the same number of questions?...May be could you give more details or change the way to explain it.

In the next sentence: page 3; line 68. “These two instruments were developed to measure healthcare professionals and students’ knowledge and attitudes regarding children’s pain (13)”. We find the reference of the PNKAS, what about the PHPKASRP reference? Could you explain this better?

When talking about instrument validation, I would give more information about whether or not PHPKASRP is validated, etc.

Page 4; line 84. References for PHPKASRP are again needed.

2.2.1.3 Figures and tables

The information in Tables 1 and 3 are explained in the text. Table 1 could be omitted by completing the information in the text. However, Table 3 can be kept, since it serves as a synthesis of the results.

2.2.1.4 Methods

Page 6; paragraph line 128-133.There is no specific reference to cultural and semantic factors. Despite the emphasis that has been given in the introduction by providing references.

They talk about: comprehensiveness, objectivity, organization and relevance defined as “comprehensive: that issue containing important information to reach the objective of the study, stated in a comprehensible manner; Objective: that issue which is easy to understand; organization: the disposition of the issues and alternatives as well as their content; Relevant: that question which is related to achieving the goal of the research”

Page 8; line 182. Again, the article reference of the revised version (PHPKASRP instrument) is missed.

2.2.1.5 Results, discussion, conclusions

The results on page 10 should reference Table 3, where the results are summarized.

The authors acknowledge both the limitations and the conclusions that, as future lines, a validation of the criterion and the construct is required.

2.2.1.6 Statistical analysis

This type of validity requires basic statistics. Experts and / or researchers use in evaluating the relevance of a scale that requires descriptive analysis.

Reviewer #2: This descriptive methodological study was carried out with the objective of evaluating and comparing the content validity of two instruments of knowledge and attitude about pediatric pain.

The study was well written and relevant, however, my main concerns are related to Validity analysis. The lack of review of the validity of the content of these instruments and the lack of reliability analysis (internal consistency, test-retest, intra-evaluator). In addition, the analysis of casual agreement among experts, with the Kappa statistical coefficient.

Type of study: Review the written form of the type of study. "Descriptive methodological study with content validity".

Review confusing writing in the first paragraph of the results.

Reviewer #3: Dear Editor

Thank you for the opportunity to review this manuscript. The manuscript aims to evaluate and compare the content validity of the PNKAS and its revised version, the PHPKASRP from a Ghanaian perspective. I have read the manuscript with careful attention. I have several major and minor comments for better improvement. The authors are expected to make point-by-point response to the comments.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Review manuscript PONE-D-20-05637 VMS.docx

Click here for additional data file.

Submitted filename: PLOS ONE.docx

Click here for additional data file.

Submitted filename: PONE-D-20-05637 Review.docx

Click here for additional data file.

6 Jun 2020

Response to Review Comments

Thank you for taking time off your busy schedules and providing us constructive feedback to further improve our manuscript. Please find below our responses to the review comments.

Overall Comments from Reviewer 1

This descriptive methodological study was carried out with the objective of evaluating and comparing the content validity of two instruments of knowledge and attitude about pediatric pain.

The study was well written and relevant, however, my main concerns are related to Validity analysis. The lack of review of the validity of the content of these instruments and the lack of reliability analysis (internal consistency, test-retest, intra-evaluator). In addition, the analysis of casual agreement among experts, with the Kappa statistical coefficient.

Response

Thank you for the feedback. The review of validity and reliability of the instruments have already been provided on page 4, lines 75-87.

Specific Comments from Reviewer 1

Comment

Type of study: Review the written form of the type of study. "Descriptive methodological study with content validity".

Response

The study type has been amended on page 5, line 103.

Comment

Review confusing writing in the first paragraph of the results.

Response

Thank you for the feedback. The review of validity and reliability of the instruments have already been provided on page 4, lines 75-87.

Overall Comments from Reviewer 2

The manuscript aims to evaluate and compare the content validity of PNKAS and its revised version, PHPKASRP from a Ghanaian perspective. I have read the manuscript with careful attention. First of all, I was expecting to find the revised the PNKAS and PHPKASRP) as appendices. I think it is worthy to include them as appendices since they are the final products of the study. I have other comments for better improvement.

Response

Thank you for the feedback. The revised instruments have been provided as appendices (refer to S1 Appendix and S2 Appendix).

Specific Comments from Reviewer 1

Comment

The title of the manuscript reads as if the study compares completely different instruments. However, PHPKASRP is a revised version of PNKAS, and it would be not surprising to find high similarity between the two versions in terms of content validity. Therefore, it would be irrational making a comparison between two versions of the same survey. Further, the title does not indicate that the assessment of content validity was performed in Ghanaian context.

I would suggest revising the title to address these two comments. For example “Evaluating the content validity of two instruments used in measuring pediatric pain knowledge and attitudes in Ghanaian context”.

Response

As suggested, changes have been made to the titles (long and short) as found on page 1, lines 1-7.

Comment

Since the keyword “psychometric evaluation” might indicates reliability, validity, and/or responsiveness, the authors only examined the content validity. I believe this keyword would be misleading. Please replace it with “content validity”. Furthermore, I would suggest including “Ghana” as a keyword.

Response

Changes have been made on page 3, line 54.

Comment

Line 24-25. “we compared the content validity of two instruments used in measuring pediatric pain knowledge and attitudes”

Again, this sentence reads as if the authors compared completely different instruments, and it does not indicate that the evaluation was conducted in Ghanaian context. I would suggest recast this sentence as “we evaluated the content validity of two versions of an instrument used in measuring pediatric pain knowledge and attitudes in the Ghanaian context”.

Response

Changes have been made at the abstract and sections of the manuscript. Refer to page 2, lines 24-25, 28-29 and page 5, line 106.

Comment

Line 28-30. “Thirteen (13) pediatric experts in Ghana assessed the content validity of two instruments: the 42-item Pediatric Nurses' Knowledge and Attitudes Survey Regarding Pain (PNKAS) and the 41-item Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (PHPKASRP).”

Same here. I suggest recast this sentence as “Thirteen (13) pediatric experts in Ghana assessed the content validity of the 42-item Pediatric Nurses' Knowledge and Attitudes Survey Regarding Pain (PNKAS) and its revised version, the 41-item Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (PHPKASRP).

If you accept this suggestion, please modify the rest of abstract and manuscript to be consistent with these changes.

Response

Changes have been made at the abstract and sections of the manuscript. Refer to page 2, lines 24-25, 28-29 and page 5, line 106.

Comment

Line 31. “four-point likert scaled”. Please capitalize the letter “L” in “Likert”.

Response

As suggested, this has been changed on page 2, line 32.

Comment

Line 66-68. “Key among them are Pediatric Nurses’ Knowledge and Attitudes Survey regarding pain (PNKAS) and its revised shorter version, the Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (PHPKASRP)”

I know the PNKAS was published by Manworren RC (reference 13). However, was the PHPKASRP also published in the same study (reference 13)? Either way, you need to cite the study/studies that published the PNKAS and PHPKASRP in the above sentence.

Further, since the difference between the two versions in terms of number of items is one item only (42 vs 41), I am not sure if it is appropriate to describe the PHPKASRP as “revised shorter version”. I suggest deleting “shorter” form the above sentence.

Response Citations have been respectively placed by each instrument, refer to page 4, lines 71-73. Information on the PHPKASRP was provided by the instrument developer (Manworren RCB) through email communication. According to the instrument developer, she is yet to publish information about PHPKASRP.

Also, the word “shorter” has been removed from the sentence, refer to page 4, line 71.

Comment

Line 68-70. “These two instruments were developed to measure healthcare professionals and students’ knowledge and attitudes regarding children’s pain [13]”

This sentence reads as if Manworren RC (reference 13) published the two instruments in their study. However, I am not sure if this is true. Please make sure and correct the reference accordingly.

Response

Modifications have been made on page 3, line 70 and page 4, lines 71-74.

Comment

Line 72. “Following the development of the PNKAS instrument”

No need to follow the PNKAS or the PHPKASRP with the word “instrument” since the last word both terms is “survey”. Please delete “instrument” from this sentence and throughout the manuscript if you accept.

Response

All instances of the word “instrument(s)” which followed the PNKAS and or PHPKASRP have been removed from the manuscript.

Comment

Line 73-78. “Face and content validity of the instrument was established by five pain management experts in the United States of America. Internal consistency of the PNKAS instrument has also been assessed by two distinct groups with a reported acceptable Cronbach’s alpha value of 0.72 among 247 pediatric nurses and 0.77 among 88 members of a children’s nursing organization. Test-retest reliability analysis among 12 clinicians (6 nurses and 6 child life specialists) recorded a correlation coefficient of 0.67, signifying an acceptable level of instrument stability”.

Too much information are mentioned in this section. You may simply report the psychometric properties without going into further details about the number of participants or experts and their genders.

Response

As recommended, this statement has been summarised concisely; refer to page 4, lines 75-79.

Comment

Line 85. “(PHPKASRP instrument)”

Same here. Please see my comment above about the word “instrument”.

Response

As earlier indicated, all instances of the word “instrument(s)” which followed the PNKAS and or PHPKASRP have been removed from the manuscript.

Comment

Line 85-87. “Content validity of the PHPKASRP has been established by national content experts comprising of physicians, pediatric nurses and pharmacists in the United States of America”. Please cite the reference for this sentence.

Response

Citation to this statement has been provided on page 4, line 87.

Comment

Line 87-89. “According to the instrument developer (Manworren, R.C.B.), the instrument has been translated into other languages and permitted for use in many organizations around the world.”. Same here. Please cite the reference for this sentence.

Response

Citation to this statement has been provided on page 4, line 89.

Comment

Line 96-98. “As part of plans to use a pediatric pain knowledge and attitude instrument as a tool in assessing the effectiveness of a short-course educational program for nursing students and nurses”

According to this sentence, the content validity of the two versions was examined for the purpose of this course. Will be the course conducted in Ghana? Are all of the audience from Ghana? Or others from different countries will be included?

Response

Clarifications on this have been provided at page 5, line 98.

Comment

Line 98. “the current study sought to evaluate and compare the content validity”. Again, comparing the content validity of two versions of the same instrument would be irrational. I suggest narrowing your aim to “evaluating” without “comparing”.

Response

Suggested amendment has been made on page 5, line 98.

Comment

Line 103. “This descriptive methodological study”

According to Portney and Watkins (Foundations of Clinical Research: Applications to Practice. Vol. 2. Prentice Hall Upper Saddle River, NJ; 2000), validity studies are described as exploratory methodological studies. Please correct the design.

Response

The study type has been amended on page 5, line 103.

Comment

Line 110-111. “The experts were chosen based on their level of training, clinical and/ or teaching experience in pediatrics”.

The criteria of selecting the experts mentioned here is wide and lacks of accuracy. What do you mean by “their level”? Did you have minimum level of experience? Please be more specific.

I think it is very important to be more careful in describing how the experts were selected because they are the main element of this study.

Response

Specifications about the expertise have been provided on page 5, lines 111-112.

Comment

Line 135. “Both individual item-level and scale level content validity indices”

There are several issues in the manuscript regarding the use of abbreviations as will be stated in some of the following comments. As a rule of thumb, you should explain each of your abbreviation the first time it appears in the main text, and then use that abbreviation instead of the complete term in the rest of the manuscript.

In this sentence, you mentioned the individual item-level content validity (I-CVI) and the average scale level content validity (S-CVI/Ave) for the first time. Therefore, you should have introduced their abbreviations and use them instead of the complete term in the rest of your manuscript.

Line 149. “Chicago, IL”

The correct citation of the SPSS version 25 is (Armonk, NY). Please refer to this webpage for more information (https://www.ibm.com/support/pages/how-cite-ibm-spss-statistics-or-earlier-versions-spss).

Response

As suggested, amendments have been made throughout the manuscript; refer to page 6, line 134 for an example.

The appropriate citation for SPSS version 25 have been provided on page 7, line 147.

Comment

Line 182-184. “The Pediatric Nurses' Knowledge and Attitudes Survey Regarding Pain (PNKAS) is a modification of Nurses' Knowledge and Attitudes Survey Regarding Pain (NKASRP)”.

You do not need to use the complete terms since you have already introduced their abbreviations in the introduction (line 67-68).

Response

Suggested amendment has been made at page 7, line 156.

Comment

Line 191-193. “Ethical approval for the study with reference number CHRPE/AP/574/18 was provided by the Committee on Human Research, Publications and Ethics (CHRPE), School of Medical Sciences (SMS), Kwame Nkrumah University of Science and Technology (KNUST)”.

One the other hand, you do not need to state the abbreviations of the Committee on Human Research, Publications and Ethics, School of Medical Sciences, and Kwame Nkrumah University of Science and Technology since they were only mentioned once in the manuscript.

Response

As suggested, these abbreviations have been removed; refer to page 8, lines 189-190.

Comment

Line 205. “Majority of them were female (69%)”. Please correct the percentage to (69.2%) to be consistence with Table 1.

Response

Suggested modification has been provided on page 9, line 201.

Comment

Table 1. Is there is any reason why you used the median instead of the mean to report the central tendency?

Response

The rationale for the use of median has been provided on page 7, lines 147-149.

Comment

Line 215-216. “The average relevance CVI for the scale (S-CVI/Ave) was 0.87”.

Same to my previous comments about the abbreviations. Use the abbreviation without introducing it again. Please correct accordingly.

Response

As suggested, changes have been made throughout the manuscript. An instance of this can be seen on page 9, lines 209, 214 among others.

Comment

Line 232. “individual-item content validity index (I-CVI)”. Same here regarding the use of complete term.

Response

Modifications have been provided throughout the manuscript. Instances of this can be seen on page 9, line 219 and page 10, line 223.

Comment

Table 2. Some issues on this table need to be addressed:

The table is lengthy and I think it would be better to include it as a supplemental table instead of containing it in the manuscript.

There are four cells that are missing in the last column (Action Taken; Revised Form) of the table.

You mentioned five items were removed from each of the PNKAS and PHPKASRP. It is not clear from the table which items are they. Please specify them.

It is not clear why some items were retained or not revised although their scores were less than 70 and some of them received comments from the experts. For example, the question about respiratory depression and the use of opioids, Questions 18, the question about Anxiolytics, sedatives and barbiturates, Question 24 of the PHPKASRP, and Question 38 of the PNKAS (Q25 on the PHPKASRP). I think the readers deserve to know the reasons behind ignoring the low scores and experts’ comments in making the decision of retaining or not revising those items. Otherwise, the ratings and comments of the experts are meaningless.

Response

As suggested, Table 2 has been designated as S1 Table in the supplementary document file named “S1_Table.docx”.

Missing information has been provided at the initial four empty cells of the now “S1 Table”.

Five items were modified from each of the instruments and “not removed”; these items have been specified on page 11, lines 249-251.

Factors which influenced the decisions made on items have been provided on page 6, lines 138-140.

Comment

Line 254-256. “In consultation with the instrument developer (Manworren, R.C.B.), the researchers addressed all comments from the experts through a review process”

This a bit confusing. In the methods section, line 139-142, the authors stated, “items which were < 0.70 were eliminated, revised or kept based on feedback and its importance to the construct under investigation. Items which had an I-CVI value of ≥ 0.70 were kept or modified based on theoretical relevance and/ or the feedback received from the participating pediatric experts”. Nothing were mentioned in the methods about involving the developer in the decision-making process.

Response

This has been rectified on page 6, lines 138-140.

Comment

Table 3. What do you mean by “universal agreement”? How this outcome was calculated?

Response

Definition and calculation of “universal agreement” has been provided on page 6, lines 141-142.

Comment

Line 311-314. “In line with the recommendations of Delgado-Rico and colleagues [43], decisions on items (i.e., elimination, modification or conservation) were made on the basis of the content validity indices, feedback given by experts, inputs from the instrument developer and the contribution of the items to the overall construct under investigation”.

Here you mentioned the involvement of the developer in the decision-making process. Again, nothing was mentioned about the developer involvement in the methods. Further, I think you need to cite Delgado-Rico and colleagues as well in your methods since you followed their recommendations in your decisions making.

Response

As earlier indicated, these concerns have been addressed, refer to page 6, lines 138-140.

Comment

Line 318-319. “we recommend the use of the both instruments (PNKAS and PHPKASRP) as simple, easy to use”

There is no evidence of simplicity nor easiness of use that can be drawn from you results.

Response

These descriptions have been removed; refer to page 12, lines 288-289.

Comment

Line 355. “Both instruments are simple, easy to use”

Same here, you need to show the evidence or delete this sentence.

Response

Suggested modifications have been made on page 13, lines 304-305.

Comment

I do not believe it is required to include the publishers in the reference (e.g., Lippincott, Williams & Wilkins, Elsevier, Hindawi ..etc.). The authors need to format their references according to the journal’s instructions (https://journals.plos.org/plosone/s/submission-guidelines#loc-references).

Response

All instances of the “publishers” have been removed to comply with the journal requirements; refer to pages 14-18.

Comment

There are several places throughout the manuscript with extra spaces between the words (e.g., line 39, line 41). Please correct the text.

Response

Extra spaces have been removed from the entire manuscript.

Overall Comments from Reviewer 3

The manuscript compared the content validity of two instruments used in measuring nurse pediatric pain knowledge and attitudes.

As strong points of the article, it should be noted that they have had a wide participation of experts. Furthermore, it appears that the author of the questionnaire, Manworren, has been involved in the counseling process.

As weaknesses, the authors acknowledge both in the limitations section and in the conclusions that, as future lines, a validation of the criterion and the construct is required.

From my point of view, we must differentiate between (Lobiondo-Wood G, Haber J.; 2013) :

• The content validity. Content validity evaluates qualitatively whether the questionnaire covers all the dimensions of the phenomenon that wants to measure, since it is considered that an instrument is valid in its content if contemplates all related aspects with the concept that measures. The researcher begins by defining the concept and identifying the attributes or dimensions of the concept. The items that reflect the concept and its domain are developed.

• The apparent validity. It is a subtype of content validity. It is a face validity, which is a rudimentary type of validity that basically verifies that the instrument fives the appearance of measuring the concept. It is an intuitive type of validity in which colleagues or subjects are asked to read the instrument and evaluate the content in terms of whether it appear to reflect the concept the researcher intends to measure. Or if the elements included in an instrument are relevant.

• Criterion validity. It is the degree of correlation between an instrument and another measure of the variable under study that serves as criterion or reference.

• Construct validity. It is understood as the degree to which an instrument measures the bipolar evaluative dimension for which it was designed.

Actually, in the present article, what they do is to measure apparent validity. It is important because the acceptance of a scale by several people gives consistency when using it. However, apparent and content validity is a relevant method especially when designing an instrument. It is not so important when the instrument has been previously validated and used in different areas.

On the other hand, and as the authors say, these tools have already been validated and used in various studies. Knowing that its content validity has already been reviewed as they describe and explain. In that case, why have they revised the apparent validity again? Why haven't they gone a step further? Why have the criteria not been applied and reviewed?

For these reasons, I think the article does not reach the level required for a journal like PLOS ONE, and should be rejected.

From my point of view, the approach and content is well developed, but due to the characteristics of the tools used, these require a criterion validity, a construct validity, or a cross-cultural validation.

The authors summarize the main research question and key findings. Even, the authors identify other literature on the topic and explain how the study relates to this previously published research.

However, I would like to make some specific suggestions in the next point.

Response

Thanks for the feedback. As you already know, a valid instrument is a reliable one but a reliable instrument does not guarantee validity. Thus, validity is an important quality expected of instruments. As you rightly point out and explain, there are several types of validity (content, construct, criterion, discriminant among others). The choice on the type of validity to be assessed depends on the aim of this enterprise. Our reasons for assessing content validity as the starting point for other types of validity and reliability in the future have been provided in the manuscript (on page 2, lines 25-27; page 4, lines 92-95 and page 5, lines 96-99). It is significant to also mention that, the changes which have been made in the instruments would have been missed if we had not assessed their content validity as the starting point for other types of validity and reliability assessments to be explored in the future.

Specific Comments from Reviewer 3

Comment

As a suggestion, and to contextualize more the article from the first moment, the title could refer to Pediatric Healthcare Providers' (HCPs) or pediatric nurses as well as keywords.

In fact, HCPs are referenced every time in the page 3, line 61 paragraph.

Another more clarifying explanation is found in page 3 and line 68: “These two instruments were developed to measure healthcare professionals and students’ knowledge and attitudes regarding children’s pain [13].” But it is still a personal appreciation.

Response

The titles (long and short) and keywords have been amended; refer to page 1, lines 1-7 and page 3, line 54.

Comment

Page 2; line 24. The introduction does not set the stage adequately. As in the title, it is required to specify who the study is aimed at. To miss this information may decontextualize and imply the study population.

In the abstract, the authors explain why the study matters and put the research in context properly. However:

In page 2; line 25; the authors clarify that “This was considered necessary due to the universal differences in culture, semantics and healthcare resources in different parts of the globe”, but in the following paragraph:

In page 2; line 31; they specify that the experts only will check the relevance and clarity of the items will be reviewed without mention culture, semantics and healthcare resources. If it is so important, because you mention it at the beginning of the abstract, could the experts have been asked about the changes due to the cultural factor? Has any change been made due to semantic and cultural changes?

Response

Thanks for the feedback. We however disagree with this comment as the introduction provides the background information and justification for the current study. We appreciate your suggestion to include the target population as part of the title but this would make it too lengthy as the PNKAS was specifically designed for nurses whilst the PHPKASRP was directed at pediatric healthcare professionals in general.

Changes have been made to the assessable areas to reflect what happened in practice; this has been provided on page 6, lines 130-133.

At the end of the validation process, five items each were modified on the PNKAS and PHPKASRP as outlined on page 11, lines 249-251. Some of these changes were related to semantics (for instance, question 37 of PNKAS and questions 11 and 37 of PHPKASRP) and cultural changes (for instance, questions 7 and 36 of PNKAS and questions 10 and 36 of PHPKASRP).

Comment

Content validity is a relevant method especially when designing an instrument. It is not so important when the instrument has been previously validated and used in different areas. However, when an instrument is translated into another language, if the explored concepts are supposed to change significantly from one culture to another, it may be useful to recheck the face validity.

Moreover, in page 4, line 92 It is said that validity is not the property of an instrument, but depends on the interpretation related to the context and participants. However, knowledge of health is based on science, evidence, principles, theories, and is universal. Therefore, they do not depend on the interpretation of people and cultures.

Content validity evaluates qualitatively whether the questionnaire covers all the dimensions of the phenomenon to be measured, since an instrument is considered to be valid in its content if it considers all aspects related to the concept it measures. For this, it is necessary to have a clear idea of the conceptual aspects to be measured. And in this case, since the instrument had previously been validated and used in different areas, the content was already available.

Response

Thanks for the feedback. As you already know, a valid instrument is a reliable one but a reliable instrument does not guarantee validity. Thus, validity is an important quality expected of instruments. As you rightly point out and explain, there are several types of validity (content, construct, criterion, discriminant among others). The choice on the type of validity to be assessed depends on the aim of this enterprise. Our reasons for assessing content validity as the starting point for other types of validity and reliability in the future have been provided in the manuscript (on page 2, lines 25-27; page 4, lines 92-95 and page 5, lines 96-99). It is significant to also mention that, the changes which have been made in the instruments would have been missed if we had not assessed their content validity as the starting point for other types of validity and reliability assessments to be explored in the future.

Comment

Page 3; line 67. This is what I cannot understand. In this section, the questions that arise are: When talking about the revised version (PHPKASRP). Is the article review being done for the first time in this article? Or has this version already been created before? Why do you call short version if they have almost the same number of questions?...May be could you give more details or change the way to explain it.

Response

Changes have been made as found on page 4, line 71.

Comment

In the next sentence: page 3; line 68. “These two instruments were developed to measure healthcare professionals and students’ knowledge and attitudes regarding children’s pain (13)”. We find the reference of the PNKAS, what about the PHPKASRP reference? Could you explain this better?

Response

The reference for PHPKASRP has been provided on page 4, lines 72-73.

Comment

When talking about instrument validation, I would give more information about whether or not PHPKASRP is validated, etc.

Response

Information on the validity of PHPKASRP has been provided on page 4, lines 85-87.

Comment

Page 4; line 84. References for PHPKASRP are again needed.

Response

Citation of this instrument has been provided on page 4, lines 87.

Comment

The information in Tables 1 and 3 are explained in the text. Table 1 could be omitted by completing the information in the text. However, Table 3 can be kept, since it serves as a synthesis of the results.

Response

As suggested, Table 1 has been omitted whilst Table 3 has been retained as the only table within the manuscript. Table 2 has been designated as S1 Table in the supplementary file named “S1_Table.docx”.

Comment

Page 6; paragraph line 128-133.There is no specific reference to cultural and semantic factors. Despite the emphasis that has been given in the introduction by providing references.

They talk about: comprehensiveness, objectivity, organization and relevance defined as “comprehensive: that issue containing important information to reach the objective of the study, stated in a comprehensible manner; Objective: that issue which is easy to understand; organization: the disposition of the issues and alternatives as well as their content; Relevant: that question which is related to achieving the goal of the research”.

Response

Changes have been made to reflect what occurred in practice, refer to page 6, lines 130-133.

Comment

Page 8; line 182. Again, the article reference of the revised version (PHPKASRP instrument) is missed.

Response

Citation to this instrument has been provided on page 8, lines 182-183.

Comment

The results on page 10 should reference Table 3, where the results are summarized.

The authors acknowledge both the limitations and the conclusions that, as future lines, a validation of the criterion and the construct is required.

Response

References to the now “Table 1” has been provided on page 10, lines 230-231.

Comment

This type of validity requires basic statistics. Experts and / or researchers use in evaluating the relevance of a scale that requires descriptive analysis.

Response

Due to the confusion introduced by the mention of “by pediatric experts”, this phrase has been removed to enhance clarity. Refer to page 7, line 152.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

23 Jul 2020

PONE-D-20-05637R1

Evaluating the content validity of two versions of an instrument used in measuring pediatric pain knowledge and attitudes in the Ghanaian context.

PLOS ONE

Dear Dr. Kusi Amponsah,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR:

Please, put efforts to answer properly the reviewers and to improve the manuscript as suggested.

==============================

Please submit your revised manuscript by Sep 06 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Matias Noll, Ph.D

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

Reviewer #3: All comments have been addressed

Reviewer #4: All comments have been addressed

Reviewer #5: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Yes

Reviewer #4: Partly

Reviewer #5: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: (No Response)

Reviewer #5: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I just have to congratulate the authors for their work on modifications. They have carefully read and responded to each and every reviewer review and comment. I am very satisfied with the final work, and among all I think that the first version has improved a lot.

Reviewer #2: I thank the authors for the improvements they have made to the manuscript. However, I still have some concerns related to my original comment in the first review.

Reviewer #3: Dear Editor

Thank you again for the opportunity to review this manuscript. The manuscript aims to evaluate and compare the content validity of PNKAS and its revised version, PHPKASRP from Ghanaian perspective. The authors have done an excellent work to address the comments I raised in the first review. However, there are still some relatively minor comments that need to be addressed. Please find them below.

Introduction

Lines 70-73. “Key among them are Pediatric Nurses’ Knowledge and Attitudes Survey regarding pain (PNKAS) [13] and 71 its revised version, the Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (PHPKASRP) [Manworren RCB, personal communication, August 16, 2018].”

According to the authors, the developer is yet to publish the PHPKASRP. However, it has been almost two years since the developer provided the authors with the PHPKASRP. I’m wondering why the PHPKASRP was not published yet and the reasons behind that, although, according to the authors, “it has been translated into other languages and permitted for use in many organizations around the world”, and was described as a “key” instrument.

I think the PHPKASRP, until this date, does not reach the level to be a “key” instrument since there is no single publication about it, except for the abstract (ref. 17), or about its psychometric properties in English and other languages.

Lines 85-89. “Content validity of the PHPKASRP has been established by national content experts comprising of physicians, pediatric nurses and pharmacists in the United States of America [Manworren RCB, personal communication, August 16, 2018]. According to the developer, the instrument has been translated into other languages and permitted for use in many organizations around the world [Manworren RCB, personal communication, August 16, 2018].

Further, I do not think it would appropriate to state such information based on personal communication alone and not based on published evidence. The authors need to remove these statements or cite them properly.

Discussion

Lines 282-285. “In line with the recommendations of Delgado-Rico and colleagues [43], decisions on items (i.e., elimination, modification or conservation) were made on the basis of the content validity indices, feedback given by experts, inputs from the instrument developer and the contribution of the items to the overall construct under investigation”.

I still think you need to cite Delgado-Rico and colleagues in your methods since you followed their recommendations in your decisions making. The authors need to explain the reasons for not citing them if they disagree with that.

Appendices

There are several spelling mistakes in “S1 Appendix. Revised Pediatric Nurses’ Knowledge and Attitudes Survey regarding pain (r-PNKAS)”. For example, items 8, 10, 22, 23, and 33. Please correct them and check the rest of this document and other documents carefully.

Reviewer #4: The authors did a great job in adjusting the manuscript in response to the reviewers' comments, but a few issues remain. A main issue that remains for me is that the current data presented on its own isn't very strong and I still believe the current manuscript would be stronger if the authors could add data on further validation studies with the modified instruments.

Minor comments:

1) In the introduction, p. 4 line 84, it is unclear to me that the PHPKASRP is a variation of the previously discussed instrument. That only becomes clear to me on p. 8 line 180. Therefore I would suggest including this information earlier on.

2) The data analyses section does not provide any details on what is being done with the comments made by the participants on appropriateness to the culture, semantics (comprehensibility, simplicity, grammatical construction) and healthcare resources available in the Ghanaian context. It only becomes clear in the results that the purpose of these comments is to redesign these instruments. More detail on how these comments were systematically dealt with to redesign the instruments is needed.

3) I find it strange that the development of the PNKAS and PHPKASRP instruments is detailed at the end of the methods sections, I would prefer to see this more upfront, might even fit in the introduction.

Reviewer #5: Dear authors,

I highly recommend you to discuss and take in account previous articles that purpose questionnaires for evaluation of Back Pain. In a quick search I fond some articles that can be improve your introduction as well as the discussion, as follow:

Spanish translation, cross-cultural adaptation and validation of the Argentine version of the Back Pain Attitudes Questionnaire Pierobon, A., Policastro, P.O., Soliño, S., (...), Raguzzi, I.A., Villalba, F.J. 2020 Musculoskeletal Science and Practice

46,102125

Is There Equivalence between the Electronic and Paper Version of the Questionnaires for Assessment of Patients with Chronic Low Back Pain? Azevedo, B.R., Oliveira, C.B., Araujo, G.M.D., (...), Pinto, R.Z., Christofaro, D.G.D. 2020 Spine

45(6), pp. E329-E335

Back Pain and Body Posture Evaluation Instrument (BackPEI): Development, content validation and reproducibility Noll, M., Tarragô Candotti, C., Vieira, A., Fagundes Loss, J. 2013 International Journal of Public Health

58(4), pp. 565-572

Psychometric Study and Content Validity of a Questionnaire to Assess Back-Health-Related Postural Habits in Daily Activities Monfort-Pañego, M., Miñana-Signes, V. 2020 Measurement in Physical Education and Exercise Science

Validation of the Japanese Version of the Fremantle Back Awareness Questionnaire in Patients with Low Back Pain Nishigami, T., Mibu, A., Tanaka, K., (...), Stanton, T.R., Moseley, G.L. 2018 Pain Practice

18(2), pp. 170-179

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Vicente Miñana-Signes (PhD)

Body Languages Didactics Department

Teacher Training Faculty

University of Valencia

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

Reviewer #5: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PONE-D-20-05637R1 Review.docx

Click here for additional data file.

1 Oct 2020

Overall Comments from Reviewer 1

Reviewer #1: I just have to congratulate the authors for their work on modifications. They have carefully read and responded to each and every reviewer review and comment. I am very satisfied with the final work, and among all I think that the first version has improved a lot.

Response

Thank you.

Overall Comments from Reviewer 2

Reviewer #2: I thank the authors for the improvements they have made to the manuscript. However, I still have some concerns related to my original comment in the first review.

Response

Thanks for accepting to review our manuscript. Responses to the additional concerns have been provided below.

Overall Comments from Reviewer 3

Reviewer #3: Dear Editor

Thank you again for the opportunity to review this manuscript. The manuscript aims to evaluate and compare the content validity of PNKAS and its revised version, PHPKASRP from Ghanaian perspective. The authors have done an excellent work to address the comments I raised in the first review. However, there are still some relatively minor comments that need to be addressed. Please find them below.

Response

Thank you all for the feedback. Please find below our responses to the concerns raised.

Specific Comments from Reviewers 1-3

Comment

Introduction

Lines 70-73. “Key among them are Pediatric Nurses’ Knowledge and Attitudes Survey regarding pain (PNKAS) [13] and 71 its revised version, the Pediatric Healthcare Providers’ Knowledge and Attitudes Survey Regarding Pain (PHPKASRP) [Manworren RCB, personal communication, August 16, 2018].”

According to the authors, the developer is yet to publish the PHPKASRP. However, it has been almost two years since the developer provided the authors with the PHPKASRP. I’m wondering why the PHPKASRP was not published yet and the reasons behind that, although, according to the authors, “it has been translated into other languages and permitted for use in many organizations around the world”, and was described as a “key” instrument.

I think the PHPKASRP, until this date, does not reach the level to be a “key” instrument since there is no single publication about it, except for the abstract (ref. 17), or about its psychometric properties in English and other languages.

Response

Thanks for the feedback. We do not know why the instrument developer (Manworren, R.C.B) has not yet published findings from the PHPKASRP. She however, indicated in one of our email correspondences that our work has served as a reminder for her to publish findings related to the PHPKASRP. We have amended the statement relating to the PHPKASRP as a “key” instrument on the subject. Refer to page 3, lines 66-71.

Comment

Lines 85-89. “Content validity of the PHPKASRP has been established by national content experts comprising of physicians, pediatric nurses and pharmacists in the United States of America [Manworren RCB, personal communication, August 16, 2018]. According to the developer, the instrument has been translated into other languages and permitted for use in many organizations around the world [Manworren RCB, personal communication, August 16, 2018].

Further, I do not think it would appropriate to state such information based on personal communication alone and not based on published evidence. The authors need to remove these statements or cite them properly.

Response

Amendments have been made to reflect the state of affairs regarding these issues. Refer to page 4, lines 85-88.

Comment

Discussion

Lines 282-285. “In line with the recommendations of Delgado-Rico and colleagues [43], decisions on items (i.e., elimination, modification or conservation) were made on the basis of the content validity indices, feedback given by experts, inputs from the instrument developer and the contribution of the items to the overall construct under investigation”.

I still think you need to cite Delgado-Rico and colleagues in your methods since you followed their recommendations in your decisions making. The authors need to explain the reasons for not citing them if they disagree with that.

Response

As suggested, Delgado-Rico and colleagues have been cited under the materials and methods section on page 8, lines 173-176.

Comment

Appendices

There are several spelling mistakes in “S1 Appendix. Revised Pediatric Nurses’ Knowledge and Attitudes Survey regarding pain (r-PNKAS)”. For example, items 8, 10, 22, 23, and 33. Please correct them and check the rest of this document and other documents carefully.

Response

Spelling errors have been corrected in the “S1_Appendix” and “S2_Appendix” in the revised documents.

Overall Comments from Reviewer 4

Reviewer #4: The authors did a great job in adjusting the manuscript in response to the reviewers' comments, but a few issues remain. A main issue that remains for me is that the current data presented on its own isn't very strong and I still believe the current manuscript would be stronger if the authors could add data on further validation studies with the modified instruments.

Response

Thanks for the feedback. We humbly disagree with the comments relating to the strength of the current manuscript due to the absence of other validation studies as there is no published recommendation on the number of validation studies to be included in a manuscript to qualify for the supposed strength. This notwithstanding, we acknowledge the importance of additional psychometric testing (validity and reliability) of the modified instruments which have been stated in the discussion (on page 33, lines 345-347) and conclusion (on page 33, lines 356-357) sections of our manuscript. As indicated in the recommendations, other types of validity and reliability of the modified instrument would be pursued in the future.

Specific Comments from Reviewers

Comment

In the introduction, p. 4 line 84, it is unclear to me that the PHPKASRP is a variation of the previously discussed instrument. That only becomes clear to me on p. 8 line 180. Therefore, I would suggest including this information earlier on.

Response

Information on the above have been provided earlier on at page 3, lines 67-71.

Comment

The data analyses section does not provide any details on what is being done with the comments made by the participants on appropriateness to the culture, semantics (comprehensibility, simplicity, grammatical construction) and healthcare resources available in the Ghanaian context. It only becomes clear in the results that the purpose of these comments is to redesign these instruments. More detail on how these comments were systematically dealt with to redesign the instruments is needed.

Response

Details on the application of the comments in redesigning the instruments have been provided on page 8, lines 183-189.

Comment

I find it strange that the development of the PNKAS and PHPKASRP instruments is detailed at the end of the methods sections, I would prefer to see this more upfront, might even fit in the introduction.

Response

As suggested, information on the instrument development processes has been moved upfront. Refer to page 5, lines 99-120 and page 6, lines 121-132.

Overall Comments from Reviewer 5

Comment

I highly recommend you to discuss and take in account previous articles that purpose questionnaires for evaluation of Back Pain. In a quick search I found some articles that can be improve your introduction as well as the discussion, as follow:

Spanish translation, cross-cultural adaptation and validation of the Argentine version of the Back Pain Attitudes Questionnaire Pierobon, A., Policastro, P.O., Soliño, S., (...), Raguzzi, I.A., Villalba, F.J. 2020 Musculoskeletal Science and Practice

46,102125

Is There Equivalence between the Electronic and Paper Version of the Questionnaires for Assessment of Patients with Chronic Low Back Pain? Azevedo, B.R., Oliveira, C.B., Araujo, G.M.D., (...), Pinto, R.Z., Christofaro, D.G.D. 2020 Spine

45(6), pp. E329-E335

Back Pain and Body Posture Evaluation Instrument (BackPEI): Development, content validation and reproducibility Noll, M., Tarragô Candotti, C., Vieira, A., Fagundes Loss, J. 2013 International Journal of Public Health

58(4), pp. 565-572

Psychometric Study and Content Validity of a Questionnaire to Assess Back-Health-Related Postural Habits in Daily Activities Monfort-Pañego, M., Miñana-Signes, V. 2020 Measurement in Physical Education and Exercise Science

Validation of the Japanese Version of the Fremantle Back Awareness Questionnaire in Patients with Low Back Pain Nishigami, T., Mibu, A., Tanaka, K., (...), Stanton, T.R., Moseley, G.L. 2018 Pain Practice

18(2), pp. 170-179

Response

Upon reading the suggested articles, some clarifications have been made in the discussion section. Refer to page 31, lines 297-299, 311-313.

Submitted filename: Response to Review Comments_01-10-2020.docx

Click here for additional data file.

26 Oct 2020

Evaluating the content validity of two versions of an instrument used in measuring pediatric pain knowledge and attitudes in the Ghanaian context.

PONE-D-20-05637R2

Dear Dr. Kusi Amponsah,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Matias Noll, Ph.D

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: After reading the revised manuscript, the authors have addressed all the comments I have raised. I would like to thank them for addressing my comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Hamad S. Al Amer, PT, PhD

28 Oct 2020

PONE-D-20-05637R2

Evaluating the content validity of two versions of an instrument used in measuring pediatric pain knowledge and attitudes in the Ghanaian context

Dear Dr. Kusi Amponsah:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Matias Noll

Academic Editor

PLOS ONE