Translation and validation of the Malay version of the Stroke Knowledge Test.

BACKGROUND: To date, there is a lack of published studies on assessment tools to evaluate the effectiveness of stroke education programs.
METHODS: This study developed and validated the Malay language version of the Stroke Knowledge Test research instrument. This study involved translation, validity, and reliability phases. The instrument underwent backward and forward translation of the English version into the Malay language. Nine experts reviewed the content for consistency, clarity, difficulty, and suitability for inclusion. Perceived usefulness and utilization were obtained from experts' opinions. Later, face validity assessment was conducted with 10 stroke patients to determine appropriateness of sentences and grammar used. A pilot study was conducted with 41 stroke patients to determine the item analysis and reliability of the translated instrument using the Kuder Richardson 20 or Cronbach's alpha.
RESULTS: The final Malay version Stroke Knowledge Test included 20 items with good content coverage, acceptable item properties, and positive expert review ratings. Psychometric investigations suggest that Malay version Stroke Knowledge Test had moderate reliability with Kuder Richardson 20 or Cronbach's alpha of 0.58. Improvement is required for Stroke Knowledge Test items with unacceptable difficulty indices. Overall, the average rating of perceived usefulness and perceived utility of the instruments were both 72.7%, suggesting that reviewers were likely to use the instruments in their facilities.
CONCLUSIONS: Malay version Stroke Knowledge Test was a valid and reliable tool to assess educational needs and to evaluate stroke knowledge among participants of group-based stroke education programs in Malaysia.

Introduction

Stroke is one of the top five common causes of mortality in Malaysia with a rate at 8.43/100,000 population [1], [2]. Stroke impact is not limited to physical disabilities that stroke patients experience but also burdens family caregivers. Therefore, it is important to highlight stroke prevention measures for primary or secondary prevention. In addition to the biomedical approach, stroke prevention must be emphasized via stroke education [3], [4], [5].

Stroke can be prevented by modifying unhealthy lifestyles such as poor dietary patterns, obesity, smoking, excessive alcohol intake, uncontrolled diabetes, hypercholesterolemia, and lack of exercise [6]. Previous studies proved that healthy lifestyles are very important mechanisms to prevent stroke occurrence or to reduce recurrent events. Thus, stroke prevention that highlights lifestyle modification at either primary or secondary prevention should begin with delivering knowledge to the community, patients, and family caregivers.

Many public health campaigns internationally included public education programs, stroke education for patients and family members, and group-based information programs to increase stroke knowledge and awareness. In Malaysia, most of the public education strategy has increased public understanding of prevention and control of chronic diseases such as diabetes, hypertension, and heart disease. Furthermore, this approach was also proven effective in the prevention of non-communicable diseases such as the spread of H1N1 and typhoid during floods. However, a focus of educational intervention on stroke remains lacking in Malaysia.

There must be a significant emphasis on assessing the effectiveness of stroke educational interventions. Generally, stroke education evaluation studies should include multiple outcome measures across multiple domains, but at least one measure will usually include stroke knowledge assessment. In order to evaluate the nature and extent of change in stroke knowledge, systematically constructed stroke knowledge measurement with good psychometric properties must be included in stroke evaluation studies.

Thus, this study developed and validated the Stroke Knowledge Test (SKT) in a Malay language version. Three investigation stages were completed. The first stage involved backward and forward translations from the original English tool to the Malay language. The second stage was conducting expert and stroke patients’ validation of the test items and investigation of item properties. The last stage determined reliability of the Malay version SKT.

Materials and methods

The present study was conducted in a cross-sectional manner. The source population for this study was 9 stroke experts and 41 stroke patients admitted to medical and surgical wards in Hospital Universiti Sains Malaysia. This study began in September 2011 and ended in December 2012. Ethical clearance was obtained from the Universiti Sains Malaysia, Human Ethical Committee [Ref. no. USMKK/PPP/JEPeM 254.4 (1.1)] prior to study.

The SKT instrument was adapted from a study by Sullivan and Dunton [7] consisting of 20 items on stroke risk factors, signs and symptoms, prevention, prevalence, treatment, and rehabilitation. The 20 multiple choice question (MCQ) items were constructed in a five-alternative multiple choice format which consisted of one correct option, three distracters, and an “I don’t know” option to reduce the tendency to guess. MCQ format was chosen since it has the ability to broadly sample content domain within a reasonable time limit. Each correct answer was given one (1) mark and a wrong answer was given no (0) mark. Possible scores ranged from 0 to 20. Higher score indicates good knowledge.

Data obtained from the study were entered in an Excel spreadsheet and Statistical Package for Social Sciences (SPSS) version 20.0 software. Descriptive statistics were employed to summarize the results. Data were interpreted as means and standard deviations for numerical outcomes and frequency and percentages for categorical outcomes.

Pilot study

This phase was completed before proceeding to the next study phase in order to check the compatibility of the items in the translated assessment tools. There are three procedures involved in this portion: (1) translation, (2) validation, and (3) reliability (Fig. 1).

Fig. 1

Translational validation of the Malay version SKT.

Stage 1: Translation procedure

The SKT was forward and backward translated into Malay by two bilingual experts and subsequently back translated into English by two additional bilingual experts from Language Centre of the Health Campus, USM. The language experts compared the original English instrument with the back translated Malay instrument and edited to obtain the matched Malay version. Following minor adjustments, a final English version was used to re-evaluate the Malay versions. After further discussion, the final Malay versions of the instruments were ready to be used in the study.

Overall, there were 20 MCQ items of the Malay version SKT reviewed and finalized before the questionnaire was compiled. The knowledge items were purposely translated to address the educational needs of stroke patients as well as to evaluate the knowledge outcome. An item was reconstructed to suit the study population in a Malaysian setting. The reconstructed item was about epidemiology occurrence; the researchers changed the answer choice based on the prevalence of stroke in Malaysia found in the Ministry of Health Ischemic Stroke Clinical Practice Guideline.

Stage 2: Validation procedure

Development construct of interest for the knowledge domain

The SKT construct was chosen from relevant literature to suit the study design and population setting [8], [9], [10]. There is an established literature measuring the stroke knowledge domain. However, most studies were open-ended questionnaires requiring extensive time to answer the questions. This indirectly contributes to low response rates. Furthermore, the analysis procedure takes longer since extensive coding is required when study participants answer in various forms [11], [12], [13].

Therefore, the researchers chose an instrument from an Australian study developed by Sullivan and Dunton [7] with good psychometric properties based on its published paper. The researchers chose this instrument because the original study was also conducted among stroke patients who were undergoing rehabilitation in Australia. The original author granted permission to use the instrument prior to instrument translation. The constructs pertained to general information on stroke (pathophysiology, epidemiology, signs and symptoms, prevention, complications), risk factors, treatments, rehabilitation, and emergency actions.

Validation procedure

Content validity

Content validity confirmed that the translated instruments measured what they were intended to measure and their appropriateness and relevance to the study purpose [13]. Content validity is usually undertaken by seven or more experts [14]. This validation process was completed by nine evaluators from various backgrounds in stroke care familiar with validation procedures. Each reviewer received an evaluation kit comprising cover letter, demographic information sheet, and translated instruments prior to review [7].

The researchers clearly defined the conceptual framework of the educational intervention prior to content validity estimation. Experts reviewed the SKT draft of 20 MCQ items to ensure its consistency with the conceptual framework. Expert reviewers evaluated each item on four dimensions using a dichotomous response scale: “clear=1” vs. “not clear=0.” The four dimensions were (a) item consistency to content area, (b) item wording clarity, (c) perceived item difficulty, and (d) whether (and why) they thought the item should be included in a revised version of the test [7], [15].

Expert reviewers also rated the perceived usefulness of an item using a five-point Likert scale ranging from “not useful” to “very useful.” Finally, expert reviewers stated how likely they were to use the instruments in their workplace using a five-point Likert scale ranging from “unlikely to use” to “likely to use.” Extra space was provided at the end of the evaluation kit for further comments from the experts.

Face validity

Face validity determines the instrument’s appropriateness to evaluate the construct of interest [14]. Face validity does not refer to whether the instrument measures what it actually measures; it assesses whether it is superficially valid to the examinees. In addition, face validity should be conducted among technically untrained observers [16]. It is the easiest validation process to undertake but the weakest form of validity. In this study, face validity was completed via a standardized form to evaluate the Malay version of the SKT. Ten adult stroke patients were randomly selected from two medical wards in USM Hospital and completed the face validity form. They evaluated the overall features of the instrument and whether sentences were clear, concise, easy to understand, and free from typographical errors.

Scoring method and item analysis for Malay version of the SKT

Initially, the 20 items were summed to obtain the stroke patients’ overall scores. After that, the scores were arranged in a descending manner. Then, the total number of participants in 27% of the upper group (UG) and that in 27% of the lower group (LG) who obtained the correct responses were counted. Ebel and Frisbie [17] suggested 27% as the optimal compromise between the two conditions: (i) the two groups had as many stroke patients as possible and (ii) they were as far apart as possible. Item difficulty was calculated using the following formula: Difficulty Index (P) (R)/Total number of stroke patients who attempted the question (T).

An item was considered difficult when the difficulty index value was less than 30%; the item was considered easy when the index value was greater than 70% [7]. However, 20–80% (0.2–0.8) was also an acceptable range [18], [19], [20]. Thus, for the purposes of this study, the researchers adopted a range of 20–80% (0.2–0.8) for the difficulty index.

Meanwhile, the item discrimination index measures the differences between the percentages of participants in the UG and LG who obtained the correct responses. The discrimination index was calculated using the formula D=(UG−LG)/n. The higher the discrimination index, the better the test item can be discriminated between participants with higher test scores and those with lower test scores. Based on Ebel and Frisbie [17] guidelines on classical test theory item analysis, items were categorized in their discrimination indices: (a) 0.40 and above for excellent items, (b) 0.30–0.39 for good items, (c) 0.20–0.29 for acceptable items that were subject to improvement, and (d) 0.19 or less for poor items that were to be rejected or improved by revision [17], [21].

Stage 3: Reliability testing

For reliability testing, the internal consistency was examined by Kuder–Richardson 20 (KR20) or Cronbach’s alpha for the knowledge domain. Cronbach’s alpha value of 0.50–0.70 was acceptable [22] while 0.70 or higher shows good homogeneity among the items [14], [23].

Results

The purpose of this study was to follow a systematic validation procedure for a stroke knowledge instrument to evaluate potential applications in any stroke education program [24]. A series of steps comprising content and face validation, item analysis, and reliability checking of the SKT were completed. The validation process resulted in 20 items that had acceptable content coverage, improved clarity, and relevant difficulty levels, and were appropriate to be included in the final SKT version.

The SKT were forward and backward translated into Malay by two bilingual experts and subsequently back translated in English by two bilingual additional experts. The Language Centre of the Health Campus, USM supervised the translation process. The language experts compared the original English version instrument with the back translated Malay version instrument and edited to obtain the matched Malay version. After minor adjustments, a final English version was used to re-evaluate the Malay versions. After further discussion, the final Malay version of the instrument was accepted.

Content validity

Validity is defined as the ability of the instrument to measure the attributes of the construct under study [14]. There are two types of validity: (1) translational validity and (2) criterion validity. Content and face validity comprise translational validity. On the other hand, concurrent, predictive, convergent, and discriminant are categorized under criterion validity. This study employed content validity and face validity only.

Initially, there were 33 evaluators involved in the content validation process from various backgrounds (physicians, counselor, nurses, physiotherapist, radiologist, dietitian, pharmacist, and teacher) who were also experts in stroke care and familiar with validation procedures. However, the researchers decided to analyze the data obtained from only nine (9) evaluators as they provided comprehensive recommendations.

The experts reviewed the 20 MCQ draft items on the SKT and evaluated each item on four dimensions using a dichotomous response scale: “clear=1” vs. “not clear=0.” The four dimensions were (a) item consistency to content area, (b) item wording clarity, (c) perceived item difficulty, and (d) whether (and why) they thought the item should be included in a revised version of the test [7], [15]. The experts were given one week to complete the evaluation.

The overall score from each evaluator was 80. An average content validity index of 0.901 was obtained (Table 1). This index indicated that SKT items were relevant and clear since the recommended cut-off value for content validity index is 0.75 [25]. However, some of the items were modified further as recommended by the experts since certain question structures were not clear and could create confusion (Table 2).

Table 1

Content validity index (CVI) of SKT.

Experts	Consistency	Clarity	Difficulty	Inclusion	Total score
Hematology specialist	16/20	16/20	15/20	15/20	62/80
Counselor	20/20	20/20	16/20	16/20	72/80
Nurse (Clinician)	20/20	15/20	19/20	19/20	73/80
Nurse (Educator)	20/20	19/20	19/20	19/20	77/80
Physiotherapist	20/20	18/20	18/20	20/20	78/80
Radiologist	19/20	19/20	19/20	19/20	76/80
Dietitian	19/20	20/20	19/20	20/20	78/80
Pharmacist	18/20	17/20	15/20	15/20	65/80
Teacher	20/20	14/20	20/20	14/20	68/80
Score					649/720
Content validity index (CVI)					0.901

Table 2

Amendment of questionnaire based on experts’ recommendation.

No.	Malay version		English version
	Before amendment	After amendment
1	Strok yang kerap berlaku berpunca daripada	Strok yang kerap berlaku di dalam masyarakat Malaysia berpunca daripada	The most common type of stroke occurs when
	A. Pengaliran darah ke otak tersumbat. B. Mengalami serangan jantung. C. Pendarahan di dalam otak. D. Terlalu kerap berjemur di bawah matahari. E. Saya tidak pasti.	A. Pengaliran darah ke otak tersumbat. B. Mengalami serangan jantung. C. Pendarahan di dalam otak. D. Terlalu kerap berjemur di bawah matahari. E. Saya tidak tahu.	A. The blood supply to the brain is blocked. B. You are having a heart attack. C. There is bleeding in the brain. D. You׳ve had too much sun. E. I don׳t know.
2	Berikut merupakan faktor risiko untuk mendapat strok sebanyak dua kali ganda.	Faktor berikut meningkatkan risiko anda diserang strok sebanyak dua kali ganda.	Which of the following will double your risk of stroke?
	A. Masalah lelah. B. Kencing manis. C. Senaman agresif. D. Semua di atas. E. Saya tidak pasti.	A. Anda mempunyai masalah lelah. B. Anda penghidap diabetes. C. Anda bersenam secara berlebihan. D. Semua di atas. E. Saya tidak tahu.	A. If you are asthmatic. B. If you are diabetic. C. If you exercise too much. D. All of the above. E. I don׳t know.
3	Sejenis keadaan degupan jantung tidak regular yang dipanggil Fibrilasi Atria (Atrial Fibrillation).	Sejenis keadaan degupan jantung tidak regular yang dipanggil Fibrilasi Atria (Atrial Fibrillation).	A type of irregular heartbeat known as atrial fibrillation (AF)
	A. Mengurangkan risiko strok. B. B.Meningkatkan risiko strok sebanyak 2 kali. C. Meningkatkan risiko strok sebanyak 5 kali. D. Bukan faktor risiko strok. E. Saya tidak pasti.	A. Mengurangkan risiko strok. B. B.Meningkatkan risiko strok sebanyak 2 kali ganda. C. Meningkatkan risiko strok sebanyak 5 kali ganda. D. Bukan faktor risiko strok. E. Saya tidak tahu.	A. Decreases the risk of stroke. B. Doubles the risk of stroke. C. Increases the risk of stroke by more than 5 times. D. Is not a risk factor of stroke. E. I don׳t know.
7	Tujuan rehabilitasi bagi individu yang mengalami strok ialah untuk	Tujuan rehabilitasi (pemulihan) bagi individu yang mengalami strok ialah untuk	For someone who has had a stroke, the main purpose of rehabilitation is to
	A. Memastikan mereka tidak mengambil dadah. B. Mengurung mereka di dalam hospital seberapa lama yang mungkin. C. Meningkat tahap keupayaan fungsi harian. D. Mengelakkan dari berfikir tentang strok. E. Saya tidak pasti.	A. Memastikan individu strok tidak mengambil dadah. B. Menahan individu strok di dalam hospital seberapa lama mungkin. C. Meningkatkan tahap keupayaan fungsi harian. D. Mengelakkan individu strok daripada berfikir tentang strok E. Saya tidak tahu.	A. Make sure they don’t take drugs. B. Keep them in hospital as long as possible. C. Improve their level of daily functioning. D. Keep their mind off it. E. I don’t know.
19	Sekiranya seseorang mengalami strok, bilakah anda perlu menelefon ambulans?	Sekiranya seseorang mengalami strok, bilakah anda perlu menelefon ambulans?	If someone has a stroke, when should you call for an ambulance?
	A. Telefon hanya apabila simptom masih kekal dalam tempoh 24 jam. B. Sentiasa telefon untuk bantuan ambulans dengan segera. C. Berjumpa doktor apabila keadaan mengizinkan. D. Tidak perlu menelefon untuk bantuan ambulans. E. Saya tidak pasti	A. Telefon hanya apabila simptom masih kekal dalam tempoh 24 jam. B. Segera telefon untuk bantuan ambulans. C. Berjumpa doktor apabila keadaan mengizinkan. D. Tidak perlu menelefon untuk bantuan ambulans. E. Saya tidak pasti	A. Only call if the symptoms remain after 24 h. B. Always call for an ambulance immediately. C. Just see your doctor when you can. D. You don’t need to call an ambulance. E. I don׳t know

Note: Bold item is the correct answer.

Face validity

For face validation, the SKT Malay version was administered to 10 stroke patients from USM Hospital. All respondents rated the overall items relevant to measure stroke knowledge. However, some items in the SKT were rated as difficult. The researchers realized that when the questions were difficult, patients had a higher tendency to leave it blank. Additionally, the stroke patients also suggested the researchers use an appropriate type, a larger font, simple sentences, and easy language. Based on their comments, several modifications were made to improve the items as listed in Table 3.

Table 3

Face validation based on stroke patients’ comments and modifications of the general questionnaire format.

No.	Respondents’ comments	Before modification	After modification
1	Words were too small.	Font size: Arial Narrow 11	Font size: Arial Narrow 12
2	Pages were packed with words.	No space between the stem (question) and leave (answers) of each item.	A space was provided between the stem (question) and leave (answers) of each item.
3	The use of scientific words in the answer.	One of the stem (question) had an option of leave (answers) written as “genetik.”	The option of leave (answers) was changed to “keturunan.”
4	The use of inappropriate options for an answer.	One of the stem (question) had an option of leave (answers) written as “Saya tidak pasti”	The options of leave (answers) was changed to “Saya tidak tahu”

Item analysis procedure

Socio-demography characteristics (n=41)

A cross-sectional study was completed with 41 stroke patients to examine the validity and reliability of the SKT at USM Hospital (Table 4).

Table 4

Socio-demographic characteristics of stroke patients (n=41).

Characteristics	Stroke patients
Freq. (%)	Mean (SD)	Median (IQR)
Age (years)		58.76±10.89
Age (categorical)
<35	1 (2.4)
35–45	3 (7.3)
46–55	11 (26.8)
56–65	16 (39.0)
> 65	10 (24.4)
Gender
Male	17 (41.5)
Female	24 (58.5)
Race
Malay	41 (100.0)
Marital status
Single	2 (4.9)
Married	34 (82.9)
Divorce	1 (2.4)
Widow/Widower	4 (9.8)
Educational status
No schooling	12 (29.3)
Primary	10 (24.4)
Secondary	12 (29.3)
College/University	7 (17.1)
Working status
Not working	3 (7.3)
Self-employed	15 (36.6)
Government sector	14 (34.1)
Private sector	3 (7.3)
Housewife	6 (14.6)
Monthly income status (RM amount)			800 (1500)
Monthly income status (RM amount)
<RM 400	3 (7.3)
RM 400–699	8 (19.5)
RM 700–999	11 (26.8)
RM 1000–1999	6 (14.6)
RM 2000–2999	5 (12.2)
RM 3000–3999	5 (12.2)
RM 4000–4999	3 (7.3)

Item analysis

The descriptive statistics for the knowledge domain showed no floor or ceiling effects. The score at the 25th percentile was 6, at the 50th was 9, and at the 75th was 11. The mean score was 8.44 (SD=2.98) out of 20 items. No participants gained the possible minimum (0) or maximum (20) scores for the knowledge domain. The minimum score obtained by the stroke patients was 1 while the maximum score was 14. This showed that the overall SKT Malay version feasibility was good.

The difficulty and discrimination indices examined the knowledge items of the SKT Malay version (Table 5). Of the 20 items, they were mostly between the acceptable range for the difficulty index (0.2–0.8) except for items 3, 13, 14, 15, and 18. Similarly, a few items had low discrimination indices such as items 2, 3, 4, 16, 18, and 19. Among these, items 3 and 18 were difficult. However, the researchers decided to retain the questions since they measured various aspects of stroke disease.

Table 5

Difficulty index, discrimination index, and their interpretation for SKT knowledge domain.

Item	Questions in Malay (English)		Difficulty Index	Discrimination Index
1	Strok yang kerap berlaku di dalam masyarakat Malaysia berpunca daripada	Interpretation:	0.41	0.64
(The most common type of stroke occurs when)	Acceptable item	Excellent
2	Faktor berikut meningkatkan risiko anda diserang strok sebanyak dua kali ganda?	Interpretation:	0.36	0.00
(Which of the following will double your risk of stroke?)	Acceptable item	Poor
3	Sejenis keadaan degupan jantung laju dan tidak teratur dipanggil Fibrilasi Atria (AF) akan	Interpretation:	0.00	0.00
(A type of irregular heartbeat known as atrial fibrillation (AF))	Difficult item	Poor
4	Manakah antara peringkat umur berikut PALINGBERISIKO mengalami strok?	Interpretation:	0.36	0.00
(Which age group is more at risk of stroke?)	Acceptable item	Poor
5	Tanda–tanda amaran kejadian serangan strok kecil (TIA) akan hilang	Interpretation:	0.23	0.46
(The warning signs of transient ischemic attack (TIA) disappear)	Acceptable item	Excellent
6	Antara berikut, manakah tanda-tanda awal serangan strok?	Interpretation:	0.41	0.27
(Which of the following is a warning sign of stroke?)	Acceptable item	Acceptable
7	Tujuan rehabilitasi (pemulihan) bagi individu yang mengalami strok ialah	Interpretation:	0.41	0.82
(For someone who has had a stroke, the main purpose of rehabilitation is to)	Acceptable item	Excellent
8	Pengambilan ubatan Aspirin boleh mengurangkan serangan strok secara	Interpretation:	0.64	0.55
(Taking aspirin assists in preventing stroke by)	Acceptable item	Excellent
9	Anda berisiko tinggi untuk mendapat strok sekiranya	Interpretation:	0.64	0.36
(You are at greater risk of stroke if)	Acceptable item	Good
10	Setelah mengalami strok kecil (TIA),	Interpretation:	0.64	0.73
(Once you have suffered a transient ischemic attack (TIA))	Acceptable item	Excellent
11	Pembedahan kadangkala membantu untuk mengelakkan kejadian strok berulang melalui	Interpretation:	0.32	0.64
(Surgery can sometimes help to prevent another stroke by)	Acceptable item	Excellent
12	Apakah kaedah rawatan yang terdapat bagi individu yang mengalami strok?	Interpretation:	0.41	0.46
(What method of treatment is available for people who have had a stroke?)	Acceptable item	Excellent
13	Faktor risiko UTAMA penyebab strok ialah	Interpretation:	0.82	0.36
(The most important known risk factor for stroke is)	Easy item	Good
14	Secara anggaran berapa ramaikah rakyat Malaysia mengalami serangan strok setiap tahun?	Interpretation:	0.18	0.36
(Approximately how many Australians are affected by stroke every year?)	Difficult	Good
15	Sekiranya anda mengambil alkohol secara berlebihan, anda akan	Interpretation:	0.18	0.36
(If you drink alcohol excessively you are)	Difficult	Good
16	Antara tanda-tanda berikut, manakah merupakan contoh masalah FIZIKAL akibat serangan strok?	Interpretation:	0.50	0.10
(Which of the following is an example of a physical disability caused by stroke?)	Acceptable item	Poor

Following this, the internal consistency coefficient for the knowledge domain was calculated using the Kuder Richardson (KR20) value. The KR20 value for the knowledge domain was 0.58, which was considered moderate. The researchers were unable to conduct test–retest reliability since stroke patients had a short stay on the ward of between two and three days during the hospitalization period.

Discussion

It is hoped that the Malay version SKT contributes towards stroke education assessment and evaluation meant to measure stroke patients’ knowledge levels. Furthermore, this tool can also be used to identify individual learning needs of stroke patients and their caregivers. Validity is defined as the extent to which an instrument measures what it is supposed to measure [13], [14]. Reliability, on the other hand, refers to the consistency and repeatability of an instrument [13], [14].

The validation aspects included in this study were content validity, face validity, and item analysis. Meanwhile, the reliability testing involved internal consistency. Construct validity was not completed for the SKT since knowledge items were not indicated for factor analysis; they were abstract concepts that required operational definition and clustering into common factors. Furthermore, the researchers also did not conduct criterion validation for this tool since there was no published tool yet in Malaysia comparable to this MCQ version of the SKT. Furthermore, most of the tools used in many research studies were self-developed by the authors.

The item analysis on SKT knowledge items showed that most items had appropriate values of difficulty and discrimination indices. The difficulty indices showed that 70% of the items were between acceptable and excellent and 30% of the items were considered poor items. On the discrimination index, 75% of the items were considered acceptable, 20% were difficult, and 5% were easy. Compared to initial content validation, experts had also rated similar items to number five as unsuitable to be included in the final version due to its difficulty level [7]. However, the researchers decided to retain the questions since the items reflected the overall picture of the knowledge on stroke.

In this study, 20% of the items were rated difficult which may indicate a lack of related knowledge among the majority of stroke patients. However, these poor items were related to the prevalence of diseases such as diabetes mellitus and atrial fibrillation, epidemiology on stroke disease, alcohol intake, and smoking status. Meanwhile, 40% of the items that had low difficulty indices had a discrimination index around 33%. These findings were similar with the original author which found that this area had the most answered as “I don’t know” by the respondents.

This was common because stroke patients usually lacked knowledge in these areas; they probably had not been exposed to such information before the stroke. Therefore, researchers should recognize an area that warrants special attention during patient education interventions. This specific topic should be emphasized during patient educational interventions for the stroke patients and their family members. A good measure would be to provide the fundamental information for the establishment of interventions successfully tailored to the needs of the target population.

Although reliability testing is necessary, it was not a sufficient component of the instrument validity. This is because reliability may change when it is conducted with a different population. Reliability was regarded as the ability of an instrument to consistently measure an attribute over time. The researchers had used KR-20 to establish the internal consistency reliability of knowledge test as it is preferable for use with measures with dichotomous variables (e.g., 1 for correct response and 0 for incorrect response [7], [25]) while Cronbach’s alpha coefficient is indicated for determining the internal consistency reliability of a measure with variables scored such with Likert scales [27], [28].

Validation study findings showed that knowledge (KR20=0.58) had moderate internal consistency. Even though this was fairly acceptable, the knowledge domain was considered a good test as KR20 tends to result in more conservative estimates than Cronbach’s alpha [28]. Enhancing the reliability of a questionnaire can be done by increasing the number of items [27] and increasing the sample size might produce a different internal consistency result. KR-20 is a frequently used method for determining internal consistency if the items are dichotomously coded. Basically, the computation requires three pieces of information, namely the number of items, the mean, and the standard deviation.

The reliability of the SKT instrument was rated at moderate level for this preliminary study; however, according to Downing and Haladyna [15] and Sullivan and Dunton [7], this level of internal consistency is acceptable for a new instrument and should not be a barrier for its use. However, a larger scale study is needed to determine if there is a significant impact on the interpretation of internal consistency estimates for the instrument.

Researchers need to conduct test–retest reliability relevant for cognitive and trait scales not expected to change over time [14]. The minimal requirement for conducting test–retest reliability was at least two weeks to six months. However, it is not appropriate for states that were expected to change over time, such as attitude, mood, or knowledge following an intervention [14]. Therefore, this study did not carry out test–retest reliability since researchers expected a change in the knowledge score over time during the intervention study. Furthermore, the short hospital stay among the stroke patients prevented the researchers from completing test–retest reliability measurements.

Conclusions

The Malay version SKT had good content coverage, acceptable item properties, and positive expert review ratings. Thus, it was relevant to evaluate stroke knowledge level outcomes. Furthermore, this instrument should be tested among other population groups in the future. The authors highly recommend that test–retest reliability be conducted with an easily reachable study population.

Conflict of interest

The authors declare no conflict of interest related to this study.