A discrepancy between methods and the interpretation of results: a comment on Umar et al. (2019).

In African Health Sciences Vol 19, Issue 2, Umar et al.1 report on their survey of clinical radiography students' perceptions of eight different methods of delivering anatomical information in relation to nine learning aims, in order to “assess the use of ultrasound scanning as an anatomy teaching aid, and to determine the teaching methods clinical medical radiography students consider most effective for understanding human anatomy in Northern Nigeria” (Umar et al. (2019)1.

There are serious discrepancies between the methods and interpretation of results in the paper by Umar et al.1. These are described below.

(1) Preference by radiography students for eight methods of delivering anatomical information, reported in Table 1 of Umar et al.

Umar et al.1 used a questionnaire similar to those used in previous studies by Bowman et al.2 and others3–4. The questionnaire was structured as eight columns, with a delivery method specified at the top of each column (column 1 = ultrasound scans; 2 = lectures; 3 = textbooks; 4 = anatomical models; 5 = computer programs; 6 = 3D radiology imaging; 7 = laboratory videos; 8 = animal dissection) and nine rows, where each row gave a different learning aim: (row 1 = to impart anatomical information; 2 = to provide information for other science courses; 3 = to encourage learning from experience; 4 = to develop team skills; 5 = to follow complex instructions; 6 = to appreciate anatomical variation; 7 = to provide a background for a clinical discipline; 8 = to provide an anatomical vocabulary; 9 = to provide a 3D appreciation). This gave a matrix containing 72 squares (Table 1 of this paper). For each of the eight delivery methods, every student scored its effectiveness in achieving each of the nine learning aims by placing a number from one to eight in the appropriate square in the matrix “…where the number one indicated there was an excellent match between aim, and method and the number eight indicated that the delivery method did not achieve the aim. On a scale of one to eight, one was the “best fit” between delivery method and learning aim and eight was the “worst fit”….. So the number four was a better fit than five and so on.” (Umar et al.1, Methods, page 2284).

Table 1

the design of the questionnaire used by Umar et al.1, based on similar questionnaires used by others2–4. Each student gave their perceptions of the effectiveness of every method for delivering information in relation to each of the nine learning aims by placing a number from 1 – 8 in every square of the matrix.

	Delivery method
Aims	Ultrasound	Lectures	Textbooks	Anatomical models	Computer programs	3D radio imaging	Lab videos	Animal dissection
Impart anatomical information	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Information for other science courses	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Encourage learning from experience	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Develop team skills	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Follow complex instructions	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Appreciate anatomical variation	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Provide background for clinical discipline	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Provide anatomical vocabulary	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Provide 3D appreciation	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8	Score from 1 – 8
Column totals (each has a possible range from 9 to 72)	Total score for ultrasound	Total score for lectures	Total score for textbooks	Total score for anatomical models	Total score for computer programs	Total score for 3D radio imaging	Total score for lab videos	Total score for animal dissection

Umar et al.1 obtained data from 92 students. For each student the nine scores they gave for each delivery method were summed. As an example, if a student gave a delivery method (e.g. lectures), their highest preference (i.e. a score of one) for each of the nine learning aims, then the aggregate score for that delivery method would be 9.0 (e.g. the column total for ‘lectures’ in Table 1 of this paper). If a student gave a particular delivery method their lowest preference (i.e. a score of eight) for every learning aim, then the aggregate score for that method would be 72 (see Umar et al.1, Results, page 2284). For each delivery method the column totals (as shown in Table 1 of this paper) were used to calculate the means of all 92 responses: these means could range from 9 to 72.

Umar et al.1 report the means for each of the eight delivery methods in the first row of their Table 1 on page 2284. Ultrasound scan had the lowest (32.48) mean and 3D radiology imaging the highest (48.17). Therefore, from their methods (where students were told to give low scores to indicate excellent matches between delivery methods and learning aims, and high scores to indicate poor matchesbetween delivery methods and learning aims), a low mean should indicate a good match and a high mean a poor match. However, Umar et al.1 have interpreted the means in Table 1 to indicate the opposite: “3D radiological imaging being the most preferred method overall (48.17), and ultrasound the least (32.48)”. The interpretation by Umar et al.1 is in direct contradiction to the scoring procedure specified in their methods.

(2) Preference by radiography students among eight methods of delivering anatomical information for each learning aim reported in Tables 2 – 4 of Umar et al.

The discrepancy described above has also occurred in the interpretation of summary data presented in Tables 2, 3 and 4 by Umar et al.1 who separately examined how well each learning aim (i.e. each of the nine rows of the questionnaire) was achieved by each delivery method, which was by calculating a mean score for every combination of delivery method and aim. For example, for the aim “To encourage learning from experience” the scores within the cells of the third row of the questionnaire (see highlighted cells in Table 1 of this paper) were used to calculate eight separate means(from the 72 questionnaires) for each of the eight delivery methods. Since students were instructed to give low scores to delivery methods that were good matches to a learning aim and high scores if they were not, then low values for means in the summary data in Tables 2 – 4 of Umar et al.1 will indicate good matches between learning aims and delivery methods, and high values of means will indicate poor matches. Here too, however, Umar et al.1 have interpreted these scores in the opposite way to that specified in their methods and have stated in the results: “Here the mean score possible across the 92 students could range from 8.00 (if all students gave a particular method their highest preference) to 1.00 (if all students gave a particular method their lowest preference)”. As an example, for the learning aim “To develop team skills” in Table 3 of Umar et al.1, ultrasound, with the lowest mean of 3.50, was assigned the lowest preference. But from the methods given in Umar et al.1 the lowest score should indicate the highest preference. The same discrepancy between methods and interpretation of results applies to all the means in Tables 2 to 4 of their paper.

Summary

The discrepancy between the methods and interpretation of results needs to be explained and clarified by Umar et al., because, at present, no conclusions can be drawn from their survey of Northern Nigerian radiography students' perceptions of different methods of teaching human anatomy.

To the Editor, African Health Sciences

For each of the eight delivery methods, the students ranked the nine learning aims by placing a number from one to eight in the appropriate square in the matrix “… where the number one indicated there was an excellent match between aim and method and the number eight indicated that the delivery method did not achieve the aim. On a scale of one to eight, one was the “best fit” between delivery method and learning aim and eight was the “worst fit”.……so, the number four was a better fit than five and so on.” (Umar et al.1, Methods, page 2284).

The students ranked the delivery methods against the learning aims. however, for a delivery method to be ranked with lower preferences (For example ranked 8), does not mean by statistical implication that delivery method will have higher scores as a result of the number “8”. Likewise, when a delivery method is ranked with higher preferences (for example ranked 1) does not mean when subjected to statistical testing, will have lower scores as a result of the number “1” and so on.

Moreover, the statement above applies to all the tables listed below as captured by Umar et al.,

“(1) Preference by radiography students for eight methods of delivering anatomical information (Table 1)

(2) Preference by radiography students among eight methods of delivering anatomical information for each learning aim as reported (Tables 2 – 4)

(3) Summary. Table 5 give the summary of the delivery methods with learning aims. A post hoc analysis was carried out to see which block differs from the other significantly. 3D radiology has the highest score and ultrasound scan has the least score. Furthermore, among the delivery methods with highest and lowest preferences, 3D radiology imaging has the highest score. Consequently, among delivery methods with lowest preferences, ultrasound scan has highest score. This indicates that survey of Northern Nigerian radiography students' perceptions of different methods of teaching human anatomy” preferred 3D Radiology imaging, than ultrasound scan in understanding human anatomy.