Use of online machine translation for nursing literature: a questionnaire-based survey.

BACKGROUND: The language barrier is a significant obstacle for nurses who are not native English speakers to obtain information from international journals. Freely accessible online machine translation (MT) offers a possible solution to this problem. AIM: To explore how Japanese nursing professionals use online MT and perceive its usability in reading English articles and to discuss what should be considered for better utilisation of online MT lessening the language barrier.
METHOD: In total, 250 randomly selected assistants and research associates at nursing colleges across Japan answered a questionnaire examining the current use of online MT and perceived usability among Japanese nurses, along with the number of articles read in English and the perceived language barrier. The items were rated on Likert scales, and t-test, ANOVA, chi-square test, and Spearman's correlation were used for analyses.
RESULTS: Of the participants, 73.8% had used online MT. More than half of them felt it was usable. The language barrier was strongly felt, and academic degrees and English proficiency level were associated factors. The perceived language barrier was related to the frequency of online MT use. No associated factor was found for the perceived usability of online MT.
CONCLUSION: Language proficiency is an important factor for optimum utilisation of MT. A need for education in the English language, reading scientific papers, and online MT training was indicated. Cooperation with developers and providers of MT for the improvement of their systems is required.

INTRODUCTION

Utilising research is critical for nurses in any field [1-3]. Because evidence-based practise (EBP) is increasingly being encouraged in many countries, nurses who are not native English speakers need to read internationally published research papers written in the English language [4]. The problems created by language differences can become a serious barrier to success for nurses in non-English-speaking countries, such as Japan [4-6]. English reading skills are perceived as highly important [3], and the need to read articles written in the English language has been recognised among Japanese nurses [7-9]. Nevertheless, no association between the reading of English technical literature and the language barrier has been examined.

It is more important for Japanese nurses to obtain nursing information quickly, to provide quality care, than it is to spend time mastering the English language [10]. The use of a technology such as machine translation (MT) may be an important step toward achieving this goal [10]. MT is a system in which text in one language is automatically translated into another language [11], and has long been used as an aid to worldwide multilingual communication. The MT concept originated in the United States, as early as the 1930s. During the 1950s, the development of computers led to the practical realisation of this concept [12, 13]. The emergence of the Internet allowed MT systems to be made available via various websites, and they have become popular and increasingly convenient. Online MT systems are freely accessible, and although many of their translations are beset with shortcomings, and leave much room for improvement, they are generally valuable and practical [14, 15]. Among the Japanese, it has been indicated that people use MT systems to avoid the task of looking up unknown words one by one in the dictionary, and/or because they only want a rough idea of what the source texts contain, rather than detailed translations [15]. Thus the current function of MT can be described as ancillary in nature, and is limited to this type of usage.

Many studies have been conducted on evaluating MT performance with various language pairs as well as on MT users. Among online MT systems, Google Translate (one of the major systems) has generally been reported to outperform the others [14, 16, 17]. Evaluation methods and processes, as well as the usability of translations, have been investigated for healthcare documents such as medical records, health information, and patient educational materials, indicating conditional usability [18, 19]. Previous studies on MT users have indicated various points of view regarding the use of online MT and wide-ranging perceptions of usability by general users [20-24]. Some user feedback on MT translations via the Internet has demonstrated a wide range of opinions, varying from gratitude to harsh criticism of the output [20]. Surveys on MT use by general Internet users (which includes a large portion of online MT users) have been conducted in Japan and different ways of using this tool have been suggested [22]. Also, Japanese English learners have reported the perceived usefulness of online MT in a large number of subjects [23]; moreover, students whose level of English proficiency was intermediate or higher were able to assess the quality of the MT systems on websites more critically [24]. However, few studies have been conducted to examine the usability of these systems by medical and healthcare professionals. Furthermore, no specifically nursing-oriented investigation of this type has been conducted, in spite of the obvious need for English language reading capabilities to enhance the practise in non-English-speaking countries such as Japan. It is currently unknown how Japanese nursing professionals, whose native tongue is one of the minor languages in academia, perceive online MT tools and their usability. Therefore, a knowledge base must be developed to investigate the practicality of online MT in relation to the perceived language barrier presented by internationally published nursing literature.

AIM

Our aim was to explore how Japanese nursing professionals use online MT and perceive its usability in reading English articles and to discuss what should be considered for better utilisation of online MT lessening the language barrier. More specifically, we attempted to investigate the following research questions: How do Japanese nurses use online MT and perceive its usability? What factors are associated with their online MT use and the perceived usability?

METHODS

Study Participants and Data Collection

A cross-sectional exploratory survey was conducted with 250 Japanese nursing participants, using a self-administered questionnaire. Our objective was to select a study population of native Japanese speakers with the necessary nursing experience to understand the clinical contexts of technical papers. Assistants and research associates[1] at nursing universities were assumed to be an appropriate choice because they had already had professional experience both as clinical nurses and as teachers, usually with more experience in clinical settings than in educational or research environments. Nevertheless, it was anticipated that these people would have some idea of how the research might be utilised.

In Japan, there were 200 4-year nursing programs as of February, 2012, and the number of assistants and research associates across Japan was assumed to be over 2,000. Two to four individuals were randomly selected from the website of each university, for a total of 500 candidates. The actual participants were recruited by mail. Each candidate was sent the questionnaire and a cover letter asking for his/her participation, and 498 of them received it. The survey was completed by each participant and anonymously returned by mail. In total, 250 individuals responded to the survey, a response rate of 50.2%. The data were collected from March to April of 2012.

Measures

The questionnaire included background questions, previous and current online MT usage, and perceived usability of online MT (Tables and ). The number of technical articles read within approximately the previous 3 months was asked and scored on a five-point Likert scale (1: 0, 2: 1–5, 3: 6–10, 4: 11–20, and 5: 21 or more). The frequency of language barrier perception was also asked about and scored on a five-point Likert scale (1: never, 2: rarely, 3: occasionally, 4: often, and 5: always). For the question regarding previous online MT use, it could be answered Yes or No. The frequency of online MT use was rated on a three-point Likert scale (1: rarely, 2: occasionally, and 3: always/almost always). The perceived usability of online MT was measured by answers on a five-point Likert scale (1: not usable at all, 2: not very usable, 3: neither usable nor unusable, 4: usable to some degree, and 5: very usable). The question of participants’ experience using Internet search engines that offer online MT had the following choices: 1) Yahoo, 2) Google, 3) Bing, 4) Excite, 5) Nifty, 6) Livedoor, 7) OCN, 8) So-net, 9) other, and 10) do not remember the names of the search engine(s).

Respondents who answered that they had never used online MT were asked to select a reason from among the following: “I do not need it”, “I do not think it is usable”, “I didn’t know there were online MT services”, and “any other reason(s)” (to be specified). In addition, those who had used online MT, but stopped using it could answer “I do not use it anymore”, optionally selecting a reason from “I do not need it”, “ I do not think it is useful”, or “any other reason(s)” (to be specified).

Data Analysis

The data were analysed using the Statistical Package for the Social Sciences (SPSS), version 20. Descriptive statistics, the t-test, analysis of variance (ANOVA), the chi-square test, and Spearman’s correlation were used. Raw data were analysed after calculating the means, standard deviations (SDs), distributions, and percentages for questionnaire items. The chi-square test was used to identify the differences in the number of English articles read and perceived language barrier according to the respondents’ demographic data, where the number of English articles read was measured based on the binary responses and “6 or more”, and perceived language barrier was measured based on the binary responses “less” and “more”. Also, the chi-square test was used to determine the differences in perceived usability, measured based on the binary responses “not usable” and “usable”, according to the participants’ characteristics. The t-test and ANOVA were used to detect the differences in the frequency of online MT system use according to participant characteristics. A post hoc examination was performed, using a multiple comparison procedure (Tukey’s HSD test). The relationships between professional experience and the outcome variables were examined using a Spearman’s correlation coefficient.

Ethical Considerations

The Ethical Review Board of the University of Tokyo approved this study. Participants were informed that all information would be treated confidentially, and that they could contact the researcher if they had any questions.

RESULTS

Participants’ Characteristics

The respondents were predominantly women, three-fourths of whom were research associates, with a most frequently reported age group of 30s (Table ). Most respondents had a master’s degree. Approximately 45% of the respondents reported their English language proficiencies, based on their scores or grades from the following tests: the Test of English for International Communication (TOEIC), the Test of English as a Foreign Language (TOEFL), the Society for Testing English Proficiency (STEP) test, and the United Nations Association’s Test of English (UNATE). The English proficiency levels were operationally defined and divided into two categories: above and below intermediate (Table ). Above intermediate refers to a score of 600 or above on the TOEIC, 2nd grade on the STEP test, or a grade of B or above on the UNATE, while scores and grades below these thresholds were referred to as below intermediate [25]. The mean number of years of clinical experience was more than twice the mean number of years of teaching (Table ).

More than 50% of the participants responded that they had read one to five technical articles in English language within approximately the previous 3 months (Table ), and more than 20% reported having read no article in English. Although they amounted to less than 4%, a few nurses had read more than 21 articles in English. More than 80% said they often or always feel a language barrier when reading nursing articles in English, and, overall, as many as 98% of the respondents perceived a language barrier (Table ).

Previous and Current Use of Online MTs and Perceived Usability

More than 70% of respondents indicated that they had indeed used MTs (Table ). Among the 65 individuals who said they had never used an online MT, 9 (9%) indicated “I do not need it”, 36 (53%) reported “I do not think it is useful”, and 12 (18%) responded “I never knew there were online MT services”. Eight (20%) respondents specified other reasons (e.g., “I cannot learn to read English with MT”, “Reading English by myself is somewhat faster and more efficient for me”, “I would rather use a commercially available MT software package”, “I never read articles written in any foreign languages”, “I do not have any opportunities to be involved in research activities”, and “I am lazy about using online MT”). As for the frequency of using online MT, occasional users were predominant (Table ). Additionally, 31 (18%) respondents who had previously used online MTs said that they did not use them anymore. Among the reasons why they stopped using MTs, 3 answered that they “do not need it”, and 28 answered that online MT was “not useful”.

Several online MT systems were used by the nursing professionals in this study. The respondents chose Internet search engines that offer online translation tools within Japan, and Yahoo (n = 106), Google (n = 100), and Excite (n = 102) were remarkably popular compared with Nifty (n = 14), Bing (n = 17), Livedoor (n = 16), OCN (n = 16), So-net (n = 13), and others (n = 19). Eleven respondents answered that they did not remember the names of the Internet engine(s) they had used.

Nearly half of all respondents evaluated online MT as “usable to some degree”, followed by “not very usable”, “neither usable nor unusable”, and “very usable”. The percentage that answered “not usable at all” was the lowest (Table ).

Differences in the Number of English Articles Read and Perceived Language Barrier According to Participant Characteristics

Significant differences in the number of English articles read were observed according to job title and the level of English proficiency (Table ). For each of these, participants at higher levels tended to read more articles in English. Also, the amount of teaching experience was weakly correlated with the number of articles read. There were significant differences in the perceived language barrier according to academic degree and English proficiency level. Participants with lower academic degrees and lower English proficiencies were more likely to feel the language barrier when reading articles written in English.

Differences in the Frequency of Use and the Perceived Usability of Online MTs According to Participant Characteristics

There was a significant difference in the frequency of online MT use according to the perceived language barrier (Table ). The results of the post-hoc multi-comparison procedure (Tukey HSD test) revealed that the difference was significant between the “always” and “never–occasionally” categories of the frequency of online MT use (mean difference = .552, SE = .187, P = .011 [P < .05], CI = .11 –1.00). There were no statistically significant differences in perceived usability according to respondent characteristics.

DISCUSSION

Participants’ Online MT Use and its Perceived Usability

A large number of participants indicted that they had previously used online MTs when reading technical papers. It is perhaps only natural that people would enjoy the convenience of automatic translations on websites in this era of the Internet. However, this result implies that many nurses require help when reading technical English language papers in their own fields. Accordingly, it indicates that online MT is much in demand among nursing professionals, highlighting the necessity of language support for them. Also, the comments written on the questionnaire indicated that they tended to use online MT services to grasp the basic meaning of the original text, similar to the results of a previous study [15]. In addition, some participants used multiple online MT systems for the same text, to identify the most useful translation (which is an advantage of the free accessibility of online translators). Some reported that they use online MT systems in combination with a dictionary of technical terminology, indicating that the respondents do not accept translations from online MT systems without question. Also, from the fact that 43% of the respondents used online MT “occasionally” when reading technical literature, it is assumed that they did not regard it as adequate for full-time use, although their perception was that this tool was somewhat helpful.

More than half of our respondents perceived that online MT systems are useful, and hence the general tendency was toward relatively positive impressions. In a previous study, university students rated online MT performance, and more than 30% of the respondents judged it favourably [21]. More of our subjects reported positive perceptions, and in one previous study, online MT received a far better evaluation [23]. The reasons for these differences among study populations are not known but could involve the factors used in the study samples, respondent needs, and the kinds of source texts.

Factors Affecting the Frequency of Online MT Use and its Perceived Usability

The perceived language barrier was associated with frequency of use. Respondents who felt more of a language barrier were likely to use online MT systems more often. This may indicate that MT is relatively advantageous for users with higher language proficiency. Because Japanese users with extremely poor English language skills cannot judge MT performance by looking at a source text written in English, it is difficult for them to utilise MT, and thus they will use it less often. Hence, this result appears to be consistent with previous studies that have indicated the importance of English language ability for the effective use of MT [15, 24]. The perceived language barrier was associated with the participants’ academic degree (Table ). In Japan, nurses have more opportunities to read English in graduate programs. In such programs, nurses might become accustomed to reading research articles or presenting their own research projects, eventually improving their English skills. In fact, some graduate nursing programs in Japan offer a course in reading English [26].

In contrast, no significant correlations were found between perceived usability and participant background information. Factors that could be associated with perceived usability include motivation for use, type of use, and the expected effectiveness of online MT systems, in addition to the demographic attributes of participants. To obtain a beneficial outcome for users, factors related to usability, such as willingness to use, and specific ways and expected benefits of using online MT services, should be investigated in future studies.

Practical and Educational Implications

The comments written on the questionnaire by the participants included their expectations for and suggestions to developers and providers to improve the systems for the benefit of nursing and healthcare professionals. Specifically, they wanted better translations of technical terms. Their awareness of language difficulties when reading and writing English was also often mentioned in connection with desired improvements in MT performance. Their feedback should be helpful to MT providers for improving translations in nursing fields, as well as beneficial to nursing professionals. Furthermore, our respondents pointed out the important issue of transmitting nursing information from Japan in English. They were concerned with producing internationally interactive nursing personnel resources. Thus, online MT systems are expected to produce better translations not only from English to Japanese, but also from Japanese to English.

English education for nursing professionals must evolve. In general, Japanese users of MTs have some knowledge of the English language, but are uncertain of their proficiency [15]. In our study, the percentage of respondents who felt more of a language barrier when reading English articles was significantly high, indicating the need to improve English education. English education using MT systems may be helpful. However, a negative aspect of MT in foreign language education is that students may accept an output without verifying its correctness, and stop trying to improve their own language skills [24]. Previous studies have indicated some effectiveness in the use of MTs in English language education. It has been claimed that the use of MT lessens students’ aversion to reading and writing English sentences, and helps to train them in logical thinking [27]. It will be important to provide nursing professionals with opportunities to receive training in effective ways of using online MT systems. Also, Japanese nurses need to have training to learn how to specifically read scientific papers in English because this type of document has its own rules for readers. Knowing these rules can save time and reduce inconvenience. These educational opportunities should be provided during post-graduate/continuing professional training (rather than solely as a part of their primary education), in the form of integrated and organisational programs that have yet to be established in the nursing workplaces of Japan. The understanding and collaboration of managerial level nurses will be required to make this continuing education a reality, and we will need to investigate their opinions and attitudes regarding education in the English language and reading scientific papers as part of the occupational development of nursing professionals.

Limitations

Our sample was limited to two types of nursing faculty members, therefore, their academic degrees may have been higher than those typically held by nurses in clinical settings. Nursing professionals from a broader range of settings and levels will be necessary for a deeper understanding of general tendencies. In addition, the sample might have contained a disproportionate number of respondents that were already interested in MT services and/or users of online MT systems.

The English language test scores and grades reported by our respondents did not appear to accurately represent their English reading skills. The participants’ English language proficiencies were judged solely in terms of scores and grades on general language tests. Japanese English language examinations usually measure overall skills, and half of the score is based on listening and/or speaking skills. Therefore, the total score on one of these examinations could be lower if the examinee’s listening or speaking skills are poor, even if his/her reading proficiency is higher. In the present study, the participants’ reading and writing skills should have been directly assessed, and analyses based on these assessments are needed. Also, recall bias could have been introduced when the participants were asked about the number of articles read within approximately the previous 3 months.

CONCLUSIONS

Online MT had been used by a sizeable percentage of the study participants as an aid to reading international nursing journals. The level of the perceived language barrier was related to the likelihood of using online MTs. For optimal use of online MT, it is necessary to improve users’ language skills. We should consider the English language education of nurses, as well as the possibility of using online MTs in English education, and providing training in reading scientific papers and the use of MT tools. It will also be a role of our work to promote user cooperation with the developers and providers of MT systems to improve MT dictionaries with regard to nursing-specific terms. Future studies should include a wider range of nurses, assessing English proficiency precisely, an investigation of the usability of online MT considering possible influencing factors (such as willingness to use the systems and specific ways and expected benefits of using this technology), investigations of the need for research education on using international journals, and promoting postgraduate English education according to the perceptions of nurses at the managerial level.

Issues related to reading articles published in English should be seriously considered, to be resolved not only in Japan, but also in many other non-English-speaking countries. The present study will provide incentive for further exploration of possible linguistic support for nurses via advanced Internet-based technology, leading to more effective use of MT tools, and eventually more opportunities for nurses to access internationally published nursing information and obtain updated knowledge for better care delivery.

Table 1.

Participant Background Information (nr = No Response)

Variables			n	%
Sex		Men	22	8.8
		Women	226	90.4
		nr	2	.8
Title		Assistant	59	23.6
		Research Associate	189	75.6
		nr	2	.8
Age group		20s	19	7.6
		30s	135	54
		40s	81	32.4
		50s	13	5.2
		60s	2	.4
Academic degree		Diploma	4	1.6
		Associate	6	2.4
		Bachelor	47	18.8
		Master	171	68.4
		Doctoral	20	8.0
		nr	2	.8
English language proficiency (n = 113)		≥ intermediate	54	47.79
		≤ intermediate	59	52.21
		N	Mean	SD
Professional experiences (years)	Clinical	246	8.84	5.26
	Teaching	248	4.32	3.67
		Answers	Value	%
Number of articles read in English within approximately 3 months (n = 247)		0	57	23.1
		1–5	143	57.9
		6–10	28	11.3
		11–20	10	4.0
		21 ≤	9	3.6
Frequency of perceived language barrier (n = 245)		Never	1	.4
		Rarely	2	.8
		Occasionally	42	17.1
		Often	74	30.2
		Always	126	51.4

Table 2.

Use of Online MT and Perceived Usability

Variables	Answers	n	%
Previous online MT use (n = 248)	Yes	183	73.8
	No	65	26.2
Frequency of online MT use (n =143)	Rarely	49	34.3
	Occasionally	62	43.4
	Always/almost always	32	22.4
Perceived usability (n =182)	Not usable at all	3	1.6
	Not usable very much	51	28.0
	Neither usable nor unusable	25	13.7
	Usable to some degree	91	50.0
	Very usable	12	6.6

Table 3.

Frequency of Online MT Use and Perceived Usability According to Participant Characteristics

Variables		Numbers of Articles Read in English					Perceived Language Barrier
		n	0-5 (%)	≥ 6 (%)	χ2	P	n	Less (%)	More (%)	χ2	P
Title	Assistant	59	55 (93.2)	4 (6.8)	7.713	.005	55	6 (10.9)	49 (89.1)	2.484	.115
	Research Associate	186	143 (76.9)	43 (23.1)			188	38 (20.2)	150 (79.8)
Age group	≤ 30s	152	127 (83.6)	25 (16.4)	1.709	.191	149	24 (16.1)	125 (83.9)	1.295	.255
	≥ 40s	95	73 (76.8)	22 (23.1)			96	21 (21.9)	75 (78.1)
Academic degree	≤ bachelors	56	50 (89.3)	6 (10.7)	3.359	.067	58	4 (7.5)	49 (92.5)	5.097	.024
	≥ masters	189	148 (78.3)	41 (21.7)			190	40 (21.1)	150 (78.9)
English level	≤ intermediate	59	53 (89.9)	6 (10.1)	6.026	.014	56	8 (14.3)	48 (85.7)	4.629	.031
	≥ intermediate	53	38 (71.7)	15 (28.3)			53	17 (31.5)	37 (68.5)
		n	r*	P**			n	r*	P**
Professional experience	Clinical	243	-.045	.485			241	.14	.029
	Teaching	245	.186	.004			243	-.112	.018

Spearman’s correlation coefficients.

Significance level is 1%.

Table 4.

Relationship Between Numbers of Articles Read in English and Perceived Language Barrier, According to Participant Characteristics

Variables		Frequency of Online MT Use					Perceived Usability
		n	Mean	SD	t	P	n	Unusable (%)	Usable (%)	χ2	P
Title	Assistant	28	2.18	.819	.471	.638	38	17 (44.7)	21 (55.3)	.053	.818
	Research Associate	115	2.1	.73			143	61 (42.6)	82 (57.3)
Age group	≤ 30s	94	2.2	.742	1.87	.064	113	47(41.6)	66(58.4)	.399	.528
	≥ 40s	49	1.96	.735			69	32(46.4)	37(53.6)
Academic degree	≤ bachelors	26	2.23	.815	.827	.414	37	19 (51.4)	18 (48.6)	1.219	.269
	≥ masters	115	2.09	.732			143	59 (41.3)	84 (58.7)
English level	≤ intermediate	20	2.20	.696	1.016	.314	33	16 (48.5)	17 (51.5)	.402	.526
	≥ intermediate	39	2.00	.725			46	19 (41.3)	27 (58.7)
		n	M	SD	F	P*	n	Unusable (%)	Usable (%)	χ2	P**
Number of articlesa	zero	26	1.81	.749	.462	.631	33	15 (45.5)	18 (54.5)	.852	.653
	1-5	87	1.87	.728			113	51 (45.1)	62 (54.9)
	≥ 6	27	2.00	.832			33	12 (36.4)	21 (63.6)
Language barrierb	Never-Occasionally	18	2.00	.716	5.195	.007	25	13 (52.0)	12 (48.0)	.879	.644
	Often	40	2.27	.717			55	24 (43.6)	31(56.4)
	Always	84	1.72	.752			101	42 (41.6)	59 (58.4)
		n	r	P***			n	r	P***
Professional experiences	Clinical	142	.048	.574			175	.128	.093
	Teaching	42	.052	.542			176	-.072	.341

ANOVA p-value.

Kruskal-Wallis test p-value.

Spearman’s correlation coefficient p-value.

Number of professional articles read in English within approximately the previous 3 months.

Frequency of language barrier perception when reading nursing articles in English.