Performance of an automatic translator in translating medical abstracts.

The aim of the present paper was to quantitatively assess the performance of a free online translator in translating medical texts from English into Italian. Effectiveness was evaluated and established by analyzing the number and percent of translation errors with respect to the original medical abstracts. Furthermore, the present study analyzed the total number and the percent of translation errors and their correlation with lexical density, which was assessed using Text analyzer. The mean number of total translation errors was 21. The mean percentage of total translation errors was 15%. The mean lexical density was 66.7%. A direct correlation was found between total translation errors and the lexical density. The findings of the present study indicate a fairly good performance of Google translator in translating words in highly academic writings such as medical abstracts. However, an effective translation is not a matter only of finding a correspondence between words in the source language and the target language, many other aspects are just as important.

Introduction

Translation of highly specialized medical papers has been growing due to exponential medical progress. Most medical articles accepted by international medical journals are in English. Therefore, Italian professionals and specialists working in the medical fields need to read, write and understand medical papers in English. Most professionals have sufficient knowledge of the particular topics, the English language and specialized medical language so that they are capable of reading medical literature themselves. However, in some cases it may be necessary to use a translator, due to the fact that many professionals are not English language native speakers, and thus, they are unable to understand the language correctly and the language richness. Furthermore, although translations are carried out by native speakers who translate into their mother tongue, the background knowledge of the specific topics may be insufficient and knowledge of the specialist language may be incomplete (Halliday, 1978; Jacobson, 1989; Neubert, 2000). Specific problems are posed by the particular register and lexicon, and by the many linguistic conventions and norms adopted by this particular language community (Halliday, 1988; Bell, 1991). The resulting translations are often less than satisfactory, and sometimes clearly wrong (Halliday, 1973; Jacobson, 1989; Daniele, 2005).

Many theories have been developed on how to translate. Although many researchers and experts more or less agree on the techniques, the processes and the deep mechanisms involved with translating standard language, much controversy arises from technical translations and particularly medical translations (Newmark, 1981; Darian, 1982; Daniele, 1999). A common idea is that translations have to be performed complying with the original style of the authors (Taylor, 1998). On the other hand, others argue that translating consists mainly in expressing the same concepts in a different language (Bassnett, 2013). It seems that no unanimous position can be taken. Instead, it is crucial to remain open and versatile toward the process of translating that might simultaneously involve a number of different techniques.

The type of translation seems to be very important. Many authors have discussed different kinds of translations. Peter Newmark distinguishes between a semantic and a communicative, poetic translation. According to Newmark, a communicative translation is one that addresses the reader in an understandable way, while remaining faithful to the source text. On the other hand, a semantic translation appears to be more rigidly bound to the source text, tending to be more precise and highly literal (Newmark, 1988, 1991, 1993). Poetic translations are closer to re-writing, because the translator has to convey the deep meaning the author is expressing, but this must be done in a way that respects poetic conventions and also the sounds and the feelings the original author intended to elicit in the reader (Jacobson, 1959; Dastjerdi and Hakimshafaaii, 2008). As a result, in these conditions a specific style must be respected (Taylor, 1998). On the other hand, technical translations seem to be more like a simple transfer of the corresponding words into the target language. This would appear to be a more mechanical type of translation, and the outcome is a word-for-word style (Fischback, 1998). The main problems posed by translations of this type do not necessarily derive from the source language, nor from the target language, but from the translator.

In recent years, translation studies have shifted their focus from the translation to the translator. Today it is widely agreed that the best translations are those by translators who translate into their mother tongue (Ulrych, 1999; Hatim and Mason, 2005). In addition, translators need to be fully proficient in the source language, and most researchers highlight the fact that it is crucial for translators to have a good understanding of the subject matter. In other words, the translator has to be sufficiently familiar with the specific terminology, and even more importantly, to have a good knowledge of the specific concepts, processes, situations and phenomena the specialized language is communicating (Robinson, 2012; Hatim and Mason, 2014). Unfortunately to date not many professionals are translators, and so in order to overcome the obvious obstacles encountered when dealing with medical translations, cooperation between the translator and the professional seems an acceptable solution (Daniele, 2013). This represents a good compromise that combines effective language utilization and correct medical concepts (Jacobson, 1989; Gramley and Patzold, 1994; Daniele, 2000). Recent studies have reported on the effectiveness of online translators in translating various types of texts (Ellender, 2012; Somers, 2012; Taleghani and Pazouki, 2018). Mathematicians, computer scientists and engineers have published a number of papers discussing the construction and development of specific machines for medical translations (Shultz et al., 2013; Guzmán et al., 2017). However, no such machine seems to be currently available for widespread use. Therefore, to date only standard translators can be used.

The aim of the present paper is to quantitatively assess the performance of a free online translator in translating medical texts from English into Italian. Effectiveness was evaluated and established by analyzing the number and the percentage of total translation errors with respect to the original medical texts.

Materials and methods

To address these issues, medical translations from English into Italian were performed using a free online translator, and an extensive analysis of medical literature was carried out. A total of 115 abstracts were evaluated. The abstracts were taken directly from the Pubmed database, and were randomly selected using a number of search terms (Table 1) in order to cover a range of subjects in the fields of medicine and surgery. The only criterion for inclusion was that abstracts were to be in English, and written by proficient users of English (the authors of the papers included were all from English-speaking countries), in order to avoid language errors in the original texts. For each search term, the first five abstracts meeting this criterion were downloaded.

Table 1

Search terms.

AIDS	Myopia
Alzheimer's Disease	Obesity
Breast Cancer	Otitis
Cardiac Disease	Parkinson's Disease
Cervical Cancer	Prostate Cancer
Colon-Rectal Cancer	Psoriasis
Diabetes	Pulmonary Disease
Kidney Disease	Rheumatoid Arthritis
Liver Diseases	Sleep Disorders
Low Back Pain	Stroke
Lymphoblastic Leukemia	Thyroid Disease

All abstracts were translated from English into Italian using the free online Google translator. The errors and the number of translation errors resulting from the translations were analyzed directly by the author of the present study. The number of total translation errors included all words, even when they were repeated. All English abstracts were analyzed using Text analyzer, which is an online text analysis tool for detailed text statistics. Table 2 depicts how texts can be selected and brought into the Text analyzer, and Table 3 shows the options provided by the analyzer. For the purposes of the present study only the total number of words and the lexical density were taken into account (Table 4). For each abstract, the only items that were considered for the present analysis were: (1) total word count (TWC) and (2) the lexical density (LD). In each translated abstract: (1) the number of total translation errors (NTTE) and (2) the percentage of total translation errors (PTTE) were calculated. Furthermore, the relation between NTTE and LD was analyzed. All calculations were carried out using Windows Excel.

Table 2

Text analyzer characteristics - Text selection.

Writing directly - Enter text (copy and paste is fine) here

Taking text from website - or read it from a website URL (plain text.TXT)

Table 3

Text analyzer characteristics - Analysis options available.

Number of characters (including spaces)

Number of characters (without spaces)

Number of words

LD

Number of sentences

Number of syllables

Abstracts were downloaded directly from the Pubmed and then they were copied onto the Text analyzer.

Table 4

Text analyzer characteristics – Analysis options used in the present work.

Number of words

LD

Results

Initially a total of 115 abstracts were analyzed and translated from English into Italian. Four abstracts were not included in the analysis. One abstract was excluded from the calculations because it was too short as compared to the other abstracts, as it consisted of only 19 words and would have biased the analysis. The same problem would have arisen with another abstract consisting of 332 words that was excluded because it was too long in comparison with the other abstracts. Two other abstracts were excluded from the calculations because they had abnormal LD.

As a result, a total of 111 abstracts were included in the present analysis. The longest abstract included in the analysis consisted of 233 words, while the shortest contained just 46 words (233-46). The mean number of words was 143. The highest LD was 84.7% and the lowest was 59.4% (84.7%–59.4%). The mean LD was 66.7%. The highest NTTE identified in the study was 38, while the lowest was 10 (38-10). The mean NTTE was 21. The highest PTTE was 34.8%, while the lowest was 6.3% (34.8%-6.3). The mean PTTE was 15%.

A direct correlation was found between NTTE and LD (Fig. 1). A correlation was established between the increase in LD and NTTE. The curve indicates that when LD rises from 54% to 83%, so increasing by 54%, NTTE proportionally goes up by 46%, increasing from 13% to 19%.

Fig. 1

Lexical density vs. percent of total translation errors.

Discussion

The present study analyzed the total number and the percentage of total translation errors and their correlation with LD in an attempt to assess the performance of the free online Google translator in translating medical abstracts. The Prague School and Halliday have furnished theories concerning translations, and the problem of what translators should or should not know has also been posed (Halliday, 1973, 1978, 1988). Nida and Wilss set forth their three-phase theory: the first phase is represented by the analysis of the source text, then it is transferred into the target language, and finally the receptor language is restructured (Nida, 1974; Wilss, 1982). Wilss argued that: “The translator first deciphers the STL macro- and micro-contextually, then undertakes to coordinate individual elements of the SLT with the TL on a one-to-one correspondence or a non-one-to-one correspondence. In other words, he decides on an inter-lingual transfer strategy and then, finally, by means of synthesizing operations in the TL, produces the TLT” (Wilss, 1982). This translation process seems applicable to medical translations as it represents the method that enables translators to obtain acceptable translating results (Halliday, 1988; Daniele, 2015). However, in the present study translations were not carried out by human translators, but by machines. While machines do not “decide on an inter-lingual transfer strategy and then, finally, by means of synthesizing operations in the TL, produce the TLT” and do not “decipher the STL macro- and micro-contextually”, they certainly do “coordinate individual elements of the SLT with the TL on a one-to-one correspondence or a non-one-to-one correspondence” (Wilss, 1982).

Medical language could be defined as a specialized language used by a particular community of practice including medical practitioners and to a certain extent also paramedics and researchers. The particular features of medical language can be highlighted in different contexts such as: medical practitioners talking to other doctors, and doctors talking to paramedics in different situations and environments. Much work has been devoted to the particular relationship that is developed during doctor-patient interaction, when doctors talk to their patients (Cicourel, 1985; Fairclough, 1989; Sarangi, 2002; Candlin and Candlin, 2003; Daniele, 2007). Furthermore, all the varieties of written forms of communication also belong to medical language. Clearly, all these different written and oral medical genres have their own features and peculiarities, which deserve specific and detailed separate analyses. The present study is limited to the language of medicine in academic writing. In this context, the language is significantly different from standard language, since it belongs to a specific language community that comprises medical practitioners and researchers writing in medical journals addressed to the international scientific community. Moreover, medical language is endowed with many peculiarities regarding grammar and linguistic structures; lexicon, terminology, style, and syntax, but more importantly, medical language is governed by language conventions and norms (Halliday, 1989a, Halliday, 1989b; Taavitsainen and Pahta, 2000). One of the most widely studied aspects of medical language is terminology and lexicon, and directly related to these, the extensive use of nominalization in medical texts (Hyland and Bondi, 2006; Maglie, 2009; Snow and Uccelli, 2009; Nagy and Townsend, 2012). Nominalization is a type of word formation in which a verb or an adjective is used as a noun. In functional grammar, nominalization refers to the derivation of a noun phrase from an underlying clause (Halliday, 1989a, Halliday, 1989b, 1994). Another specific lexical feature of medical writings is pre-modification and compounding. Pre-modification is the creation of strings of words ending with a head noun. The head noun is preceded by a number of words called modifiers that ‘modify’ the head noun. The pre-modification process involves removal of all function words, including verbs (Duan and Gu, 2010; Kirkman, 2012). This process, together with compounding, which is the formation of medical terms starting from more than one lexical item is used in medical writings for the purposes of concision. In this way, the whole writing acquires more fluidity and is therefore more accessible. Nominalization together with pre-modification and compounding all tend to reduce the number of function words and make the text more ‘dense’ with lexical words (Pèrez Ruiz, 2006; Crespo, 2011; Zorita and Sandoval, 2016). This is the reason why the present paper correlated the PTTE with the LD.

Previous studies have been carried out on Google translator in translating medical English. However, the studies have investigated this specialized language mainly in the context of doctor-patient communication in multicultural and multi-lingual settings (Walker, 2011; Davies, 2014). In the present study the mean PTTE was 15%. This is a surprising, unexpected, positive result meaning that machine translators are good translators of words, regardless of the direct correlation between translation errors and LD. It must be pointed out that the texts analyzed in the present study are highly academic, technical texts with a mean LD of 66.7%, which per se makes a text more difficult to understand and thus translate. The performance of the online Google translator seems to be fairly effective because of the intrinsic characteristics of medical language used in abstracts, which is rich in lexical words. Indeed, most of these latter are scientific terms that have a Greek-Latin root both in English and Italian, therefore a direct correspondence between them is certainly plausible. The matching of words in the two languages seems to be heavily relied on by machine translators, having been set mainly for this specific activity. The present study is intended primarily as a quantitative analysis, but a few examples of mistranslations are noteworthy. First, many pre-modified clauses were mistranslated in relation to the order of the words, failing to provide the correct translation (Table 5). Second, different categories of mistranslations of words were observed: a) sub-technical terms; b) words having a different meaning in standard language and specialized language: c) words that do not appear to have a translation equivalent in Italian; d) discrete items in English that apparently have a corresponding word in Italian with a different meaning, so an explanation is needed in order to translate them correctly (Tables 6, 7, 8, and 9). Third, other important mistranslated items were verbs and noun gender, as well as tenses (not included in the present analysis). Both these problems are strictly related to the characteristics of Italian, which unlike English has feminine and masculine, and with no direct correspondence of the tenses between the two languages. Finally, Tables 10 and 11 report sample paragraphs showing that although the words were all translated correctly, the meaning was still not transferred from English into Italian in a way that produces an effective target language text, and this is because the machine cannot “decide” and “decipher”; fortunately, these remain prerogatives of the human brain.

Table 5

Translation sample – Pre-modified clauses.

Original	Google Translator	Translation by the author of the present study
Biopsy-confirmed localized low-risk ​prostate cancer	Carcinoma prostatico localizzato a basso rischio confermato biopsia	Tumore alla prostata, a basso rischio, localizzato, e confermato con biopsia prostatica
Oral health information	Informazioni sanitarie orali	Informazioni sulla salute della bocca
MRI brain revealed right frontoparietal leptomeningeal enhancement	Il cervello della risonanza magnetica ha rivelato un miglioramento leptomeningeo anteriore-parietale	La risonanza magnetica del cervello ha rivelato una localizzazione del mezzo di contrasto a livello del lobo fronto-parietale e delle leptomeningi
…in a large cohort of patients prescribed long-term opioids	…in un'ampia coorte di pazienti con oppioidi a lungo termine prescritti…	…in un'ampia coorte di pazienti ai quali erano stati prescritti oppioidi a lungo termine….

Table 6

Translation sample – Sub-technical terms.

Original	Google Translator	Translation by the author of the present study
Proportion	Percentuale	Proporzione
Coaptation	CoaptazioneCoabitazioneCoabitamento	Combaciamento
Paper	Document	Articolo
Complication	Complicazione	Complicanza
Modelling	ModellizzazioneModellazione	Modellamento

Table 7

Translation sample – Terms having a different meaning in standard language and specialized language.

Original	Google Translator	Translation by the author of the present study
Leaflets	Volantini	Lembi Valvolari
Chords	Accordi	Corde Tendinee
Background	Sfondo	Premessa
Failure	Fallimento	Insufficienza
Research settings	Impostazioni di ricerca	Condizioni di ricerca

Table 8

Translation sample – Words that could be used as loan words.

Original	Google Translator	Translation by the author of the present study
Imaging	Immagini	Imaging
Odds ratio	Odds ratio	Rapporto di probabilità
Hazard ratio	Hazard ratio	Rapporto di rischio

Table 9

Translation sample – Words requiring an explanation.

Original	Google Translator	Translation by the author of the present study
Chronic analgesia	Analgesia cronica	Trattamento cronico con analgesici
Freedom from biochemical failure	Libertà da fallimento biochimico	Paziente privo di insufficienza biochimica
Caregivers	Fornitori di cura	Persone che si prendono cura del paziente
Referral	Rinviare	Il paziente che è stato inviato da un medico ad un altro

Table 10

Translation sample.

Original Text	Google Translator	Translation by the author of the present study
The predictive validity of vertebral fracture assessment (VFA) on bone density lateral spine images to identify prevalent vertebral fractures in routine clinical practice has not been established. Our objective was to estimate the associations of prevalent vertebral fracture identified on VFA images in routine practice with incident hip, all non-vertebral, major osteoporotic, and clinical vertebral fractures, using the Manitoba Bone Density database.	Non è stata stabilita la validità predittiva della valutazione della frattura vertebrale (VFA) sulle immagini di colonna vertebrale della densità ossea per identificare le fratture vertebrali prevalenti nella pratica clinica di routine Il nostro obiettivo era stimare le associazioni di frattura vertebrale prevalente identificate sulle immagini VFA nella pratica di routine con fratture dell'anca incidente, tutte le fratture vertebrali maggiori osteoporotiche e cliniche, utilizzando il database Manitoba Bone Density.	Non è stata stabilita la validità predittiva della valutazione delle fratture vertebrali (VFA) effettuata analizzando la densità ossea attraverso radiografie laterali della colonna vertebrale per l'identificazione delle fratture vertebrali prevalenti nella pratica clinica di routine. Il nostro obiettivo era di stimare le associazioni tra le fratture vertebrali prevalenti identificate attraverso le radiografie nella pratica di routine, con le fratture dell'anca, con tutte le fratture non-vertebrali e con le fratture vertebrali osteoporotiche importanti e con le fratture sintomatiche, utilizzando il database Manitoba Bone Density.

Table 11

Translation sample.

Original Text	Google Translator	Translation by the author of the present study
Fifteen studies were included in this systematic review. Increasing age and history of cardiac disease were found by most studies to be positively associated with risk of ​cardiac ​complication.	Quindici studi sono stati inclusi in questa revisione sistematica. L'aumento dell'età e della storia della malattia cardiaca sono stati riscontrati dalla maggior parte degli studi per essere positivamente associati al rischio di complicazioni cardiache.	Quindici studi sono stati inclusi in questa revisione sistematica della letteratura scientifica. La maggior parte degli studi ha riscontrato che l'aumento dell'età e una anamnesi di malattia cardiaca sono associati positivamente al rischio di complicanze cardiache.

Conclusions

The present paper quantitatively assessed the performance of Google translator in translating medical abstracts. Medical abstracts are highly academic writings that are written in a language that is significantly different from standard language, since it is endowed with many peculiarities regarding grammar and linguistic structures. One of the most prominent characteristics of this specialized language is LD, which is the result of an extensive use of nominalization and compounding. LD poses problems in both understanding and translating this specialized language. Indeed, the present analysis showed that mean LD of the original abstracts was 66.7%. Surprisingly, the evaluation of the Google translated abstracts yielded a mean NTTE of 21 and a mean PTTE of 15%, showing a fairly good performance of Google translator in translating words, regardless of LD. However, a direct correlation was found between NTTE and LD in the translated abstracts; NTTE increases together with LD.

In conclusion, the findings of the present study indicate a fairly good performance of Google translator in translating words in highly academic writings such as medical abstracts. However, an effective translation is not a matter only of finding a correspondence between words in the source language and the target language, many other aspects are just as important. This was a preliminary report on the performance of online translators and only a limited number of the problems correlated with translating highly specialized medical abstracts have been pointed out. Further investigations are necessary to gain more insight into the phenomena that were only mentioned in the present paper and which deserve further investigation. Future studies will continue to assess the translation of medical articles that will further highlight the weak points of medical translations and the strategies that should be adopted to improve them.

Declarations

Author contribution statement

Franca Daniele: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.