Face and Content Validity of the MacArthur Competence Assessment Tool for the Treatment of Iranian Patients.

BACKGROUND: There is not a valid Persian tool for measuring the decision-making competency of patients. The aim of this study is to evaluate the face and content validity of the MacArthur Competence Assessment Tool for the treatment of Iranian Persian-speaking patients.
METHODS: To assess the validity of the Persian version of the tool, a self-administrated questionnaire was designed. The Lawshe method was also used for assessing each item. Content validity ratio (CVR) and content validity index (CVI) were used to assess the content validity quantitatively. According to the experts' judgment, questions with a CVR ≥0.62 and CVR <0.62 were maintainable and unmaintainable, respectively.
RESULTS: The questions were designed in a manner to achieve the desirable result (CVR ≥0.62). The CVI scale (S-CVI) and CVI (S-CVI/Ave) were 0.94 (higher than 0.79). Thus, the content validity was confirmed.
CONCLUSIONS: Since capacity assessments are usually based on physician's subjective judgment, they are likely to bias and therefore, with this suitably validated tool, we can improve judgment of physicians and health-care providers in out- and in-patient cases.

Introduction

Experts in bioethics are concerned with the insufficient understanding of the information within the informed consent operational procedures.[1] Informed decision-making is one of the main bases for ethical practice of medicine for both diagnosis and treatment of diseases.[2] As far as competent decision-making is concerned, autonomy is known as a fundamental concept in medicine.[3]

Informed consent assumes that “rational” decision-making can construct a specific type of information supply.[4] Some studies have shown that comprehensibility of the informed consent form is poor and patients like to receive more information.[56]

Amini et al. in 2009 showed that the level of apprehending the inpatients’ informed consent was inappropriate among patients hospitalized in hospitals affiliated to Tehran University of Medical Sciences.[7] Similar studies also showed that understanding level of the patients was weak[89] or moderate.[10] Asghari et al. in 2012 considered 36 ethical issues among hospital staff of Tehran University of Medical Sciences. They found that the main reasons of the importance of ethical issues and ethical consultation attributed to the necessary information pertaining to informed consent, determination of patient's competence for decision-making, and decision-making for incompetent patients.[11]

Informed consent for treatment comprises disclosure, voluntariness (no force), and decision-making capacity dimensions.[12] Capacity is specific to certain decisions.[13]

Legal and bioethics experts agree that decision-making capacity is constituted of four main abilities: (1) Understanding the relevant information, (2) appreciation of the situation and possible consequences, (3) ability to reason with the information and weight options as logically as possible, and (4) ability to communicate the choice.[14] If a patient lacks any one of these abilities, he/she is probably considered incompetent in making decision about treatment options.[15]

If a clinician is doubtful about a patient's capacity to make a medical decision, he/she may follow some procedures to become certain of it:first, a formal evaluation of the capacity may be made by an expert clinician. This evaluation is typically based on clinical interview, medical record, and mental status. Some ethicists believe that physicians’ judgment can be effective in determining the golden standard.[12] Since these assessments are usually based on the physician's subjective judgment, they are likely to be biased and unreliable.[16] Second, the expert may decide based on clinical interview, standardized neuropsychological tests, and forensic assessment instruments.[12]

Not having a valid Persian tool is one of the challenges in this area. If the patient or relatives complain to the court, in some cases, due to lack of documentation related to the capacity components, either the physician or the patient is not capable enough to prove their claims.

Aids to capacity evaluation

Some authors believe that the aids to capacity evaluation (ACE) is the best instrument because it can be performed in <30 min. This instrument is also available online that includes training materials.[17] By searching the Persian-language databases for Persian versions of assessment tool for capacity consent to treatment, we could only find one translated tool and one unique study, which was performed on patients admitted to surgical wards of Imam Khomeini Hospital of Tehran using the Persian version of the ACE. This semi-structural tool allows physicians to put patient into four groups of definitely capable, probably incapable, probably capable, and definitely incapable. Merits of this tool include shortened duration of the test and its adaptability to different patients with different treatments.[18] There are two weaknesses related to ACE. First, if a patient is considered as probably capable or probably incapable, the patient's competence or incompetence is not lastly definite. Second, the Persian version of this tool has been translated and used without any validity process.

The MacArthur Competence Assessment Tool for Treatment

As far as competency to consent to treatment and research is concerned, in 2005, ten different tools were compared in terms of strengths, weaknesses, and psychometric properties of the existing measures. The results revealed that the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) has been present in most diagnostic categories compared to any other instrument. Hence, it can be considered as a golden standard in this respect. Although these tools have not been tested in large samples, it has been repeatedly proven that these measures may be useful in detecting and tracing the different types of incapacity.[19] The MacCAT-T is one of the frequently used tools for assessing decision-making capacity.[20] This tool, along with clinical interview and patient history, can assess the entire dimensions of patient capacity and can assist with deciding on patients’ decision-making competency or incompetency in a particular time for a particular treatment.[21] This tool is a standardized assessment tool[19] with certified validity and reliability[22] and is considered as the most empirical support.[23] Despite being time-consuming and requiring special training for the physicians,[22] application of this tool is of value, especially when patient capacity is doubted.[24] Efficiency and usefulness of this tool have also been asserted in other studies among the Persian-speaking patients.[25]

The MacCAT-T was first developed by Grisso and Appelbaum in 1998. By implementing the tool on forty schizophrenic and schizoaffective patients aged 25–50 years and then comparing it with a control group, they concluded that this tool owns optimal reliability and effectiveness in the psychotic patients. This standardized tool has been developed for assessing patient capacity to consent to treatment in clinical settings. Most components of this tool are originated from modified and manipulated items of other common existing research tools. This tool is also a semi-structured interview for guiding physicians to assess patient capacity to consent to treatment.[26] It seems to be the most practical way for cases of ambiguous competency. Therefore, researchers believe that it will mostly be applicable to situations involving legal procedures to determine competency and in cases that psychiatrists are to come to formal assessments.[19]

The current study was prospected to evaluate the face and content validity of Farsi version of the MacCAT-T among the Iranian Persian-speaking patients.

Methods

This psychometric and descriptive study was conducted to measure the face and content validity of the Persian version of MacCAT-T in 2015 in Iran. In the first place, we contacted the instrument developer (Professor Thomas Grisso) to get the permission of use, and then, translation and cultural adaptation of the current tool were developed for Iranian patients based on the Wild's model. They include translating from the original language into the target language (Persian) by three persons separately, who are qualified in English; combination and reconciliation of the three initial translations and merging them into a single translation after modifying differences and conflicts; translation of the final translated tool into English by a person qualified in English to ensure the accuracy of the MacCAT-T translation process in the Persian version; and comparing and matching the translated tool with its original version to detect inconsistencies in the two tools. Back translation is defined as a match between the translated version and the original one.[25]

Face validity can be done both quantitatively and qualitatively. Experts’ viewpoints are adopted to qualitatively evaluate the face validity of a tool.[27] Moreover, judgment of the capacity and competency is mainly done based on the level of patients’ mental health and this is mainly referred to psychologists and psychiatrists in Iran. Therefore, to certify the face validity of the current tool, seven psychiatrists, along with three clinical psychologists, were chosen from the faculty members of Kashan, Isfahan, and Tehran Universities of Medical Sciences. Experts got involved in or were included in the study with regard to their education and experience and were excluded from the study if were not enthusiastic to participate in the study.

Later on, an initial questionnaire was developed based on the main and Persian versions of the tool through consultations with the methodologists and psychometrics. This questionnaire was designed so as to see whether different dimensions of the capacity could be obtained using English and Persian versions of the tool. Then, the initial questionnaire, along with the main and Persian versions of the tool, was given to ten experts using the Delphi method. Their opinions were received and the questionnaire was changed accordingly. In the view of some experts, one of the long questions in which multiple items were posed was modified. Next, the final questionnaire along with the main and translated versions was again handed down to the experts so as to certify its content validity.

Questions were scored based on their relation with the Persian version of the MacCAT-T.

Items 1–16 were based on a 5-score Likert scale ranging from A (completely disagree, score 1), B (disagree, score 2), C (no idea, score 3), D (agree, score 4), and E (completely agree, score 5).

In addition, items 17–20 were based on a 4-score Likert scale ranging from A (expression is complicated, score 1), B (needs some modifications, score 2), C (appropriate, but needs review, score 3), and D (very simple and easy, score 4). Furthermore, questions 21–23 were scored from A (completely appropriate, score 4), B (needs some modifications, score 3), C (appropriate, but needs review, score 2), and D (completely inappropriate, score 1).

Overall, the least and the highest scores for the questionnaire were 23 and 108, respectively. “Understanding” dimension obtained the highest score with seven questions and scores of 7–35. Furthermore, “appreciation” with two questions and scores of 2–8 along with “reasoning” and “evidencing a choice” each with one question and scores of 1–4 received the lowest score. To sum up, the more the score obtained by items, the more experts agreed on using the MacCAT-T among Iranian patients.

Qualitative method of confirming the content validity was done using two indices of content validity ratio (CVR) and content validity index (CVI). The former was applied to make certain that the most significant and accurate content was chosen and the latter was used so as to make sure that the items were developed as best as possible for measuring the content.[28]

After collection of the questionnaires, CVR and CVI were used to assess the content validity quantitatively. So far, the Lawshe method has been the most common method for measuring content validity. Using the Whalts and Bassel's method, the CVI is measured by summing up the scores for each item with the highest score divided by the number of experts. This coefficient represents the validity of only one question. To measure the total validity of the questionnaire, mean CVI of all questions is measured which is known as scale CVI (S-CVI).

The Lawshe method[29] was used to evaluate each item in the present study. Based on the decision-making table of CVR, in the case of CVR ≤0.62, the questions were modified so that CVR ≥0.62 was attained.[30] Two questions were modified accordingly. Using Whalts and Bassel's method, the CVI was measured by summation of the agreed scores for each item receiving 3 and 4 or 4 and 5 scores (highest score) divided by the number of experts. To assess the total validity of the test, mean validity coefficients of all questions were calculated, and if the S-CVI was above 0.79, the scale content validity was verified.[31]

Ethical considerations

We obtained the permission to use and validate the MacCAT-T instrument from one of the main modelers (Professor. Thomas Grisso) through E-mail contact. This study was approved by Kashan University of Medical Sciences’ Ethics Committee (Code No. 2467).

Results

This study was conducted at Tehran University of Medical Sciences in 2015. Experts were asked to complete a self-administrated questionnaire. The initial question was “We (physicians) think that you are suffering from (the name of the disease or disorder). If you do not think so or if you doubt your involvement, we are willing to know about your assumption. Do you feel the disease within you? And if you deny or doubt the presence of the disease, what are your reasons?” The question was then changed according to the experts’ comment: “We (physicians) think that you are suffering from (the name of the disease or disorder). You may disagree or be doubtful about your involvement. In the case of disagreement or uncertainty about the presence of the disease, what is your reason?” Another question was omitted due to its obscurity. The question was: “Do you feel the disease within you? And in the case of disagreement or uncertainty about the presence of the disease, what is your reason?” This procedure was repeated for seven times till it reached the saturation level. Ultimately, the final version of the questionnaire included 23 questions.

The final questionnaire included four questions about “necessity of the tool and its acceptance and feasibility” and four questions about “capacity-related abilities.” Moreover, with regard to the dimensions of the capacity, seven questions were concerned with “understanding,” two questions with “appreciation,” one question with “reasoning” and “evidencing a choice,” and one question with “appearance and tabling” and “scoring” and “interpretation.” The final questionnaire covered 23 items in four domains (items 1–5, 6–9, 10–16, 17–18, 19, 20, 21, 22, and 23 were related to the general, capacity abilities, understanding, appreciation, reasoning, evidencing a choice, appearance and tabling, scoring, and interpretation areas, respectively).

Experts’ demographic characteristics that have filled out the questionnaire are summarized in Table 1.

Table 1

Demographic characteristics of the experts

Seven experts were psychiatrics while three of them were clinical psychologists. Six experts had 10–20 years of work experience. Mean work experience of the experts was 16.4 years.

Tables 2 and 3 summarize the content validity of the tool. According to the experts’ judgment, questions with a CVR above or equal to 0.62 and below 0.62 were maintainable and unmaintainable, respectively. Therefore, the questions were designed in a manner to achieve a desirable result (CVR ≥ 0.62).[31]

Table 2

Results of evaluating the overall content validity of the MacArthur Competence Assessment Tool for Treatment instrument by the experts

Table 3

Results of evaluating the overall content validity of the MacArthur Competence Assessment Tool for Treatment instrument based on the questions

According to Table 2, the experts reported the questions of the questionnaire as relevant up to 87%–100%.

Table 3 summarizes the CVR and CVI results by each question. CVR range was 0.8–1, while the range of CVI was 0.87–1. Therefore, it was found that all the questions were valid.

Since the S-CVI in this study was above 0.79 (0.94), validity of this tool was confirmed.

Discussion

Results of the present study supported the face and content validity of the MacCAT-T. This validity was also satisfactory in all the four areas of understanding, appreciation, reasoning, and expressing a choice. There are two alternatives to evaluate patients’ decision-making capacity. The first alternative is subjective and based on the experts’ or physicians’ judgment (clinical physicians) which sometimes seems unreliable. The second alternative may be carried out by means of tools which seem more accurate and to the point.

Due to the significance of patients’ decision-making capacity and lack of a valid tool in this respect in Iran, this study was carried out to measure validity of the MacCAT-T. Through our research, we could only find two studies in which validity of different versions of MacCAT-T was confirmed.

The first study was directed in 2011 by Hernando Robles et al.[32] in Spain. Validity of this Spanish study was optimally supported. Evidently, we had no access to the complete English text of the study and only its abstract was available. Validity of the tool was analyzed by 15 experts, while we had employed only 10 experts in this respect. According to the texts, multiplicity of experts is not necessary. The necessity is that the experts are qualified enough to evaluate items and individuals under study.[31] The second study was conducted in 2011 by Bilanakis et al.[20] in Greece. The MacCAT-T was translated into Greek and the translated version was validated among psychiatric patients. Thirty-nine patients were divided into three groups and each group was evaluated and scored by one particular expert. Two-third of the patients were evaluated and scored through listening to interviews of other colleagues.

Fallahzadeh et al. in 2015 determined a mean content validity of 96% using a panel of ten experts.[33] Instead, in 2014, Abdishahshahani et al. introduced a mean content validity of 90% by a panel of ten experts.[34] Liu et al. in 2016,[35] Lin et al. in 2015,[36] and Rahmani Bilandi et al. in 2015[37] determined the mean content validity using ten experts.

On the other hand, the MacCAT-T is the most comprehensive and applicable tool for this purpose according to majority of the experts.[19] Another difference is the presence of cultural adaptation after the forward and back translations in our study rather than the above-mentioned studies. It was concluded that all dimensions, items, and methods of interpretation and scoring of this tool are adaptable to the Persian patients.

Limitations

If we could select our experts among the forensic medicine, medical laws, etc., in addition to psychiatrists and psychologists, it would be more appropriate for the study; however, due to the executive limitations, we were not allowed to do this.

Conclusions

Psychometric findings demonstrated that this tool can be used for measuring patient's decision-making capacity in Iran. Since this tool owns a valid face and content validity, it can be utilized efficiently by physicians and health-care providers. This tool can also be used to assess the prevalence of incompetency among the Persian psychiatric and nonpsychiatric patients.

It is recommended that, in the future studies, inter-rater reliability of the instrument should be calculated and proved after implementation of the Persian version of this tool. Then, the incidence of incapacity for treatment decisions among the Iranian psychiatric and nonpsychiatric patients could be estimated.

Financial support and sponsorship

This study was part of a PhD thesis entitled “comparative study on Islamic principles and medical, legal, and ethical criteria in relation to patient's capacity and competency in medical decision making and developing propositions to improve current condition” supported by the deputy of research of Tehran University of Medical Sciences under the supervision of Professor. Bagher Larijani.

Conflicts of interest

There are no conflicts of interest.