Test-retest reliability of the questionnaire in the Sasang constitutional analysis tool (SCAT).

BACKGROUND: The Sasang constitutional analysis tool (SCAT) is an integrated Sasang constitutional analysis system developed by the Korea Institute of Oriental Medicine. This study aimed to evaluate the reliability of a questionnaire for measuring personality and pathophysiological symptoms that is one of the components of the SCAT.
METHODS: In this study, data were collected from university students in their twenties. Tests were administered twice, with an interval of 4 weeks between tests. Test-retest data from 176 students were collected and used for analysis. Internal consistency reliability was analyzed by using Cronbach's alpha coefficient, and test-retest reliability was analyzed by using Spearman's rank correlation coefficient.
RESULTS: Cronbach's alpha coefficient was 0.788 for personality, 0.511 for eating habits, 0.718 for digestion, 0.667 for heat- or cold-wise penchant, and 0.612 for water ingestion. Spearman's rank correlation coefficients, which were used to assess correlations between test and retest results, ranged from 0.444 to 0.828.
CONCLUSION: The internal consistency and test-retest reliability of the SCAT questionnaire were found to be satisfactory.

Introduction

Sasang constitutional medicine is a Korean traditional medical system introduced by Jema Lee (1837-1900). In Sasang constitutional medicine, people are divided into four Sasang constitutional types: Tae-Eum, So-Eum, So-Yang, and Tae-Yang. Different treatment principles and prescriptions are suggested depending on individuals’ Sasang constitutional types, even if these individuals have the same diseases. Therefore, in this medical system, precise diagnosis of Sasang constitutional type is the most important concern. Longevity and Life Preservation in Eastern Medicine, the original source describing Sasang constitutional medicine, states that Sasang constitutional type is determined by synthesizing various clinical information about an individual, including face, body shape, voice, biopsychological traits (emotional tendencies and inclinations), and pathophysiological symptoms (symptomatic presentation).1, 2

To support the objectivity of Sasang constitutional type diagnosis in research and clinical applications, studies have been conducted to develop an objective and validated diagnosis tool. The objective study of Sasang constitutional diagnosis began with the development of the Questionnaire of Sasang Constitution Classification (QSCC) in 1993.3, 4 The QSCC was redeveloped in 1995 as the QSCC II, and the QSCC II has been used as a diagnostic tool for many studies. However, the QSCC II includes a total of 121 items and has limitations with respect to the use of this questionnaire alone to diagnose Sasang constitutional type. The QSCC II also exhibits a problem related to diagnosis of the So-Yang type. Several questionnaires, such as the revised QSCC II, have been developed to address this problem, but these questionnaires retain the aforementioned limitations.7, 8 There have been several studies conducted to extend the use of Sasang constitutional medicine in other populations besides Korean. These studies investigated the reliability and validity of Sasang type classification questionnaires among Mongolian, American, Japanese and Vietnamese.9, 10, 11, 12, 13, 14

In 2009, Korea Institute of Oriental Medicine developed an integrated analysis tool for diagnosing Sasang constitutional type using comprehensive information, such as questionnaire results, face, body shape, and voice analysis. This tool was developed based on data from the Korea Constitutional Multicenter Bank (KCMB), which has accumulated Sasang constitutional type identification and intervention data from certified clinical specialists. This Sasang constitutional analysis tool (SCAT) has been developed into a web-based integrated Sasang constitutional analysis system via beta testing and is utilized in several Korean medical institutions.16, 17

However, responses to the questionnaire used in the SCAT, one of the components of this tool, may vary depending on age and gender. Moreover, insufficient data for younger individuals were used in the development of the SCAT. Therefore, it is necessary to study the reliability and validity of the questionnaire for younger subjects. The purpose of this study was to evaluate the reliability of this questionnaire via test-retest reliability analysis.

Methods

Subjects and procedures

In this study, data were collected from university students after the study procedures were approved by the relevant institutional review board (approval no. 1040647-201706-HR-018-03). An online survey was conducted from June 15, 2017, to July 27, 2017, for individuals who voluntarily agreed to participate in this study, and a recruitment announcement was posted in the lobby on the first floor of Daejeon University College of Korean Medicine. All subjects signed a written informed consent form for participation in this study. For test-retest reliability analysis, tests were conducted twice, with an interval of 4 weeks between tests. Test-retest data from 176 students were collected and used for analysis.

First test (test): The subjects responded to the questionnaire by accessing the online test site mentioned in the recruitment announcement.

Second test (retest): Four weeks after subjects participated in the first test, the online retest site address link was distributed to these subjects via email or mobile device.

Methods

General characteristics

Sex, age, height, and body weight data were collected to determine subjects’ general characteristics.

Questionnaire in the SCAT

The questionnaire included in the SCAT developed by Korea Institute of Oriental Medicine was used. This questionnaire is a self-reported questionnaire consisting of 35 items, including 12 personality items and 23 pathophysiological symptom (素證) items. The personality items, which measure Yin-Yang temperament, are scored using a three-point Likert scale (1 = bold, 2 = medium, and 3 = delicate); subjects are asked to select one of the three answers for each item. The pathophysiological symptom (素證) items, which are used to assess typical symptoms and lifestyle during the preceding 6 months, consist of 2 items on eating habits, 5 items on digestion, 4 items on perspiration, 2 items on defecation, 1 item on sleep, 4 items on heat- or cold-wise penchant, 2 items on water ingestion and 3 items on pathological symptoms. For the pathophysiological symptom (素證) items, subjects are asked to select one of 3 to 7 answers, with the exception of 4 items on which multiple responses can be provided.

Results for the analysis of Sasang constitutional type based on this questionnaire can be derived when questionnaire responses are entered into a SCAT program. These results are presented as probability values for the four Sasang constitutional types. A combined probability value is obtained by applying an equation to assess the questionnaire responses and combining the results with probability values for the four Sasang types obtained from analyses of face, voice and body shape. The Sasang constitutional type with the highest probability value among the four Sasang constitutional types is ultimately determined to be the outcome of the Sasang constitutional analysis.

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics 20.0 (IBM, Armonk, NY) with a significance level of 0.05. General characteristics of the subjects were expressed as the mean ± standard deviation for continuous variables and as frequency (%) for categorical variables. Internal consistency reliability was analyzed by using Cronbach's alpha coefficient, and test-retest reliability was analyzed by using Spearman's rank correlation coefficient.

Results

General characteristics

The 176 subjects included 102 (58%) males and 74 (42%) females. The mean age of the subjects was 24.89 ± 3.70 years, and the mean BMI was 21.88 ± 3.19 (Table 1).

Table 1

General characteristics of the participants

	Male	Female	Total
Number (%)	102 (58%)	74 (42%)	176 (100%)
Age	25.40 ± 4.31	24.19 ± 2.49	24.89 ± 3.70
Height	175.02 ± 4.82	161.75 ± 4.18	169.44 ± 7.99
Weight	70.47 ± 10.02	53.26 ± 7.49	63.24 ± 12.41
BMI	23.00 ± 3.12	20.34 ± 2.61	21.88 ± 3.19

Data are expressed as number (%) and as the mean ± standard deviation for categorical and continuous variables, respectively.

Internal consistency reliability

Cronbach's alpha coefficient, which can be used to analyze internal consistency reliability, was calculated for 12 personality items and 13 pathophysiological symptom 素證 items. Cronbach's alpha coefficient was 0.788 for personality, 0.511 for eating habits, 0.718 for digestion, 0.667 for heat- or cold-wise penchant, and 0.612 for water ingestion (Table 2).

Table 2

Internal consistency reliability of the SCAT questionnaire

Item	Cronbach's α#	Cronbach's α
Personality		.788
Q1	.768
Q2	.785
Q3	.755
Q4	.763
Q5	.747
Q6	.764
Q7	.789
Q8	.785
Q9	.768
Q10	.747
Q11	.794
Q12	.804
Pathophysiological symptoms (素證)
Eating habits		.511
Q13	-
Q14	-
Digestion		.718
Q15	.583
Q16	.750
Q17	.646
Q18	.726
Q19	.605
Heat- or cold-wise penchant		.667
Q27	.676
Q28	.492
Q29	.498
Q30	.687
Water ingestion		.612
Q31	-
Q32	-

Cronbach's α without the insignificant items

Test-retest reliability

Spearman's rank correlation coefficient, which can be used to analyze test-retest reliability, was used to assess test-retest correlations for 12 personality items and 17 pathophysiological symptom 素證 items. The correlation coefficients for these items ranged from 0.444 to 0.802 (Table 3).

Table 3

Test–retest Reliability of the SCAT Questionnaire

	Item	Spearman rho
Personality
	Q1	0.589***
	Q2	0.698***
	Q3	0.695***
	Q4	0.614***
	Q5	0.735***
	Q6	0.707***
	Q7	0.651***
	Q8	0.642***
	Q9	0.755***
	Q10	0.649***
	Q11	0.652***
	Q12	0.750***
Pathophysiological symptoms (素證)
	Q13	0.513***
	Q14	0.762***
	Q15	0.642***
	Q16	0.700***
	Q17	0.649***
	Q18	0.577***
	Q19	0.706***
	Q20	0.725***
	Q21	0.742***
	Q22	0.669***
	Q25	0.444***
	Q27	0.784***
	Q28	0.802***
	Q29	0.704***
	Q30	0.707***
	Q31	0.690***
	Q32	0.764***

p < 0.001.

Discussion

The SCAT is an integrated Sasang constitutional analysis system developed by Korea Institute of Oriental Medicine based on constitutional information for 4300 people who were diagnosed by experts. Data for an individual's face, voice, body shape, and questionnaire results are entered into a SCAT program, Sasang constitution-related variables are extracted, and probability values for the four Sasang constitutional types are analyzed and provided. The SCAT is characterized by the provision of detailed and quantified information with respect to face, voice, body shape, and questionnaire responses, which is needed when a Korean medicine doctor diagnoses an individual's Sasang constitutional type.16, 17

This study was conducted to evaluate the reliability of the questionnaire component of the SCAT. The internal consistency reliability and test-retest reliability of questionnaire items were analyzed. Cronbach's alpha was used to determine whether the questionnaire items measure the same construct, and Spearman's rank correlation coefficient (Spearman's rho) was used to confirm the stability of the questionnaire items over a 4-week period.

Cronbach's alpha coefficient was assessed for 12 personality items and 13 pathophysiological symptom 素證 items for which internal consistency could reliably be analyzed. Cronbach's alpha was 0.788 for personality, 0.511 for eating habits, 0.718 for digestion, 0.667 for heat- or cold-wise penchant, and 0.612 for water ingestion (Table 2). In general, Cronbach's alpha coefficients of at least 0.6 are thought to be indicative of good reliability. Therefore, this questionnaire was confirmed to exhibit internal consistency for all items except for those related to eating habits. Given that better internal consistency reliability is typically observed as the number of examined items increases, it appears possible that the internal consistency reliability of eating habit items is relatively low because there were only two items on eating habits. In the future, we can devise ways to improve internal consistency by increasing the number of eating habit items.

Spearman's rank correlation coefficient was used to assess test-retest correlations for 12 personality items and pathophysiological symptom 素證 items, and the correlation coefficients for these items ranged from 0.444 to 0.802 (Table 3). In general, correlation coefficients between 0.4 and 0.7 and between 0.7 and 0.9 indicate moderate and strong correlations, respectively. Therefore, given that the time interval between the two tests was 4 weeks, it can be confirmed that the questionnaire answers were relatively stable.

The results of this study demonstrated the internal consistency of the SCAT questionnaire measuring personality and pathophysiological symptoms and showed that the responses to each item remained stable over a 4-week period. However, the validity of the questionnaire was not evaluated in this study. Therefore, the sensitivity and specificity of the questionnaire should be evaluated in a future validation study. In addition, the study subjects were limited to university students in their twenties who have prior knowledge of Korean medicine and Sasang constitutional medicine. As a result, the capacity to generalize from the results of this study is limited and there might be a risk of bias due to the predefined subject group. Thus, additional reliability studies should be conducted for other age groups without prior knowledge of Korean medicine and Sasang constitutional medicine.

Conflict of interest

The authors declare no conflict of interest.