Questionnaire development and validation for designing a health telemonitoring system for frail elderly people.

BACKGROUND: Advancements in information technology have been instrumental in successful recent developments in telemonitoring systems. In this regard, there is a lack of development of valid and reliable tools to determine the requirements and applications of telemonitoring systems used to provide health care for frail elderly people living at home, specifically in a national setting.
METHOD: A cross-sectional study was carried out in 2018. The statistical population was 15 geriatric and gerontology professionals and 15 health information management experts. Then, content validity ratio (CVR), Cronbach's alpha, and correlation coefficient were calculated for measuring content validity, internal consistency and external reliability (through the test-retest method) respectively. SPSS software was used to analyze the collected data.
RESULTS: Based on the identified items, a draft questionnaire was developed. Using the validity analysis in two stages, 37 items were removed, and 60 items were approved as the essential system requirements. The final questionnaire was organized into five sections with content validity index 99% and internal reliability (Cronbach's alpha coefficient 0.9). Furthermore, the external reliability results of the questionnaire showed that this instrument has a desirable correlation coefficient (r = 0.85, p-value<0.05).
CONCLUSION: Considering the desirable validity and reliability of the questionnaire developed, it is recommended to telemonitoring system designers to determine the usages and requirements of health monitoring systems for frail elderly people living at home. The verified instrument is suitable for use in countries with the same living conditions and level of development as Iran.

Introduction

The ageing populations around the world are rising.[1-4] This phenomenon has significant consequences for the community and elderly people. One of the most challenging results is the increasing prevalence of frailty among elderly.[5] Frailty is a syndrome of the elderly; its symptoms include dependency, vulnerability, chronic disease, reduced quality of life, long-term hospitalization and death.[5-11] Usually, frail older adults have one or more chronic conditions and require continuous care and monitoring of their health status.[12]

In recent years, the development of information and communication technology (ICT) has provided new ways to improve the quality of life for the elderly and monitor their health status.[13] Telecare, telemedicine and telehealth are components of these innovative methods and are often used.[14] Telemonitoring is one of the telehealth methods that has widespread applications in the area of health status monitoring.[15] In other words, telemonitoring is essential to providing satisfactory care to elderly people and their families; that is, it permits elderly people with chronic diseases or who are very frail to live independently in their own home providing direct contact to specialists, relatives and caregivers.[16] Healthcare providers need to access suitable systems that provide novel ways in which to manage care for age-related chronic diseases such as stroke, heart disease, osteoporosis and chronic obstructive pulmonary disease (COPO). Telemonitoring is the best option for addressing increasing healthcare costs associated with care of frail elderly patients.[17] Home telemonitoring aims to decrease the number of visits and costs, improve quality of life and allow patients to stay in their own homes via monitoring of vital signs such as blood pressure, blood sugar, pulse, temperature and oxygen saturation.[12,18]

There are several challenges in the use of health telemonitoring systems including complications such as low level of education, physical weakness and multiple chronic diseases of frail elderly.[19,20] When developing and designing a health telemonitoring system for frail elderly, recognizing their various requirements and the applications of this system are important, particularly for system designers.[21,22] In this regard, provision of a questionnaire to determine the requirements and applications of such systems is essential. Previous studies in other countries have surveyed the validity and reliability of questionnaires in various related topics such as an “online fall-risk questionnaire for non-frail community-dwelling elderly persons”,[23] “Pfeffer questionnaire for the Brazilian elderly population”,[24] “the questionnaire for health-related resource use in the elderly population”[25] and “international physical activity questionnaire (IPAQ) in elderly adults”.[26] However, there is no specific tool to determine monitoring system requirements for the elderly in Iran. The aim was to develop and validate a questionnaire defining the requirements of a health telemonitoring system for frail elderly living at home in Iran according to the technical and communication infrastructure available in this country.

Methods

This study was conducted in three main phases including item generation and development, content validity ratio (CVR), and internal consistency and test–retest reliability, which are addressed in separate sections.

Item generation and development

This was a cross-sectional study conducted in 2018. We aimed to develop and validate a questionnaire to determine the requirements of an Iranian health telemonitoring system for frail elderly at home. Firstly, the items of the questionnaire were derived from a review study. A comprehensive review of scientific databases, including PubMed, Scopus, Web of Science, and Google Scholar, was undertaken and was searched using the following keywords: frail elderly, home care, telehealth, telemonitoring and health information system. The searched articles published between 2013 and 2017 were first scanned for duplicate articles, which were removed. Then, titles and abstracts of articles according to the exclusion and inclusion criteria were investigated. In the next phase, the full text of the articles was studied separately and the requirements of the existing systems of health telemonitoring systems for elderly at home were determined. A total of 1727 records were retrieved by searching the databases above. After removal of duplicates and investigation of the full texts, 21 articles met the inclusion criteria and were included in the final review. These papers involved studies of the elderly aged 65 and older who were deemed frail due to at least one chronic disease. PRISMA was used for data collection and analysis, and the results were presented in Figure 1. The results of the literature review were used to recognize items; 97 items were identified for the draft version of the questionnaire.

Figure 1.The process of PRISMA for data collection and analysis.

In the next phase, the identified items were translated into Persian and categorized into five sections. The items of the draft questionnaire were designed based on the MOSCOW scale according to the requirements of the system, and those achieving early benefits were prioritized. MOSCOW method which (Table 1) is a prioritization technique used in management and software development, promoted by Dynamic Systems Development Method (DSDM) community. The importance of this method is that the items can be used directly to discuss what is significant for the different stakeholders.[27,28]

Table 1.

Abbreviation MoSCoW, which is an acronym derived from the first letter of each of four prioritization categories (Must have, Should have, Could have, and Won't have).

M	MUST have this requirement
S	SHOULD have this requirement if at all possible
C	COULD have this requirement if it does not affect anything else
W	WON'T have this requirement this time but would like in the future
"o"	The two lower case o’s are there to make the acronym work

Possible responses to the questionnaire included: “Must have”, “Should have”, “Could have” and “Won't have”. Subsequently, the initial design of the questionnaire for face validity was reviewed and approved by a panel of five people including two geriatric and gerontology professionals and three health information management experts. The aims of the questionnaire were considered by the specialists in order to determine the main requirements of a health monitoring system for Iranian frail elderly living at home.

Content validity ratio

In the first step, according to the arrangement of questions in the two areas of geriatric medicine and health information management, the questionnaire was completed; that is, each part relating to a specific field was completed by five geriatric medicine professionals, and the other four sections were completed by five health information management specialists. Completed questionnaires were analyzed based on expert responses and calculaing the CVR using the following formula: CVR= (Ne − N/2)/(N/2). And “N” is the total number of specialists which were 29 in this step of study. In the questionnaire, “Must have” and “Should have” were chosen as essential options. We calculated CVR in two steps for each item. Based on the above formula, at the first step, CVR was calculated for each item and was compared with the approved scale. In this manner, if the value obtained for each item was larger than the threshold value listed in ref. [29], the content validity of that item was verified. In the second step, those items in which the proportion of validities was less than the threshold values were selected for further discussion and the items returned to the experts again; then, based on approved threshold, the final items of the questionnaire were determined.

Internal consistency and test–retest reliability

Internal consistency shows how all of the items measured in a scale the different aspects of a similar property and presents the tendency to be consistent in repeated measurements. Test–retest is a type of external reliability analysis that involves matching the results of the same measurements to the same group of respondents under the same conditions at different time intervals.[30] In this study, internal consistency method (Cronbach's alpha) was used to assess the reliability of each item of the questionnaire, and the test–retest method was used to measure the correlation between the answers (external reliability) of the questionnaire under the same conditions at different times. In this stage, the statistical population of the study was 15 geriatric and gerontology professionals and 15 health information management experts.

Results

Based on the findings of the literature review, 1727 articles were retrieved from the searched databases. After removing the duplicates and reviewing the articles, 21 were finally selected and carefully examined. As a result, the parameters and requirements of existing health telemonitoring systems for the elderly at home were consolidated into 97 items, and a draft version of the questionnaire was developed comprised of five sections. These included a variety of usage/applications (18 items), functional requirements (19 items), non-functional requirements (27 items), device types (18 items) and communication infrastructure (15 items), as shown in Table 2. The draft questionnaire was reviewed and approved by the geriatric medicine specialists and health information management professionals.

Table 2.

Total items of the draft questionnaire from obtained literature review.

Communication infrastructure	Devices	Non-functional requirements	Functional requirements	Usages/applications (M = monitoring)
Bluetooth	Activity hub	Agent based design	Alert system	Blood coagulation M
Cellular network	Broad band router	Animated guide	Antinational guide	Blood pressure M
Ethernet	Camera	Applying audio message	Creation user profile	Blood sugar M
GPR	Game controller	Applying cloud service	Detection of emergency situation	Body temperature M
GPS	HTC radar	Applying email	Displaying/feedback information for the users	Chest impedance M
GSM	Intel health guide	Applying HTTPS	Ease of use for elderly	Diet tracking M
Internet	IOT device	Applying POP – UP	Education for the users	Electrocardiogram M
LR-WPANs	Land line phone	Applying SNOMED CT	Graphical representation of outputs	Electromyogram (EMG) M
MMS	Medical sensors	Applying video linked	Historical summary calculation	Galvanic skin response (GSR) M
RFID	Motion detector	Applying videoconference	Medication management	Heart rate M
SMS	PC & laptop	Applying virtual environment	Motivation for the users	Medication compliance M
Telephone line	Pulse-Doppler radar	Applying web portal	Personalized services	Oxygen saturation M
Wi-Fi and Wi-Max	RFID reader	Central data base	Playing remotely game	Physical activity M
WLAN	Smart phone	Contact with HIS ability	Recommendation generation	Respiration M
Zig bee	Speaker& microphone	Data reduction	Reminder system	Restlessness M
	Tablet	EPR& PHR development	Social and emotional support	Self-assessment of psychological
	Web – cam	Information diagnosis and analysis	Ubiquitous information access	Self-assessment of symptoms
	X bee PRO S2B Module	Intelligent performance	Use colour for notification	Weight M
		Management of apps	Use Google TV
		Open and extensible architecture
		Push notification
		Real time monitoring
		Security and privacy
		Service UPnP technology based design
		Use guideline and standard
		User-friendliness
		Web-based design

GPRS: General Packet Radio Service; GPS: Global Positioning System; GSM: Global System for Mobile communications; LR-WPANs; low-rate wireless personal area networks; MMS: Multimedia Messaging Service; RFID: radio-frequency identification; SMS: short message service (SMS); Wi-Fi: technology for radio wireless local area networking of devices based on the IEEE 802.11 standards; Wi-Max: Worldwide Interoperability for Microwave Access; WLAN: wireless local area network.

Content validity ratio analysis

The results of the CVR calculation in the first step showed that 97 items of the questionnaire were completed by five specialists. The CVRs for 37 items were about 99%. This value is the critical for a panel of five experts stating thier agreement for each item.[29] In the second round, in order to improve the questionnaire, experts received more information about each item through face to face conversation, and after further discussion, the questionnaire was revised and the CVR recalculated, as shown in Table 3. Finally, 60 items with a CVR of over 99% were approved. The final questionnaire contained 60 essential items, including 12 items for applications, 14 items for functional requirements, 13 items for non-functional requirements, 10 items for devices and 11 items for communication infrastructure, as shown in Table 4.

Table 3.

Changes in questionnaire items in the first and second stages of the calculation of CVR.

	First step	Second step
Changes in items	Removed items due to low CVR	Added items after re-discussion
	Usage/application items	Usage/application items
	Oxygen saturation M Respiration M Electrocardiogram M Physical activity M Body temperature M Blood coagulation M Restlessness M Chest impedance M Galvanic skin response (GSR) M Electromyogram (EMG) M Diet tracking M Self-assessment of symptoms Self-assessment of psychological	Oxygen saturation M Respiration M Electrocardiogram M Body temperature M Physical activity M Diet tracking M Self-assessment of symptoms
	Functional items	Functional items
	Education for the users Social and emotional support Playing remotely game Antinational guide Use colour for notification Historical summary calculation Recommendation generation Use Google TV	Social and emotional support Use colour for notification Recommendation generation
	Non-functional items	Non-functional items
	Information diagnosis and analysis EPR& PHR development Applying virtual environment Management of apps Applying videoconference Applying Web portal Applying Audio message Applying Video linked Applying e-mail Animated guide Push Notification Applying cloud service Applying SNOMED CT Contact with HIS ability Open and extensible architecture Agent based design Service UPnP technology based design Applying POP – UP Applying HTTPS	Information diagnosis and analysis Applying virtual environment Applying videoconference Push Notification Open and extensible architecture Applying Audio message
	Device items	Device items
	Web – cam Intel health guide IOTdevice Game controller Motion detector Xbee PRO S2B Module HTC radar	Motion detector
	Communication Infrastructure items	Communication Infrastructure items
	Zigbee LR-WPANs Ethernet Cellular network	Ethernet
Average CVR of construct	Application items: 9.4/18=0.52	Application items: 12/12=1
	Functional items: 13/19= 0.68	Functional items: 14/14=1
	Non-functional items: 11.8/27=0.43	Non-functional items: 13/13=1
	Device items: 10.4/18=0.57	Device items: 10/10=1
	Communication items: 13/15=0.86	Communication items: 11/11=1
Construct validity of the whole questionnaire	0.59	1

Table 4.

Final items of the questionnaire after calculating the validity and reliability of the questionnaire.

Communication infrastructure	Devices	Non-functional requirements	Functional requirements	Usages/applications (M= monitoring)
Bluetooth	Activity hub	Applying virtual environment	Alert system	Blood pressure M
Ethernet	Broad band router	Audio message	Creation user profile	Blood sugar M
GPRS	Camera	Central data base	Detection of emergency situation	Body temperature M
GPS	Land line phone	Data reduction	Displaying/feedback information for the users	Diet tracking M
GSM	Medical sensors	Information detection and analysis	Ease of use for elderly	Electrocardiogram M
Internet	Motion detector	Intelligent performance	Graphical representation of outputs	Heart rate M
MMS	PC & laptop	Open and extensible architecture	Medication management	Medication compliance
SMS	Smart phone	Push notification	Motivation for the users	Oxygen saturation M
Telephone line	Speaker& microphone	Real time monitoring	Personalized services	Physical activity M
Wi-Fi and Wi- Max	Tablet	Security and privacy	Recommendation generation	Respiration M
WLAN		Use guideline and standard	Reminder system	Self-assessment of symptoms
		User-friendliness	Social and emotional support	Weight M
		Web-based design	Ubiquitous information access
			Use colour for notification

GPRS: General Packet Radio Service; GPS: Global Positioning System; GSM: Global System for Mobile communications; LR-WPANs; low-rate wireless personal area networks; MMS: Multimedia Messaging Service; RFID: radio-frequency identification; SMS: short message service; Wi-Fi: technology for radio wireless local area networking of devices based on the IEEE 802.11 standards; Wi-Max: Worldwide Interoperability for Microwave Access; WLAN: wireless local area network.

Reliability analysis

To measure the internal reliability of the questionnaire, Cronbach's alpha coefficient was calculated. The questionnaire was completed by 15 geriatric medical professionals and 15 health information management specialists giving a Cronbach's alpha value of 0.9. Moreover, to assess the external reliability of the questionnaire, a test–retest method was used. The questionnaire was completed by 10 specialists in two phases separated by a 1-week interval, and the correlation coefficient value was 0.85 (p-value<0.05).

Discussion

Despite the growing development of telehealth systems for the elderly living at home, few studies have been conducted that to identify the requirements of these systems for the frail elderly.[31-32] Furthermore, there was no reports of an instrument used to specifically measure telemonitoring system requirements for the frail elderly in Iran.

In a similar study, Barcelo et al., developed a five-question questionnaire. The questionnaire was presented to 20 clinicians to test its face validity. Feasibility was pre-tested with 10 non-institutionalized subjects who completed the questionnaire twice: once alone or with the help of a proxy, and again with the help of the researcher.[33] According to the study’s findings, the final questionnaire was compiled with 60 items and the CVR was 99%; Cronbach's alpha coefficient was 0.9 and a desirable correlation coefficient was achieved (r=0.85). Another study examined the construct validity and reliability of the Turkish language version of the seven-dimension ageing perceptions questionnaire short version (APQ-S) in a sample of Turkish community-dwelling elderly migrants in the Netherlands. The results of this study showed that the Turkish language version of the APQ-S was a psychometrically valid and reliable instrument for the assessment of ageing perceptions among the elderly.[34] Demiris et al. developed a questionnaire to assess patients’ impressions of the risks and benefits of home telecare. A preliminary 20-item questionnaire was developed and tested on 32 subjects. This related article proved to be of acceptable reliability (Cronbach’s alpha of 0.8) and validity; however, three items appeared to be redundant. The final 17-item version was tested twice on 10 more subjects and was found to have high test–retest reliability.[35] Moreover, in a related study, researchers developed the instrument for measuring quality of care from the perspective of non-institutional elderly people. This related study produced a self-administered questionnaire on quality of health care from the perspective of elderly people.[32]

According to the results of this study using the final questionnaire, the items related to the various types of applications of health telemonitoring systems for frail elderly living at home include monitoring of blood pressure, heart rate, blood glucose, weight, oxygen saturation, respiration, body temperature and physical activity. Lamprinakos et al. in a similar study to establish a health monitoring system for the elderly identified the importance of monitoring vital physiological signs such as blood pressure, heart rate, blood glucose, weight, oxygen saturation, respiration, body temperature and physical activity.[36] Further, we established the essential functional requirements for a health telemonitoring system aimed at frail elderly people, including “the detection of emergency situations”, “alert system”, “displaying information for the users” and “ubiquitous access”. The essential items related to non-functional requirements for these systems included “central database”, “information analysis”, “user-friendliness”, “intelligent performance” and “security”. The findings of the present study are consistent with the findings of another study aimed at creating and evaluating a health management system for the elderly.[37]

Kojima et al., in a related study, asked whether a questionnaire-based method using the Chinese version of the veteran-specific activity questionnaire (VSAQ) was applicable to measure exercise tolerance of Chinese elderly patients with coronary heart disease. The results of this study showed that the Chinese version of the VSAQ confirmed its validity and equivalence to the original version, especially when assessing people living with coronary heart disease and older adults. The results showed that the VSAQ is a valuable tool to evaluate exercise tolerance.38 Furthermore, this study identified the necessary devices for implementation and operation of the systems, including medical sensors, smart phones, cameras, personal computers and tablets, as well as the infrastructure requirements, such as internet, Bluetooth, Wi-Fi, Wi-Max and GPRS. A study by Hussain et al. in 2015 aimed to design an emergency and health care platform for the elderly and people with disabilities in smart cities. The devices and infrastructure requirements used in this study were in accordance with the requirements of the final questionnaire of the present study.[39]

As the target group of the system is the frail elderly population living in Iran, many of whom have several chronic diseases, the monitoring of vital signs is especially important. This function is known as the initial application of the home telemonitoring system. The use of telecommunication tools such as cameras, webcams and video communication in elderly people’s homes might cause a sense of lack of privacy, which may be exacerbated by Iranian culture and religion-related considerations. Thus, these tools were not included as system requirements. Furthermore, due to physical weakness and widespread lack of information literacy in the elderly,[19,20] features such as use of the web portal, email and computer games are not mentioned as initial requirements of the system. The main advantage of the current questionnaire is to focus on a specific age group, that is, elderly people. Moreover, the use of this questionnaire by designers of health telemonitoring systems will provide an improved understanding of the needs of frail elderly people in different dimensions. It is suggested that future studies could focus on conducting a pilot study of the questionnaire in Iran’s elderly care projects. Also, this tool can be used to develop a health monitoring system for Iranian frail elderly people.

Conclusion

We developed and validated a questionnaire for a health telemonitoring system for frail elderly people. The questionnaire was divided into five sections: applications, functional requirements, non-functional requirements, devices and communication infrastructure. This questionnaire could be used by healthcare system designers to determine the usages and requirements of health telemonitoring systems for frail elderly people living at home. The questionnaire is in accordance with Iranian culture and the existing technical and communication infrastructure in Iran.