Patient Continuity of Care Questionnaire in a cardiac sample: A Confirmatory Factor Analysis.

OBJECTIVE: Even though continuity is essential after discharge, there is a lack of reliable questionnaires to measure and assess patients' perceptions of continuity of care. The Patient Continuity of Care Questionnaire (PCCQ) addresses the period before and after discharge from hospital. However, previous studies show that the factor structure needs to be confirmed and validated in larger samples, and the aim of this study was to evaluate the psychometric properties of the PCCQ with focus on factor structure, internal consistency and stability.
DESIGN: A psychometric evaluation study. The questionnaire was translated into Swedish using a forward-backward technique and culturally adapted through cognitive interviews (n=12) and reviewed by researchers (n=8).
SETTING: Data were collected in four healthcare settings in two Swedish counties. PARTICIPANTS: A consecutive sampling procedure included 725 patients discharged after hospitalisation due to angina, acute myocardial infarction, heart failure or atrial fibrillation. MEASUREMENT: To evaluate the factor structure, confirmatory factor analyses based on polychoric correlations were performed (n=721). Internal consistency was evaluated by ordinal alpha. Test-retest reliability (n=289) was assessed with intraclass correlation coefficient (ICC).
RESULTS: The original six-factor structure was overall confirmed, but minor refinements were required to reach satisfactory model fit. The standardised factor loadings ranged between 0.68 and 0.94, and ordinal alpha ranged between 0.82 and 0.95. All subscales demonstrated satisfactory test-retest reliability (ICC=0.76-0.94).
CONCLUSION: The revised version of the PCCQ showed sound psychometric properties and is ready to be used to measure perceptions of continuity of care. High ordinal alpha in some subscales indicates that a shorter version of the questionnaire can be developed. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The process of translation and cultural adaption is validated using cognitive interviews with patients.

The 721 patients included in the confirmatory factor analysis is a sufficient sample size for evaluating the 27 items.

Using polychoric correlations is an appropriate method when data are ordinal.

Introduction

Continuity of care is considered a key indicator of quality of care1 and includes the seamless transition, consistent communication, coordination and coherence between healthcare settings and providers.2–4 Continuity of care has been defined as ‘the extent to which a series of healthcare services is experienced as connected and coherent and is consistent with a patient’s health needs and personal circumstances’.5 Three dimensions compose continuity of care: informational continuity, interpersonal continuity and management continuity. Informational continuity is the availability of clinical and psychosocial information across encounters and professionals. Interpersonal continuity is the subjective experience of a caring relationship between patients and their healthcare professional. Management continuity is the effective collaboration of teams across care boundaries to provide seamless care.5 6

Continuity of care is most at risk during the transition of the patients from hospital to home.4 7 For cardiac patients, discharge from hospital to home is a critical time involving both physical and psychological challenges as they often need to perform self-care, including monitoring and managing symptoms, handling complex medical regimen, implementing lifestyle changes and psychosocially adapting to a new diagnosis, complications or deterioration.8 Further, the period after hospitalisation due to cardiac conditions often involves follow-up at another caregiver, making coordination and continuity of care essential.9 10 However, patients have described the discharge process as fragmented and expressed that the clinical care pathways are hard to understand.11 12 Inadequate information transfer from hospital to primary care is described in several studies13–18 and can jeopardise patient safety.19 20 A quantifiable measure of patients’ perceptions of continuity of care can allow healthcare professionals or researchers to detect insufficiencies, examine continuity of care over time and evaluate the impact of interventions. Self-report questionnaires are the most common way to systematically collect information about patient experiences,21 and in efforts to improve the quality of care, such evaluations are essential.22 The most frequently used questionnaires that quantify continuity of care are the Usual Provider of Care,23 the Continuity of Care Index24 and the Sequential Continuity.25 None of these are conceptually sound since they do not include all dimensions of continuity of care as previously stated, but they focus on the number of physicians that provide patient service; further, they address outpatient settings exclusively.6 There are questionnaires that encompass the multidimensionality of continuity of care, that is, the Nijmegen Continuity Questionnaire26 and the Chao Perception of Continuity.27 28 However, neither of them explicitly addresses the period related to hospital discharge.26 27

The Patient Continuity of Care Questionnaire (PCCQ) is a generic questionnaire developed to cover all three dimensions of continuity of care described by Haggerty et al5 6, and it addresses both the period before and after hospital discharge. The original English version of PCCQ was developed and psychometrically evaluated in Canada. An exploratory factor analysis of 27 items suggested a six-factor solution; however, two items did not load on any factor but were placed in one factor each.29 Even though the original PCCQ showed acceptable internal consistency with Cronbach alpha of the factors ranging from 0.68 to 0.88, there were some limitations regarding the validation of the questionnaire. First, the sample size was smaller than recommended.30 Second, the previous validation did not take the ordinal nature of data into account. Treating ordinal data as continuous can result in that the association between an item and a factor seems weaker than it really is, which can lead to that pseudo factors reflecting item difficulty rather than the underlying concept are identified.31 Third, according to the explorative factor analysis, two items did not load on any factor.29 Exploratory factor analysis is generally a descriptive procedure, and the result needs to be confirmed in new samples. Despite those limitations mentioned previously, PCCQ has the potential to become a clinically useful self-report questionnaire to measure patients’ perceptions of continuity of care after hospitalisation.

A validated, clinically feasible questionnaire can be used to evaluate continuity of care and highlight areas where improvement is needed in order to facilitate continuity of care, and it is valuable to continue the validation of PCCQ, by confirming the factor structure in a sufficiently large sample using methods taking the ordinal nature of the questionnaire into account. Therefore, the aim was to evaluate psychometric properties of the PCCQ with focus on factor structure, internal consistency and stability.

Methods

Design and setting

A psychometric evaluation study using consecutive inclusion of patients discharged from one university hospital, two county hospitals, and one district hospital in central Sweden was conducted. Data were collected 2017–2019.

Study participants

Patients were eligible for inclusion if they had been hospitalised due to angina (International Classification of Diseases(ICD): I20.0, I20.1, I20.8 and I20.9), acute myocardial infarction (ICD: I21.0, I21.1, I21.2, I21.3, I21.4 and I21.9), atrial fibrillation (ICD: I48.0, I48.1, I48.2 and I48.9) or heart failure (ICD: I50.0, I50.1, I50.9 and I42.0). The inclusion criteria were (1) 18 years or older and (2) hospitalised for more than 24 hours. The exclusion criteria were (1) dementia with severe cognitive decline, (2) not able to read or understand Swedish, (3) expected survival of <3 months, (4) discharged to a nursing home or (4) resident in another county.

Data collection

Eligible patients were contacted by mail 4–6 weeks after discharge. In addition to the PCCQ, the patients received study information, a written informed consent form, questions on patient demographics and a postage prepaid envelope. One reminder was sent after 4–5 weeks. The final response rate was 52% (n=725). Patients responding to the questionnaire tended to be younger and were more often men compared with the non-responders. The most common diagnosis in the non-responders was heart failure. For test–retest analysis, the first 500 patients were consecutively invited to complete the PCCQ once again within 2 weeks, and 58% (n=289) of the patients responded. Medical charts of included patients were reviewed for length of hospital stay, comorbidities and healthcare utilization 30 days prior to and 30 days after the index hospitalisation.

Questionnaire

The original version of PCCQ was psychometrically evaluated in 204 orthopaedic and family medicine patients; the mean age was 64.9 years (SD 17.4); and 59.8% were female. The most primary diagnosis in family medicine patients was pulmonary or respiratory, and in the orthopaedic patients, the most common reason for hospitalisation was fracture.29

The 27 items of the PCCQ are scored on a 5-point Likert-type scale with the following response options: 1 (strongly disagree), 2 (somewhat disagree), 3 (hard to decide), 4 (somewhat agree) and 5 (strongly agree). In addition, there is an option to respond, ‘not applicable’ (N/A) to each item. The first section of the PCCQ includes items that address the period prior to hospital discharge, and the second section addresses the postdischarge period. The questionnaire has six suggested subscales: ‘information transfer to patients’, with six items; ‘relationships with providers in hospital’, with seven items; ‘relationships with providers in community’, with four items; ‘management of written forms’, with three items; ‘management of follow-up’, with three items; and ‘management of communication among providers’, with four items. Subscale scores are created by calculating the average scale scores; that is, all subscales have a possible score range between 1 and 5. Higher scores indicate that the patient perceived high continuity of care.29

Translation and cultural adaption

The PCCQ was not available in Swedish, and to ensure that the translated version of the PCCQ measures the same concept as the original version, the process of translation and cultural adaption was guided by the method described by Wild et al, which includes forward–backward translation, harmonisation and cognitive interviews. The backward translation and the harmonisation are essential to ensure that the translated version has the same content and conceptual basis as the source questionnaire, and make it possible to compare the results between different countries or cultures.32 In the first step in the translation process, the constructor was contacted and approved the translation of the questionnaire. In the next step, a forward translation was made by a native Swedish-speaking translator fluent in English. The backward translation was done by a native English-speaking translator fluent in Swedish who was blinded to the original version of the questionnaire. The forward and backward translated versions of the questionnaire were compared, and discrepancies were resolved among the two translators and the researchers. This resulted in a prefinal version of the questionnaire that was pilot tested through cognitive interviews using a think-aloud method33 34 with 12 patients who had been hospitalised due to heart failure. The cognitive interviews were conducted by the first and second authors. The patients found the content of the items to be relevant but found the phrasing of some items to be unclear. To increase the readability and culturally adapt the questionnaire to a Swedish setting, the phrasing of a few items was modified by the researchers in consultation with the constructor. Six of the patients who participated in the cognitive interview reviewed this version which led to additional changes of the wording in some of the items. The questionnaire was then reviewed for content validity and wording of items and instructions by an expert panel of eight researchers and nurses. Based on the experts’ advice, some changes in the wording of the instruction, response options and items were made. The middle response option was reworded into ‘neither agree nor disagree’.

Patient and public involvement

There was no patient or public involvement in the study design.

Data analysis

Descriptive statistics were used to present sociodemographic and clinical characteristics of the sample. Mean and SD were used for continuous data, while median and quartiles were used for ordinal data. The item responses were treated as ordered categorical data.

Item statistics were evaluated in terms of score distribution, floor and ceiling effects, and missing data. Floor and ceiling effects were evident if >15% of the item responses were distributed to the extremes.35 36 A ceiling effect occurs when patients’ scores cluster toward the high end of the measure and refers to the inability of the questionnaire to record if the patient’s perception of the measurement is higher than the questionnaire’s highest response option.37 The distribution of scale scores was evaluated with skewness and kurtosis statistics. The normal distribution has kurtosis and skewness values close to 0, respectively. If skewness was <−1 or >1, the distribution of scale scores was considered highly skewed and was analysed using non-parametric statistics.38

The factor structure of the PCCQ was evaluated with confirmatory factor analysis (CFA).39 Since the response categories in PCCQ are of Likert-type, they were considered to be ordinal; the robust estimator weighted least squares with means and variances (WLSMV) were used.31 The WLSMV estimator is based on a polychoric correlations matrix, taking the ordinal nature of data into account.40 The response option N/A was coded as missing. Missing data were handled by pairwise deletion, and the CFA was thereby based on 721 observations. The baseline CFA model was the six-factor model with 27 items suggested by Hadjistavropoulos and colleagues.29 Based on the findings, compete CFA models were evaluated, including a second-order model. To evaluate the fit of the models, the following recommended criteria were used: root mean square error of approximation (RMSEA) ≤0.06, Comparative fit index (CFI) of ≥0.95, Tucker-Lewis index (TLI) of >0.95 and standardised weighted root mean square residual (SRMR) of <0.08.41 Factor correlations were evaluated and a correlation of >0.7 was considered as strong, 0.3–0.7 as moderate and <0.3 as weak.42 Internal consistency was evaluated using an ordinal variant of Cronbach’s alpha described by Zumbo and colleagues.43 44 This reliability coefficient is computed much like the traditional Cronbach alpha but is based on a polychoric correlation matrix rather than a variance–covariance matrix. To be able to compare internal consistency with prior studies on PCCQ, the traditional Cronbach alpha is presented as well. Alpha values for scales used in research should not be smaller than 0.7.45 In addition, composite reliability, also known as a coefficient omega (ω), was also calculated. The CR was estimated as

where (Σλj)2 is the squared sum of the standardised factor loadings, while the Σεj is the sum of the residual variance.46 47

The test–retest reliability of the questionnaire was tested in a subsample of 289 patients, using intraclass correlation coefficients (ICC), two-way mixed-effects model and absolute agreement).48 An ICC of 0.70–0.90 represents good reliability and an ICC of ≥0.90 represents excellent reliability.49 Test–retest reliability is an important aspect of the questionnaire used to evaluate an intervention or groups of patients over time and indicates that any changes in the scale scores over time are due to changes of the perception of the continuity of care, not due to instable measures.50

Statistical analyses were conducted in IBM SPSS V.25 for Windows, Mplus V.7.4 (Muthén & Muthén, Los Angeles, California, USA) and R V.3.5.1 (the R Foundation for Statistical Computing, Vienna, Austria).

Results

A total of 725 patients participated in the study. The mean age was 72.4 years (SD=10.4) and 34% (n=245) were women. The main diagnoses at index hospitalisation were acute myocardial infarction 33% (n=237), atrial fibrillation 27% (n=194), heart failure 20% (n=145) and angina 20% (n=141). The most common comorbidity was prior myocardial infarction (44%, n=319), followed by heart failure (31%, n=223), stroke (8%, n=61) and chronic obstructive pulmonary disorder (7%, n=58). A total of 13% (n=95) of the included patients had been readmitted within 30 days after the index hospitalisation. Four of the patients had missing data on all items and were not included in the CFA analyses.

Item statistics

Ceiling effects (ie, extreme scores>15%) were evident for all items, except for item 22; however, all response options were used. There were missing responses in all items, but only four (items 3, 25, 26 and 27) were left unanswered by more than 5% of the patients. The N/A option was used most frequently in item 22 (‘As far as I am aware, the healthcare staff at the hospital communicated well with people from the municipality (eg, home help service and home healthcare) about my care’) and item 26 (‘As far as I am aware, the required forms were sent to all the applicable places/care providers’), where a total of 51% and 18%, respectively, used the N/A option (table 1).

Table 1

Item statistics of the Patient Continuity of Care Questionnaire (n=725)

Item number	Items	n	Median	q1–q3	Item score distribution %						Missing data (%)
					1	2	3	4	5	N/A
1	Information about diagnosis	706	5	4–5	1.0	1.0	6.2	20.0	68.0	1.2	2.6
2	Information about prognosis	694	4	3–5	4.1	4.6	15.6	24.8	43.6	3.0	4.3
3	Information about non-acute symptoms	685	4	3–5	7.4	6.6	22.1	26.2	24.1	8.0	5.5
4	Information about acute symptoms	702	4	3–5	5.7	4.6	13.1	23.9	44.3	5.4	3.2
5	Information about medication	706	5	4–5	2.6	3.7	10.2	28.0	49.8	2.3	3.3
6	Advice about physical activity	701	4	3–5	14.5	3.7	17.9	21.4	30.9	9.0	2.6
7	Information about follow-up appointments	708	5	2–4	4.1	3.3	9.9	21.8	55.6	2.9	2.3
8	Information about treatment after discharge	700	4	3–5	13.7	5.1	16.0	19.7	26.1	16.0	3.4
9	Understood expectations	698	4	3–5	3.2	3.6	17.0	26.5	38.9	7.2	3.7
10	Knew about situation and medical condition	704	5	4–5	1.9	3.4	9.7	24.8	54.2	3.0	2.9
11	Confidence in healthcare staff before discharge	709	5	5–5	0.3	0.8	5.0	17.1	73.7	1.0	2.2
12	Satisfied with information	707	5	4–5	1.5	3.3	10.1	23.4	58.3	0.8	2.5
13	Satisfied with emotional support	708	5	4–5	1.7	2.2	11.0	22.6	56.6	3.6	2.3
14	Satisfied with opportunities to ask questions and talk	711	5	4–5	1.4	3.2	8.0	25.7	57.7	2.2	1.9
15	Healthcare staff communicated well with each other	709	5	4–5	1.4	2.8	13.8	24.1	53.7	2.1	2.2
16	Plan for follow-up arranged and explained	700	4	3–5	8.4	5.7	17.2	25.5	32.6	7.2	3.4
17	Felt sufficiently prepared for discharge	709	5	4–5	4.3	4.1	11.9	20.4	54.6	2.5	2.2
18	Healthcare staff know medical condition after discharge	693	5	3–5	3.2	3.3	14.3	17.9	43.2	13.7	4.4
19	Confidence in healthcare staff after discharge	699	5	4–5	3.0	3.0	13.5	17.8	45.4	13.7	3.6
20	Satisfied with information after discharge	696	5	3–5	4.3	3.3	14.5	18.5	41.1	14.1	4.0
21	Satisfied with opportunities to ask questions after discharge	696	4	3–−5	4.3	5.5	12.4	21.2	38.2	14.3	4.0
22	Hospital communicated well with municipality	695	3	1.5–4	11.2	2.8	15.9	6.1	9.0	51.0	4.1
23	Information from hospital to follow-up doctor	694	4	3–4	7.3	3.6	19.7	13.7	35.7	15.7	4.3
24	Consistent information from all healthcare staff	692	4	3–5	8.4	4.0	21.1	19.6	31.9	10.5	4.6
25	Required forms were filled in	686	4	3–5	7.0	2.3	24.4	11.3	38.3	11.2	5.4
26	Required forms were sent	686	4	3–5	9.1	2.2	26.1	10.8	28.7	17.8	5.4
27	No forms went missing at discharge	688	4	3–5	8.4	1.8	25.9	8.6	36.6	13.7	5.1

N/A, not applicable.

Factor structure

The baseline model demonstrated unsatisfactorily model fit according to RMSEA (>0.06) and TLI (<0.95), while CFI and SRMR showed an acceptable model fit (table 2). Evaluation of the modification index identified item 15 (‘The healthcare staff communicated well with each other while I was in the hospital’) as a problematic indicator for ‘management of communication’ and item 24 (‘I have received consistent information about my care from all healthcare staff’) as a problematic indicator for information transfer to patients.

Table 2

Goodness-of-fit indices for the confirmatory factor analyses of the Patient Continuity of Care Questionnaire (n=721)

Models	Goodness of fit			RMSEA
	χ2	df	P value	RMSEA	90% CI	CFI	TLI	SRMR
Baseline model	1552.504	309	<0.001	0.075	0.071 to 0.078	0.95	0.94	0.06
Model 2 (final model)	833.994	309	<0.001	0.049	0.045 to 0.053	0.98	0.98	0.04
Second-order model	1171.810	318	<0.001	0.061	0.057 to 0.065	0.97	0.97	0.05

Baseline model: model as presented by Hadjistavropoulos et al.29 Model 2: item 15 moved to factor relations in hospital, item 6 moved to factor information transfer to patients, and item 24 moved to factor management of communication. Second-order model: factor structure as in model 2, adding a second-order factor.

CFI ≥0.95, RMSEA≤0.06, SRMR<0.08, TLI>0.95.

CFI, Comparative Fit Index; df, degrees of freedom; RMSEA, root mean square error of approximation; SRMR, standardised weighted root mean square residual; TLI, Tucker Lewis Index.

As the next step, we scrutinised the factor structure of the baseline model and compared it to the concept and the dimensions of continuity of care. These conceptually based considerations guided the revision of the factor structure that was evaluated in model 2: item 15 is conceptually related to how the patient experienced the relations to the healthcare personal in hospital, and for that reason, it was examined as an indicator for ‘relations in hospital’. Item 24 is conceptually related to that care received from different providers is connected in a coherent way, and therefore it was examined as an indicator for management of communication. Of conceptual reasons, we also found item 6 (‘I received advice about physical activity and exercise’) as a poor indicator for management of communication and conceptually more relevant for information transfer to patients. In the evaluation of model 2, all criteria for a good model fit was met (table 2). The standardised factor loadings ranged from 0.679 to 0.943 (table 3). The factor correlations ranged between 0.547 and 0.874 (p<0.001) (table 4). The distribution of scale scores was close to normal in all subscales except for information transfer to patients’, relations in hospital, ‘relations in community’ (table 3).

Table 3

Standardised factor loadings, reliability and scale scores for the final model (model 2) of the Patient Continuity of Care Questionnaire (n=721)

		Information transfer to patients	Relations in hospital	Relations in community	Management of forms	Management of follow-up	Management of communication
1	Information about diagnosis	0.746
2	Information about prognosis	0.805
3	Information about non-acute symptoms	0.786
4	Information about acute symptoms	0.799
5	Information about medication	0.748
6	Advice about physical activity	0.705
9	Understood expectations		0.837
10	Knew about situation and medical condition		0.847
11	Confidence in healthcare staff before discharge		0.849
12	Satisfied with information		0.912
13	Satisfied with emotional support		0.851
14	Satisfied with opportunities to ask questions and talk		0.886
15	Healthcare staff communicated well with each other		0.774
17	Felt sufficiently prepared for discharge		0.771
18	Healthcare staff know medical condition after discharge			0.912
19	Confidence in healthcare staff after discharge			0.938
20	Satisfied with information after discharge			0.924
21	Satisfied with opportunities to ask questions after discharge			0.904
25	Required forms were filled in				0.934
26	Required forms were sent				0.943
27	No forms went missing at discharge				0.905
7	Information about follow-up appointments					0.782
8	Information about treatment after discharge					0.778
16	Plan for follow-up arranged and explained					0.841
22	Hospital communicated well with municipality						0.679
23	Information from hospital to follow-up doctor						0.766
24	Consistent information from all healthcare staff						0.860
Reliability
Ordinal alpha		0.84	0.95	0.95	0.94	0.84	0.82
Cronbach's alpha		0.84	0.91	0.92	0.91	0.78	0.78
Composite reliability		0.89	0.95	0.96	0.95	0.84	0.81
ICC (95% CI)		0.86	0.94	0.91	0.87	0.83	0.76
		(0.86–0.90)	(0.93–0.95)	(0.90–0.93)	(0.84–0.89)	(0.80–0.86)	(0.71–0.81)
Scale distribution
Mean		3.99	4.37	4.11	3.71	3.83	3.36
Skewness		−1.03	−1.50	−1.17	−0.63	−0.91	−0.63
Kurtosis		0.62	2.23	0.74	−0.44	0.16	−0.64

ICC, intraclass correlation coefficient.

Table 4

Factor correlations based on model 2 and the second-order model of the Patient Continuity of Care Questionnaire (n=721)

	Information transfer to patients	Relations in hospital	Relations in community	Management of forms	Management of follow-up	Management of communication
Relations in hospital	0.790
Relations in community	0.700	0.710
Management of forms	0.547	0.595	0.650
Management of follow-up	0.863	0.854	0.839	0.684
Management of communication	0.692	0.758	0.873	0.874	0.861

All factor correlations were statistically significant at a level of p<0.001

Finally, a second-order factor analysis were conducted to examine if the causal construct, that is, continuity of care is a function of the first-order latent factors from model 2. This second-order model demonstrated reasonable model fit with RMSEA close above 0.06 and satisfactory model fit in all other fit indices (table 2). The factor loadings for the second order model, that is, between the higher order factor and the six factors ranged between 0.750 and 0.964. Thus, the findings support the that the six factors from model two are all important aspects of continuity of care.

Reliability

The internal consistency for the final measurement model evaluated by ordinal alpha was excellent for all factors and ranged between 0.82 (management of communication) and 0.95 (relations in hospital and relations in community). The corresponding internal consistency measured with traditional Cronbach alpha ranged between 0.78 and 0.92 (table 3).

The composite reliability was satisfactory as well and ranged from 0.81 (management of communication) to 0.96 (relations in community) (table 3).

According to the ICC, test–retest reliability for the six factors ranged between good (information transfer to patients, management of forms, management of follow-up and management of communication) and excellent (relations in hospital and relations in community) (table 3).

Discussion

This study evaluated the psychometric properties of the PCCQ using appropriate statistical analyses for ordinal data and sufficient sample size. After minor revisions, the previous suggested factor structure was overall confirmed. All subscales demonstrated satisfactory internal consistency and test–retest reliability in the present sample.

Translation and cultural adaption

The process of translation and cultural adaptation brought some challenges, especially due to semantic equivalence (eg, do words mean the same thing) and idiomatic equivalence (eg, language used for casual communication),51 and some items needed rephrasing. The cognitive interviews were a guide for how to reword phrases for cultural adaptation. For instance, findings from the interviews revealed that the translation of ‘provider’ was perceived as the hospital or region, and instead the phrase ‘healthcare personnel’ was suggested. The expert panel advised additional changes in the wording of the instruction, response categories and items which further strengthened the content validity of the translated version of PCCQ.

Item and scale statistics

All response options were used for all items, which strengthens the validity and reliability in this sample.50 A ceiling effect was evident in 26 of 27 items. Ceiling effects are common in measures of patient’s subjective perception of the care experience, in particular, in questionnaires based on Likert-type response options.37 However, the distribution of item responses is highly dependent on the sample and can therefore vary between different settings or groups. It is possible that the ceiling effect is due to the patients participating in this study being satisfied with the continuity of their care. Notable are that since the subscales were skewed and the items were on ordinal level, subscale scores should be analysed using non-parametric analyses.

The PCCQ is currently constructed with a 5-point Likert-type scale, including the N/A option. The advantage of including N/A as a response option is that patients can use it when the item is not applicable to their situation instead of being forced to take a stand. This might result in fewer items being omitted.50 However, Streiner and colleagues argue against the use of ‘N/A’ as a response option since N/A implies that the patient has not taken a position on the statement, and from a psychometric view, the item still remains unanswered. Rather than using an N/A response option, the rule of thumb should be to use items that are relevant to the sample studied,50 which can be obtained by, for instance, the use of item banks.52 Even though not including a N/A option might result in patients omitting items that are not relevant to their situation, we recommend exclusion of the N/A option in future validations of the PCCQ.

The evaluation of the original factor structure demonstrated an acceptable model fit in all indices except RMSEA. Since the RMSEA was >0.06, some conceptually based revisions in the factor structure was advocated.41 53 The changes resulted in improved model fit, which indicates that the new factor structure of the revised PCCQ is a more accurate description of the concept of continuity of care, and the questionnaires’ ability to measure information transfer to patients, relations in hospital and management of communication has improved. However, since it is the constructors’ advice to calculate the mean score rather than the sum score for the subscales, the changes in the subscales did not affect the scale score significantly.

The evaluation of the second-order factor model strengthens the construct validity and imply that all the six factors are relevant to the construct of continuity of care. This finding is in line with a recent concept analysis, proposing that continuity of care occurs only when all dimensions are aligned and integrated.54

The ordinal alpha for the subscales was higher than the recommended >0.7.50 However, Streiner argues that alpha>0.9 may be a reflection of that some items are redundant.45 The subscales relations in hospital, relations in community and management of forms had an alpha higher than 0.9, and therefore deletion of items might be considered in any future validation. On the other hand, this problem was not addressed by the patients in the cognitive interviews. One important limitation with the alpha coefficient is the assumption that all factor loadings are the same for all items. It has therefore been criticised as a measure of both reliability and internal consistency.55 In contrast, composite reliability take consideration to the variation in factor loadings.46 47 In the present study, both ordinal alpha and composite reliability were similar. However, the traditional Cronbach alpha coefficient was lower than both ordinal alpha and composite reliability. This stresses the importance to use appropriate statistics for ordinal data.

The analysis of test–retest reliability demonstrated satisfactory stability in this sample, indicating that any changes in the scale scores over time are due to changes of the perception of the continuity of care, not due to instable measures.

Methodological aspects

This study included a sufficiently large group for a CFA. Brown has shown in simulation studies that a sample size of 150–200 can be enough in non-complex models.31 The study population consisted of cardiac patients from various hospital settings and diagnoses, and the results therefore reflect a wide variety of experiences of continuity of care, which have advantages in terms of generalisability and usability of the questionnaire.

There were multiple sites including study participants. It can be seen as a strength of the study that different types of hospitals were included. On the other side, one can argue that this also might influence the independence of the data. However, the number of sites was too small for multilevel latent variable modelling.

In studies that use mail survey to collect data, it has been found that a possible response bias is that patients who find the research topic important or relevant to their situation are more likely to respond to a questionnaire.56 In this sample, response bias might be due to illness when non-responders were older and diagnosed with heart failure. Further, due to expected differences in gender and mean age in different diagnoses, a selection bias cannot be disregarded. Furthermore, the generalisability of this study might have been affected by the response rate since almost half of the patients that filled the inclusion criteria did not respond to the questionnaire. However, the response rate is in line with what can be expected in postal surveys, and the generalisability is strengthened by the fact that the sample is consecutively included and that patients are included from four different hospitals and different diagnoses.

A related problem is the large number of missing data. This can to a large extent be explained by the fact that many patients used the N/A response option. To handle this problem in the CFAs, a pairwise deletion was considered more appropriate compared with a listwise deletion to not drop too many observations. Nevertheless, both methods can result in biassed parameter estimates; in particular, if data are missing not at random, that is, that the missing data on a given variable are caused by the subject level of that variable.57 In these cases, direct maximum likelihood estimators are commonly recommended. However, the WLSMV estimation is a more appropriate method to handle ordered categorical indicator variables and is often recommended in the literature.31 Thus, it cannot be excluded that our procedure to handle missing data has biassed the results.

The internal consistency was evaluated with ordinal alpha since it has been shown to estimate reliability more accurately than Cronbach’s alpha when ordinal response scales are used.44 Notably, the ordinal alpha values were higher compared with traditional Cronbach’s alpha in all factors. It is well known that statistical methods treating ordinal data as continuous can result in attenuated estimates of associations between variables and suggest factors that are artefacts of item difficulty rather than reflections of how the concept is constructed.31 58 This stresses the importance to use appropriate statistical methods to evaluate self-rating scales based on ordinal data, such as Likert-type response options.

It is difficult to decide on an optimal time interval for the test–retest and recommendations range from 2 to 14 days. If the interval is too short, there is a risk for recall bias, and if the interval is too long, things might have changed.50 Even though we chose an interval of 14 days, and some patients did not answer until another 1–2 weeks, the ICC ranged from good to excellent.

Conclusion

The revised measurement model of PCCQ demonstrated sound psychometric properties regarding factor structure, internal consistency and test–retest reliability in this Swedish sample of cardiac patients. The PCCQ can thereby be used to measure patients’ perceptions of factors influencing continuity of care. The high ordinal alpha for some factors indicates that a shorter version of the questionnaire should be considered in future revisions.