Measuring improvement in health-status with the Oxford Participation and Activities Questionnaire (Ox-PAQ).

Background: The Oxford Participation and Activities Questionnaire (Ox-PAQ) is a recently validated 23-item generic patient reported outcome measure that is theoretically grounded in the World Health Organisation International Classification of Functioning, Disability and Health (ICF), as well as fully FDA compliant. The measure comprises three domains, Routine Activities (14 items), Emotional Well-Being (5 items) and Social Engagement (4 items), and has been shown to be both valid and reliable. The aim of this study was to make a preliminary assessment of the sensitivity to the change of the Ox-PAQ.
Methods: Three hundred and seventy three people with either chronic pulmonary obstructive disease, motor neurone disease, multiple sclerosis or Parkinson's disease completed the Ox-PAQ electronically on two occasions, six months apart. Participants completed an additional four questions during the second administration of the Ox-PAQ in order to assess any relevant changes since baseline. For each of the three Ox-PAQ domains, the minimally important difference (MID) and effect size (ES) was calculated from those participants indicating small, but meaningful positive improvement.
Results: Two hundred and ninety respondents were entered into the final analysis, 77.74% of the original sample. For those indicating "slight improvement" in Routine Activities (n=19), an MID of 7.51 and an ES of 0.32 was calculated. For Emotional Well-Being (n=26), the MID was 10.77 and the ES 0.44. For Social Engagement (n=16), the MID was 5.47 and the ES 0.28.
Conclusion: Data presented provides MID improvement and effect sizes for Ox-PAQ domains that will act as preliminary guidance for those powering clinical trials assessing activity and participation. Future research with larger samples will provide further sensitivity to change data for the Ox-PAQ.

Background

The Oxford Participation and Activities Questionnaire (Ox-PAQ) is a 23-item patient reported outcome measure (PROM) specifically designed for generic use in clinical trials and interventions where the primary focus is on the improvement and/or maintenance of participation and activity. The measure is theoretically grounded in the World Health Organisation International Classification of Functioning, Disability and Health (ICF),1 and its development2 was fully compliant with current best practice guidelines such as those provided by the United States Food & Drug Administration (FDA).3 The Ox-PAQ comprises three domains, Routine Activities (14 items), Emotional Well-Being (5 items) and Social Engagement (4 items), and psychometric analyses indicate that it is both valid and reliable.4

An important attribute of any given PROM is sensitivity to change, ie, the ability to detect meaningful changes in health status over time.5,6 Given that the purpose of many clinical trials and interventions is to measure improvement in a given condition, this short report aims to provide preliminary evidence of the ability to detect positive change through administration of the Ox-PAQ. Specifically, analyses aim to determine the minimally important difference (MID) and effect size (ES) for each of the three Ox-PAQ domains from those participants indicating small, but meaningful, positive improvement.

Methods

Ethical approval for the study was granted by the Medical Sciences Inter Divisional Research Ethics Committee of the University of Oxford (ref: R44281/RE001).

Participants

Three hundred and seventy three people with a diagnosis of either chronic obstructive pulmonary disease, motor neurone disease, multiple sclerosis or Parkinson’s disease were recruited via their respective support organizations in the United Kingdom; the British Lung Foundation, Motor Neurone Disease Association, Multiple Sclerosis Society and Parkinson’s UK. Written informed consent was obtained electronically from all participants included in the study.

Materials

An e-based version of the Ox-PAQ was administered via Qualtrics survey software.7 As outlined in the Introduction, the measure comprises three domain; Routine Activities, Emotional Well-Being and Social Engagement. Higher scores indicate greater problems with the relevant domain. Following migration from its paper-based equivalent, the e-based Ox-PAQ has been shown to be both usable and acceptable to respondents.8

Procedure

In response to advertisements placed by their relevant support organizations, participants contacted a member of the research team via email and were subsequently forwarded a unique link to the survey.

The Ox-PAQ was administered to participants on two occasions, six months apart (Time 1 and Time 2). At Time 2, four transition items9 were included, one for each of the three Ox-PAQ domains and one as an indicator of overall health. Participants were asked to indicate their degree of change using one of five response options; “much better”, “slightly better”, “the same”, “slightly worse”, “much worse”. Demographic data were captured prior to completion of the Ox-PAQ.

Statistical analysis

Data were analyzed using IBM SPSS Statistics Version 22.10 Means and standard deviations (SDs) were calculated for relevant demographic variables. Mean change scores were calculated for each domain of the Ox-PAQ in order to establish minimally important differences (MIDs).11 Effect sizes were subsequently calculated as the difference between mean domain scores at Time 1 and Time 2, divided by the standard deviation of domain scores at Time 1.12,13

Results

Participants who completed the Ox-PAQ at both Time 1 and Time 2 and with no missing data for at least one Ox-PAQ domain were entered into the final analysis, equating to two hundred and ninety respondents (77.74% of the original sample). The sample consisted of 122 males (42.1%) and 168 females (57.9%). The mean age was 61.87 years (SD 10.82) and mean age at diagnosis 53.03 years (SD 13.53). For those indicating “slight improvement” in Routine Activities (n=19) an MID of 7.51 and an ES of 0.32 was calculated. For Emotional Well-Being (n=26), the MID was 10.77 and the ES 0.44. For Social Engagement (n=16), the MID was 5.47 and the ES 0.28. Further details can be viewed in Table 1.

Table 1

Mean Ox-PAQ domain scores at Time 1 and Time 2 for respondents who indicated they were “slightly better” with subsequent minimally important differences and effect sizes

Ox-PAQ domain (n)	Mean (SD) at Time 1	Mean (SD) at Time 2	Mean change/minimally important difference	Effect size
Routine Activities (n=19)	43.04 (22.98)	35.53 (20.20)	7.51	0.32
Emotional Well-Being (n=26)	48.27 (24.41)	37.50 (20.16)	10.77	0.44
Social Engagement (n=16)	33.98 (19.36)	28.51 (19.63)	5.47	0.28

Abbreviation: SD, standard deviation.

Discussion

This short report provides a preliminary assessment of MID improvement in Ox-PAQ domain scores, alongside effect size calculations for each domain. The MID is regarded as the smallest change that respondents perceive as important,11 and the MIDs reported here should be sought in studies incorporating the Ox-PAQ to evaluate changes over time in activity and participation.

Traditionally, effect sizes of 0.2, 0.5 and 0.8 have been regarded as indicating small, medium and large degrees of change, respectively.14 However, these values are generally regarded as a guide, and ideally effect size interpretation should be based on analyses of the measurement properties of any given questionnaire. Consequently, the analyses presented here suggest effect sizes for the Ox-PAQ domains fall between 0.28 and 0.44, indicating small to medium degrees of change that are subjectively meaningful to respondents completing the Ox-PAQ.

It is acknowledged that the sample sizes for the reported analyses are small, although it is noteworthy that many similar studies include small numbers who reflect sensitivity to change in their analyses.15–17 Additional research with larger samples will provide more precise MIDs and effect sizes for the Ox-PAQ. Further research is also planned to determine what constitutes a minimally important negative change.

Conclusion

Data presented provide MID improvement and effect sizes for Ox-PAQ domains that will act as preliminary guidance for those powering clinical trials which assess activity and participation. Future research with larger samples will provide further sensitivity to change data for the Ox-PAQ.