Design and performance of a multi-centre randomised controlled trial and economic evaluation of joint tele-consultations [ISRCTN54264250].

BACKGROUND: Appropriate information flow is crucial to the care of patients, particularly at the interface between primary and secondary care. Communication problems can result from inadequate organisation and training, There is a major expectation that information and communication technologies may offer solutions, but little reliable evidence. This paper reports the design and performance of a multi-centre randomised controlled trial (RCT), unparalleled in telemedicine research in either scale or range of outcomes. The study investigated the effectiveness and cost implications in rural and inner-city settings of using videoconferencing to perform joint tele-consultations as an alternative to general practitioner referral to the hospital specialist in the outpatient clinic.
METHODS: Joint tele-consultation services were established in both the Royal Free Hampstead NHS Trust in inner London, and the Royal Shrewsbury Hospitals Trust, in Shropshire. All the patients who gave consent to participate were randomised either to joint tele-consultation or to a routine outpatients appointment. The principal outcome measures included the frequency of decision by the specialist to offer a follow-up outpatient appointment, patient satisfaction (Ware Specific Questionnaire), wellbeing (SF12) and enablement (PEI), numbers of tests, investigations, procedures and treatments.
RESULTS: A total of 134 general practitioners operating from 29 practices participated in the trial, referring a total of 3170 patients to 20 specialists in ENT medicine, general medicine (including endocrinology, and rheumatology), gastroenterology, orthopaedics, neurology and urology. Of these, 2094 patients consented to participate in the study and were correctly randomised. There was a 91% response rate to the initial assessment questionnaires, and analysis showed equivalence for all key characteristics between the treatment and control groups.
CONCLUSION: We have designed and performed a major multi-centre trial of teleconsultations in two contrasting centres. Many problems were overcome to enable the trial to be carried out, with a considerable development and learning phase. A lengthier development phase might have enabled us to improve the patient selection criteria, but there is a window of opportunity for these developments, and we believe that our approach was appropriate, allowing the evaluation of the technology before its widespread implementation.

RCT Entities:   Population Interventions Outcomes

Background

Between 6% and 10% of patient episodes in primary care result in a referral for specialist opinion [1], accounting for most of the patients seen for the first time in hospital outpatients. This process requires effective communication between all parties involved, with deficiencies leading to a range of problems [2-5]. The current model of referral often leads to duplication of investigation and unnecessary follow-up, and there is dissatisfaction by all concerned [6-8]. A number of models have evolved to overcome these difficulties, including the domiciliary visit and, more recently, the outreach clinic [9]. The latter was expected to lead to better communication, with educational and clinical benefits, but studies suggest this was rarely achieved because of the lack of GP involvement [10], and there are also cost implications. In a study in the Netherlands, three general practitioners accompanied patients to an orthopaedic clinic and sat in on the consultations [11]. Reported benefits included important reductions in hospital follow-up appointments, tests and investigations, and improvements in health status for patients in the intervention group one year following referral. The general practitioners made significantly fewer referrals following the study. However, such an arrangement presents obvious practical and resource problems, making implementation unlikely. Video-conferencing technologies allow virtual meetings between patients and practitioners, and several studies have investigated teleconferenced consultations in a range of clinical areas including dermatology, orthopaedics, cardiology, and psychiatry [12]. Studies have reported high levels of patient satisfaction, though the validity of these findings has been questioned [13]. Acceptability to patients was confirmed in a feasibility study which we carried out, using tele-consultations across a range of clinical specialities [14]. A subsequent pilot study involving general practitioners from inner city practices and 132 patients referred to specialists in orthopaedics, neurology, gastroenterology, and otorhinolaryngology, indicated the feasibility of recruiting patients, confirmed that joint tele-consultations were acceptable across a range of specialities, and suggested that patients' satisfaction might be higher than with conventional outpatient appointments [15] Thus, it appeared that joint tele-consultations might have the potential to make an important impact on the quality and effectiveness of referral from primary to secondary care. However, a large RCT was required to determine reliably the impact of this technology on patient welfare, use of health service facilities for investigation and treatment, and the economic implications for primary care and hospitals, patients, and the nation.

Methods

A randomised controlled trial was designed to assess the effectiveness and cost implications of joint tele-consultation compared with routine outpatient consultations, in urban and rural settings. The hypotheses were as shown:

Compared with conventional outpatients, joint tele-consultations will:

· have a positive impact on patient satisfaction

· have a positive on patients health status

· reduce hospital follow-up appointments

· reduce numbers of medical interventions, tests and investigations

· reduce costs incurred by the patients' attending outpatient appointments

· reduce the economic costs to the nation through less time off work

· incur no increased cost for the NHS

· be more cost-efficient than physical outreach clinics

It was decided to investigate whether there were systematic differences between joint tele-consultations in rural and urban settings. The Royal Free Hampstead NHS Trust, in London, serves general practitioners in inner city and urban settings. The Royal Shrewsbury Hospitals Trust in Shropshire serves general practitioners almost exclusively in rural settings. The project teams recruited and trained specialists in each trust and general practitioners who referred patients to specialists participating in the trial. The numbers of participating general practitioners and hospital clinicians at each of the sites, together with their specialties are shown. (Table 1) A subsequent study demonstrated that the participating GPs were representative of their peers in their use of information technologies [16].

Table 1

Participating general practitioners and hospital clinicians

	London	Shrewsbury	Total
Number of GPs	74	60	134
Number of practices	15	14	29
Number of consultants by specialty:	12	8	20
Endocrinology	1	0	1
ENT	2	2	4
Gastroenterology	1	2	3
General Medicine	0	2	2
Neurology	0	1	1
Orthopaedics	2	0	2
Rheumatology	2	0	2
Urology	4	1	5

The tele-consultations used PC-based technology (Intel Business Video Conferencing version 5) and ISDN2 links. The specialists provided protected appointments for joint tele-consultation and routine outpatient appointments for trial patients. To ensure comparability in the two arms of the trial, waiting times of no more than eight weeks were established for patients in both arms of the trial. In most cases, the specialists were unable to provide dedicated tele-consultation clinics, but generally offered appointments at the beginning or end of their routine outpatient clinics.

Baseline measures consisting of demographic variables and a measure of perceived health status (SF12 [17] for adults and Child Health Questionnaire for subjects aged under 16 years) were collected on a questionnaire sent with the appointment letter. The referring general practitioners completed a Duke Severity of Illness Inventory (DUSOI) for each patient [18]. The outcome measures immediately following the index consultation included the Ware Specific Visit questionnaire (patient satisfaction) [19] and the Patient Enablement Inventory (PEI) [20]. Hospital specialists completed a questionnaire on a range of aspects of the consultation. The six months assessment included a patient questionnaire containing the SF12 and Child Health Questionnaire questions, and questions about health care resources used over the previous six months. In addition, information was collected from general practice and hospital records about activity for six months following the index consultation. Data included the number of visits to the general practitioner, the nurse or other clinical practice staff. Data were also recorded on the number of home visits and referrals from the practice, radiological investigations, blood tests and laboratory investigations and other tests and investigations. A record of drug prescriptions issued in the 6 months prior to and following the index consultation was downloaded from the practices' databases. Equivalent data was recorded on hospital activity; in addition the numbers of visits as an outpatient or inpatient were recorded, together with the length of inpatient stays, number of visits for day surgery, visits to accident and emergency departments, and other hospital contacts. Interventions that could reliably be identified (using criteria developed by the research team after extensive discussion) as specific to the specialty to which the patient had been referred, were designated as attributable to the index consultation for the final analysis, as were all of the non specific items carried out within four weeks of the consultation. We did not include any specific measures to test diagnostic accuracy, but the six month follow-up period was deemed sufficient to detect any major adverse outcomes.

The trial was as inclusive as possible, but patients requiring urgent assessment, private patients and patients with significant difficulty communicating in English were excluded. All other patients referred by the participating general practitioners to specialists taking part were eligible for recruitment. The exclusion criteria were extended shortly after the start of the trial to exclude patients referred for a specific investigation (such as a hearing test). The small number of such patients who had already been recruited were retained for inclusion in the final analysis. Ethical approval was granted by the relevant Local Research Ethics Committees. General practitioners were encouraged to seek consent from their patients when the decision was made to refer. Those patients eligible for the trial for whom consent had not yet been sought, were sent a patient information leaflet and a consent form by the project staff. Eligible patients for whom consent had been obtained were randomly allocated to either the joint tele-consultation or the routine outpatient consultation arm of the trial. Computerised randomisation in permuted blocks of sizes 4 and 6 (arranged unpredictably) was stratified by centre, practice and specialty.

Power calculations were carried out on the basis of an intention-to-treat analysis, considering patients in the groups as originally randomised. The decision by the specialist whether or not to offer a follow-up appointment was selected as the primary outcome, with further analyses to assess differences in the effect of telemedicine on repeat appointments in both the urban and rural settings, and between five pre-defined specialty groups. Patient-centred outcomes and measures of health care resource use were selected as secondary outcomes. These included 6-month mental and physical health scores, patient satisfaction and patient enablement, together with tests and investigations, prescriptions, contacts with the general practice, hospital outpatient visits, accident and emergency contacts and inpatient stays, focusing on those items attributed to the index consultation. The sample size was based on the study of joint consultations in orthopaedics, carried out by Vierhout et al. [21]. It was decided to calculate the numbers needed to give power to detect a 20% reduction (from 60% to 40%) in booked hospital outpatient follow-up appointments between the randomised groups, both overall and separately for each of the five specialty groups, with 90% power and 5% significance. It was calculated that 250 patients were needed in each specialty. Taking account of the imbalance in numbers between specialties, and the estimate of 30% of missing outcome data, the required total sample size was calculated to be 1950 patients.

Economic evaluation focused on actual resource use and was designed to measure the total costs to the NHS of the two consultation modalities. This included the immediate costs of the consultations, and the consumption of NHS resources in the 6-month period following the index consultation. Non-participant observation of a number of consultations allowed us to estimate the labour time devoted to consultations, and to record the duration of the different tasks in the consultation. Each resource item was costed from data supplied by the two trusts and the NHS Reference Costs 2000. Prescription costs were obtained from the GP's computerised record systems. The economic evaluation also investigated the costs to patients of the respective consultation methods, using data on a number of economic variables collected from a patient questionnaire, and one-way sensitivity analysis used to assess the generalisability of the results to the assumptions made in the costing process.

Results

It was estimated that the recruitment phase of the trial would be 12 months. However, it became evident that, despite careful arrangements to optimise recruitment, rates of entry to the trial were slower than expected. This was due to a combination of factors, including inconsistency by general practitioners in actively seeking patients' consent in the surgery, and limited availability of specialist appointments. Thanks to additional funds from the trial's sponsors, the recruitment phase was extended to 18 months. The participating general practitioners referred a total of 3170 patients, who satisfied the entry criteria. Of these, 1040 (33%) failed to provide consent or actively refused to participate in the trial. The age and sex of these patients were not substantially different from those included in the trial (Table 2).

Table 2

Characteristics of those 1040 patients eligible for the trial but who did not provide consent

N = 1040	Number (%)
Site	London	797 (77%)
	Shrewsbury	243 (23%)
Specialty	Orthopaedics	174 (17%)
	Urology	194 (19%)
	ENT	272 (26%)
	Gastroenterology	226 (22%)
	Endocrinology	60 (6%)
	Rheumatology	57 (5%)
	General Medicine	36 (3%)
	Neurology	21 (2%)
Sex
	Male	508 (49%)
	Female	532(51%)
Age (years)*
	Mean (SD)	44 (21)

*age at date of referral letter (missing for 10 patients)

A further 36 patients were randomised but could not be included in the trial; the majority of these were London patients referred for a rheumatological opinion before a specialist became available to participate in the study. A total of 2094 patients were recruited and randomised, 862 in Shrewsbury, and 1232 in London. It was subsequently discovered that five of these patients (4 in London and 1 in Shrewsbury) had been erroneously allocated to the opposite arm of the trial from that indicated by the randomisation programme. These patients were nonetheless retained in the trial, as there was no evidence that misallocation had resulted from anything other than administrative error which had occurred randomly and without bias. The numbers and basic characteristics of those allocated to the intervention and control arms in the different specialty groups are shown (Table 3).

Table 3

Patients included in the trial (N = 2094)a

Number (%) /mean (SD)	Randomised group
	Tele-medicine N = 1051	Standard N = 1043	Total N = 2094
LONDON (N = 1232)
Specialty
Orthopaedics	185 (30%)	190(31%)	375 (30%)
Urology	117(19%)	113(18%)	230 (19%)
ENT	109(18%)	111 (18%)	220(18%)
Gastroenterology	117(19%)	121 (20%)	238 (19%)
Endocrinology	42 (7%)	28 (4%)	70 (6%)
Rheumatology	46 (7%)	53 (9%)	99 (8%)
Sex
Male	301 (49%)	291 (47%)	592 (48%)
Female	315(51%)	325 (53%)	640 (52%)
Age (at randomisation in years)
Mean (SD)	49 (20)	48 (20)	48 (20)
Age grouping
Adults	578 (94%)	577 (94%)	1155(94%)
Children (<16 years)	38 (6%)	39 (6%)	77 (6%)
WALES (N = 862)
Specialty
Neurology	43 (10%)	38 (9%)	81 (9%)
Urology	29 (7%)	35 (8%)	64 (8%)
ENT	186(43%)	179 (42%)	365 (42%)
Gastroenterology General Medicine	98 (22%) 79 (18%)	95 (22%) 80 (19%)	193 (23%) 159(18%)
Sex
Male	208 (48%)	217(51%)	425 (49%)
Female	227 (52%)	210(49%)	437(51%)
Age (at randomisation in years)
Mean (SD)	47(21)	48 (22)	48(21)
Age grouping
Adults	384 (88%)	385 (90%)	769 (89%)
Children (<16 years)	51 (12%)	42 (10%)	93(11%)

a excludes 36 patients randomised in error

Of the patients recruited to the trial, a total of 1,902 (91%) completed and returned the baseline questionnaire. The information on the characteristics of the patients recruited to the trial showed good balance between the treatment and control groups (Table 4). Descriptive analysis of patients in London and Shrewsbury indicated some lack of equivalence in relation to access to cars, marital status, age on leaving education and ethnic origin, confirming the expected difference between these contrasting urban and rural populations (Table 5). It was not always possible to schedule an appointment suitable for all three parties, and, as a result, 28 of the patients randomised to the tele-medicine group failed to be seen in a tele-consultation. All of the practices and hospital specialists participated in the trial until its conclusion.

Table 4

Baseline questionnaire information a (N = 1902)

	Randomised group
Number (%)	Tele-medicine N = 950	Standard N = 952	Total N = 1902b
Car availability (n = 1892)
Yes	696 (74%)	681 (72%)	1377 (73%)
No	250 (26%)	265 (28%)	515(27%)
Marital status (n = 1890)
Married / Co-habiting	600 (64%)	564 (59%)	1164(62%)
Single	172(18%)	189(20%)	361 (19%)
Divorced	97 (10%)	121 (13%)	218(11%)
Widowed	76 (8%)	71 (8%)	147 (8%)
Currently in full time education ? (n = 1888)
Yes	33 (3%)	46 (5%)	79 (4%)
No	912(97%)	897 (95%)	1809(96%)
If no, age left education (n = 1657)
Less than 16 years	405 (49%)	395 (48%)	800 (48%)
16 to 22 years	307 (37%)	313(38%)	620 (38%)
Older than 22 years	117(14%)	120 (14%)	237 (14%)
Ethnic group (n = 1889)
White	848 (90%)	835 (88%)	1683 (89%)
Black Caribbean /African / other	20 (2%)	34 (4%)	54 (3%)
Indian / Pakistani / Bangladeshi	22 (2%)	33 (3%)	55 (3%)
Chinese	5 (<1%)	4 (<1%)	9 (<1%)
Other Asian	13 (1%)	12 (1%)	25 (1%)
Other	35 (4%)	28 (3%)	63 (3%)
Employment status (n = 1895)
Student	38 (4%)	44 (5%)	82 (4%)
Full time worker	296(31%)	304 (32%)	600 (32%)
Part time worker	134 (14%)	111 (12%)	245 (13%)
Unemployed	84 (9%)	87 (9%)	171 (9%)
Retired	270 (29%)	285 (30%)	555 (29%)
Other	125 (13%)	117(12%)	242 (13%)

a For children less than 16 years of age responses to these questions are those of the parent or guardian b Denominators for each item differ due to non-completion of particular questions

Table 5

Baseline questionnaire information by study sitea (N = 1902)

	Study site
Number (%)	London N = 1102	Shrewsbury N = 800	Total N = 1902b
Car availability (n = 1892)
Yes	664 (60%)	713 (89%)	1377 (73%)
No	431 (40%)	84(11%)	515(27%)
Marital status (n = 1890)
Married / Co-habiting	574 (53%)	590 (74%)	1164(62%)
Single	269 (25%)	92 (12%)	361 (19%)
Divorced	159(14%)	59 (7%)	218(11%)
Widowed	90 (8%)	57 (7%)	147 (8%)
Currently in full time education ? (n = 1888)
Yes	52 (5%)	27 (3%)	79 (4%)
No	1043 (95%)	766 (97%)	1809(96%)
If no, age left education (n = 1657)
Less than 16 years	318(35%)	482 (66%)	800 (48%)
16 to 22 years	401 (43%)	219(30%)	620 (38%)
Older than 22 years	204 (22%)	33 (4%)	237 (14%)
Ethnic group (n = 1889)
White	892 (82%)	791 (99%)	1683 (89%)
Other	201 (18%)	5 (1%)	206(11%)
Employment status (n = 1895)
Student	54 (5%)	28 (4%)	82 (4%)
Full time worker	352 (32%)	248(31%)	600 (32%)
Part time worker	133 (12%)	112(14%)	245 (13%)
Unemployed	123(11%)	48 (6%)	171 (9%)
Retired	293 (27%)	262 (33%)	555 (29%)
Other	140 (13%)	102 (12%)	242 (13%)

a For children less than 16 years of age responses to these questions are those of the parent or guardian b Denominators for each item differ due to non-completion of particular questions

Discussion

This multi-centre trial is unparalleled in telemedicine research both in the number of patients recruited and the range of outcomes examined. It has demonstrated that it is feasible to undertake a major randomised controlled trial of telemedicine in an urban and a rural centre. The size was necessary to take account of dilution effects and to allow estimates to be made about the effect in individual specialties. The study thus has unrivalled potential to assess the impact of telemedicine at the interface between primary and secondary care. It has been suggested that telemedicine is only cost effective when the distances involved are relatively large [22] ; the inclusion of urban and rural arms of the trial, both of which achieved satisfactory levels of recruitment, should provide information to test this hypothesis.

For many patients, the condition underlying their trial consultation was not their only health problem. Such co-morbidity could have a substantial effect on the trial outcomes, although the attribution criteria developed to distinguish between outcomes related to the index consultation and the rest, were clear, consistent and reproducible. For many outcomes attribution was straightforward (for example gastrointestinal endoscopy in relation to a gastroenterology consultation). For others, particularly the generic laboratory tests (for example, full blood count, urea and electrolytes), attribution was according to the timing of the tests in relation to that of the index consultation, and this may lead to inaccuracies.

Exclusion criteria were limited to secure inclusion of as many patients as possible and to ensure standardisation of recruitment, thus avoiding difficulties in interpretation of the results. This, combined with the arrangements for randomisation, meant that the GPs had no significant control over who would be seen in tele-consultations, and we found no evidence of bias in GP recruitment. This situation was artificial, and it is probable that in actual clinical practice, GPs and specialists would wish to exert greater control over selection of patients. The trial may thus fail to reflect the likely outcome, were this technology to be implemented in a non-experimental setting, possibly underestimating the potential benefits by including unsuitable referrals. Information about criteria for defining cases most suitable for joint tele-consultation should emerge from the trial data, and the qualitative study being carried out in parallel with this study.

There were substantial difficulties in scheduling joint appointments, the staff in the project offices often having to make several phone calls followed by letters of confirmation. In any future mainstream joint tele-consultation service, there would need to be greater availability and flexibility of consultant appointments, accompanied by online booking facilities to allow easier matching of availability of the different parties.

The trial required that GPs should be present in the tele-consultations. This was predicated on the supposition that the presence of all three parties would improve outcomes for patients, and was based on evidence from a number of studies, in particular from the RCT of joint consultations carried out in the Netherlands [23]. However it seems likely that beyond the confines of the study, GPs may act as they have in real outreach clinics, limiting their contact with the hospital specialist in order to save time [24]. This would not necessarily invalidate the use of joint tele-consultation, but it might have an impact on the dynamic of the consultation, and thus on the outcomes for all involved. The qualitative study referred to earlier may shed light on this issue. Unless the trial demonstrates significant negative health outcomes and/or major increased costs associated with joint tele-consultations, there should be further studies with different participants (for example, doctor-to-doctor consultations, and consultations with the practice nurse instead of the GP, or with the patient alone) and in different settings (such as outpatient follow-up appointments).

Conclusions

We have designed and performed a major multi-centre trial of tele-consultations in two contrasting centres. Many problems were overcome to enable the trial to be carried out, with a considerable development and learning phase. A lengthier development phase might have enabled us to improve the patient selection criteria, but there is a window of opportunity for these developments, and we believe that our approach was appropriate, allowing the evaluation of the technology before its widespread implementation.

Authors' contributions to the study

PW, AH and RH were involved in developing the original idea for funding and were co-applicants on the successful funding proposal. PW was responsible for the overall direction of the project. RH co-ordinated and led the assembly and production of the proposal, and collaborated with JB in data collection and processing. ST and JB provided statistical advice, and JB was responsible for data cleaning and analysis. JR was involved in the development of the economic framework for the study, and together with PJ provided health economics advice. PJ undertook analysis of economic data. All the authors were members of the Steering Group and contributed to the drafting of the paper.

Competing interests

None declared.

Contributors

The Vitual Outreach Project Group includes

Julie Barber, Will Clayton, Rosemary Currell, Kim Fleming, Paul Gardener, Andy Haines, Robert Harrison, Paul Jacklin, Carol Jarrett, Rushmi Jayasuriya, Leo Lewis, Sophie Parker, Jenny Roberts, Simon Thompson, Paul Wainwright, Paul Wallace

Pre-publication history

The pre-publication history for this paper can be accessed here:

Additional file

Partcipating clinicians, nursing, administrative and management staff

Click here for file