| Literature DB >> 24384896 |
Lisa Douet1, Ruairidh Milne, Sydney Anstee, Fay Habens, Amanda Young, David Wright.
Abstract
OBJECTIVES: The objective of this study was to assess whether National Institute of Health Research (NIHR) Health Technology Assessment (HTA)-funded randomised controlled trials (RCTs) published in the HTA journal were described in sufficient detail to replicate in practice.Entities:
Keywords: AUDIT; STATISTICS & RESEARCH METHODS
Mesh:
Year: 2014 PMID: 24384896 PMCID: PMC3902303 DOI: 10.1136/bmjopen-2013-003713
Source DB: PubMed Journal: BMJ Open ISSN: 2044-6055 Impact factor: 2.692
Replicability criteria for interventions, developed from an initial design by Schroter et al3
| Checklist criteria | Descriptor of criteria where appropriate |
|---|---|
| 1. Setting | |
| Is it clear where the intervention was delivered? | |
| 2a. Recipient—inclusion | |
| Is it clear who is receiving the intervention?—inclusion criteria | Clear inclusion criteria in the journal |
| 2b. Recipient—exclusion | |
| Is it clear who is receiving the intervention?—exclusion criteria | Clear exclusion criteria in journal |
| 2c. Recipient—baseline characteristics | |
| Do you know all that you need to about the patients? (eg, which drugs they are taking, what they were told, etc)? | Baseline characteristics of participants provided in journal |
| 3. Provider | |
| Is it clear who delivered the intervention? | |
| 4. Procedure | |
| Is the procedure (including the sequencing of the technique) of the intervention sufficiently clear to allow replication? | |
| 5. Intensity | |
| Is the dose/duration of individual sessions of the intervention clear? | |
| 6. Schedule | |
| Is the schedule (interval, frequency, duration or timing) of the intervention clear? | Frequency of intervention, length of session |
| 7a. Materials—physical | |
| Are the physical materials used adequately described? | |
| 7b. Materials—informational | |
| Are the informational materials used adequately described? | |
| 8. Missing | |
| Is the description of the intervention complete? | |
| 9. Control | |
| Is it clear what the control group received during the study? | |
Intervention type of NIHR HTA-funded trial RCTs included in the study
| Type of intervention | N (%) |
|---|---|
| Drug | 15 (15.3) |
| Radiotherapy | 1 (1.0) |
| Surgery | 9 (9.2) |
| Diagnostic | 8 (8.2) |
| Education and training | 3 (3.1) |
| Service delivery | 19 (19.4) |
| Psychological therapies | 11 (11.2) |
| Vaccines and biologicals | 3 (3.1) |
| Devices | 12 (12.2) |
| Physical therapies | 7 (7.1) |
| Exercise | 1 (1.0) |
| Complementary therapies | 2 (2.0) |
| Mixed or complex | 6 (6.1) |
| Other* | 1 (1.0) |
| Total | 98 |
*Other refers to an intervention using larval therapy.
HTA, Health Technology Assessment; RCT, randomised controlled trial.
Examples of poor reporting of intervention elements within the HTA journal series, taken verbatim form the journal
| Checklist item | Examples of poor reporting | Reason why rated as incomplete | Examples of good reporting | Reason why rated complete |
|---|---|---|---|---|
| Inclusion criteria | ‘…patient identification was retrospective. Searches were conducted on practice databases using either repeat prescriptions alone or repeat prescriptions plus diagnostic terms… GPs then sent letters to suitable patients, providing information about the trial’ | No details given about the searches and the criteria patients were screened with | Inclusion criteria for trial patients were: | Very detailed patient criteria listed |
| Exclusion criteria | ‘GPs were given a ringbinder file with information and instructions about the trial and, within each, a number of recruitment packs. The packs contained the paperwork required to complete the recruitment of each patient, this was: a reminder of the inclusion/exclusion criteria for the study….’ | No details given about the exclusion criteria | Reasons for exclusion (yes/no) | Detailed patient exclusion criteria listed |
| Provider | ‘All services had staff who were trained and experienced in family therapy, but not necessarily family interventions specifically for eating disorders’ | No details about the staff providing the interventions or the training they received | Eight counsellors (six females and two males) took part in the trial (one worked at two practices) and all were BACP accredited or eligible for BACP | States who delivered the intervention and their training |
| Procedure | ‘Generally home-based rehabilitation services provide, as a minimum, physiotherapy and occupational therapy in the patient's own home. Services can be specialised (eg, in stroke rehabilitation) or be provided for patients with a range of disabilities’ | No details about the services provided to patients and variation between centres | The content of the CBT programme included (complete course description contained within an appendix): | Key aspects of the intervention summarised in the text and a full description of the intervention is detailed in the appendices |
| Intensity and schedule | ‘Patients come to the day hospital where the rehabilitation service is provided for a full or half day. Usually ambulance transport is provided to bring patients into the service and return them home after a session’ | No details of the length or number of sessions | Psychological treatment was based on existing protocols (references included) and distributed over six 50-minute sessions, with printed information sheets provided after each session | The length and number of sessions is included as well as the details of each session |
| Materials—physical | ‘The acupuncture point prescriptions used were individualised to each patient and were at the discretion of the acupuncturist’ | The prescriptions used are not detailed | ▸ 500 mg oral oxytetracycline (non-proprietary) twice daily+topical vehicle control twice daily | Each of the treatments prescribed is clearly defined |
BACP, British Association for Counselling & Psychotherapy; BP, blood pressure; CBT, cognitive behavioural therapy; ESR, erythrocyte sedimentation rate; GP, general practitioner; HTA, Health Technology Assessment; VAS, visual analogue scale.
Completion rates of NIHR HTA reports by intervention type
| Description criteria | Drugs Number of trials with complete intervention description (%) | 95% CI | Non-drugs Number of trials with complete intervention description (%) | 95% CI | Psychological Number of trials with complete intervention description (%) | 95% CI | All Number of trials with complete intervention description (%) |
|---|---|---|---|---|---|---|---|
| Number journals | 15 (15.3) | 72 (73.5) | 11 (11.2) | 98 (100) | |||
| Setting | 15 (100) | 0.80 to 1 | 66 (91.7) | 0.83 to 0.96 | 9 (81.8) | 0.52 to 0.95 | 90 (91.8) |
| Inclusion criteria | 15 (100) | 0.80 to 1 | 63 (87.5) | 0.78 to 0.93 | 10 (90.9) | 0.62 to 0.98 | 88 (89.8) |
| Exclusion criteria | 14 (93.3) | 0.70 to 0.99 | 55 (76.4) | 0.65 to 0.85 | 10 (90.9) | 0.62 to 0.98 | 79 (80.6) |
| Baseline characteristics | 14 (93.3) | 0.70 to 0.99 | 68 (94.4) | 0.87 to 0.98 | 11 (100) | 0.74 to 1 | 93 (94.9) |
| Provider | 11 (73.3) | 0.48 to 0.89 | 56 (77.8) | 0.67 to 0.86 | 10 (90.9) | 0.62 to 0.98 | 77 (78.6) |
| Procedure | 14 (93.3) | 0.70 to 0.99 | 57 (79.2) | 0.68 to 0.87 | 9 (81.8) | 0.52 to 0.95 | 80 (81.6) |
| Intensity | 13 (86.7) | 0.62 to 0.96 | 63 (87.5) | 0.78 to 0.93 | 9 (81.8) | 0.52 to 0.95 | 85 (86.8) |
| Schedule | 13 (86.7) | 0.62 to 0.96 | 59 (81.9) | 0.71 to 0.89 | 9 (81.8) | 0.52 to 0.95 | 81 (82.7) |
| Patient information | 9 (60.0) | 0.36 to 0.80 | 42 (58.3) | 0.47 to 0.69 | 6 (54.5) | 0.28 to 0.79 | 57 (58.2) |
| Physical materials | 10 (66.7) | 0.42 to 0.85 | 52 (72.3) | 0.61 to 0.81 | 6 (54.5) | 0.28 to 0.79 | 68 (69.4) |
| Intervention description complete overall | 5 (33.3) | 0.15 to 0.58 | 22 (30.6) | 0.21 to 0.42 | 3 (27.3) | 0.10 to 0.57 | 30 (30.6) |
Some criteria are not applicable, therefore denominator less than total number of journals. 95% CI: no continuity correction.
HTA, Health Technology Assessment.