User-generated quality standards for youth mental health in primary care: a participatory research design using mixed methods.

OBJECTIVES: To develop user-generated quality standards for young people with mental health problems in primary care using a participatory research model.
METHODS: 50 young people aged 16-25 from community settings and primary care participated in focus groups and interviews about their views and experiences of seeking help for mental health problems in primary care, cofacilitated by young service users and repeated to ensure respondent validation. A second group of young people also aged 16-25 who had sought help for any mental health problem from primary care or secondary care within the last 5 years were trained as focus groups cofacilitators (n=12) developed the quality standards from the qualitative data and participated in four nominal groups (n=28).
RESULTS: 46 quality standards were developed and ranked by young service users. Agreement was defined as 100% of scores within a two-point region. Group consensus existed for 16 quality standards representing the following aspects of primary care: better advertising and information (three); improved competence through mental health training and skill mix within the practice (two); alternatives to medication (three); improved referral protocol (three); and specific questions and reassurances (five). Alternatives to medication and specific questions and reassurances are aspects of quality which have not been previously reported.
CONCLUSIONS: We have demonstrated the feasibility of using participatory research methods in order to develop user-generated quality standards. The development of patient-generated quality standards may offer a more formal method of incorporating the views of service users into quality improvement initiatives. This method can be adapted for generating quality standards applicable to other patient groups. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

For well over a decade, the National Institute for Health and Care Excellence (NICE) has developed a series of quality standards for the National Health Service to adopt in the four countries in the UK. These are based upon the best available evidence for three dimensions of quality: clinical effectiveness, patient safety and patient experience. NICE has encouraged the involvement of service users and carers in their quality standard development groups as well as including clinicians, managers and public health specialists. Patient and carer groups are also involved through an intensive consultation process. NICE quality standards relating specifically to young people cover depression in children and young people and focus on assessment and monitoring.1 Elsewhere, quality standards have been generated for primary care management of mental health problems in adults and children but have not involved young people in their development.2 3

Common mental health problems are relatively prevalent in young people aged 16–25.4 However, young people consult their general practitioners (GPs) infrequently5 even when their mental health problems are severe.6 When they do consult, they are unlikely to have their mental health problems identified.7 Young people report concerns about confidentiality, lack of respect for their health concerns and shortness of consultations in primary care8 and often associate their GPs only with physical health problems.9 GPs also report difficulties when managing this group of patients citing resource issues and training needs.8 Some GPs are uncomfortable making a diagnosis of depression in young people10 and are reluctant to discuss psychological problems with young people even when they perceive them to be present.11 Quality standards which address these problem areas may enable practitioners to improve the way services are delivered.

Features of quality care for youth-friendly health services developed by the Royal College of General Practitioners, the Royal College of Paediatrics and Child Health and the Department of Health include training, ease of access and ensuring confidentiality.12–14 But, the extent to which these quality standards are the responsibility of primary care is ill-defined. In addition, they lack focus on mental health problems or the involvement of young people other than inviting consumer organisations to comment on draft reports. Developing quality standards for primary care mental health that reflect the needs of young people is important because they represent a hard to reach group, whose use of primary care may be influenced by early experiences.15–17

Incorporating the patient experience into quality assessment of healthcare18 is a central tenet of government policy but it is difficult to assess how well this happens in practice. Patients and health professionals differ on what they consider to be important features of high quality care.19 Patients may not speak openly in development groups nor have their concerns acted upon.20 Young people have additional factors affecting their experience of healthcare. They may lack the confidence to challenge an adult in a position of authority such as a health professional who comes wrapped with all the traditional cultural and historical trappings of ‘expertise’. These issues are even more pronounced for different groups in society. The needs of young people with mental health problems tend to be assessed and defined for them by others—specifically adult health professionals—and as a result they tend to have little influence in the design and delivery of services.21

This study adopted a participatory approach to quality standard development to minimise the impact of different stakeholder groups working together. We involved young service users as focus group cofacilitators, in data analysis, project meetings and as an expert panel consisting entirely of young service users to enhance data collection and gain more indepth information.22 This is a novel approach to quality standard development as patients are usually represented in the minority and not the majority. This study aimed to produce young user-generated quality standards for primary care mental healthcare.

Methods

Study design

Decisions about what to research and how data are collected and analysed are not usually made by patients but by funders, clinicians and researchers. Treating research as a shared exercise between researcher and patient shifts the balance of power so that it becomes more equal.23 In a participatory research project this is done through reflexivity, disclosure and involving patients as coresearchers. This model of participatory research is often referred to as ‘cooperative enquiry’ between coresearchers with different perspectives resulting in new interpretations.24 In this study, young service users (coresearchers) worked on research tasks alongside the research team which included GP academics (AT, MA, HL), primary care researchers (JM) and service user researchers (TG, DR). Shared experiences of help seeking were disclosed among the research team, coresearchers and participants in an effort to reduce the differences in status. This encouraged the coresearchers to conduct their research tasks and incorporate their experiences into data collection and analysis.

This study was conducted in five stages (outlined in figure 1) adopting focus groups, interviews, participatory research groups and nominal group technique. Focus groups and nominal groups were adopted to allow young service users to work in partnership with other coresearchers. The guidelines adopted during focus and nominal group work which stress the importance of respect for one another, as well as confidentiality among the group, served to build trust among everyone in the group. Young service users were able to ask questions, write cofacilitator notes, contribute to debriefing meetings after the focus groups and comment on the themes developed from the analysis. Contributing to data generation and analysis enabled young service users to be meaningfully involved in the development of the quality standards. The nominal groups also complemented the participatory research design as they enabled participants to interact with one another. Sharing reasons for allocating a particular ranking encouraged young service users to share their views in a safe environment with people who had similar experiences. Being a member of the nominal group empowered young services users to make meaningful decisions about the results of the study.

Figure 1

Flow chart of the four phases in the development of the quality standards.

Study participants and setting

Young people aged 16–25 from community settings and primary care participated in focus groups and interviews. Data collection took place at sixth form colleges (educational institutions where students aged 16–19 typically study for advanced level qualifications), a university, a drug and alcohol drop-in centre, a hostel and a research institute. A second group of young people (referred to as young service users) also aged 16–25 who had sought help for any mental health problem from primary care or secondary care within the last 5 years were trained as focus groups cofacilitators, developed the quality standards and participated in nominal groups. Young service users were recruited from primary care, child and adolescent mental health services, service user groups and student counselling groups. The study was conducted across four south London Primary Care Trusts, the local unit for the management of primary care and community services (Southwark; Lambeth; Lewisham and Croydon), which were superseded by Clinical Commissioning Groups on 1 April 2013.

Phase I—from young service user to coresearcher: recruitment and training

Young service users were recruited to take part in the coresearcher training. They were aged 16–25 and had sought help for a mental health problem from a GP or mental health professional within the last 5 years. Letters were sent to all 16–25 year olds who had a diagnosis of depression or anxiety from two GP practices. Posters and fliers were also placed at three child and adolescent mental health services, a student counselling service, a homeless shelter and a supported housing project inviting 16–25 year olds to contact TG by text message, email or phone. The snowballing technique was also used to recruit coresearchers. Coresearchers were trained by TG, DR and AT for their role as focus group cofacilitators and paid for their time (box 2).

Phase II—developing the quality standards: focus groups with young people in the community

Three pilot focus groups and three interviews with students and young people from a local homeless shelter and a local Muslim women's group were conducted to develop the topic guide, confirm that feedback to participants was feasible, and inform sampling and recruitment strategies. As most of the evidence on young people's views about the role of primary care and seeking help for mental health problems is based on students, we decided to widen the scope and include the views of more vulnerable young people such as those who were unemployed, at risk of drug abuse (type of drug not specified) and homeless.25 26 Sampling was also based upon ethnicity27 28 and gender5 6 which are known to affect help seeking for mental health problems. A majority of the sample were either in education or unemployed, so we also included employed young people to ensure a wide range of views and make comparisons. Some young people worked as administrators, one was an advisor for a mental health support charity and another was a researcher for an international health charity. During the recruitment stage, TG met with each group of participants to discuss the topic area and ensure they were comfortable discussing their views and experiences of seeking help for a mental health problem. TG discussed her own experiences of counselling during University and spoke about how it was important to conduct more research in this area. Each focus group was facilitated by TG (an experienced female qualitative researcher), cofacilitated by a coresearcher, and lasted between 45 and 60 min (see table 1 for demographic details). In the hostel and drop-in clinic group, the key worker was present during the focus groups. All but one focus group spoke openly about their views and experience of seeking help for mental health problems. The Asian female group felt it was a sensitive topic and so individual interviews were conducted with those focus group participants who agreed to take part. Preliminary interpretations of the data were fed back to the participants during a second focus group to ensure respondent validation and develop the coding frame (see box 1).31 34 All but two participants from the first set of focus groups participated in the second set of focus groups. Feeding back results through second focus groups enabled detailed further discussion of the topic resulting in theoretical saturation.35

Table 1

Focus group demographics

Group/demographic	Sixth form college	Sixth form college	Asian females	Employed	Students	Hostel residents	African-Caribbean males
Males to females	All female	All male	All female	1:5	1:7	4:4	All male
Age range	16–17	16–17	16–17	21–23	19–25	19–25	16–19
Self-reported ethnicities	Black- CaribbeanBlack- AfricanMixedCaucasian	Mixed Black- CaribbeanIndianBlack- CaribbeanIrish	PakistaniBurmeseGuyaneseIndianIndian-Pakistani	CaucasianBlack- AfricanBlack- Caribbean	Caucasian Bangladeshi	Black- AfricanBlack- CaribbeanSpanishKosovan	MixedBlack-AfricanBlack-Caribbean
Occupations	All studying for A levels	All studying for A levels	All studying for A levels	All working full/part time	All studying for degrees	TrainingWorkingLooking for work	TrainingWorkingLooking for workOther
Focus group setting	Sixth form college	Sixth form college	Sixth form college	Research institute	University	Hostel	Drop-in service
Number of participants	7	7	8	6	8	8	6
Percentage with self-reported mental health problems	43%	28.5%	57%	50%	62.5%	75%	50%

All focus groups were recorded, transcribed verbatim, edited by TG and supplemented by field notes. An initial coding framework was developed from the topic guide, key themes from the literature and the pilot work.29 30 The framework was validated by participants’ agreement with the preliminary interpretations.31 32 The coding frame was extended by reapplying it to all the transcripts through an iterative process of data abstraction. Software (Max QDA) was used to index and retrieve data. Data contained in each code were continually scrutinised to ensure consistency. Transcripts were coded independently by TG, two coresearchers, a GP academic (HL) and other researchers (JM and DR). Disagreements and alternative interpretations of the data were discussed and the coding frame was modified.33

Phase III—developing the quality standards: participatory research groups with coresearchers

From the coding framework developed in the previous phase, codes considered most relevant to primary care were identified by two coresearchers, three GPs on the steering group (including HL) and TG. Participatory research groups facilitated by TG, AT, DR and JM were held with coresearchers to develop quality standards. The coresearchers read through all of the data represented by the codes and summarised the data into key words and phrases. These were then grouped together by similarity and refined into one or two sentences by the coresearchers and facilitators (see example in online supplementary figure S2). A pilot nominal group of 11 young service users (who were not involved in generating the quality standards from the focus group data) were asked to read through each standard and make amendments to avoid duplication and ensure clarity. We also wanted to test the feasibility of using nominal group technique with young service users.36 The pilot nominal group individually ranked each statement in order of importance on a 5-point scale (1—very important, 2—important, 3—no opinion, 4—not important, 5—not at all important). Importance was defined as ‘the extent to which the standard is necessary when suffering from emotional distress’. We chose nominal group technique rather than the Delphi process so that consensus could be reached as a result of group interaction and not individuals working in isolation. Encouraging group interaction was a vital component of the participatory methodology employed for involving young service users in the research process. It was also important to allow the young service users to ask questions of the facilitator as most had not been involved in such a process before.

Phase IV—Developing the quality standards: reaching consensus among young service users

The quality standards were presented by TG to a second expert panel of 12 young service users. As with the pilot nominal group, the group individually ranked each statement in order of importance on a 5-point scale (1—very important, 2—important, 3—no opinion, 4—not important, 5—not at all important). Importance was defined as ‘the extent to which the standard is necessary when suffering from emotional distress’. Individual rankings from the first round were tabulated, anonymised and presented back to the group. Members extensively discussed reasons for their rankings. DR, AT and JM observed the group and kept a record of what was said to ensure completeness of data. The group was then asked to rerank each statement. The results were presented back to the youth expert panel who all agreed that the final set of quality standards accurately represented their preferences.

There were a number of ethical issues related to involving young service users as researchers that we needed to consider.37 Asking young people to cofacilitate focus groups and interpret data represented for the most part a new role in which they had little or no experience. We provided clear information about what the roles entailed and how they would be supported through their involvement (appropriate cofacilitator training, feeding back data and research results, and organising data analysis meetings). We carefully judged how much new information they could take in, and incorporated methods of group work that would keep them interested including ice breaker and group cohesion exercises. The impact of taking part in research tasks on the emotional well-being of the young service users was closely monitored by TG and the research team. TG held debriefing sessions after each focus group to discuss what the participants had talked about and how it affected the coresearchers.

Results

Phases I and II: recruiting young services users and collecting qualitative data

Twelve young service users took part in the focus group cofacilitator training sessions. Seven focus groups were conducted with 50 young people both with and without mental health problems each cofacilitated by young service users. Three interviews with Asian women were conducted to supplement the focus group data. Thematic analysis of the focus group and interview data analysis resulted in 17 codes and 49 subcodes representing a range of topics including confidentiality, counselling, features of an ideal service, reasons for and for not seeking help, other places for help, self-help, gender differences, medication, stigma, and experiencing stress in comparison with having a mental health problem (see online supplementary table S2).

Phase III: participatory research groups

From the coding framework developed in phase II, nine codes and three subcodes considered most relevant to primary care were identified (see online supplementary table S2). Two participatory research groups were held with five coresearchers. The participatory research groups yielded a total of 49 quality standards from the focus group data (detailed in online supplementary table S3). The quality standards represent the following four areas of primary care: quality standards 1–6 refer to issues of practice organisation and skill mix; quality standards 7–18 represent broader treatment options and better referral protocols; quality standards 19–36 refer specifically to the consultation process, and include putting young people at ease, confidentiality and suggestions for improving interaction with young people; and quality standards 37–49 illustrate features of an ideal service such as improving accessibility and environmental preferences of a primary care practice. Of the 49 quality standards reviewed by the pilot nominal group, 27 were included, amendments were made to 19 and three were removed due to duplication. The amendments resulted in quality standards that were more succinct, better aligned to the terms used by young service users and with an appropriate emphasis as dictated by the young service users. The nominal group technique enabled young service users to work together with one another and the research team sharing experiences and discussing rankings within the group illustrating that it was a feasible method to use with this group. The pilot nominal group process resulted in 46 quality standards to be presented to the final nominal group in phase IV.

Phase IV: Final nominal groups

The nominal group scores of the 12 young service users converged between rounds 1 and 2 in relation to each quality standard illustrating the importance of group interaction and reranking.36 Agreement on which standards should constitute the final quality standards was established by including only those standards where all the individual rankings were either 1 or 2 (very important or important). These rankings were calculated from the round 2 scores to incorporate the impact of group interaction. Sixteen standards met these criteria for agreement (figure 3). The standards apply to both practice and patient levels of service representing the following aspects of primary care: better advertising and information about local services and confidentiality policies; mental health training provision; access to different treatment options; improved referral protocol; and specific questions and reassurances during a consultation (see table 2).

Figure 2

Example of how codes were developed into quality standards for ranking by the Youth Expert Panel.

Table 2

The final quality standards*

Quality standard	PrL	PL	O	A
1. Primary care practitioners should advertise that they are available to help young people with mental health difficulties (1)	x		x
2. Young people should be able to see a mental health specialist at the GP surgery (3)	x			x
3. Primary care practitioners must have up-to-date information about other local services to give to young people (4)	x		x
4. Primary care practitioners should have up-to-date information about local specialist mental health services (5)	x		x
5. All primary care practitioners should have mental health training (6)	x		x
6. Primary care practitioners should discuss other treatment options before prescribing medication to a young person (8)		x	x
7. Access to talking therapies needs to be made easier as an alternative to medication for mental health problems (10)	x		x
8. Referral times for young people should be as short as possible to reduce the stress experienced while waiting for an appointment (13)		x	x
9. For referrals within the GP surgery, primary care practitioners should keep the young person up to date with the progress of the appointment (16)		x	x
10. If making a referral, primary care practitioners should provide information for young people about what to expect at the first appointment (14)		x	x
11. Primary care practitioners should discuss ways that you can help yourself overcome the symptoms of mental health problems (20)		x		x
12. Counsellors need to be more interactive with young people rather than just listen and write notes (19)		x	x
13. Primary care practitioners should have a strict confidentiality policy, which they state at the beginning of the consultation preventing them from disclosing information to members of the young person's family (25)		x	x
14. Primary care practitioners should appreciate that young people can feel embarrassed to seek help and should reassure them that mental health problems are common (23)		x		x
15. Primary care practitioners should ask questions about young persons’ relationships and support network (27)		x		x
16. An ideal place for help with mental health problems would offer a variety of treatments for young people (38)	x		x

*This is the final set of quality standards—the number in parentheses refers to the original number in online supplementary table S3.

A, amended O, original; PL, patient level; PrL, practice level.

Figure 3

Number of standards grouped by level of agreement. Agreement was based upon all 12 individuals ranking the standard as either ‘very important’ or ‘important’.

Discussion

Sixteen quality standards for primary care mental health were developed using nominal group technique with 28 young service users. To the best of our knowledge, no other quality standards have been generated using participatory research methods with young service users as the sole stakeholder group. This novel approach to quality standard development led to different aspects of quality care for youth mental health in primary care being generated that have not been previously reported. The inclusive sampling strategy enabled the engagement of young people from a wide range of socio-economic groups. Young people were also recruited through GP practices and a range of other services so that those who did not access help from primary care were also included. These hard to reach patients are often not included in other studies.

Focus on treatment options and communication during consultation

The quality standards are similar to primary care mental health quality standards in the areas of training,2 3 ensuring confidentiality and referral protocol.3 NICE quality standards also state that practitioners should be competent in delivering appropriate interventions,39 that ‘systems are in place to confirm a diagnosis’ and that information about treatment options is provided.1 Keeping skills up to date, advertising services and managing confidentiality also feature in quality standards for general youth healthcare developed by the Department of Health,12 the Royal College of General Practitioners13 and the Royal College of Paediatrics and Child Health.14

The user-generated quality standards differ from existing quality standards for young people in the areas of treatment options (quality standards 5 and 7) and communication during consultation (quality standards 11, 12, 14 and 15). Quality standards 5 and 7 call for greater consideration to be given to talking therapies as a form of treatment compared with medication. Despite the relatively new Improved Access to Psychological Therapies (IAPT) programme, access to talking therapies for young people remains restricted for two reasons. First, IAPT for under 18 year olds is limited to existing Child and Adolescent Mental Health services and is not routinely available within primary care. Second, NICE quality standard for depression in people aged 18 and over states that access to low-intensity psychological interventions should be limited to people with subthreshold, mild or moderate depression.39 This limits access to high-intensity psychological therapies and other talking therapies for young people (aged 18–24) with symptoms that do not meet the thresholds defined by NICE guidance.

Both sets of quality standards for primary care mental health refer to the importance of good communication3 and an ‘awareness of stigmatising feelings’ during consultation.2 These are quite general including ‘GPs should communicate effectively with both parents and children about the child's emotional and behavioural health’.3 The user-generated quality standards provide more specific questions and reassurances that encompass good quality care for young people with mental health problems, for example, ‘Primary care practitioners should ask questions about a young person's relationships and support network.’

Ensuring quality assessment is meaningful to patients

Patient reported outcome measures (PROMs) and patient reported experience measures are increasingly being used to assess quality of primary care. However, it has been noted that PROMs used in the IAPT outcomes framework reflect professionals’ concerns and lack those of users who prefer to focus on recovery rather than severity of mental health problems.40 Other frequently used measures such as the Global Assessment of Functioning and General Health Questionnaire have also been criticised for not adequately capturing the experiences of mental health service users.41 In addition, there is little evidence to suggest that surveys about young people's experience of mental health services have an impact upon service development.21 This may be because practitioners find it difficult to translate survey results into a change in their practice or the way a service is delivered. User-generated quality standards may be a more effective way of incorporating patient experience into quality assessment and service development.

Involving patients as interviewers and in questionnaire development is a core feature of ‘user research’ in the mental health field42 43 leading to different questions being asked that may be more closely aligned to the needs and experiences of people receiving services.44 For this study, in addition to different questions being asked, new quality standards were developed that focus on aspects of care not previously reported, notably, treatment options and communication during consultation.

Limitations

Although the use of nominal group technique ensures the user-generated quality standards have face validity, there are a number of other criteria which have not been assessed. Campbell et al argue that quality standards should be ‘tested for acceptability, feasibility, reliability, sensitivity to change and (content) validity’.38 Acceptability has been assessed by nominal group agreement of the results and focus groups and interviews with practitioners (to be published). The latter provided an opportunity to examine the issues that may arise during implementation. The remaining criteria would need to be assessed through piloting the user-generated quality standards for determining their validity when applied to service evaluation in routine practice.

Adopting participatory research methodology to develop quality standards

There were a number of challenges that we negotiated throughout this project associated with participatory research methodology. The recruitment process was lengthy and involved meeting with each of the 28 young service users individually to discuss the project. Ensuring that the young service users remained engaged and were fully supported required significant researcher time. It was important to not overload young service users while respecting their roles as coresearchers. This process was managed by seeking verbal feedback on the experience of being a coresearcher throughout the study.

Managing expectations about what was possible to achieve was another important ethical issue. We reminded the young service users that there were other barriers that may affect the impact of the quality standards such as the GPs’ knowledge of mental health problems and resources to provide other treatment options. Involving young service users in the research process required the research team members to examine their own roles and negotiate shared decision-making with young service users. This was not always easy to achieve and required the authors to be self-aware and critical of the obstacles affecting their ability to share power and control over the research with the young service users.45

Incorporating user-generated quality standards into practice

The next phase would be to incorporate the piloted quality standards with existing toolkits to aid practices in improving the quality of care provided to young people with mental health problems.46 This could be achieved by a working group of young patients and practitioners from a number of practices followed up by an evaluation. Such a programme of work could be embedded within existing networks such as the Royal College of General Practitioners’ Adolescent Health Group with learning events across practices through quality improvement collaboratives. A collaborative approach between patients and practitioners is pivotal as practitioners are generally supportive of those quality standards that they feel are useful and that they have some influence over.47 In this study, GP academics were involved in data analysis and facilitated the participatory research and nominal group meetings.

Conclusions

The results illustrate that exploring young service users’ definitions of quality in primary care mental health leads to new interpretations as well as similarities with previously published standards. This builds upon our knowledge of what encompasses an optimal primary care service and can inform patient-centred practice. It is vitally important to fully involve young service users in the development of quality improvement initiatives that assess their experiences and needs. This study ultimately raises the question whether young service user views can have an impact on policy and commissioning decisions and practice and, if so, how that can occur. We have illustrated that through the use of a participatory research design, young service users’ views can be meaningfully incorporated into quality indicator development and can thereafter form part of the evidence base on which policy is based.