How do stakeholder groups vary in a Delphi technique about primary mental health care and what factors influence their ratings?

BACKGROUND: While mental health is a core part of primary care, there are few validated quality measures and little relevant internationally published research. Consensus panel methods are a useful means of developing quality measures where evidence is sparse and/or opinions are diverse. However, little is known about the dynamics of consensus techniques and the factors that influence the judgements and ratings of panels and individual panelists.
OBJECTIVES: (1) To describe differences in panel ratings on the quality of primary mental health care services by patient, carer, professional and managerial panels within a Delphi procedure; and (2) to explore why different panels and panelists rate quality indicators of primary mental health care differently.
DESIGN: Two round postal Delphi technique and exploratory semi-structured interviews. PARTICIPANTS: 115 panelists across 11 panels. Eleven panelists were subsequently interviewed.
RESULTS: 87 of 334 indicators (26%) were rated face valid by all 11 panels. There was little disagreement within panel ratings but significant differences between panels. The GP panel rated the least number of indicators valid (n = 138, 41%) and carers the most (n = 304, 91%). The way in which panelists interpreted and conceptualised the indicators and their definition of quality of mental health care affected the way in which participants made their ratings.
CONCLUSIONS: Stakeholders in primary mental health care have diverse views of quality of care and these differences translate into how they rate quality indicators. Exploratory interviews suggest that ratings are influenced by past experience, expectations, definitions of quality of care, and perceived power relationships between stakeholders.