Towards sustainable healthcare system performance in the 21st century in high-income countries: a protocol for a systematic review of the grey literature.

INTRODUCTION: There is wide recognition that, if healthcare systems continue along current trajectories, they will become harder to sustain. Ageing populations, accelerating rates of chronic disease, increasing costs, inefficiencies, wasteful spending and low-value care pose significant challenges to healthcare system durability. Sustainable healthcare systems are important to patients, society, policy-makers, public and private funders, the healthcare workforce and researchers. To capture current thinking about improving healthcare system sustainability, we present a protocol for the systematic review of grey literature to capture the current state-of-knowledge and to compliment a review of peer-reviewed literature. METHODS AND ANALYSIS: The proposed search strategy, based on the Preferred Reporting Items for Systematic Review and Meta-Analyses guidelines, includes Google Advanced Search, snowballing techniques and targeted hand searching of websites of lead organisations such as WHO, Organisation for Economic Cooperation and Development, governments, public policy institutes, universities and non-government organisations. Documents will be selected after reviewing document summaries. Included documents will undergo full-text review. The following criteria will be used: grey literature document; English language; published January 2013-March 2018; relevant to the healthcare delivery system; the content has international or national scope in high-income countries. Documents will be assessed for quality, credibility and objectivity using validated checklists. Descriptive data elements will be extracted: identified sustainability threats, definitions of sustainability, attributes of sustainable healthcare systems, solutions for improvement and outcome measures of sustainability. Data will be analysed using novel text-mining methods to identify common concept themes and meanings. This will be triangulated with the more traditional analysis and concept theming by the researchers. ETHICS AND DISSEMINATION: No primary data will be collected, therefore ethical approval will not be sought. The results will be disseminated in peer-reviewed literature, as conference presentations and as condensed summaries for policy-makers and health system partners. PROSPERO REGISTRATION NUMBER: CRD42018103076. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This protocol provides a systematic framework for the review of grey literature including methods for document identification, selection, quality appraisal, data extraction, synthesis and interpretation, which could serve as a guide for future grey literature reviews on this topic.

The framework enables inclusion of the opinions, policy and strategy about healthcare system sustainability that would otherwise be excluded in traditional systematic reviews of peer-review literature.

Non-English language documents will be excluded and therefore our review will omit some opinions, ideas or strategies for healthcare system sustainability.

It is unlikely that our review will detect all relevant documents because grey literature searching is not supported by international databases containing systematically collected and catalogued document details.

We envisage reviewing high-income systems’ views on sustainable healthcare; a separate review may be needed for low-income and middle-income countries’ sustainability efforts.

Introduction

International organisations including WHO, Organisation for Economic Cooperation and Development (OECD) and World Economic Forum (WEF) have recently identified significant challenges to the long-term durability, performance and sustainability of healthcare systems.1–3 Ageing populations, increasing rates of chronic and complex disease, growing cost pressures from new medical technologies and medicines, wasteful spending on low-value care, inefficiencies due to system fragmentation and limited use of data and evidence to support reform have been identified as threats to health system performance and sustainability.2 4 A synthesis of knowledge about how to respond to these challenges, and which reforms are most likely to be effective in improving the sustainability of healthcare systems, is very much needed.

Objective

We propose to undertake a systematic review of the non peer-reviewed literature (the ‘grey literature’) to identify potential solutions for healthcare system resilience, longevity and performance—sustainability. Here, we provide a protocol that sets out our approach and methods.

Rationale

Our proposed review builds on and extends a recent systematic review of the sustainability literature published in peer-review journals.5 Much of the knowledge about healthcare strategy, policy and recommendations associated with creating sustainable systems of care is not published in the peer-review literature. Instead, this information mostly resides in documents published online or in print by international organisations such as WHO, OECD, WEF, national or provincial government departments, think tanks or national public policy institutes. Documents from such institutions are unlikely to be published in peer-review literature but rather manifest as opinion pieces, reports, submissions, policy briefs and policies, strategy documents and white papers. Systematically reviewing the grey literature has proven to be a useful research strategy elsewhere, to capture the current thinking about emerging, policy-relevant issues.6 7

Mahood et al 8 urged that grey literature review groups publish review protocols with sufficient detail to ‘… ensure that reviews follow explicit methodology to be systematic, transparent and reproducible’.8 This review protocol will enable us to maintain a systematic approach to the review process (document selection, quality appraisal, data extraction, analysis and conceptual synthesis) and will provide a template for other researchers interested in undertaking similar systematic reviews of the grey literature.

This review is an endeavour of the recently established Australian National Health and Medical Research Council Partnership Centre in Health System Sustainability (PCHSS), which brings together over 100 participants including leading academic groups in Australia, healthcare system partners and consumer partners who work together to improve healthcare system sustainability in Australia. The review will set out the current international concepts for addressing and supporting healthcare system sustainability. There are diverse views on whether, and the extent to which, health systems are sustainable. One view is that as economic progress continues, consumers will choose to allocate their resources to more healthcare; if the proportion of GDP allocation to healthcare consequently grows, then health systems will concomitantly grow.9 Another perspective is that all countries have a health system and they are expected to do so in the future, and therefore, by definition, health systems are sustainable.10 This does not of course say whether these health systems are good or bad, effective or ineffective, value for money or not and so on. This perspective simply says that each county will have a health system of sorts. Yet another view is that health systems are embedded in inequitable societies, and healthcare, becoming increasingly more expensive, will be sustainable for wealthy groups and not for poorer populations.11 Thus, health systems will become partially sustainable. Any of these alternative views of the future are possible and there are many others. Our view is that we need more information to understand these different perspectives.

Research questions

The overarching question we wish to answer is: According to international and national thought leaders, and considered opinions and analyses, what does a sustainable healthcare delivery system look like and how can we move towards healthcare system sustainability?

We also aim to answer the following specific questions:

Can healthcare system sustainability be defined and measured, and if so, how?

What do strategy and policy documents from jurisdictions, regions and health systems, and those published by reputable and authoritative national and international bodies tell us about the attributes of sustainable healthcare systems?

Are healthcare systems performing in a resilient and sustainable way?

What solutions have been proposed in the grey literature to maintain sustainability of healthcare systems?

What are the most important factors that contribute to successful solutions for improving healthcare system performance in sustainable ways?

Which solutions have been evaluated and shown to be effective, credible and applicable to the international context (ie, spreadable across contexts and scalable)?

What are the identified gaps in knowledge and next steps to create more sustainable systems?

Methods and analysis

Search strategy and information sources

Google Advanced Search is helpful in identifying grey literature and is often used; however, search results can be incomplete and searching is laborious.12 ‘Hand searching’ that targets specific organisations’ websites has been shown to be more efficient in identifying relevant target documents, and is an invaluable adjunct to Google Advanced Search.12 To ensure comprehensiveness of the search strategy, we will use both search methods for documents published between 1 Jan 2013 and 31 March 2018 inclusive. This time period was chosen to capture the current ‘state-of-the-evidence’6 on health system sustainability and current thinking. Key search terms were identified during a PubMed search of the peer-reviewed literature and following consultation with a university librarian with search strategy and database expertise (table 1).13 For the hand search of the grey literature we will target websites of organisations and governments including the websites of international health authorities and institutes, national and provincial Departments of Health, in high-income countries classified as category I by the OECD.14 The leading international and national organisations targeted include among others, WHO, OECD, WEF, the Commonwealth Fund, the King’s Fund, universities and institutes with expertise in healthcare system policy, economics and health services research (box 1). The documents will be stratified by their geographical scope: international and national.

Table 1

Google Advanced Search strategies and key phrases used to search websites of target organisations

Search strategy	Exact word or phrase		Any of these words
1	Health system	AND	Sustainable(ility) OR resilience(ent)
2	Health system performance	AND	Sustainable(ility) OR resilience(ent)
3	Health system improvement	AND	Sustainable(ility) OR resilience(ent)

In addition to the targeted hand searching, searches will be undertaken using Google Advanced Search according to the search terms outlined in table 1. A snowballing strategy will be used, where the reference sections of publications pertaining to sustainability of healthcare systems will be searched to identify additional documents. Experts in the field will be also consulted for recommended grey literature publications for potential inclusion.

Online searches will be conducted by targeting sources and organisations likely to have published on healthcare systems sustainability and performance including:

International organisations concerned with health and healthcare systems, such as WHO, the King’s Fund (the UK), the Commonwealth Fund (the USA), Organisation for Economic Cooperation and Development (OECD), the World Economic Forum, the International Society for Quality in Healthcare, the European Observatory of Health Systems and Policies, the World Innovation Summit for Health, the Institute of Global Health Innovation, the National Academy of Medicine (formerly the Institute of Medicine).

Government websites (OECD category 1, high-income countries).

Corporate organisations who consult on healthcare systems and health economics including Deloitte, PricewaterhouseCoopers, KPMG, McKinseys Group, Ernst and Young.

Research institutes operating at national or international level including but not limited to: London School of Economics Health and Social Care, Health Systems Evidence, McMaster University, Wiser Healthcare, National Choosing Wisely programmes.

Scope and context of documents to be included

This systematic review focuses on the healthcare system defined as the institutions, facilities and actors involved in delivering healthcare services.3 In this context, the following sectors are included as a component of the healthcare system (whether publicly or privately funded):

Primary care (general practice, community health centres and clinics);

Secondary, tertiary and quaternary hospitals;

Rehabilitation services and facilities;

Aged care sector;

Mental healthcare sector.

In this review, we acknowledge that the sustainability of the healthcare system may be improved in many different ways and we have taken a broad and inclusive view. Documents will be included that discuss any of the following concepts which align with the OECD strategies for reducing waste and increasing sustainability,1–4 or any other concepts that support healthcare system sustainability:

Reducing clinical waste:

Introduce robust information systems to identify low-value care, inefficiencies, waste;

Implement reporting systems of adverse events (transparency, learning from mistakes, safety);

Implement evidence-based behaviour change campaigns;

Develop and implement clinical guidelines;

Monitor unwarranted variation in care delivery;

Provide financial incentives, for example, withholding payments for ‘never-do’ events; introduce technology assessment programmes;

Link payment systems to value-based outcomes rather than volume of care provided.

Reducing hospital operational waste (increase efficiency):

Unnecessary hospital visits;

Inefficient hospital processes;

Longer than necessary hospital stays.

Reducing operational waste—use of pharmaceuticals (increase efficiency):

Supporting patients to get the right information about effective medicines;

Changing payment incentives, eg, reward for prescribing generics;

Improving procurement systems—establish hospital consortia to purchase drugs—bulk discounts from pharmaceutical companies;

Integrating and coordinating care delivery to reduce waste through fragmentation.

Governance-related waste and administrative costs (increase efficiency):

Single payer systems;

Paper-less prescriptions;

Quality improvement programmes.

Fraud, abuse, corruption and integrity violations:

Transparency in private health sector and its business practices;

Regulation, eg, banning certain procedure;

Financial disincentives for inappropriate practices.

Appropriately skilled and appropriately deployed workforce working to full capacity of license.

Eligibility criteria

The selected documents will be limited to book chapters, reports, policy statements, government policy documents and submissions published online or in print in English or translatable into English. Opinion pieces published in peer-review literature will also be included. The publication period is limited to January 2013–March 2018.

Documents will be deemed relevant to the main area of interest if they are specifically related to healthcare system performance sustainability including threats, challenges and drivers of sustainability; frameworks or policy responses for improved sustainability; formulation, implementation and evaluation of interventions for improving sustainability. Documents will be excluded if their primary focus is on the diagnosis or management of outcomes relating to a specific disease; population health prevention initiatives (eg, immunisation programmes, smoking cessation programmes, etc); sustainability/performance of disaster management or emergency preparedness; foreign aid, foreign investment or environmental sustainability. Documents that do not contain substantial content relating to healthcare system performance sustainability or associated interventions will be excluded. Documents related to health systems in low-income countries or conflict zones, or aspects of health system reform which are exclusive to a particular national political situation such as Brexit in the UK or the Affordable Care Act in the USA will also be excluded. Documents will be stratified according to scope. International documents are those that cover more than one nation, usually from international organisations such as the OECD or WHO, while national documents are those that have national scope in any of the OECD high-income countries.

Selection process

We will use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines as an overarching framework for screening and selection of documents. A PRISMA flow chart detailing document identification, screening and inclusion will be produced.15

The first 10 pages (100 Google results list) retrieved from Google Advanced Search will be assessed to determine eligibility for inclusion by screening titles by two independent reviewers. Similarly, the titles of documents from targeted hand searching will also be assessed for relevance. Search results from Google Advanced Search and the targeted hand searching will be combined and duplicate documents will be excluded.

The research team will meet to discuss the inclusion and exclusion criteria to ensure a common understanding and interpretation of the criteria. A 10% sample of all documents will be independently screened by three blinded reviewers according to the inclusion and exclusion criteria (boxes 2 and 3). The reviewers will review the executive summaries, abstracts or the first two pages of text in each document. We chose three reviewers to ensure consistency and credibility to the review process because grey literature documents are generally more difficult to assess and require a higher level of interpretation and value judgements than peer-reviewed literature where results are neatly encapsulated in an abstract. The level of agreement among the three reviewers will be evaluated using Cohen’s κ analysis. Discrepancies will be discussed among the three reviewers until a consensus is reached. If agreement cannot be reached, additional team members will be consulted.

English language or translatable into English.

Published January 2013–March 2018.

Grey literature: any document not usually published as a peer-reviewed article, which contains mainly expert opinion, knowledge synthesis or recommendations, including:

Position statements, white papers, submissions.

Policies and policy briefs.

Annual reports.

Guidelines and recommendations.

Theses/dissertations.

Book chapters.

Opinion pieces/essays published in peer-reviewed journals.

Relevant to the healthcare system: the healthcare system describes the institutions, facilities and actors involved in delivering healthcare services. In our review, the following are included as part of the healthcare system (whether publicly or privately funded):

Primary care (general practice, community health centres and clinics).

Secondary, tertiary and quaternary hospitals (all models including hospital in the home).

Rehabilitation services and facilities.

Aged care sector.

Mental healthcare sector.

Relevant to the review topic: includes information about health system sustainability including at least one of these topics:

Healthcare system performance sustainability.

Financing and efficiency.

Healthcare system threats or challenges.

Indicators and/or measures of healthcare system sustainability.

Solutions for sustainability and resilient performance of healthcare systems.

Relevant healthcare systems in high-income countries as defined by the Organisation for Economic Cooperation and Development (OECD).

Authoritative: published by a reputable organisation that has authority, eg:

Global organisations (WHO, OECD, Commonwealth Fund).

Governments/government departments.

Non-government organisations.

Research institutes and universities.

Corporate entities such as KPMG, McKinsey, Deloitte, etc.

Public health groups, eg, Public Health Association of Australia.

Doctors’ groups (eg, medical colleges).

Advocacy groups including consumer groups.

Charitable foundations or published by known experts.

Credible: opinions presented in the document have basis in evidence, eg:

The document has been endorsed by other credible and authoritative organisations.

The document draws on credible evidence (eg, published papers, reports, policies, cases for change, etc).

A reference list or bibliography of credible sources is provided.

Foreign language publication not translatable to English.

Published before January 2013, or after March 2018.

Peer-reviewed literature reporting results of studies; presentations or lectures; media press releases, newsletters, newspaper articles, blogs.

Not relevant to the healthcare system context as defined in this protocol; outside of the scope of the healthcare delivery system (eg, population prevention interventions).

Not relevant to the review topics, eg, focused on specific diseases or populations, pertains to foreign aid, foreign investment, environmental sustainability, disaster management or preparedness.

Not relevant to high-income countries as defined by Organisation for Economic Cooperation and Development category 1 countries, (eg, focused on low-income and middle-income countries).

Not authoritative—published by individuals or organisations not recognised as experts or authorities in healthcare system sustainability or reform.

Not credible, with evidence of bias through selective use of evidence, omission of evidence or misrepresentation of evidence sources or lack of use of any evidence.

Quality appraisal and risk of bias

After investigating available tools and guidelines for the critical appraisal of grey literature, we found two widely used tools: the Johanna Briggs Institute Checklist and the Accuracy, Authority, Coverage, Objectivity, Date, Significance (AACODS) tool.16 17 In addition to assessing authority and credibility as described in box 1, we will also assess objectivity during the full-text review, to identify systematic bias according to a modified AACODS definition. We will assess objectivity by:

Conducting a qualitative assessment of the logical construction of the opinion or argument according to cited literature, data, case descriptions and other expert opinion.

Assessing whether a balanced view is presented and supported by literature, data or other expert opinion.

Detecting bias due to selective use of evidence that supports the opinion or argument presented, while ignoring non-supporting evidence.

Checking whether the authors acknowledge limitations and knowledge gaps of the sources used to support their formulation of opinions, arguments or recommendations.

Data management

Lists of document titles, authors, details of place and year of publication, relevant organisation that published the documents and the document URL will be stored in an Excel spreadsheet. This spreadsheet will also be used to manage the document selection process, by including eligibility criteria as additional columns; the final decisions for inclusion or rejection and each of the three reviewers’ decision will also be entered here.

When assessing the 10% sample of documents, a separate spreadsheet will be used by each of the three reviewers independently. The final selection choices for each reviewer will be provided to a statistician for calculation of the level of agreement and the κ score.

The final set of included documents will be entered into EndNote, the reference management software, for ease of referencing future manuscripts and other publications.

Data collection process and data items

After screening of document summaries, a full-text review of all included documents will be undertaken. The reviewers will meet prior to undertaking the full-text review to discuss the definitions and scope of data elements to be extracted to ensure a common understanding. The reviewers also will meet on a regular basis during full-text review phase to discuss any difficulties in interpretation.

A data extraction form has been developed in Microsoft Excel (Office365) to systematically extract data elements from the selected documents, about the following topics:

Definitions of sustainability and evidence for operationalising the definitions.

Description of the identified sustainability problem including but not limited to:

Fragmentation, complexity, limited information sharing; absence of interdisciplinary approaches;

Practice variation, low-value care, fraud;

Funding failures, perverse incentives, market failures, affordability;

Consumer expectations/demands and consumer health literacy.

Attributes of a sustainable healthcare systems, whether already implemented or recommended.

Description of proposed solutions/initiatives, programmes or reforms for healthcare system sustainability.

Healthcare setting where the change for sustainability was, is or will be implemented (primary care, community care, acute care, across sectors (and which ones), etc).

Description of the outcomes and measures or benchmarks of healthcare system sustainability (whether tested or intended for testing).

Description of important factors identified as integral to the sustainability of solutions/initiatives according to Scheirer18 including:

Ability to modify the solution during and after implementation to meet specific needs;

Affordability of the solution (for patients, governments, facilities);

Demonstrated likely benefits—positive evaluation outcomes (for patients, governments, facilities);

Strong champion(s) who support(s) the solutions/initiatives;

Strong existing capacity to implement the change (eg, availability of appropriately skilled workforce; facilities; developed or developing relationships) or strategies are in place to build capacity;

Support from other organisations or using programmes already established elsewhere;

Alignment with current priorities of governments, institutions, facilities;

Funding arrangements and funding stability to support the initiative over time;

Other important factors.

Evidence of cost-benefit or efficiency.

Identified gaps in knowledge and next steps.

It is unlikely that grey literature documents will contain specific outcomes or quantitative data that can be extracted and analysed. Instead, we will extract descriptive summary data on each of the above items.

Data synthesis

The authors will undertake qualitative thematic analysis of extracted data elements to identify common concepts. Documents will also be analysed using text data-mining techniques, via Leximancer V.4.5—an automated content analysis software program.19 Leximancer was chosen as the data-mining tool because the program does not simply perform a raw count of words or phrases but produces a detailed and meaningful list and concept map of the significant and inter-related concepts and themes.20 The concept map will highlight the unknown collective perceptions from multiple information sources that may not have been recognised by the review team a priori. Associations among the different strata of sources will allow examination of the alignment or divergence of concepts according to the information source. Concepts will be grouped and the inter-relationships and linkages among the concepts will be identified using the concept maps generated by the text data-mining software. The text data-mining software will provide a broad overview of the concepts and themes while the researcher-conducted content analysis will provide more nuanced understanding of the concepts and meanings. We will triangulate between the concepts identified using text data-mining and the concepts identified through summative content analysis undertaken by the researchers. Any new concepts identified by the text data-mining software will be tested against the summative content analysis to see whether the concepts hold true. Specific terms that are meaningful and relevant to the research questions, and which were identified through the traditional content analysis by the researchers, may then be inputted into the text data-mining software to determine their frequency and connections to other identified concepts.

The different strata of documents—international and national—will be analysed to identify the main concepts in each stratum, and to determine how the concepts identified align or diverge across the different strata. Reasons for divergent concepts will be identified using extracted data.

The interpretation of results will be supplemented and strengthened by consultation with known national and international experts in health system sustainability. The PCHSS includes within its governance structure a Scientific Advisory Forum and an International Advisory Forum, in addition to lead investigators, all of whom are recognised experts.

Strength of evidence

Levels of evidence are difficult to assess for grey literature documents. We will assess individual documents for authority, credibility and risk of bias using the AACODS tool and the Joanna Briggs checklist as described above. We will conduct a qualitative assessment of these factors for the final set of documents included in the review to provide an overall synthesis of the quality of opinions presented.

Dissemination

The completed review will be published in an open-access peer-reviewed journal and the results will be presented at relevant international conferences. The results will also be directly disseminated to the 20 investigators and 40 health system partners involved in the PCHSS and will provide an understanding of current knowledge to support further research in health system sustainability. Links to the final results paper will be published on the PCHSS website, the Australian Institute of Health Innovation website and the websites of all PCHSS partners including the Consumer Health Forum of Australia (CHF). Brief, summary resources targeted to system partners and to consumers will be developed and made available via these websites.

Patient and public involvement

The CHF is a partner in the PCHSS and representatives from the CHF are aware of the proposed grey literature review and this protocol. We plan to liaise with representatives of the CHF when the review is complete to produce a resource on healthcare system sustainability for health consumers.

Conclusion

Maintaining a durable, resilient and sustainable healthcare system to withstand impending and ongoing challenges while providing effective and efficient healthcare that is safe and of high quality is a significant challenge to governments, health services, funders (private and public), policy-makers, healthcare providers and health consumers. This review will describe the attributes of sustainable healthcare systems, and the outcomes and measures used to determine their successful performance. By analysing the concepts and interventions proposed, developed or implemented internationally, we will inform potential interventions for healthcare system reform and modelling of highly performing healthcare delivery systems. This protocol provides a framework structure for the identification, selection, quality appraisal, data extraction and analysis of key sources of grey literature in healthcare system sustainability. The framework may inform future similar reviews undertaken by us or other researchers, enabling comparisons of changes over time.