Questioning the ethics of evidence-based practice for Indigenous health and social settings in Australia.

Australian government planning promotes evidence-based action as the overarching goal to achieving health equality for Aboriginal and Torres Strait Islander populations. However, an inequitable distribution of power and resources in the conduct of evidence-based practice produces a policy environment counterintuitive to this goal. This context of contemporary evidence-based practice gives legitimacy to 'expert practitioners' located in Australian governments and universities to use Western guidelines and tools, embedded in Western methodology, to make 'evidence' informed policy and programming decisions about Aboriginal and Torres Strait Islander populations. This method for decision making assumes a positional superiority that can marginalise the important perspectives, experiences and knowledge of Aboriginal Community Controlled Organisations and their processes for decision making. Here we consider the four steps of an evidence review: (1) developing a review question; (2) acquiring studies; (3) appraising the evidence and (4) assessing the evidence, as components of wider evidence-based practice. We discuss some of the limitations across each step that arise from the broader context within which the evidence review is produced. We propose that an ethical and just approach to evidence-based review can be achieved through a well-resourced Aboriginal community controlled sector, where Aboriginal organisations generate their own evidence and evidence is reviewed using methods and tools that privilege Aboriginal and Torres Strait Islander ways of knowing, doing and being. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Globally, evidence-based practice is increasingly applied by policymakers in governments and universities to make evidence-based decisions for Indigenous populations.

Indigenous scholars and collectives have raised concern with current evidence-based practice—bringing attention to the ethics and values of practice, as well as the epistemological injustices of applying Western informed methodologies in cross-cultural settings.

In Aboriginal and Torres Strait Islander health and social settings in Australia, evidence-based review practice as currently practised is not culturally appropriate nor equitable. The broader environmental and organisational context of evidence-based practice decision making is of particular concern.

Equitable, culturally appropriate and methodologically robust approaches to evidence-based practice require Aboriginal and Torres Strait Islander organisations to be resourced to lead evidence reviews and evidence-based decision making. They also require practice methodologies and tool for reviewing evidence to be value informed and inclusive to Aboriginal and Torres Strait Islander peoples’ worldviews and ways of knowing, doing and being.

We advocate for greater resourcing and participation by Aboriginal Community Controlled Organisations in evidence-based review and evidence-based policy decision making.

Introduction

Aboriginal and Torres Strait Islander people are the first peoples Indigenous to Australia. They are a culturally intra-diverse group of people from over 250 nations, who share many intertwined recent histories and socialised experiences.1 Under colonialism for over two centuries, Australian governments have made many ongoing attempts to manage, police and govern Aboriginal and Torres Strait Islander peoples’ lives and territories, including through official policies of protectionism, biological absorption and cultural assimilation.2 3 While Australian universities, as sites of racialised knowledges, have provided ‘evidence’ and the logics to government policy.4 However, despite ongoing attempts to control Aboriginal and Torres Strait Islander peoples’ lives, many Aboriginal and Torres Strait Islander individuals and organisations continue their long history of enacting sovereignty, actively resisting and asserting their rights, including the right to self-determination.2 3

In recent decades, Australian governments have turned their attention to the promotion and application of ‘evidence-based’ administration, policy, programmes and services.5 In Aboriginal and Torres Strait Islander health and social policy planning settings, evidence-based action has been identified by the Commonwealth government as the overarching goal to achieve health equality and improved life expectancy outcomes.5 However, despite this strong endorsement, Aboriginal and Torres Strait Islander researchers and service delivery organisations, including the Victorian Aboriginal Child Care Agency (VACCA), have expressed real concerns that contemporary Western evidence-based practice, as currently practised, has the potential to marginalise Aboriginal and Torres Strait Islander ways of knowing, doing and being.6–9 In particular, they are concerned that contemporary Western evidence-based practice places too strong a value on Western methods and scientific rigour of research evidence, with little to no consideration of a more inclusive and self-determining research methodology and ethics of practice, which ensure research evidence is culturally ‘safe, respectful, responsible, high quality, of benefit to Aboriginal and Torres Strait Islander people and communities and of benefit to research’.10 Further, there are concerns about the ethics of evidence-based practice where limited attention has been afforded to the social constructionist nature of evidence production and evidence review.6–9 11

Pluralities of worldview and methodologies for producing evidence

Both research and evaluation as ‘evidence’ and evidence-based practice are conducted from within a cognitive frame, where human decision making is imperative to research methodology in how theories are placed, why methods are selected and how evidence is developed or reviewed.12 13 As such, evidence methodologies are biased to the worldview of the humans who create them, with their cultural and social backgrounds, political views, values, experiences and social positions shaping their way of knowing, doing and being.11 12 14 15 For Aboriginal and Torres Strait Islander peoples, their unique histories, viewpoints and traditions have produced ways of knowing, doing and being that are different from that of non-Aboriginal and Torres Strait Islander peoples, who are members of the dominant, colonising culture.11 15 Recognising these pluralities of worldviews is crucial in the neo-colonial context of Australia, where predominately Western experts, methods and frameworks for producing evidence and reviewing evidence are privileged and resourced, and Aboriginal and Torres Strait Islander peoples’ ways of knowing, doing and being have been continually devalued and dismissed by the dominant culture.

For decades, Indigenous scholars globally have increasingly positioned and advocated for Indigenous methodologies as better ‘ways of doing research’. By Indigenous methodologies, they refer to methodologies applied by Indigenous people and collectives to produce research, where Indigenous worldviews inform the methods, theoretical approaches and underpinning.14 16 Examples of Indigenous methodologies include anti-colonial,17 Indigenous standpoint,18 decolonising,19 Indigenist,20 Indigenous quantitative21 and data sovereignty methodologies.22 23

For Indigenous people and collectives (in Australia and globally), there is a diversity of worldviews which informs Indigenous peoples’ methodologies for generating and reviewing evidence. Indigenous methodologies, although diverse, can share some commonalities that arise from populations sharing like sociocultural–political–historical contexts. These methodologies for generating evidence foreground Indigenous knowledges, however, are not exclusionary to Western methods or approaches.14 15 Table 1 presents characteristics of ‘evidence’ generated from known Indigenous methodologies14 16–22 and positions these against the traditional positivist Western methodology that has been used for generating evidence for over a century. These pluralities of methodologies and the weighting afforded to Western methodologies are poorly recognised in contemporary evidence-based practice.6 12

Table 1

Characteristics of evidence produced from Indigenous methodologies compared with traditional Western methodologies used by dominant cultures

Towards Indigenous methodology	Towards a dominant Western methodology
Knowledge is produced through an anti-colonial/decolonising /sovereign/self-determining lens (emancipatory)	Knowledge is produced towards a colonial lens (surveillance and management)
Interrogates systems of power	Maintains systems of power
Knowledge is from lived experience, based on realities of people who experience them	Knowledge is produced from observation, measured and quantified
Knowledge is subjective, and based on values and opinions of those who create it	Knowledge is objective presented as a single truth
Considers knowledge to be socially produced, including as a means of oppression or emancipation	Considers knowledge is presumed to be/presented as neutral and impartial
Knowledge is collective, all are knowers	Qualified Individuals (researchers) are knowers
Narrative (storying, yarning) methods are highly legitimate ways to produce knowledge	Knowledge is obtained by method, with quantifiable methods considered as highly legitimate ways of producing knowledge
Challenges dominant discourses, traditions, beliefs or assumptions of understanding	Relies on dominant discourses, traditions, beliefs or assumptions as a starting point for understanding, does not make these explicit
Self-representation can capture Indigenous excellence, strength and survival of both people and culture and issues of importance to communities including racism and colonisation	Outsider representation may focus on the deficits and problems of Indigenous people and culture
Considers contexts specific to Indigenous people	Social–cultural–medical–political–historical experiences are de-contextualised and Western experiences normalised
Evidence is political it is generated for Indigenous advancement and change	Evidence has the purpose of solving or responding to identified problem/s and can be profited from
Oriented to Indigenous ways of understanding (holistic)	Oriented towards Western understandings (disease/problems)

In this paper, we examine the evidence review process as a component of contemporary evidence-based practice. Describing contemporary methods for evidence-based review including those of the Australian government, we draw attention to the limitations and biases of applying these to Aboriginal and Torres Strait Islander health and social settings. We then propose recommendations for how the evidence-based review process can be made more equitable and of benefit to Aboriginal and Torres Strait Islander people and organisations.

What is evidence-based practice?

Evidence-based practice is a reductionist and structured method for research translation (or knowledge exchange) that has its origins in evidence-based medicine. Developed in the 1990s by clinical epidemiologists, it aims to promote better clinical decision making through the systematic appraisal of clinical research by decision makers.24 In the clinical setting, it is associated with improved patient outcomes, increased safety, less variation in patient outcomes and reduction of healthcare costs.25

By virtue of its systematic method, evidence-based practice is proposed to minimise clinician bias that comes from cognitive decision making.26 However, contemporary frameworks, such as those by Satterfield et al (2009), illustrate that human decision making lies at the core of evidence-based practice (figure 1). This framework places decision making within a wider environmental and organisational context where ‘best quality evidence’, ‘population values and preferences’ and ‘practitioner experts’ are all contributing equally to decisions.27 In this paper, we focus on the ‘environmental and organisational context’ and the determination of ‘best quality evidence’ in the Aboriginal and Torres Strait Islander health and social settings.

Figure 1

Framework for evidence-based practice. Adapted from Satterfield et al (2009).27

Evidence-based practice in Aboriginal and Torres Strait Islander health and social settings

In the last two decades, evidence-based practice has become transdisciplinary, with application beyond clinical medicine, to the public health and social service sectors and increasingly to Aboriginal and Torres Strait Islander settings.5 28

Prima facie, the implementation of programmes with known success is appealing in Aboriginal and Torres Strait Islander health and social settings. Evidence-based decision making presents an appealing counternarrative to reactive ‘policy on the run’ that has epitomised decades long ideological driven government-led decision making. Through its iterative and stepped method, evidence-based practice provides a method with transparency around decision making processes and programming. With its focus on prescriptive implementation, planning and manualised delivery, evidence-based practice offers the promise of dependable results and rigorous method, something desirable in a polity marked by a desire to improve health and social outcomes. There is also the supposed safety provided in implementing interventions that have demonstrated no prior harmful effects in context. Yet, despite these benefits there are also real concern regarding its current application in Aboriginal and Torres Strait Islander health and social settings.

All evidence-based decision making occurs within an ‘environmental and organisation context’(figure 1).27 In the context of Australia, it is dominant governments and universities that are resourced and powered to produce research and evaluation evidence and review these, not Aboriginal Community Controlled Organisations (ACCOs). ACCOs, like the VACCA, are organisations initiated and owned by local Aboriginal community that deliver social programmes (legal, health, family supports, early child education) by and for local communities. ACCOs emerged from the 1970s as part of the ‘self-determination’ movement and operate under a model of community control where there is full ‘use of all talents within the community to come to consensus …after feeling together, perceiving together and thinking together’.29 ACCOs are premised on collective local decision making and notion that local Aboriginal and Torres Strait Islander people living in a community are the real experts with solutions to the issues impacting them.3 30 In the philosophies of both ‘self-determination’ and ‘community control’, it is the collective perspectives, experiences and knowledge of Aboriginal and Torres Strait Islander people that are legitimised and at the forefront of decision making.3 30

In contemporary evidence-based practice, ACCOs are not at the forefront of decision making, it is Western trained ‘practitioner experts’ in government and universities who systematically locate and appraise Western research and evaluation using Western guidelines and tools, and present these as ‘best quality evidence’.31 32 Where Aboriginal and Torres Strait Islander organisational ‘values and preferences’ are sought, prevailing mechanisms for participation take the form of consultation, engagement and partnership, where ACCOs most often lack real power in decision making. This regrettably can marginalise the important perspectives, experiences and knowledge of ACCOs.

Guiding methods for contemporary evidence-based practice in Australia

Methods for evidence-based practice broadly describe the stages of reviewing evidence, implementing evidence and evaluating evidence-based decision making (figure 2). In this paper, we focus on the four steps of the evidence review stage that are intended to select the ‘best quality evidence’. These four steps include (1) developing questions; (2) acquiring and searching for evidence; (3) synthesising the best available evidence through appraisal of the quality, relevance and strength of evidence and (4) applying the evidence with consideration of evidence for other populations and settings.31–34

Figure 2

The steps of the evidence-based practice cycle.

In Australia, National Health and Medical Research Council (NHMRC) documents describe the Australian government’s preferred methods for evidence review.33–35 While the Cochrane Handbook for Systematic Reviews, Critical Appraisal Skills Programme (CASP) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses are ‘global’ guidelines and critical appraisal tools used in the Australian setting. Only in recent years have tools specific to Indigenous settings in Australia and globally been developed to strengthen reporting in research36 and appraisal of evidence.6 9

Constructing an evidence review in the Aboriginal and Torres Strait Islander health and social settings

In recognising the constructionist nature of evidence production as well as evidence review, we describe some of the limitations of practice across each of the four steps of evidence review of: (1) identifying questions, (2) selecting methods to source evidence, (3) selecting methods and tools to appraise evidence and (4) assessing evidence for its applicability as it relates to Aboriginal and Torres Strait Islander health and social settings.

Step 1—developing the evidence review question

Developing the questions of an evidence review requires human framing and formulation of the question/s. Who is legitimised as a ‘practitioner expert’ matters, as does their social position within the ‘environmental and organisational context’, their proximity to the ‘population’ and their authority to speak on behalf of the ‘population’. In Australia, Western-qualified evidence-based ‘practitioner experts’ located in dominant governments and universities have traditionally been powered and resourced to ask evidence review questions. These ‘practitioner experts’ are predominately the beneficiaries of colonialism who can be socially, geographically and culturally removed from the populations they claim to know.12 37 In Australia, social proximity to Aboriginal and Torres Strait Islander people among non-Aboriginal and Torres Strait Islander people is low, with a cross-sectional survey by Walter et al revealing only 9% of non-Aboriginal and Torres Strait Islander people have daily contact with an Aboriginal and Torres Strait Islander person, and fewer in cities (5%) where most decision making takes place. Without knowing Aboriginal and Torres Strait Islander people, this can lead to ‘expert practitioners’ relying on dominant discourses, traditions, beliefs or assumptions as a basis of knowing where past stereotypes may form how a population is known in lieu of actual experience.37 This imbalance of power, that affords greater authority to Western experts and Western ways of knowing, doing and being are examples what Bhakuni and Abimbola refer to as epistemic injustice of practice.38

The NHMRC states that a good evidence-based review question has three components—‘population or problem of interest’, a study factor such as an ‘intervention’, and ‘outcome/s’ to be measured.33 34 But each of these constituent parts forming a question can differ based on the cultural positioning of the questioner, exposing the review to potential cultural/racial bias.12 In current contexts, those representing an Australian government or university are unlikely to ask the same questions as an ACCO. For example, ACCOs have strong understandings of who their service community is; they talk, live and work with Aboriginal and Torres Strait Islander people daily. Conversely, members of the dominant culture do not always have such insider knowledge.

In the past, ‘expert practitioners’ have tended to formulate questions using a deficit discourse that has problematised Aboriginal and Torres Strait Islander individuals and communities.21 These questions have centred and normalised Western social, cultural, medical and political realities and histories, meaning that social and cultural phenomena that are important and real for Aboriginal and Torres Strait Islander people are often not addressed in questions.15 Such questions may focus on individualist and assimilationist outcomes promoting equality over holistic, collectivist and justice outcomes identified as important by ACCOs.26

As identified by Larkin, there is an epistemological racism of practice where Western ‘expert practitioners’ are powered to ask questions of a colonised population.12 If the intent of an evidence review is to support interventions in an ACCO setting, then ACCOs and the communities they represent must be leading the question formation and identifying how the population, intervention and outcome are framed and designed. This is not only ethical but scientifically rigorous, as attending to cultural/racism bias that comes from ‘outsiders’ framing the lives of others is paramount to the internal validity (and external validity) of the review.

Step 2—acquiring relevant studies

NHMRC guidelines for evidence review describe the acquisition of evidence as searching for published and unpublished ‘grey’ literature.33 For published literature, these guidelines preference the retrieval of existing systematic reviews and then published primary studies as sources of evidence, through MEDLINE, and for interventions suggest using the Cochrane library.33 However, in Aboriginal and Torres Strait Islander settings, these recommendations may be limiting, as much of the innovation of the Aboriginal community controlled sector is less likely to be published or retrievable through database search as ACCOs are not often resourced to publish. There is also a longstanding racism within academia, where Australian universities have excluded Aboriginal and Torres Strait Islander people from tertiary education and where academic publishing and editorial gatekeeping processes have denied Indigenous people a platform.11 This reflects an important publishing bias. As we consider published research as evidence, research in Australia and the global context has a longstanding and continuing history of Western researchers and institutions as ‘experts’ producing outsider ‘scientific’ knowledge that has subjugated, racialised and objectified Indigenous peoples and cultures.4 In Australia, there is a history of research being conducted on Aboriginal and Torres Strait Islander people, primarily for the benefit of dominant Australians, including without consent.39 The work of Larkin speaks widely to the accuracy and relevancy of evidence that has been done on Aboriginal and Torres Strait Islander communities rather than by or with.12 ‘Outsider’ research in the Western tradition and methodology is unlikely to capture with authenticity and authority Aboriginal and Torres Strait Islander peoples’ life experiences, social and cultural environments, and aspirations. Although there are increasing examples of research interventions and programmes initiated and published by Aboriginal and Torres Strait Islander communities, such research represents a very small proportion of the total produced. This is an inequity of practice that needs redress.

To minimise publication bias, the NHMRC guidelines highlight the importance of acquiring grey literature from clinical trial registries, funding bodies or tenders for research and evaluation. Here, Western knowledges recorded and reported using scientific methods are given weight, with acquisition of other forms of evidence not even considered. These search methods may be limiting in retrieval of reports and presentations produced by ACCOs or evidence in the forms of knowledge from Elders and Aboriginal and Torres Strait Islander people that are oral and not written. Guidelines further suggest writing to ‘experts’ which they detail as principal investigators of relevant studies.33 However, in Aboriginal and Torres Strait Islander health and social settings, ACCOs, Aboriginal and Torres Strait Islander peak organisations and research bodies (ie, the National Aboriginal Community Controlled Health Organisation or Lowitja Research Institute) are ‘experts’ and should be recognised as such. As we consider the acquisition of grey literature, acquisition methods must be broadened so that the full breadth of evidence is weighted for and retrievable.

In Australia, published and grey literature is kept in repositories including the Australian Indigenous Healthinfonet and the Closing the Gap Clearinghouse held by the Australian Institute of Family Studies that collects, analyses and synthesises evaluation evidence. But these are not complete, with a study by Luke et al, revealing that government tendered evaluation relating to the Aboriginal and Torres Strait Islander health and social settings, rather than being left on the shelf, never makes it to the shelf. Of 390 evaluations tendered from 2007 to 2017 by Australian governments, they found only 25% were in the public domain and retrievable through database and repository search.40

The standard systematic methods for acquiring published and grey evidence in Aboriginal and Torres Strait Islander settings are unlikely to retrieve the full breadth of relevant knowledges.

Step 3—appraising and selecting the quality, quantity and level of studies

Appraising evidence relates to the assessment or grading of evidence for its level, quality and quantity. Level relates to the study design, quality relates to how bias (ie, information, selection and cofounding) is mitigated for on basis of study design and method, and quantity relates to the statistical power that comes from large sample sizes and fewer outcome measures. Australian guidelines33–35 for appraising evidence like global methods focus on the rigour of the research study design and methods.41 42

The NHMRC method for evidence-based practice includes a clear hierarchy for grading the quality of ‘evidence’ from ‘most biased’ to ‘leased biased’. Consistent with other global grading methods, the NHMRC prioritises the systematic review of randomised controlled trials and then randomised controlled trials (RCTs) as high level ‘gold standard’ sources of knowledge.41 42 For the RCT, this classification is assigned a high-level on account of a robust scientific method and study design that includes larger samples, statistical control and a precision that comes from double blind administration of an intervention.26

There are numerous broad critiques regarding the hierarchisation of ‘evidence’ using standard evidence-appraisal tools in Aboriginal and Torres Strait Islander health and social settings. The first is that few programmes, policies or services developed by the ACCO sector have undergone rigorous evaluation or research to demonstrate effectiveness. We also know that despite Aboriginal and Torres Strait Islander communities reporting a burden of being over-researched, an overwhelming majority of research produced in Australia has been descriptive rather than experimental research. Although there has been a steady increase in RCTs since 2000 relating to the Aboriginal and Torres Strait Islander health and social settings, most have been led by the university sector with varying degrees of ACCO participation and engagement. We found only 35 RCTs conducted in the Aboriginal and Torres Strait Islander health and social settings involving Aboriginal and Torres Strait Islander populations for the period 2000–2021. These RCTs all related to specific health areas and for ACCOs such as VACCA, whose work is focused on Aboriginal and Torres Strait Islander child and family health and social welfare, these RCTs are not relevant. Although we highlight there are limited RCTs, we are mindful that the solution is unlikely to be more RCTs, rather we propose the need for greater investment in generation of an Aboriginal and Torres Strait Islander evidence-base and genuine partnering with academia. Further, an RCT is not always practical or desirable in an ACCO setting as they require considerable financial and human resources as well as large sample sizes. For example, Marley et al in discussing the challenges from an RCT of a complex smoking intervention in the Aboriginal and Torres Strait Islander health setting, describe on the ground challenges of recruiting participants, the significant investment in training required to support staff delivering the intervention, consistent work to garner organisational support and compatibility with usual care as barriers.43

Further, RCTs are unlikely to be the best research design to address questions about complex human health and social interventions, particularly those that involve psychosocial, lifestyle or educational interventions. In contrast to clinical interventions, where biological effects are more likely to be consistent across humanity, health and social interventions are more likely subject to cultural and demographic variation.44 Greenhalgh highlights the problematic nature of RCTs which are often devoid of context, writing ‘(what makes them) boring—and sometimes unimplementable in practice—is that the technical process of stripping away all but the bare bones of a focused experimental question removes what practitioners and policymakers most need to engage with: the messy context in which people get ill, seek healthcare (or not), receive and take treatment (or not) and change their behaviour (or not)’.45 For colonised and marginalised populations, it the ‘messy’ contexts including the historical, political and sociocultural that are imperative to decision making.

Dominant tools that appraise evidence, such as the Cochrane Handbook for Systematic Reviews and CASP and the NHMRC guidleines, favour research rigour in method and design, without attention to grading the broader methodology including the research frame and ethic of evidence.6 7 Although these grading systems are crucially responsive to the biases that come from the study design and method (ie, selection bias, information bias and confounding), bias cannot be fully captured in this information alone. This focus on method overlooks the importance of context.46 In Aboriginal and Torres Strait Islander contexts there are ‘cultural’ bias or ‘racial’ bias that relate to how the studies are conducted and implemented that need to be attended to.47 These biases can arise when ‘evidence’ is grounded in a methodology, epistemology and ontology that does not relate best to the study population.48 A review by Lock et al of 11 evidence-based practice tools revealed that only the Joanna Briggs Institute Checklist for Qualitative Research asessed for cultural bias.7

Recognising the limitations of standardised tools and guidelines, several Indigenous specific evidence assessment tools both in Australia and globally have been produced to assist in the appraisal of evidence for its methodolgoical rigour.6 9 49 50 These tools prioritise and rank highly evidence that is grounded in an Indigenous ethic and Indigenous ways of knowing, doing and being. Tools developed in Australia include a critical appraisal tool by MacLean et al and the ‘Aboriginal and Torres Strait Islander Quality Appraisal Tool’ by Harfield et al.6 9 While Internationally we are aware of appraisal tools by Morton Ninomiya et al in Canada for assessing research involving First Nations populations, as well as reviews methods used by Rolleston in Aotearoa (New Zealand) that priveledge research which embeds a Kaupapa Māori methodology or philosophical framework.49 50 But to use these tools, the reporting of the ethical and methodological domains measured in these appraisal tools needs to be strengthened. The ‘CONSolIDated critERtia for strengthening the reporting of health research involving Indigenous Peoples the (CONSIDER) statement’ by Huria et al is a global tool that has been developed for this purpose.36 It specifies that research manuscripts should report—governance, relationships, prioritisation, methodologies, participation, capacity, analysis and findings and dissemination to allow better appraisal.36

Currently, there are no specific criteria for reporting of research methodology and ethic in Australia that would assist in the appraising of evidence relating to Aboriginal and Torres Strait Islander populations. At a minimum, all research evidence produced in Australia should report on levels of participation for Aboriginal and Torres Strait Islander people to enable appropriate assessment of applicability, transferability and bias in the generation of evidence. Research evidence based on programmes specifically for Aboriginal and Torres Strait Islander people should report on adherence to ethical principles for working with Aboriginal and Torres Strait Islander people.10 With this information, evidence can be assessed for its cultural safety, using tools such as those developed by MacLean et al and Harfield et al.6 9

Step 4—assessing the applicability of evidence across different populations and settings

Assessing evidence for applicability or generalisability relates to whether an intervention of known benefit is likely to have the same benefits outside of its study context, beyond the original population and setting.34

Without local evidence, Australian governments are reliant on commissioning programmes that have been developed in a North American or other global context. In the past, VACCA has been offered opportunity to put in tender applications to receive substantial funding to implement such programmes. This is a concern as these programmes may not be implementable into an ACCO setting, that is removed from the US child welfare context. Further, programmes may not be transferable across different population groups.51 Such an approach also overlooks opportunity for developing and investing in existing grass roots programmes accepted by the population and working in context that do not have a published evidence-base.

For health and social interventions, there are known sociocultural and political factors are also imperative to an interventions transposability.34 McDonald confirms that assessing evidence from other dominant populations to the Aboriginal and Torres Strait Islander contexts is complicated by determinants specific to Aboriginal and Torres Strait Islander peoples, such as colonisation, dispossession, racism, assimilation and sociocultural determinants of health.52

The limited generalisability of RCTs is a known concern, particularly as it relates to lifestyle interventions and where target populations differ greatly from study populations.53–56 Further, many RCTs are experimental and do not reflect real-world patient populations.26 This experimental nature makes it difficult to determine if causal associations demonstrated by an RCT are relevant across populations.24 Kirmayer highlights how RCT participants tend to be male, urban, white, young and more likely to have a higher level of formalised education than the wider population.26 This lack of diversity within RCTs compromises the external validity, that is the ability to generalise findings outside of the study.

The intra-diversity of Aboriginal and Torres Strait Islander populations across geographies and sociocultural settings means that even within an RCT involving an Indigenous population, the population may not be representative of the wider population. What works in an Aboriginal community in urban Melbourne or Sydney may not translate and be implementable to the remote context. Particularly where programmes are resource intense and settings are compromised by organisational factors such as training, clinic space, equipment and staff time such as the busy community control setting. The NHMRC in assessing evidence suggests considering cultural factors such as ‘attitudes to health issues, including those that may affect compliance with the recommendation’.35 We elaborate and suggest they consider further cultural factors, including how populations conceive and prioritise problems, outcomes and interventions are also important.26 The current absence of valid and accepted evidence-based practice methods to consider the cultural applicability and transferability of programmes, policy or service for Aboriginal and Torres Strait Islander populations and settings, poses serious risks as well as programmatic adaptation challenges.

Health and social welfare systems differ greatly across nations, particularly in how they are funded, resourced and administered. When programmes are taken from settings far removed and populations that are socioculturally different it means that programme adaptation is necessary. For example, the Australian Nurse Family Partnership Programme initiated in Aboriginal and Torres Strait Islander communities since 2009 was an evidence-based programme that was developed from three RCTs in the USA involving white low income women from semi-rural New York, low-income urban African-American mothers from Tennessee and Hispanic families from Colorado.57 In order, to meet the requirements of the Australian healthcare system, fit within the Aboriginal and Torres Strait Islander comprehensive primary healthcare model, be consistent with existing nursing practice in Australia, as well as be culturally safe, this programme required substantial adaptation. Evaluation of this programme has seen mixed results, finding that of the five sites, one withdrew due to lack of community acceptability, while three of the remaining four clinics reported broad benefits of the programme. The evaluation also reported high costs per client, and a need to address organisational capacity and capability of the programme if it were to be rolled out further.58 As we consider the licencing for implementing evidence-based programmes and substantial costs associated with maintaining programme fidelity, including licensing fees, recruitment and training a specialist workforce and accreditation costs, it is worth considering if there is greater value and additional health and social outcomes from programmes initiated from within the ‘environmental and organisation context’ of ACCOs.

Recommendations for improving evidence-based practice in Aboriginal and Torres Strait Islander health moving forward

We have discussed some of the limitations and bias of contemporary evidence-based review in the Aboriginal and Torres Strait Islander health and social settings. With these in mind, we make the recommendation to move to Aboriginal-led evidence-based practice where ACCOs and Aboriginal-led teams in partnership are doing the evidence-based decision making. Where partnerships are characterised by ACCO governance over evidence generation and evidence-based practice and where there are mechanisms for full and meaningful participation and opportunity to privilege important Aboriginal and Torres Strait Islander voices.

As we begin to think about Aboriginal-led evidence-based practice and use of this method in Aboriginal and Torres Strait Islander health and social settings, we acknowledge that this requires ACCOs at some level to engage with a scientific positivist frame and the epistemological thinking that Western derived methods for processing knowledge can inform decision making. Additionally, it requires an acknowledgement that the scientific method of evidence-base practice developed for clinical medicine is useful to guide health and social programming, policy and services in Aboriginal and Torres Strait Islander settings.26 This engagement with a scientific method is a critically reflective one, as we are acutely aware of the past harms of research, and in particular Western research in the context of colonisation.

While there is already considerable scholarship out there to guide thinking about Aboriginal-led evidence-based practice, including by Larkin, MacLean et al, Harfield et al and Lock et al, this is still a relatively new frontier.6–9 12 There remains much more work to ensure that approaches to evidence review are scientifically rigorous as well as ethical and responsive to Aboriginal and Torres Strait Islander ways of knowing, being and doing. As we think about evidence-based practice method in an Aboriginal and Torres Strait Islander context, we emphasise that evidence-based practice is not just the systematic process of reviewing and applying evidence. Equally important are the ‘population values and preferences’ and ‘practitioner experts’.27 We are cautious that here we only examine the evidence review and appropriateness of implementation and evaluation as components of evidence-based practice also require consideration.

Evidence-based practice in an Aboriginal and Torres Strait Islander context would be characterised by its Aboriginal-led decision making and the privileging of Aboriginal and Torres Strait Islander ways of knowing, doing and being across the stages of evidence review, that is, from developing questions, acquiring studies, appraising studies and assessing of evidence. Figure 3 proposes that decision making in evidence-based practice in an Aboriginal and Torres Strait Islander context occurs at the intersection of contributing knowledges from (1) Aboriginal and Torres Strait Islander community service users’ values and preferences; (2) Aboriginal and Torres Strait Islander community controlled practitioners, as the ‘experts’ and (3) best quality evidence, that has been determined and identified as such by practitioners at Aboriginal and Torres Strait Islander organisations asking, acquiring, appraising and assessing the evidence using tools that attend to cultural bias.

Figure 3

Framework for Aboriginal-led evidence-based practice in an Aboriginal context. Adapted from Satterfield et al (2009).27

As we consider evidence-based practice in the Aboriginal health and social service settings, we must recognise that ACCOs are underpinned by principles of self-determination and community led decision making. Evidence-based decision making should not undermine existing community mechanisms for decision making, rather, they should enhance them. The important voices of Aboriginal community service users and the practice wisdom of Aboriginal community-controlled practitioners must be central to evidence-based practice.

The conceptualisation of evidence-based practice in an Aboriginal-led ‘environmental and organisational context’ fits within a data sovereignty framework, and rights based agendas as promoted by global human rights instruments such as the UNIDRIP.58 59 It also follows broader Australian government ethics for doing research with Aboriginal and Torres Strait Islander participants, where NHMRC guidelines stipulate ethical research in Aboriginal contexts and Torres Strait Islander respects and recognises Aboriginal and Torres Strait Islander peoples rights, is of benefit for Aboriginal and Torres Strait Islander people and involves Aboriginal and Torres Strait Islander people with their full agreement and participation.10

When considering evidence-based practice in Aboriginal and Torres Strait Islander settings, we are not calling for the complete and total rejection of all Western theories, methods, research or knowledge.12 Rather, it is a call to de-centre Western practice and moving to a more inclusive and equitable approach that recognises the legitimacy and importance of Aboriginal and Torres Strait Islander peoples ways of knowing, doing and being in Aboriginal and Torres Strait Islander health and social settings (box 1).13

A commitment to a new set of principles in relation to evidence-based practice

This includes but is not limited to: self-determination across all stages of the evidence review, implementation and evaluation including the developing of questions, acquiring of studies, appraising studies and assessing of evidence; recognition and respect for the inherent value and validity of Indigenous peoples knowledge and wisdom; a culturally grounded and racial equity perspective; benefit & reciprocity and mutual accountability—this includes asking how well Western evidence building measures up to ACCOs’ expectations.

Prioritise the generation of research evidence for Aboriginal and Torres Strait Islander programmes, services and policy

The ACCO sector needs a government commitment to supporting and resourcing the ACCO sector to build its own evidence-base over the long term. This requires a range of strategies including a significant investment in evidence and knowledge building, and strengthening existing capacities for research, evidence review and evaluation.

Greater transparency around the quality of evidence that gets generated

We need minimum standards on reporting so that those reviewing evidence can attend to the methodological quality of evidence. For example, this could include a requirement that all research involving Aboriginal and Torres Strait Islander populations reports to the CONSIDER statement36 or reporting to specifically developed Australian standards.

Tools and guidelines for evidence-based practice that recognise the broader methodology of research including its cultural and ethical frame

This would see that the quality of research is determined in terms of research methodology rather than just method. This includes the use and development of culturally relevant appraisal tools such as those by MacLean et al and Harfield et al.6 9 It would also require a strengthening of current government evidence-based practice guidelines, so they have greater utility in cross-cultural contexts

Evidence-based decision making must give equal weighting to the values and preferences of the Aboriginal and Torres Strait Islander population and the expertise of the ACCO sector as it does to ‘best quality evidence’ generated from an evidence review.

This means that ACCOs as expert practitioners, together with their communities are the ones assessing the applicability and transferability of evidence with consideration of its cultural acceptability in the local setting and for their service population. This recognises the importance of culturally-centred knowledge and lived experience of community in decision making.

Use Indigenous theories and knowledge systems in evidence review

Western-based paradigms and theories might be helpful here if they can be demonstrated to be culturally aligned, or where they can be integrated with Indigenous ways of doing, but this needs to be carefully interrogated. Culturally developed and aligned methodologies and methods need to be recognised and used.

As we consider evidence-based practice in the Aboriginal and Torres Strait Islander health and social service settings, we must recognise that ACCOs are underpinned by principles of self-determination and community-led decision making, and as such, practice must be designed and conducted on terms that are appropriate and acceptable to each organisation.

Conclusions

If evidence-based practice is really a means of achieving better health outcomes for Aboriginal and Torres Strait Islander people, as Australian health and social policy planning would suggest, then there are fundamental political, ethical and methodological issues that need to be attended to. In particular, ACCOs require greater power and resourcing to both produce evidence and review evidence. The current power dynamic that sees Western ‘practitioner experts’ resourced and legitimised to do research and review it without any accountability to or genuine partnering with Aboriginal and Torres Strait Islander experts is at its core inequitable, racist and standing in the way of achieving health equity for Aboriginal and Torres Strait Islander people.