Why are you asking about gender? Negotiating survey research methods within an industry collaboration.

Objectives: This brief report describes a survey design process undertaken in collaboration with industry stakeholders from government, non-government and other applied fields. This account highlights fundamental and contested issues of knowledge creation in research, situated within the broader contemporary context of social change addressing inequality and inclusion for historically marginalised and vulnerable groups. Study design: The study comprised a non-probability survey of the Australian Alcohol and Other Drugs Workforce.
Methods: A reflective account is provided.
Results: Significant and unanticipated differences in conceptual frames and perceptions of research ethics between the research team and industry representatives emerged during the collaboration, with major implications for the validity of the research process. Conclusions: The traditional, and largely unquestioned, understanding of quantitative survey research methodology is encountering increasing challenges in light of contemporary considerations of identity, privacy and wellbeing of survey participants. Some of these differences seriously challenged conventional approaches to research methodology, quality and rigour. There is a pressing need for further exploration, discussion and debate regarding the process of knowledge creation, ownership and stewardship. Strategies to better equip the research community and their industry stakeholders to navigate issues of research veracity, integrity and rigour are urgently needed, including training and guidance on negotiate differences in values, priorities and perspectives for upcoming and established researchers.

The science of quantitative survey design is well established, including validated measures and evidence-based techniques for wording and item ordering to minimise bias and maximise reliability and validity [1]. In an ideal world, researchers simply apply these methods in survey development, informed by rigorous theories and models. Much research is conducted this way as evident in published protocols [2,3]. What is less overtly evident are the underlying assumptions, values and world views that inform and shape research questions and methods. The anthropologist Lévi Strauss [4] observed that ‘The scientist is not a person who gives the right answers, he's [sic] one who asks the right questions'.

In the current era of social change redressing inequality and embracing inclusion of historically marginalised groups, explicit and implicit power structures and dynamics are being called into question. In science, this gives rise to the query of who decides the ‘right questions’? Throughout the history of science what is considered the ‘right questions’ has shifted with transitions to new paradigms [5]. In the current era, social and political movements are increasingly contesting and informing what are the ‘right questions’ in many domains including public health [6,7].

These issues were bought to the forefront in a recent public health project involving a quantitative survey of the Australian alcohol and other drug (AOD) workforce. In the context of a competitive funding environment, collaborations with industry stakeholders such as government, non-government and private organisations are strongly encouraged or required by many funding programs. There are significant benefits from such collaborations, including improved research relevance and impact. But there are also challenges underpinned by differences in priorities, values and understandings of the role and purpose of research. In many circumstances these differences can be accommodated within a research project. For example, in qualitative research such as grounded theory differences of opinion are an accepted and integral aspect of the research process. Here we describe a situation in which the difference of opinion addressed a fundamental aspect of the quantitative research process, the structure and content of survey measures, with implications for the validity of the research process, analysis and interpretation of findings.

Our project involved an online survey of government, NGO and private sector AOD workers across all roles (e.g. treatment, research, administration) in the sector. This workforce planning and workforce development survey included questions on social and employment demographics, client characteristics, professional development, retention and wellbeing. The project was approved by three Human Research Ethics Committees and met required standards with regard to anonymous and voluntary participation. The project received federal and state government funding and was informed by informal and formal negotiations with stakeholders including representatives from government and non-government sectors, and professionals employed in policy, advocacy and service delivery. Some stakeholders intended to use the survey items and/or data in their own workforce development initiatives, hence had a significant stake in survey design and content. The discussions outlined below were conducted primarily with representatives from a single industry stakeholder.

An initial point of contention concerned the inclusion of a survey item addressing gender. The research team was asked to provide a rationale for this item, on the basis that such information was not relevant to the purpose of the survey and was overly intrusive. We successfully retained this item arguing that the gender composition of the workforce was fundamental to accurate workforce profiling and has significant implications for policies and programs addressing diversity, flexibility, parental leave and work-life balance. Scholars from gender studies and sociology have explored the ‘undoing’ or irrelevance of gender in workplaces and other contexts [8]. The research team's view was that positioning gender as irrelevant was not only highly contested in academic scholarship but did not reflect the reality of paid work for most individuals.

In contrast, a survey item addressing Aboriginal and/or Torres Strait Islander identity was uncontested. Indeed, there was significant focus from the industry representatives on ensuring the terminology conformed with contemporary language preferences. Issues of privacy and relevance were not raised with regard to questions addressing Aboriginal or Torres Strait Islander identity or a second measure of cultural and ethnic identity (country of birth). This anomaly raises important questions regarding the sociodemographic indicators that are considered essential in survey research. The research team viewed gender as an essential and standard survey item, yet this view was not shared by others who worked in policy and other applied contexts.

Similar concerns regarding intrusiveness and privacy were raised regarding items addressing whether participants had ‘personal experience of alcohol or drug problems (past or present)’ or a family member with this experience. No other information was requested. Inclusion of these questions were strongly challenged by an industry stakeholder who maintained that it ran the risk of re-traumatising participants with AOD lived experience. The research team assessed this risk to be very low, noting that the majority of survey participants were in frontline roles providing services to clients with drug or alcohol issues. The research team successfully negotiated the inclusion of these items by arguing that (1) accurate workforce data on workers with lived experience was necessary to inform and evaluate policies and programs to support these workers, and (2) not collecting information on vulnerable groups has the unintended consequence of ‘silencing’ these groups, inadvertently hiding them from view due to an absence of information and potentially risking their oversight in policy, funding and programming decisions.

A third major point of negotiation concerned a validated measure of compassion fatigue addressing burnout (e.g., feelings of depression and failure) and secondary trauma (e.g., sleep disturbance, intrusive thoughts). This measure has been used in previous research and rehabilitation programs for health professionals [9]. Industry representatives viewed these questions as inappropriate due to the risk of traumatising stressed workers. The industry stakeholder was also concerned that the questions implicitly stigmatised and discredited the clients of AOD services by exploring service providers’ negative experiences related to client contact.

The research team successfully negotiated inclusion of alternative validated measures of burnout and engagement without reference to clients. Instead items referred to feelings of exhaustion, frustration, enthusiasm and immersion that have universal application to any work role were included. Whilst these measures were adequate for the purposes of the project, there was a loss of specificity and context in the measurement of workplace mental health for those in client service roles. Some of the concerns regarding privacy and intrusiveness were addressed by including response options such as ‘prefer not to say’, free text (instead of pre-set response) and skipping items. Whilst not ideal in terms of data quality, these options were necessary compromises in the negotiations about item inclusion.

To conclude, in the process of negotiating survey design and content we encountered significant differences in perspectives and values between members of the public health research community and representatives of the policy and practice communities. Some of these differences presented a serious risk to research methodology, measurement quality and research rigour. Such challenges are likely to continue to emerge at the interface between researchers and industry stakeholders and partners. As outlined above, a key strategy to resolving these differing viewpoints and finding agreement was open discussions, including a willingness by all parties to be informed and educated on the values and practices considered standard in our respective domains (research and policy/practice).

Questions therefore arise regarding the best ways to anticipate and navigate these challenges. Possible strategies include establishing a priori some new rules of engagement at the outset of collaborations such as 1. Acknowledging that some differences in world views are likely to arise; 2. Identify agreed negotiation and conflict resolution strategies; 3. Recognise that compromise is necessary; 4. Accept that traditional research criteria may be challenged; and 5. Realise that there may be necessary trade-offs between partnerships over rigour.

These important issues of research veracity within research-industry partnerships need to be tackled early and more broadly. Given that recent changes and challenges in access to knowledge moves to empower those traditionally excluded from many societal controls, greater emphasis on ‘co-designed’ research and the general diminution of ‘expert knowledge’, these negotiations are likely to become more commonplace. Inclusion of such issues in graduate researcher training programs may be appropriate to better equip new researchers to navigate these challenges. More generally, debate is pressingly needed on the notion of ‘knowledge’ and its ownership, stewardship and veracity. These are fundamental issues of knowledge creation and ownership in contemporary society that warrant a larger stage for exploration and resolution.

Funding

.

Ethical approval

[Anonymised] University Social and Behavioural Research Ethics Committee; [Anonymised] Clinical Human Research Ethics Committee.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.