Rationale and Methodology of the 2018 Canadian Armed Forces Members and Veterans Mental Health Follow-up Survey (CAFVMHS): A 16-year Follow-up Survey: Raison D'être Et Méthodologie De L'enquête De Suivi Sur La Santé Mentale Des Membres Des Forces Armées Canadiennes Et Des Anciens Combattants, 2018 (ESSMFACM).

OBJECTIVE: Knowledge is limited regarding the longitudinal course and predictors of mental health problems, suicide, and physical health outcomes among military and veterans. Statistics Canada, in collaboration with researchers at the University of Manitoba and an international team, conducted the Canadian Armed Forces Members and Veterans Mental Health Follow-Up Survey (CAFVMHS). Herein, we describe the rationale and methods of this important survey.
METHOD: The CAFVMHS is a longitudinal survey design with 2 time points (2002 and 2018). Regular Force military personnel who participated in the first Canadian Community Health Survey Cycle 1.2-Mental Health and Well-Being, Canadian Forces Supplement (CCHS-CFS) in 2002 (N = 5,155) were reinterviewed in 2018 (n = 2,941). The World Mental Health Survey-Composite International Diagnostic Interview was used with the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) criteria.
RESULTS: The CAFVMHS includes 2,941 respondents (66% veterans; 34% active duty) and includes data on mental disorder diagnoses, physical health conditions, substance use, medication use, general health, mental health services, perceived need for care, social support, moral injury, deployment experiences, stress, physical activity, military-related sexual assault, childhood experiences, and military and sociodemographic information.
CONCLUSIONS: The CAFVMHS provides a unique opportunity to further understand the health and well-being of military personnel in Canada over time to inform intervention and prevention strategies and improve outcomes. The data are available through the Statistics Canada Research Data Centres across Canada and can be used cross-sectionally or be longitudinally linked to the 2002 CCHS-CFS data.

Introduction

Mental disorders, suicidal behavior, and other poor health outcomes are major concerns in military and veteran (i.e., a person who has served in the military) populations worldwide. In 2002, the Canadian Community Health Survey Cycle 1.2—Mental Health and Well-Being, Canadian Forces Supplement (CCHS-CFS) was conducted (N = 8,441) and provided the first mental health snapshot of the Canadian Armed Forces (CAF). The survey found a high prevalence of mental disorders, perceived need for mental health care, and unmet need for mental health care. An association between deployment-related traumatic events and adverse childhood experiences was also noted. In 2013, the Canadian Forces Mental Health Survey (CFMHS) provided an update regarding mental health of the CAF. This landmark cross-sectional survey indicated that in Canada, generalized anxiety disorder (GAD), panic disorder, and post-traumatic stress disorder (PTSD) had increased in CAF since the reported 2002 estimates. Notably, the prevalence of some of these poor health outcomes and perceived need for care are higher in the Canadian military than in the general Canadian population.

The literature regarding mental health among military personnel suffers from several key limitations. First, although cross-sectional surveys provide an important understanding of the epidemiology of mental health problems, they do not provide opportunities to understand the longitudinal course of mental disorders and predictors of incident mental disorders. Second, to the best of our knowledge, all longitudinal mental health surveys in military and veteran samples worldwide have used self-report screening tools to assess mental health symptoms. The U.S. Millennium Cohort Study and the UK cohort studies have used self-reported mental health screening tools to understand the longitudinal course of mental health symptoms. However, these measures do not provide much needed insight into the longitudinal course of diagnosed mental disorders. Key gaps in knowledge include the incidence, remittance, and persistence of mental disorders; pre- and postdeployment factors impacting mental health; and how mental disorders may influence perceived need for mental health care and help-seeking among military personnel over time. An understanding of these issues is necessary to develop preventative and early intervention programs to mitigate the negative impact of mental disorders on military personnel and their families.

The transition from the military back to civilian life is considered a high vulnerability period for an increase in mental health problems and suicidal behavior. However, we are not aware of longitudinal studies that have followed individuals during their military experience and through their transition to civilian life. The current survey was designed to address this important limitation.

The CAF Members and Veterans Mental Health Follow-up Survey (CAFVMHS) was designed to recontact the original Regular Force personnel who participated in the 2002 CCHS-CFS to create the first longitudinal Canadian military and veterans survey. Using an integrated knowledge translation approach, our team designed the survey through strong engagement of policy makers, international researchers, and key stakeholders. The CAFVMHS had the following key objectives. The first objective was to estimate the incidence, prevalence, and course of common mental health problems; suicidal behavior; physical health problems; and health service use. The second objective was to examine the predeployment factors (e.g., adverse childhood experiences, previous history of mental health problems, coping styles), deployment-related factors (e.g., combat exposure, military-related sexual assault, length/number of deployments, witnessing atrocities, injuries, work stress), and postdeployment factors (e.g., moral injury, social support) in relation to mental health problems, suicidal behavior, physical health problems, and service use. The third objective was to develop risk prediction tools for common mental health problems and suicidal behavior. Finally, the fourth objective was to examine the baseline predictors of resilience, mental wellness, and healthy transition to civilian life. The purpose of this article is to describe the study rationale, research design and methods, main variables included in questionnaire, and to provide overall comparison of the 2002 and 2018 data.

Methods

Study Design

The CAFVMHS used a longitudinal survey design with 2 time points (2002 and 2018). The target population for the CAFVMHS included all Regular Force personnel who were originally interviewed for the CCHS-CFS survey. This survey originally included a representative sample of full-time Regular and Reserve Force Class C members of the Canadian Forces. However, only full-time Regular Force members were recontacted for the CAFVMHS due to budget considerations.

Sampling Frame

The list of Regular Force respondents from the 2002 CCHS-CFS survey was used as the initial sampling frame for the CAFVMHS. Statistics Canada created a sampling frame with up-to-date contact information for those who were still actively serving members of the CAF. For respondents who had been released from the CAF (veterans), contact information was obtained from multiple sources (e.g., Veterans Affairs Canada, Department of National Defence [DND], among others). To identify those who had died since the 2002 survey, the Vital Statistics Death Registry was used. In 2016, Statistics Canada conducted the Life After Service Survey (LASS) and the CAF Transition and Well-Being Survey (CAFTWS). These 2 surveys included samples of CAF members who had retired from the military including some who were respondents in the original 2002 CCHS-CAF. To reduce respondent burden related to asking the same individuals to participate in multiple surveys, those who participated in the LASS and CAFTWS were considered ineligible for the 2018 CAFVMHS. The LASS and the CAFTWS were based on random samples, which would eliminate the possibility that this design feature would introduce selection bias. The original sampling frame was based on 5 strata defined by sex, rank, and whether the respondent was a reservist. Statistics Canada created longitudinal weights to produce estimates that were representative of full-time Regular Force members, in the original 2002 target population. Importantly, the longitudinal weighting in the CAFVMHS does not produce estimates that are representative of Regular Forces members and veterans from 2018. The adjustments made to the initial weights to create the new longitudinal weighting included adjustments to the initial sample weights to reflect those included in 2018, suppression of out-of-scope units, and adjustments of nonresponse.

Survey Development and Testing

Much of the content for CAFVMHS was repeated from the 2002 CCHS-CAF, although some new content was also added. A project steering committee was created, which included researchers from the University of Manitoba and representatives from Veterans Affairs Canada, the CAF, the DND, and Statistics Canada. This committee worked together to make decisions regarding content, and consultations were undertaken with experts in the field. Experts in the field were consulted to assist with the best practices on how to measure specific constructs. The content was selected to address the main aims of the survey, including understanding rates of new onset and persistence of mental health disorders, and risk and protective factors related to mental disorders over time. The time to administer the survey was approximately 75 minutes. The questionnaire was developed in both English and French. To comply with Statistics Canada policies and quality standards, Statistics Canada performed qualitative testing of survey content in both English and French before data collection. Qualitative testing involved testing content with selected respondents who represented the target survey population. The aim of qualitative testing was to ensure that respondents understood the survey questions in the way they were intended. In addition, qualitative testing allows respondents to provide feedback on the overall questionnaire and to understand respondent’s willingness to respond to the survey questions. Some content was revised based on findings from the qualitative testing. This process allows for problems to be identified before data collection resulting in a high-quality data. The main content areas are described below and in Table 1.

Table 1.

Content of Survey in 2002 and 2018.

Content Area	2002	2018	2018 Compared to 2002 Content
Military and demographic factors	X	X	Similar at both time points
General health	X	X	Same at both time points
WMH CIDI—DSM-IV diagnosis—major depression, panic disorder, social phobia, and generalized anxiety disorder	X	X	2002—Lifetime and past-year disorders2018—Since 2002 and past-year disorders also contains DSM-V GAD
WMH CIDI—28 traumatic events	X	X	2002—28 Lifetime and past year2018—28 Since 2002 and past year
WMH CIDI—post-traumatic stress disorder	X	X	2002—Lifetime and past year disorder2018—Since 2002 disorder; past-year probable disorder
WMH CIDI—alcohol abuse	NA	X	2002—Not assessed;2018—Since 2002 and past year
WMH CIDI—alcohol dependence	X	X	2002—Assessed for past-year disorder only2018—Since 2002 and past-year disorder
Smoking	NA	X	2018—Lifetime and current
Drug use (nonmedical)—sedatives/tranquillizers, stimulants, analgesics, marijuana/hashish, cocaine, club drugs, hallucinogens, heroin or opium, inhalants, and other illegal drugs	NA	X	2002—Not assessed2018—Lifetime and past year. Only assessed among veterans.
Medication use	X	XX	2002—past year2018—since 2002 and past year
Chronic conditions—mental disorders: attention deficit hyperactivity disorder, obsessive-compulsive disorder, mania or bipolar disorder, and personality disorders	NA	X	These diagnoses were assessed by self-report in the chronic conditions section because the reliability and validity of the CIDI for these diagnoses are poor. They were long-term conditions that have been diagnosed by a health professional and have lasted or were expected to last 6 months.
Chronic conditions—physical conditions: asthma, chronic bronchitis, emphysema, chronic obstructive pulmonary disease, arthritis, back problems, high cholesterol, high blood pressure, heart disease, stroke, diabetes, chronic fatigue syndrome, intestinal or stomach ulcers, irritable bowel syndrome, inflammatory bowel disease, migraine headaches, traumatic brain injury or concussion, cancer, epilepsy, multiple sclerosis, chronic liver disease	NA	X	The chronic conditions were self-reported. They were long-term conditions that have lasted or were expected to last 6 months and have been diagnosed by a health professional.
WMH CIDI module—substance use	NA	X	Military members were not asked questions related to substance use in 2002 and 2018. In 2018, only veterans were asked this section.
Suicidal ideation, plans, and attempts	X	X	2002—Lifetime and past year (ideation and attempts only)2018—Lifetime, since 2002 and past year (ideation, plan, and attempt)
Mental health service use	X	X	2002—Lifetime and past year2018—Since 2002 and past year
Perceived need for mental health care	X	X	2002—Past year2018—Past year
Deployment experiences—deployment experiences (DEX) formerly the DEPX	X	X	Additional deployment experiences were asked in 2018.
Moral injury	N/A	X	Only in 2018
Military-related sexual assault	N/A	X	Only in 2018
Social support	X	X	In 2002 and 2018
General stress and work stress	X	X	In 2002 and 2018
Childhood adversity	X	X	Different childhood experiences and stressors were measured in 2002 and 2018.

Note. WMH CIDI = World Mental Health Survey–Composite International Diagnostic Interview; DSM = Diagnostic and Statistical Manual of Mental Disorders; GAD = generalized anxiety disorder.

Data Collection and Processing

Data were collected by Statistics Canada using trained lay interviewers between January and May 2018. Respondents were informed about the nature of the survey and provided consent to participate. Participation in the survey was voluntary. Interviews were conducted in person using a computer-assisted personal interview with most interviews completed in respondents’ homes. Data were collected according to the Statistics Act, which provides Statistics Canada the authority to collect data while ensuring confidentiality of respondents. Data can be accessed in Canada with application and permission through Statistics Canada Research Data Centres (RDCs).

Concepts and Measures

The main concepts included in the 2018 data collection were as follows: sociodemographic and military demographic information; general health; chronic health conditions; mental disorders; suicidal ideation, plans, and attempts; military-related sexual assault; substance use; mental health service use; medication use; perceived need for mental health care; childhood experiences and stressors; social support; moral injury; deployment experiences; stress; and physical activity.

Military and demographic characteristics

Several standard questions were used to measure military and demographic characteristics. Military characteristics included rank, service environment, deployments (including before 2002, after 2002 or before, and number of deployments since 2002), date of release, and number of years served. Demographic characteristics included age, sex, marital status, household income, education, and province of residence.

General health and chronic conditions

Several indicators of general health were assessed in the CAFVMHS such as self-perceived mental and physical health, sleep, and adjustment to civilian life. Several chronic conditions were also assessed using self-reported data asking about long-term health conditions, which were expected to last or have already lasted 6 months or more and have been diagnosed by a health professional. These chronic conditions included asthma; chronic bronchitis, emphysema, or chronic obstructive pulmonary disease; arthritis; back problems; high cholesterol; high blood pressure; heart disease; stroke; diabetes; chronic fatigue syndrome; intestinal or stomach ulcers; irritable bowel syndrome; inflammatory bowel disease; migraine headaches; traumatic brain injury or concussion; cancer; epilepsy; multiple sclerosis; chronic liver disease; attention deficit hyperactivity disorder; obsessive-compulsive disorder; mania and bipolar disorder; and personality disorder.

Mental disorders

Understanding mental health over time was the primary aim of the CAFVMHS. In addition to self-reported mental disorders previously mentioned, several other mental disorders were assessed using diagnostic interviews. Several mental disorders were assessed using a Canadian adaption of the World Health Organization Composite International Diagnostic Interview (WHO-CIDI), which assessed mental disorders according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria. The WHO-CIDI has well-established validity and reliability. The same mental disorders assessed in 2002 were assessed in 2018 and included major depressive episodes, panic disorder, social phobia, GAD, PTSD, and alcohol dependence. Alcohol abuse was also assessed in 2018 but not in 2002.

Suicidal ideation, plans, and attempts

Lifetime and past-year suicidal ideation, plans, and attempts were assessed using several items. Respondents were asked whether they had ever seriously thought about attempting suicide or taking their own life, whether they had ever attempted suicide or tried to take their own life, and whether they had made a plan for attempting suicide.

Substance use

Smoking, alcohol, and drug use were assessed in CAFVMHS. Smoking was assessed using several questions asking about lifetime and current smoking. Other questions about smoking included age when smoking began, quantity of cigarettes smoked each day, and smoking cessation. Alcohol abuse and dependence were assessed using the World Mental Health CIDI diagnostic interview. Alcohol abuse and dependence were both assessed since 2002 and in the past year. Only veterans and not currently serving CAF members were asked about nonmedical drug use. These types of drugs included sedatives/tranquillizers, stimulants, analgesics, marijuana/hashish, cocaine, club drugs, hallucinogens, heroin or opium, inhalants, and other illegal drugs. Despite the confidential nature of the data collection, it may be that currently serving CAF member might be reluctant to disclose this type of information due to potential disciplinary action. Previous Statistics Canada Surveys have also not asked currently serving CAF questions of this nature.

Military-related sexual assault

Military-related sexual assault was measured using several items. If it was indicated that the traumatic event occurred, follow-up questions were asked including whether the event ever took place while deployed or in a CAF workplace and whether the persons committing the act were CAF members or DND civilian employees.

Mental health services

Respondents were asked several questions about contacts with health-care professionals. This module included questions about hospitalization and talking to a professional about emotions, mental health, or use of alcohol or drugs.

Medication use

Respondents were asked several questions about the use of medications including prescription and over-the-counter drugs to help with problems with emotions, mental health, or use of alcohol or drugs. This did not include natural health products such as herbs, minerals, or homeopathic products. Specific medications were not assessed in an effort to reduce the time to complete the questionnaire and to ease respondent burden.

Perceived need for care

Past-year perceived need for care assessed several different types of help respondents received or thought they needed for problems with emotions, mental health, or use of alcohol or drugs.

Childhood adversity

Respondents were asked to retrospectively recall several types of childhood adversity that were not assessed in the 2002 data. These experiences included exposure to intimate partner violence, physical abuse, sexual abuse, emotional abuse, physical neglect, and parent or adult living in the home with mental health problem.

Social support

Social support was assessed using several items to determine the level of affection, emotional support, and informational support in the past year.

Moral injury

Respondents of the CAFVMHS were provided with a series of statements regarding experiences at any point since joining the military to which they were asked to indicate the level of agreement. For example, these statements included seeing things that were morally wrong, being troubled by having witnessed others’ immoral acts, acted in ways that violated one’s own morals or values, feeling betrayed by leaders who were once trusted, and feeling betrayed by fellow service members who were once trusted.

Deployment experience

Several questions were asked in 2018 regarding stressful events that might have happened while on a CAF deployment. Eight potentially traumatic experiences while on deployment were included in the 2018 survey, which were the same as the questions asked in the 2013 Canadian Forces Mental Health Survey.

Stress

Several questions were asked about general stress in one’s life including contributing factors of stress, one’s ability to handle stress, and strategies for coping with stress. In addition, work stress was assessed using several items specific to stress related to work and job satisfaction.

Results

Study Sample

The 2002 CCHS-CAF sample included n = 8,441, with n = 5,155 Regular Forces personnel and n = 3,286 reservists. Of the 5,155 potentially eligible respondents, n = 856 were excluded before data collection. Reasons for exclusion included being deceased (n = 108), participant overlap with other surveys (n = 373), not being able to be located (n = 30), not being within the catchment of the 10 Canadian provinces (n = 89), and those that were randomly selected for removal due to budget constraints (n = 256). An additional n = 1,358 were excluded during data collection due to being deceased (n = 7), being outside of the 10 Canadian provinces (n = 19), not approving linkage to the 2002 CCHS-CAF data (n = 16), and refusal to participate (n = 1316). More details on attrition from the longitudinal sample and how the cohort in 2002 compares to the cohort in 2018 can be found elsewhere. Statistics Canada used computed weights to be used in statistical analyses to account for the nonresponse. Therefore, the final sample size for the 2018 CAFVMHS was n = 2,941. Of respondents who were eligible and available within the sampling frame, the response rate was 68.8%. The characteristics of the study sample at 2002 and 2018 are presented in Table 2. Of the 2,941 respondents interviewed at both time points, 87.8% were male and 12.3% were female. Over the 16-year time period, some changes in distribution were noted in some variables such as marital status, household income, and rank.

Table 2.

Sociodemographic Variables of the Sample Interviewed at Both Time Points in 2002 and 2018.

	Cohort in 2002 (n = 2,941)		Cohort in 2018 (n = 2,941)
	%a	95% CI	%	95% CI
Sex
Male	87.8	87.5 to 88.0	—	—
Female	12.3	12.0 to 12.5	—	—
Language of interview
English	71.3	69.4 to 73.0	—	—
French	28.7	27.0 to 30.6	—	—
Race/ethnicity
White	95.2	94.1 to 96.0	—	—
Non-White	4.8	4.0 to 5.9	—	—
Region of interview
Atlantic	22.1	20.4 to 23.9	—	—
Quebec	15.0	13.7 to 16.4	—	—
Central/Ontario	35.1	33.3 to 37.0	—	—
Western	27.8	26.1 to 29.7	—	—
Educational attainmentb
Secondary graduate or less	41.4	39.5 to 43.3	—	—
Some postsecondary or higher	58.6	56.7 to 60.5	—	—
Age
17 to 29 years	22.6	20.8 to 24.4	0.0	-
30 to 39 years	45.0	43.1 to 47.0	8.0	6.8 to 9.4
40 to 49 years	28.8	27.3 to 30.4	31.5	29.7 to 33.4
50 years and older	3.6	3.1 to 4.2	60.5	58.6 to 62.4
Marital status
Married/common-law	71.8	69.9 to 73.6	82.7	81.2 to 84.2
Separated/widowed/divorced	8.1	7.1 to 9.1	11.0	9.8 to 12.3
Single, never married	20.2	18.5 to 22.0	6.3	5.3 to 7.4
Total household income
$0 to $49,999	19.6	17.9 to 21.5	4.5	3.8 to 5.4
$50,000 or higher	80.4	78.6 to 82.1	95.5	94.6 to 96.3
Rank
Junior noncommissioned	56.1	55.4 to 56.8	26.5	24.7 to 28.4
Senior noncommissioned	22.1	21.6 to 22.7	47.8	46.0 to 49.7
Officer	21.8	21.3 to 22.2	25.7	24.8 to 26.6
Environment
Land	49.4	47.4 to 51.4	47.1	45.0 to 49.1
Air	31.9	30.2 to 33.6	34.4	32.6 to 36.3
Sea	18.8	17.2 to 20.4	18.5	17.0 to 20.2
Veteran status
Active member	100.0	—	34.5	32.7 to 36.4
Veteran	0.0	—	65.5	63.6 to 67.3

Note. n = 2,941.

a The cohort in 2002 in this table is a subsample of the original 2002 sample representing the sample respondents included at both time points.

b Changes in categories of educational attainment make comparisons from 2002 to 2018 not possible.

Discussion

Over the past several decades, many surveys have been conducted worldwide to understand the health and well-being of military personnel. This is important because of the essential work of military personnel and the high risk of mental health problems associated with such work. However, most of these studies have been cross-sectional. This a unique longitudinal survey of active and veteran Regular Force military personnel with 2 waves of data collected 16 years apart, in 2002 and in 2018. This is the first Canadian longitudinal data set focused on mental health, deployment-related traumatic events, and trauma among Regular Force members and veterans of the CAF. The high response rate of nearly 70% and the use of sampling weights create a sample that is representative of full-time personnel from the original 2002 sample. Although the data may not be representative of the CAF in 2018, a great strength of these data is that the 2018 data do remain representative of the CAF from baseline in 2002. These data will provide an important opportunity to understand the long-term mental health outcomes of CAF members using high-quality data and will be able to inform clinical practice and policy.

Although the data are unique and the data collection process is of high quality, there remain several limitations of the CAFVMHS. First, although it is a 16-year follow-up of a representative 2002 CAF sample, the 2018 data are not representative of all 2018 active duty Regular Force personnel and veterans. Second, these data are not generalizable to Reserve Force members since reservists were not included in the 2018 data collection. Third, mental disorders were assessed using lay interviewer-based diagnoses, which may not be equivalent to diagnoses made by clinicians. Fourth, the long period of follow-up (16 years) may be associated with recall errors. As well, with only 2 waves of data collection, details of the course of mental illness within the 16-year period are not possible to assess. Fifth, physical health conditions were not measured at baseline, which means that changes in physical health conditions over time cannot be assessed. However, to address this limitation, in 2018, respondents were asked to report age of onset. Finally, information on illicit drug use was not assessed among currently serving personnel.

The CAFVMHS data will provide the opportunity for researchers to ask numerous novel questions with respect to the health and well-being of our CAF. The CAFVMHS data can be accessed in Canada with permission through Statistics Canada RDCs across the country. These data are useful to inform clinical and policy decisions with the goal of improving mental health of military personnel. There are many advantages to prospective and longitudinal data collection. Future military data collections should be planned with multiple waves of data collection to further advance knowledge and improve health outcomes for military personnel.