A knowledge transfer study of the utility of the nova scotia seniors' mental health network in implementing seniors' mental health national guidelines.

BACKGROUND: The Canadian Coalition for Seniors' Mental Health (CCSMH) developed national best-practice guidelines in seniors' mental health. Promoting adoption of new guidelines is challenging, as paper dissemination alone has limited impact on practice change.
PURPOSE: We hypothesized that the existing knowledge transfer (KT) mechanisms of the Nova Scotia Seniors' Mental Health Network would prove useful in transferring the CCSMH best-practice guidelines.
METHODS: In this observational KT study, CCSMH best-practice guidelines were delivered through two interactive, case-based teaching modules on Depression & Suicide, and Delirium via a provincial tele-education program and local face-to-face sessions. Usefulness of KT was measured using self-report evaluations of material quality and learning. Evaluation results from the two session topics and from tele-education versus face-to-face sessions were compared.
RESULTS: Sessions were well attended (N = 347), with a high evaluation return rate (287, 83%). Most participants reported enhanced knowledge in seniors' mental health and intended to apply knowledge to practice. Ratings did not differ significantly between KT session topics or modes of delivery.
CONCLUSIONS: The KT mechanisms of a provincial seniors' mental health network facilitated knowledge acquisition and the intention of using national guidelines on seniors' mental health among Nova Scotian clinicians. Key elements of accelerating KT used in this initiative are discussed.

INTRODUCTION

Knowledge transfer (KT) is an important theoretical construct in continuing medical education, yet research on the clinical application of KT in medicine, including seniors’ mental health, remains limited. The Canadian Coalition for Seniors’ Mental Health (CCSMH) reviewed KT and exchange resources. Of 492 KT resources, only 20 (4%) were specific to seniors’ mental health.1 Current literature offers some insight about effective and ineffective KT approaches. A standard approach of KT is dissemination of new information through publication in academic journals or mail-outs. Experts agree this has little impact on practice change.2

The CCSMH developed guidelines promoting best practice in the areas of depression, delirium, suicide, and mental health issues in long-term care homes.3 Since their publication in 2006, 25,000 copies of the guidelines were disseminated across Canada and internationally.

Recognizing that paper dissemination of guidelines has little impact on practice change,2 better mechanisms for KT are needed. Successful KT requires several interacting key factors: 1) a supportive context or environment; 2) content that is practice based, high quality, and relevant to recipients; and 3) effective facilitation that is appropriate to recipients.4

At the organizational level, effective KT requires leadership, good facilitation, and active participation of stakeholders including academics, planners, nongovernmental organizations, consumer groups, service providers, and consumers (Chambers, Larry W, [President and Chief Scientist, Élisabeth-Bruyère Research Institute, Ottawa, ON]. Conversation with: J Kenneth Le Clair [Professor, Division of Geriatric Psychiatry, Queen’s University, Kingston, ON], April 2010). For successful KT in networks, “change champions,” leaders who take interest in promoting a cause within an organization, are needed to facilitate implementation of new, evidence-based practices.5 Good facilitation can overcome barriers to uptake of evidence-based practice,6,7 and face-to-face interaction is critical for KT success.8 Formal networks are more likely to experience KT than organizations operating outside of a formal network.9

The Nova Scotia Seniors’ Mental Health Network (NSSMHN) is a formal network with a mandate to increase provincial capacity for seniors’ mental health care. Sponsored by the provincial Department of Health and Wellness, it is unique within Canada both in mandate and in composition, linking government, clinicians, academics (researchers and educators), nongovernmental organizations, and patient advocates in the area of seniors’ mental health.

Several other elements recommended for successful KT are found in the NSSMHN model. These include a supportive context through regular meetings of stakeholders, and inclusion of district representatives who facilitate clinical capacity building through face-to-face education, train-the-trainer opportunities with links to an academic centre, and an established provincial tele-education program.

We assessed the capacity of this network to move CCSMH guidelines beyond paper dissemination and promote KT in Nova Scotia. The primary hypothesis was that existing KT mechanisms of the NSSMHN would provide useful KT vehicles. Secondary hypotheses were that evaluations would not differ significantly between session topics or between methods of KT delivery, and that participation in this project would engage local change champions of the NSSMHN and strengthen KT capacity.

METHODS

Design

This pilot project was an observational study using existing KT mechanisms of the NSSMHN and was evaluated using the network’s standard tele-education feedback form. CCSMH leaders attended two regularly scheduled network meetings to cement a partnership between the organizations. The following four network KT mechanisms were used, and the usefulness of each was assessed.

KT Mechanism 1: NSSMHN Capacity to Engage Local Change Champions

By providing a supportive context, project leaders (authors MB and KLC) promoted discussion about KT among the membership during regular NSSMHN meetings, to engage change champions.

KT Mechanism 2: NSSMHN Capacity to Develop Relevant, Interactive Material (Knowledge Translation)

Provincial experts translated the CCSMH guidelines into a clinically relevant format, through two case-based interactive modules: Depression in the Elderly and Delirium in the Elderly. The depression module referenced a KT tool called “Tool on Depression: Assessment and Treatment for Older Adults,” a quick-reference pocket card created by the CCSMH and the National Institute for the Care of the Elderly (NICE). Network members reviewed the modules to ensure clinical relevance for the target audience, which was urban and rural mental health team members, and hospital and nursing home clinicians. The number of target audience members was not estimated.

KT Mechanism 3: NSSMHN Tele-education Sessions and Train-the-Trainer Opportunities (Knowledge Exchange)

The modules were presented, via the tele-education program, to mental health teams in all nine district health authorities in Nova Scotia, in April and June 2008. Change champions of the NSSMHN promoted the sessions locally by liaising with mental health teams to encourage attendance. They facilitated the sessions, providing participants with copies of the presentation and CCSMH guidelines, and distributing and collecting evaluation forms. They used the sessions as a train-the-trainer opportunity to prepare themselves to deliver face-to-face sessions using the modules.

KT Mechanism 4: Face-to-Face Educational Sessions (Knowledge Exchange)

To address gaps in the KT reach of the tele-education program, face-to-face sessions were held in the nine district health authorities (primarily in hospitals and nursing homes). Between April and July 2008, nine identified change champions within the NSSMHN—three geriatric psychiatrists, one psychologist, and five community mental health nurses—facilitated face-to-face sessions using the two modules. Facilitators received training kits with materials necessary to deliver the presentations.

Outcome Measures

The primary outcome measure of KT was a self-report evaluation of participants, with questions that measured usefulness of several KT mechanisms under investigation, relevance of material, and quality of sessions through both tele-education and face-to-face sessions. The survey-response Likert scale ranged from “poor” (score of 1) to “excellent” (score of 5).

To further assess KT effectiveness, participants were asked whether they were likely to use the material in the future. Responses on the Likert scale used ranged from “completely disagree” (score of 1) to “strongly agree” (score of 5). They also listed one or two new concepts they learned in the sessions, and one or two ways they would change their practice as a result of the KT session.

A secondary measure was a self-report questionnaire distributed to the NSSMHN change champions regarding the experience and their engagement in the initiative.

Statistical Methods

From the rating scales, results were analyzed by calculating mean scores ± standard deviation (SD). To compare results from the two topic sessions, responses from 10 questions common to the two evaluations were added together for a total score for each individual participant, and means and SDs on the total scores were calculated. Data were included from all evaluations in which ≥80% of the common questions were completed. Depression versus delirium sessions and tele-education versus face-to-face formats were compared. Independent t-tests were conducted for comparing total scores in each subgroup. All data are presented as mean ± SD.

Return rates of evaluations were calculated as a percentage. Written comments were categorized, counted, and presented as a percentage of participants.

RESULTS

Participants

In total, 347 participants attended the KT sessions, with 186 participants attending the Depression in the Elderly sessions (35 via tele-education session, 151 via face-to-face sessions) and 161 attending the Delirium in the Elderly sessions (59 via tele-education session, 102 via face-to-face sessions). The five professional groups most commonly attending the sessions were (in descending order) as follows: nursing staff, Department of Health care coordinators, social workers, psychiatrists, and psychologists.

Self-Report Evaluation Result

Of the 347 participants, 287 (83%) completed self-report evaluation forms. Response rates were similar between the Depression (157, 84%) and Delirium sessions (130, 81%).

The relevance of the material content and quality of the tele-education and face-to-face presentations were rated highly overall, with participants rating most aspects of the presentations as either “good” (score of 4) or “excellent” (score of 5) (for mean scores, see Table 1).

TABLE 1

Depression & suicide risk, and delirium presentations: evaluation mean scores*

Please rate the following aspects of the session	Depression & suicide risk mean ± SD	Delirium mean ± SD	Overall mean ± SD
Was the content of the presentation interesting?	4.57±0.61	4.60±0.49	4.58±0.56
Did the presentation enhance your current knowledge of the subject?	4.46±0.57	4.53±0.59	4.49±0.58
How well did the presentation meet the learning objectives?	4.47±0.64	4.57±0.51	4.52±0.58
How would you rate the quality of the learning materials presented? (i.e., slides, handouts)	4.59±0.61	4.64±0.54	4.61±0.58
How would you rate audiovisual/sound quality?	4.58±0.61	4.41±0.64	4.50±0.62
Will you apply what you have learned to the care of your patients?	4.47±0.56	4.60±0.53	4.53±0.55
Was the session of suitable length?	4.32±0.72	4.46±0.57	4.39±0.66
Was there adequate time for questions, answers, and discussion?	4.39±0.70	4.47±0.57	4.43±0.64
Were participants encouraged to be actively involved?	4.56±0.65	4.59±0.59	4.57±0.62
How would you rate the presentation overall?	4.59±0.62	4.67±0.50	4.63±0.57

5-point Likert scale used (1 = poor, 2 = fair, 3 = average, 4 = good, 5 = excellent).

Most participants reported that both KT sessions enhanced their current knowledge of the subject (mean score 4.49) and expected to apply this knowledge to patient care (mean score 4.53) (see Table 1). Most participants either “somewhat” agreed or “strongly” agreed that the sessions resulted in greater awareness of the contents of the guidelines (see Table 2).

TABLE 2

Depression & suicide, and delirium presentations: guideline utilization evaluation mean scores*

Presentation	Mean ± SD
Depression & Suicide
I am now more aware of the CCSMH guidelines on depression and suicide in the elderly.	4.53±0.69
I am now more likely to screen for depression and suicide risk factors in the elderly.	4.50±0.69
I am more likely to refer to the CCSMH guidelines to help answer questions about depression and suicide in the elderly.	4.53±0.68
I am more likely to refer to the NICE brochure on depression to help answer questions about depression in the elderly.	4.57±0.61
Delirium
I am now more aware of the CCSMH guidelines on delirium in the elderly.	4.58±0.55
I am now more likely to screen for delirium in the elderly.	4.54±0.67
I am more likely to refer to the CCSMH guidelines to help answer questions about delirium in the elderly.	4.62±0.64

5-point Likert scale used (1 = strongly disagree, 2 = somewhat disagree, 3 = neutral, 4 = somewhat agree, 5 = strongly agree).

CCSMH = Canadian Coalition for Seniors’ Mental Health; NICE = National Institute for the Care of the Elderly.

Most respondents agreed they were more likely to screen for delirium, depression, and suicide risk factors in the elderly, and were more likely to refer to the guidelines and the NICE brochure on depression to help answer future questions (see Table 2).

There were no statistically significant differences in overall participant ratings between the depression and delirium modules, whether seen via tele-education or face-to-face sessions (45.0±4.8 versus 45.5±3.9, p = .2). There were also no statistically significant differences between tele-education and face-to-face session ratings regardless of topic (44.9±4.1 versus 45.4±4.6, p = .39).

Participants identified 1–2 concepts learned in each session. The two most common concepts were 1) the role of medications (19, 10%) and 2) electroconvulsive therapy (19, 10%) in depression; and 1) the role of medications (19, 12%) and 2) the importance of assessment (15, 9%) in delirium. Participants listed 1–2 practice changes they would make. The two most common planned practice changes in the depression sessions were 1) to screen for depression (52, 28%) and 2) to monitor for adverse antidepressant reactions (9, 5%); and in the delirium sessions were 1) to screen for delirium using the guidelines (28, 17%) and 2) to perform a delirium work-up (11, 7%).

Change Champion Feedback

The nine change champions who facilitated face-to-face sessions experienced increased comfort in teaching and more networking opportunities where they could discuss the cases and learn about local resources. All believed the project strengthened the KT capacity of the NSSMHN. Project challenges included difficulty getting frontline clinicians to attend due to busy schedules, inadequate time for questions, and the diversity of learning needs among participants.

DISCUSSION

Translating best-practice guidelines into clinical care is challenging to accomplish and evaluate.2 It is critical for healthcare professionals to remain current in best practice, and more research is needed to understand keys to successful KT and improve future efforts. The results of this project provide early support of the provincial health network model as a useful KT vehicle in seniors’ mental health.

This project appeared to reach our target audience, which consisted of provincial mental health teams, and hospital and nursing home clinicians. It was not surprising that nurses were the most represented professional group, given their role in providing direct care. While family physicians are aware of the tele-education program, they may not have been targeted for participation by local change champions.

Participants reported high quality and relevance of the format and content of the materials, suggesting that the interactive, case-based module format was an engaging way to review best-practice guidelines with a wide range of clinicians. The most commonly listed learning concepts and practice changes accurately reflected the content of the CCSMH guidelines, indicating that participants learned and retained key guideline content.

Within a provincial network, academic links and a tele-education program have an important role in supporting KT. Tele-education, which provided train-the-trainer opportunities for local change champions, appeared to be an efficient and successful means of providing KT, with well-attended sessions rated as highly as face-to-face sessions.

By providing a supportive environment for KT, the NSSMHN increased access to local and regional change champions, and increased its capacity to develop future learning opportunities. Evidence of a high level of engagement by change champions in the initiative included high participant response rates of evaluations, and the absence of significant statistical differences in ratings between tele-education and local face-to-face sessions. Face-to-face sessions reached a broader audience of clinicians, allowing for networking to occur at the local level. This has strengthened the network, by increasing clinician awareness of the mandates and resources of the NSSMHN and CCSMH.

Study limitations are related to the cross-sectional design and reliance on self-report measures. Due to the need for CME credit and to track disciplines, collection of evaluation forms was not anonymous. This could lead to reporting bias. Follow-up interviews with in-depth chart reviews would provide more objective evidence of KT to practice and whether clinicians changed practice consistent with reported intentions.

Despite the limitations, this study provides preliminary evidence that the formal health network model might promote KT and offers a useful model for other provinces to consider. With the seniors population expected to double in Canada over the next 25 years, efficient and effective models to promote best practice will be important to meet their mental health needs.