Using a knowledge translation framework to implement asthma clinical practice guidelines in primary care.

Quality problem International guidelines establish evidence-based standards for asthma care; however, recommendations are often not implemented and many patients do not meet control targets. Initial assessment Regional pilot data demonstrated a knowledge-to-practice gap. Choice of solutions We engineered health system change in a multi-step approach described by the Canadian Institutes of Health Research knowledge translation framework. Implementation Knowledge translation occurred at multiple levels: patient, practice and local health system. A regional administrative infrastructure and inter-disciplinary care teams were developed. The key project deliverable was a guideline-based interdisciplinary asthma management program. Six community organizations, 33 primary care physicians and 519 patients participated. The program operating cost was $290/patient. Evaluation Six guideline-based care elements were implemented, including spirometry measurement, asthma controller therapy, a written self-management action plan and general asthma education, including the inhaler device technique, role of medications and environmental control strategies in 93, 95, 86, 100, 97 and 87% of patients, respectively. Of the total patients 66% were adults, 61% were female, the mean age was 35.7 (SD = ± 24.2) years. At baseline 42% had two or more symptoms beyond acceptable limits vs. 17% (P< 0.001) post-intervention; 71% reported urgent/emergent healthcare visits at baseline (2.94 visits/year) vs. 45% (1.45 visits/year) (P< 0.001); 39% reported absenteeism (5.0 days/year) vs. 19% (3.0 days/year) (P< 0.001). The mean follow-up interval was 22 (SD = ± 7) months. Lessons learned A knowledge-translation framework can guide multi-level organizational change, facilitate asthma guideline implementation, and improve health outcomes in community primary care practices. Program costs are similar to those of diabetes programs. Program savings offset costs in a ratio of 2.1:1.

Quality problem

The Global Initiative for Asthma and the Canadian Asthma Consensus Guidelines set the national standard for the diagnosis, assessment and treatment of asthma in Canada [1-4]. These guidelines synthesize asthma literature, present evidence-based best practices and establish control criteria.

Asthma care based on guideline recommendations leads to well-controlled asthma in the majority of patients [1-5]. However, international publications identify that many patients do not receive evidence-based care, that a majority do not have controlled asthma and that asthma exacerbations commonly result in urgent physician visits, absenteeism, emergency room visits and hospitalization [6-14]. The findings of a needs assessment conducted in Essex County, with a population of 375 000 and an asthma prevalence of 10%, aligned with this evidence suggest a significant guideline-to-practice gap in our region [15]. We hypothesized that closing this gap using a knowledge translation approach would improve health outcomes.

The Canadian Institutes of Health Research (CIHR) defines knowledge translation as ‘a dynamic and iterative process that includes synthesis, dissemination, exchange and ethically sound application of knowledge to improve health, provide more effective health services and strengthen the health system’. Conceptually, ‘knowledge translation’ is the process that connects the researcher to the knowledge user, converts knowledge into actions and links research to clinical practice. The CIHR has developed a knowledge translation framework that presents a practical ‘knowledge-to-action’ cycle composed of process elements that are common to 31 planned-action models. In this framework, a series of ‘action phases’ follow knowledge creation to convert medical knowledge to clinical actions [16-18] (Fig. 1).

Figure 1

CIHR knowledge-to-action cycle. Source: Graham et al., 2006. Reprinted with permission from John Wiley & Sons, Inc.

Knowledge translation was a substantial challenge in primary care private practice in our region. The administrative, clinical and human resource requirements to implement asthma guidelines did not exist. The project is presented as a quality improvement ‘case study’ highlighting the CIHR ‘knowledge-to-action’ framework.

Choice of solutions

A steering committee comprising primary care physicians, asthma specialists, registered respiratory therapists, registered nurses, pharmacists, hospital administrators and information technology experts engineered health system change in a multi-step phased approach described by the CIHR knowledge translation framework [16] (Fig. 1). Knowledge translation actions were targeted at multiple levels within the healthcare model: the individual patient, the practice, and the health system level (Table 1). The key deliverable from the process was a patient level guideline-based interdisciplinary asthma program.

Table 1

Multi-level community knowledge translation actions

Level of intervention	Knowledge translation (KT) actions
Project planning
Community-based quality improvement project approach	Established a multidisciplinary community advisory group
	Collaboratively created a community plan the Essex County Community Asthma Care Strategy
	Identified key barriers to the implementation of asthma guidelines
	Identified key guideline interventions for implementation within the project
	Collaboratively developed infrastructure tools and a healthcare model to address the identified barriers
	Pilot testing of project tools and program operations with tool refinement
Health system level	Organizational
Barrier: Primary care without a common organizational structure, standardized KT tools and sufficient human resources	Asthma Research Group (Windsor Essex County Inc.) is registered as a community non-profit corporation to lead the initiative
	Community organizations (6) sign an operating agreement
	A project coordinator is hired. Healthcare professionals from a variety of backgrounds are trained as asthma educators
Solution: infrastructure innovation focused on asthma KT, professional training and developing standardized tools	Electronic
	An electronic infrastructure is created collaboratively with the University of Windsor including: (i) a web-based communication and scheduling tool to support project administration, (ii) an educator software program for patient assessment, education and decision support and (iii) an automated recall appointment reminder system
Practice level
Barrier: no common model for implementing guidelines, quality improvement, sharing human resources and sharing knowledge tools	Accepted the Global Initiative for Asthma and Canadian Asthma Consensus guidelines as the guiding document for best practices
	Guideline objectives (6) were incorporated into the care model.
	The asthma educator is placed centrally in an inter-disciplinary care model as a guideline content expert.
Solution: creating an asthma management program and asthma care days	Care is integrated into the primary care practice with all elements delivered on-site where the patient normally receives care
	The educator uses the software program created for the project to standardize the intervention, track performance indicators and for action plan decision support
	Self-management education is a key element of the care model
	Automated recall notices for follow-up appointments
Individual patient level
Barrier: practitioner resources limit access	Regular physician review of controller medication and asthma control
Solution: inter-disciplinary care based on six guideline recommendations	Self-management education including a written action plan
	Objective measurement of lung function with spirometry
	Education on environmental control
	Education on role of medications
	Review and instruction on inhaler device technique

All full-time primary care physicians (n= 155) with an office-based practice serving the region were notified about the project by a general mailing from the local medical society, by colleagues and by the project staff [19]. Patients with a physician diagnosis of asthma were enrolled between October 2004 and November 2006. Physicians identified patients by recall, after a scheduled clinical encounter, and by a billing system audit where available. Posters were placed in physician waiting rooms.

We prospectively evaluated the asthma program comparing pre-intervention with post-intervention health outcomes. Symptom profile, healthcare utilization and absenteeism data were obtained by structured interview, administered from an electronic template, by asthma educators at every clinical evaluation. In addition, a patient questionnaire structured to parallel the standardized interview was administered by mail at study close at a minimum of 6 months after the enrollment date. Asthma symptom profile was based on symptoms over the preceding 4 weeks. Absenteeism was defined as a day missed at school or work because of asthma.

The key project deliverable was a regional guideline-based interdisciplinary asthma program. The primary outcome of interest in our benefits evaluation was the annual rate of urgent health services utilization. Secondary outcomes included the implementation frequency of six guideline-based care elements, the rate and proportion of patients with asthma-related absenteeism and the proportion of patients with symptom-defined disease control.

Patients who returned for follow-up were compared with those who did not using chi-square tests and unpaired t-tests. Pre- and post-intervention asthma symptoms, drug therapy and patient characteristics were compared by McNemar's chi-square test. Pre- and post-intervention data on healthcare utilization and absenteeism were annualized based upon the follow-up interval and analyzed using Wilcoxon signed rank tests. A sensitivity analysis was conducted to evaluate the potential effect of patients lost to follow-up. The project was approved by the Research Ethics Board at Hotel-Dieu Grace Hospital in Windsor, Canada.

Implementation

Multi-level health system change was a necessary first step before patient level implementation. Action phases of the CIHR ‘knowledge-to-action’ framework are identified in the following headings. The ‘initial barriers assessment’ from the CIHR framework has been paired with the implementation solution to emphasize the diagnostic nature of the barriers assessment process (Fig. 1).

CIHR: adapt knowledge to local context

The Essex County Community Asthma Care Strategy was developed in 2002 by a multi-disciplinary steering committee. The needs assessment, asthma management program, knowledge tools, electronic infrastructure and an inter-disciplinary care model were developed, pilot tested and refined in 2003–04. The demonstration project reported here began patient enrollment in 2004 and closed in 2006. Provincial funding was awarded in 2006; the program is ongoing. Guideline-based patient care objectives with a strong evidence base were selected for implementation (Table 1).

CIHR: select, tailor, implement interventions

Primary care in the region comprises solo practitioners and small group practices that do not have a unifying organizational structure to facilitate collaboration. We created a community-based infrastructure to support the implementation of the project. ‘Organizational infrastructure’: Asthma Research Group (Windsor Essex County) was registered as a not-for-profit corporation with a community board of governors. A multi-disciplinary steering committee advised the board. Six community organizations including professional associations, hospitals and the University of Windsor signed an operating agreement to implement the project which was executed by a full-time project coordinator. ‘Electronic infrastructure’: two web-based forums were created, one for asthma educators and one for the board. These tools connected educators, primary care offices, the board and the project coordinator who were located in different organizations across the region. Functionality included scheduling educators into asthma care days, delivering program notices, confirming appointments, voting and scheduling meetings. There were 1326 web-postings related to these functions during the study interval.

In a needs assessment conducted prior to project implementation (n= 29), 83% of physicians indicated that they did not have time to teach patients self-management. An environmental scan revealed that there was one certified asthma educator providing hospital-based services 1 day per week. Whereas, self-management education and regular clinical review are core guideline-based recommendations, the requisite human resources did not exist [1–4, 20]. Nineteen healthcare professionals were recruited from a variety of backgrounds (pharmacists, respiratory therapists and registered nurses) to complete college level training as asthma educators and 10 (53%) participated in the demonstration project. Thirty-three physicians (21%) were recruited from 19 sites across the region.

CIHR: knowledge products/tools: asthma management program and care model

Primary care in the region did not have a common infrastructure to facilitate the development and sharing of human resources and knowledge tools. We created a shared inter-disciplinary care model and asthma-specific knowledge tools. ‘Care model’: family physicians scheduled patients into ‘asthma care days’. Asthma educators were assigned to these days in 33 physician offices across the region utilizing a web-based tool developed for the project. The educator completed the elements of the asthma management program that are described in detail in Table 1, individual patient level. The educator encounter was 60–75 min and was followed immediately by a physician evaluation with the collaborative development of a final management plan (7–10 min). One 30-min follow-up visit was scheduled within 3 months and further appointments at the discretion of the provider.

Global Initiative for Asthma and Canadian Asthma Consensus guidelines are the evidence-based guidelines that were selected as foundational knowledge tools for the project. There were no local guideline-based tools adapted for, or accepted by, primary care practitioners in the community. We created a set of practical knowledge tools to standardize the intervention and capture outcomes including an electronic asthma educator assessment and teaching tool (AsthmED). AsthmED functionality included data collection; standardized asthma-related assessments and teaching reminder prompts for the educator; decision support for asthma action plans; encounter report generation; remote program updates and secure encrypted transmission of remote patient data to a dedicated centralized server. AsthmED served as an electronic medical record for the project (904 patient encounters), is compliant with national privacy legislation and was the central tool of the Asthma Management Program [21].

CIHR: sustain knowledge use

The demonstration project was transitioned to a provincial primary care asthma program in 2006. To date 60 physicians (participation rate = 39%) and 1261 asthma patients have participated. This demonstration project received $260 000.00 Canadian dollars to support all elements of the project including development, implementation and evaluation ($501/patient). The primary care asthma program provides services to 500 patients/year with a maintenance cost of $290/patient.

Evaluation

From October 2004 to November 2006 1240 invitation letters were mailed to patients identified with asthma; 519 patients were enrolled (42%). ‘Clinical follow-up visits’: a majority of patients 312 (60%) returned for a follow-up visit during the study period at a mean interval of 102 (SD = ±97) days. The average number of follow-up visits was 1.25 (SD = ±0.49). ‘Questionnaire follow-up’: 350 (67%) responded to the questionnaire administered at study close. The mean interval from enrollment to administration of the questionnaire was 22 (SD = ±7) months. A total of 237 patients (46%) completed assessments at both intervals.

Of the 519 patients, 340 (66%) were adults (>18 years) and 315 (61%) were female (Table 2). The average age was 35.7 (SD = ±24.2) years, 433 (83%) were on asthma controller therapy at the time of enrollment, and 377 (80%) had an FEV1 ≥80% predicted. Practice sites included a variety of urban and rural settings with differing socioeconomic status. Patients who returned for follow-up compared with those that did not return were not statistically different excepting three variables: they were older, had a lower FEV1 and greater use of inhaled corticosteroid and long-acting B-agonist controller therapy (Table 2).

Table 2

Baseline characteristics: comparing patients with/without follow-up

	All patients (n= 519)	Patients not returning for follow-up (n= 207)	Patients returning for follow-up (n= 312)	P value
Gender, female [n (%)]	315 (60.7)	121 (58.4)	194 (62.2)	0.395
Age [mean (SD)]	35.7 (24.2)	31.4 (22.5)	38.5 (24.9)	0.001
Caucasian [n (%)]	478/504 (94.8)	188/203 (92.6)	290/301 (96.4)	0.063
Patient reported allergic history [n (%)]	345 (66.5)	136 (65.7)	209 (67.0)	0.761
Smoking status [n (%)]				0.414
Never	359 (69.2)	149 (72.0)	210 (67.3
Former	105 (20.2)	36 (17.4)	69 (22.1)
Current	55 (10.6)	22 (10.6)	33 (10.6)
FEV1 as % predicted [mean (SD)]	94.6 (20.4)	97.1 (19.5)	92.9 (20.8)	0.029
Patient taking any controller medication [n (%)]	433 (83.4)	165 (79.7)	268 (85.9)	0.063
ICS alone [n (%)]	132 (25.4)	62 (30.0)	70 (22.4)	0.054
LTRA alone [n (%)]	17 (3.3)	7 (3.4)	10 (3.2)	0.912
Combination therapy (ICS + LABA) [n (%)]	270 (52.0)	94 (45.4)	176 (56.4)	0.014
Rescue medication in doses/day [mean (SD)]	0.597 (1.233)	0.524 (1.113)	0.646 (1.306)	0.271

SD, standard deviation; ICS, inhaled corticosteroid; LTRA, leukotriene receptor antagonist; LABA, long-acting B-agonist; FEV1, forced expiratory volume in 1 s.

CIHR: monitoring knowledge use—implementing guideline recommendations

Six guideline-based care objectives were implemented in the majority of patients and in most on their initial clinical visit. There was an increase in the proportion of patients prescribed controller therapy after participation in the program: 268/312 (86%) at baseline vs. 295 (95%) post-intervention (P<0.001). In patients with two or more symptoms outside of control benchmarks at baseline, 98% were prescribed controller therapy post-intervention (Table 3).

Table 3

Implementing guideline recommendations

Guideline recommendations	Initial clinical visit (n= 519) [n (%)]	Final clinical visit (n= 312) [n (%)]
Asthma education provided
1. Environmental control	390 (75.1)	272 (87.2)
2. Role of medications	471 (90.8)	304 (97.4)
3. Inhaler device technique	487 (93.8)	311 (99.7)
4. Written action plan	404 (77.8)	269 (86.2)
Spirometry measured	474 (91.3)	290 (92.9)
Controller therapy prescribed (n= 312)	Pre-intervention (n= 312)	Post-intervention (n= 312)	P-value
Any controller	268 (85.9)	295 (94.6)	<0.001
ICS only	70 (22.4)	65 (20.8)	0.456
ICS and LABA combination therapy	176 (56.4)	216 (69.2)	<0.001
LTRA combination therapy	83 (26.6)	106 (34.0)	<0.001
Sub-set: controller therapy in patients not in control at baseline (n= 133)
Any controller	117 (88.0)	130 (97.7)	0.002

SD, standard deviation; ICS, inhaled corticosteroid; LTRA, leukotriene receptor antagonist; LABA, long-acting B-agonist; Not in control, >2 benchmark symptoms beyond acceptable limits.

CIHR: evaluate outcomes

Asthma symptoms

These data are presented as individual symptoms and as a composite of more than one or two symptoms outside of the Canadian Asthma Consensus guidelines control criteria. The reported outcomes are based on clinical follow-up visit data at 3 months. At baseline 209/312 (67%) had one or more symptoms beyond acceptable limits vs. 112/312 (36%) post-intervention (−46%, P< 0.001) and 130/312 (42%) had two or more symptoms beyond acceptable limits vs. 52/312 (17%) post-intervention (−60%, P< 0.001). Individual symptom benchmarks including coughing, wheezing, dyspnea, chest tightness, rescue medication use and nocturnal symptoms showed similar improvements (range −33 to 50%, P< 0.001). Composite symptom improvements remained clinically and statistically significant on long-term follow-up at 22 months (P< 0.001).

Health resource utilization and absenteeism

Among 350 patients with a completed follow-up questionnaire 247 (71%) reported an urgent/emergent healthcare visit at baseline compared with 156 (45%) post-intervention during a mean follow-up interval of 22 months (−37%, P< 0.001). The mean urgent healthcare utilization fell from baseline 2.94 (SD = ± 4.36) visits/year to 1.45 (SD = ± 2.91) visits/year (−51%, P< 0.001). (Fig. 2) Similar improvements were identified for absenteeism. Health resource utilization and absenteeism from patients who completed both a clinical follow-up visit and the questionnaire at study close are summarized in Table 4.

Figure 2

Urgent healthcare utilization before and after the program. Healthcare utilization at baseline and follow-up (n= 350). Urgent visits were defined as unscheduled healthcare encounters for asthma symptoms, including unscheduled family physician, walk-in clinic, emergency department and hospital admissions. All comparisons P< 0.001 except hospital admissions P= 0.355.

Table 4

Healthcare utilization and absenteeism

Parameter	Baseline (n= 237) [n (%)]	Short-term follow-up (clinical visit) (n= 237) [n (%)]	P	Long-term follow-up (questionnaire) (n= 237) [n (%)]	P
Any urgent/emergent healthcare visit for asthma	167 (70.5)	43 (18.1)	<0.001	117 (49.4)	<0.001
Unscheduled family MD visits	151 (63.7)	28 (11.8)	<0.001	99 (41.8)	<0.001
Walk-in clinic visits	73 (30.8)	16 (6.8)	<0.001	39 (16.5)	<0.001
Emergency room visits	48 (20.3)	6 (2.5)	<0.001	28 (11.8)	0.003
Hospital admissions	16 (6.8)	2 (0.8)	<0.001	15 (6.3)	0.819
Absenteeism	91 (38.4)	21 (8.9)	<0.001	47 (19.8)	<0.001

Baseline data: visits in the prior year. Clinical visit: outcome data collected during clinical visits. A mean interval of 102 days. Questionnaire: outcome data collected by questionnaire at the end of the study. A mean interval of 22 months.

Sensitivity analysis on patient outcomes

A sensitivity analysis was conducted to evaluate the potential effect of patients lost to follow-up. The analysis considered three outcome scenarios: that all patients lost to follow-up improved (best-case), that all patients lost to follow-up were unchanged from baseline status (unchanged) or that all patients lost to follow-up deteriorated or had an urgent healthcare visit (worst-case). Improvements in symptom control, absenteeism and healthcare utilization were confirmed in all three scenarios. In the best-case and the unchanged-status scenarios all health outcome improvements were confirmed. In the worst-case scenario, reductions in total urgent healthcare visits remained significant (P< 0.001); however, improvements in symptom control (>1 symptom outside of benchmark) demonstrated only a trend toward improvement (P< 0.076).

Lessons learned

We used a knowledge translation framework to implement asthma guidelines in a region and in a practice setting that did not have the requisite administrative, clinical, and human resources for effective guideline implementation. We engineered health system change by creating a regional administrative infrastructure, by creating asthma specific knowledge tools and by implementing an interdisciplinary care model. We demonstrated the functionality of the program by implementing six evidence-based best practices in a majority of patients. Finally, we demonstrated improvements in asthma-related health outcomes including asthma symptoms, urgent healthcare utilization and absenteeism that were sustained over time. This project is a case study demonstrating how quality improvement/knowledge translation can be usefully guided by a conceptual framework and result in positive health and health system outcomes.

Despite the identified knowledge-to-practice gap internationally, there are relatively few studies evaluating the implementation of evidence-based asthma care in a primary care setting [22-26]. The pediatric asthma management program, Easy Breathing, has been implemented by primary care physicians in large health maintenance organizations and urban Medicaid populations in the USA [22, 24, 25]. A proactive asthma program was implemented in a pediatric private practice population in Australia and a provincial program for children and adults in a community healthcentre population in Canada [23, 26]. Similar to our study, these studies demonstrate improvements in urgent health service use, absenteeism and symptom control. A novel aspect of our study is implementation in private practices that operate outside of a large healthcare organization, where there is no infrastructure to support the development, implementation and evaluation of quality improvement initiatives. We created electronic knowledge tools to support program portability, scalability, resource sharing and program evaluation.

Implementing a program in a community setting without access to a large administrative data set required that we use patient self-report to measure health services utilization. Although self-reported health service utilization is common in the chronic disease management literature, and although self-report has been validated against administrative databases in other settings, utilization in this study was not validated against an administrative data set [27-29]. We did however utilize several methods to increase the accuracy of the self-reported data including: measuring acute events that are easily understood by the subject (exacerbations requiring urgent care), conducting structured interviews with qualified staff, utilizing the shortest meaningful reporting interval and repeating the same inquiry consistently on all assessments [30].

When tailoring this project to meet regional needs, we selected a design that would maximize physician participation. This design did not include a randomly assigned control group. While we are not aware of any systematic bias affecting our outcomes, without a control group we cannot exclude the possibility that bias influenced our results. Several factors support a cause-and-effect conclusion between our intervention and improved health outcomes. Our cohort was recruited from low acuity settings and we demonstrated asthma control levels that aligned with published surveys. This suggests our cohort was a valid representation of primary care and mitigates the risk that regression to the mean bias enriched our results [8, 10, 11]. There was strong internal consistency in our outcomes over time; early improvements were sustained for almost 2 years and across all health outcome measures. Finally, there was a notable external consistency between the effect sizes in our study and those of comparable controlled efficacy trials [1–4, 22–26].

Despite measures to facilitate follow-up appointments, the number of patients lost to follow-up was high, reflecting the real challenges of implementation in a community setting. We evaluated our data in two ways to assess for the impact of incomplete follow-up. First, we compared the baseline characteristics of the patients returning for follow-up with those who did not. The groups were not different on a majority of parameters; however, patients who returned for follow-up were older, or had a lower FEV1, or were more likely to be on a combination product than patients who did not return. While the retention of patients with parameters indicative of more severe asthma is clinically desirable, it may have increased the magnitude of the reported improvements. Another concern is that lost patients may over-represent patients who did not improve or who deteriorated thereby enriching the reported outcomes. We conducted a sensitivity analysis to address this concern directly. In the most likely scenario of the analysis, where we assumed that every patient lost to follow-up remained unchanged from baseline, all reported health outcome improvements were confirmed.

In Canada, the mean cost of caring for patients in an ambulatory asthma clinic population has been estimated at $2550/person/year [31]. The development, implementation and evaluation of our project cost $501/patient which is comparable to the $664/patient cost of a diabetes self-management program in our jurisdiction [32]. The ongoing primary care asthma program that followed our demonstration project is maintained at a cost of $290/patient. Estimated program cost savings can be calculated based on the post-intervention reduction in the number of urgent health visits and days absent multiplied by the average cost. Applying this simple model over the 2-year study interval, we estimate cost savings of $166 880.00 ($321/patent) on urgent health services and $145 656.00 ($281/patient) on absenteeism [33-34]. Program savings offset costs in a ratio of 2.1:1 ($602/$290), suggesting that our program may be cost effective.

Closing the gap between research knowledge and evidence-based care requires a systematic approach that is adaptable to a range of healthcare settings. A knowledge-translation framework can guide multi-level organizational change, facilitate asthma guideline implementation and improve health outcomes, with modest program expenditures in community primary care practices.

Authors’ contributions

C.L., T.S., L.P. and I.N. contributed to the conception and design of the study and to the analysis and interpretation of data. C.L. and M.O. drafted the paper. All authors were involved in critically revising the article for important intellectual content and gave final approval of the version to be published.

Funding

The work was supported by the Ontario Ministry of Health and Long-Term Care: Primary Health Care Transition Fund Grant# G03-05495. Funding to pay the Open Access publication charges for this article was provided by the Government of Ontario.

Conflict of interest

C.L. has received honoraria for speaking engagements and/or participated on advisory boards for GlaxoSmithKline, AstraZeneca, Novartis and Merck. C.L. has received honoraria from the Ontario Lung Association for continuing education program development and presentations.