Using photographic interpretation to evaluate the safety of home environments.

In the US there were 400,000 home fires resulting in 2755 deaths, 12,450 injuries, and $6.9B lost. The purpose of this study was to evaluate the content-validity of photographs taken in the home for use as an educational instrument to teach about "safe" and "unsafe" fire safety practice for adults and older adults. A total of 73 home fire safety experts were provided 27 photographs to evaluate home fire safety practice. Initially, a Krippendorff's alpha was calculated for the first 24 questions to evaluate inter-rater reliability, and differences in demographics were evaluated. Unique codes and themes for the last three questions were identified and inter-rater reliability examined. A majority of respondents were female (n = 43, 60.6%), college educated (n = 61, 83.6%), nurses (n = 25, 33.8%), or worked for a fire department (n = 21, 29.6%). Their mean age was 45.5 years and they had 11.05 years of experience. The first 24 questions had high inter-rater reliability (Krippendorff α = 0.831). No significant differences existed between the strata of the demographic variables (all p-values > 0.05). Similarly, based on the codes and themes identified, the last three questions had moderate-to-good inter-rater reliability (Krippendorff α = 0.764). Providing photographs as a 'seek-and-find' or 'What's wrong with this picture?' tools and simplified visual images is an excellent way to aid recognition of unsafe home fire safety environments. Education through non-traditional visual methods increases the possibility of change for diverse low-literacy populations.

Introduction

Nationally, 380,300 residential fires were reported in 2013 which included 2755 deaths, 12,450 injuries, and $6.9B lost (United State Fire Administration (USFA), 2015). Cooking (188,000) and heating (49,000) were the leading causes of fire (United State Fire Administration (USFA), 2015). Deaths were related to carelessness (465), electrical malfunction (335), smoking (320), and intentional reasons (320) (United State Fire Administration (USFA), 2015). Injuries were caused by cooking (4225), open flame (1150), and carelessness (1075) (United State Fire Administration (USFA), 2015). Carelessness ($1B) and electrical malfunction ($923M) accounted for the greatest fire dollar losses (United State Fire Administration (USFA), 2015). Though preventable, residential fires are significant causes of death, injury, and dollars lost in the United States.

Assessment of home fire safety (HFS) practices most frequently involved checking for fire alarm location(s), and functionality (Gielen et al., 2013, Miller et al., 2014, Parker et al., 2013, Sidman et al., 2011). Less often was tap hot water temperature tested (Parker et al., 2013), carbon monoxide alarms presence assessed (McDonald et al., 2013), presence and practice of fire escape plans determined (Yang et al., 2006), or participant knowledge regarding fire safety obtained (Parker et al., 2013). The United States Fire Administration's Home Fire Safety Checklist examines HFS from multiple practice perspectives (United States Fire Administration). The checklist has eight subscales (i.e., smoke alarms, cooking safety, electrical and appliance safety, carbon monoxide (CO) alarms, candle safety, smoking safety, heating safety, and home escape plans) with three to six items per subscale (United States Fire Administration). To date, no reported evidence of reliability or validity is available for this instrument. The practice of HFS is complex, needing a comprehensive approach using an instrument with evidence of reliability and validity.

In two previous studies by the authors on HFS education, the team studied 103 parents of newborns (with and without special needs) (Lehna et al., 2015a) and 125 older adults (community active and homebound) (Lehna et al., 2015b). Participants were asked to watch a 5-minute HFS DVD; complete a pre, post, and 2-week follow-up phone call. In addition, in-home fire safety checks using the Home Fire Safety Checklist were conducted (Lehna et al., 2015a, Lehna et al., 2015b).

At the time of the HFS check some participants of the parent study provided researchers permission to take photographs of “safe” and “unsafe” home fire hazards within their residence. Initially, to examine the content validity of the photographs, two fire inspectors independently rated the photographs (inter-rater reliability). The local fire inspectors rated all the photographs twice, two-weeks apart in time (test-retest reliability) (Lehna et al., 2015c). Inter-rater reliability between the two fire inspectors was 0.92, and the intra-rater reliability was 0.91. When community participants and HFS team members rated the photographs the inter-rater reliability rating was 0.76 using Krippendorff's Alpha Coefficient (Lehna et al., 2015c).

As community members rated the photographs for content validity, team members found that the photographs evoked discussion between older adult participants about similar situations from their past (Lehna et al., 2015c). Including photographs in a HFS education program could potentially increase knowledge retention adding to improvement in HFS practice, especially for those who were older, had lower literacy, or for whom English was a second language.

With the exception of the two studies (Lehna et al., 2015a, Lehna et al., 2015b), assessment of HFS practices remains fragmented and disjointed. Researchers mainly focus on smoke alarm placement and functionality (Gielen et al., 2013, Miller et al., 2014, Parker et al., 2013, Sidman et al., 2011). The research team is attempting to shift the paradigm to prevention (eradicating unsafe practices) from a traditional reactionary approach (fire alarms). There was a need to further evaluate the effectiveness of the photographs obtained in the two previous studies (Lehna et al., 2015a, Lehna et al., 2015b) with both a local and a national group of burn prevention experts.

Methods

Measure

For the current project focusing on photographic instrument development, 27 photographs were grouped according to the United States Fire Administration's Home Fire Safety Checklist (United States Fire Administration) and presented in two formats: as a PowerPoint presentation and as a poster. Expert participants used either presentation format for their evaluation. Participants rated whether photographs were “safe”, “unsafe”, and “unable to determine”. While n = 18 (75%) photographs were anticipated to be viewed as unsafe, n = 6 (25.0%) were anticipated to be viewed as safe. The authors purposively set the allocation rate at 3:1. The ordering of photographs was randomly determined (see Table 2).

Table 2

Photograph alphas.

Photo	Krippendorff α
1	0.777
2	0.785
3	0.821
4	0.777
5	0.874
6	0.785
7	0.816
8	0.618
9	0.621
10	0.827
11	0.906
12	0.832
13	0.833
14	0.846
15	0.832
16	0.806
17	0.786
18	0.774
19	0.706
20	0.829
21	0.809
22	0.851
23	0.853
24	0.894

`For 24 of the photographs, the 73 home fire safety respondents simply recorded if they felt the practice in the photograph was “safe” or “unsafe.” For the three additional photographs, the practice was clearly unsafe and the respondents answered an open-ended question to describe the unsafe practice. Examples of some of the unique HFS photographs include: unsafe cooking practices (see Photograph 5); unsafe candle safety (see Photograph 7); and safe fire escape practice (see Photograph 20).

Photograph 5

Unsafe cooking practices.

Photograph 7

Unsafe - Candle in shallow container; surrounding flammable materials potential fire hazard.

Photograph 20

Safe – In-home posted fire escape plan.

Procedure

This anonymous survey was institutional review board exempt. Return of the completed forms considered consent. Initially, a Krippendorff's alpha was calculated for the first 24 questions to evaluate inter-rater reliability, and differences in demographics were evaluated. Unique codes and themes for the last three questions were identified and inter-rater reliability examined (Photograph 25, Photograph 26).

Photograph 25

Unsafe - Electrical strip without surge protector, cords under mat.

Photograph 26

Unsafe - Flammable materials and chemicals on dryer.

Sample

Participants were recruited at the 2015 American Burn Association Annual Conference and Meeting. At the Burn Prevention Committee meeting, the PowerPoint presentation was presented to members and photographs were rated. The PowerPoint presentation also was emailed to all members to have experts rate at their home location. The poster was available in the exhibit room at the Burn Prevention booth for any participant to judge. Completed response sheets were either submitted in-person or by email to the principal investigator (PI).

Locally, fire inspector participants were recruited from an urban central fire state headquarters during roll call. Recruitment also occurred at a monthly Safe Kids Coalition meeting where members completed the evaluation while either viewing the PowerPoint presentation or examining the poster.

Results

A majority of respondents were female (n = 43, 60.7%), college educated (n = 61, 83.6%), nurses (n = 25, 33.8%) or worked for a fire department (n = 21, 29.6%). Their mean age was 45.5 years and they had 11.1 years of burn prevention experience (see Table 1).

Table 1

Demographics of expert panel.

Categorical variable	N = 73 (%)
Female	43 (60.7%)
Education
HS/GED	12 (16.4%)
College	42 (57.5%)
Advance degree (Masters, PhD, MD)	19 (26.1%)
Profession
Fire Department Staff	21 (29.6%)
Clinician (RN, MD)	31 (43.7%)
OT/PT	6 (8.5%)
Other (law, research, admin, etc.)	13 (18.3%)
Burn Prevention	44 (60.3%)
Continuous variable	Mean (SD)
Age	45.5 (11.8)
Yrs. experience	11.1 (8.7)

The first 24 questions had high inter-rater reliability (Krippendorff α = 0.831). Table 2 shows the individual alphas for the respective photographs.

As shown in Table 3, a great deal of agreement in response/view (safe vs. unsafe) of the individual photographs occurred (average agreement of 89.6%). No significant differences existed between the strata of the demographic variables and responses of safe vs. unsafe (all p-values > 0.05).

Table 3

Most popular responses and agreement for individual photographs.

Photograph	Most popular response	Percent of same response among panelN = 73 (%)
1	Safe	68 (93.2%)
2	Unsafe	70 (95.9%)
3	Unsafe	72 (98.6%)
4	Unsafe	69 (94.5%)
5	Unsafe	72 (98.6%)
6	Safe	62 (85.0%)
7	Unsafe	72 (98.6%)
8	Safe	62 (84.9%)
9	Unsafe	70 (95.9%)
10	Unsafe	65 (89.0%)
11	Safe	62 (84.9%)
12	Unsafe	70 (95.9%)
13	Unsafe	49 (67.1%)
14	Unsafe	73 (100.0%)
15	Unsafe	67 (91.8%)
16	Safe	49 (67.1%)
17	Unsafe	66 (90.4%)
18	Unsafe	67 (91.8%)
19	Unsafe	69 (94.5%)
20	Unsafe	70 (95.9%)
21	Unsafe	41 (56.2%)
22	Unsafe	72 (98.6%)
23	Unsafe	70 (95.9%)
24	Safe	66 (90.4%)

Based on the codes and themes identified, the final open-ended questions had moderate-to-good inter-rater reliability (Krippendorff α = 0.764). Participants who responded (n = 69, 100%), correctly identified in photograph 25 two themes, the power strip was without the surge protector (n = 59, 85.5%) and cords were running under the mat (n = 10, 14.5%). For photograph 26, participants described two themes, the fire hazard was flammable chemicals (n = 46, 67.6%) and potentially flammable materials on the dryer/washer (n = 16, 23.5%). For photograph 27, the respondents had one theme, a wiring/cord issue (overload, too many wires/cords, etc.; n = 59, 90.8%).

Discussion

This study further extends previous mixed method predominantly qualitative (ethnographic focus) support of photographic analysis obtained during HFS checks (Lehna et al., 2015c) through further validation with experts. There is a paucity research on fire safety practice within individual homes. Currently, HFS education programs lack comprehensive approaches for teaching and assessing home practice, often focusing on one or two aspects of HFS (e.g., smoke alarms, carbon monoxide alarms, scald prevention) (Gielen et al., 2013, Miller et al., 2014, Parker et al., 2013, Sidman et al., 2011, McDonald et al., 2013, Yang et al., 2006). Findings from this study, moderate to strong evidence of inter-rater reliability and validity, provide evidence to support use of “safe” or “unsafe” photographs for the basis of future HFS curriculum development (e.g., story book or PowerPoint formats). Use of multi-method education strategies has been found to be successful in teaching parents.

Reimer and Kagan in a series of three consecutive studies developed and tested the effectiveness of a burn prevention curriculum for Amish children (Reimer and Kagan, 2012a, Reimer and Kagan, 2012b, Reimer and Kagan, 2013). By survey, mothers identified burn hazards unique Amish communities (Reimer and Kagan, 2012a). Using the survey information an education curriculum was developed which involved teachers telling stories and running through scenarios with an open-viewed doll house as a game to identify home burn hazards. Change in burn prevention knowledge was measured through pre- and post-testing (Reimer and Kagan, 2012b, Reimer and Kagan, 2013). Using a curriculum based on this unique community's burn prevention needs in different formats (storytelling and open-viewed doll house) was shown to be highly effective and culturally accepted (Reimer and Kagan, 2012b, Reimer and Kagan, 2013).

Sinha and colleagues evaluated the effectiveness of “Tales of Burn Safety” comic book with two groups of school aged children, one from West Virginia (n = 74) and the other from West Bengal (n = 39). Change in student learning was evaluated by pre- and post-testing (Sinha et al., 2011). In implementing a multi-learning approach (words with supporting graphics) cues are both written and visual for learners which the authors found to be effective.

Conclusion

Using ‘seek-and-find’ or ‘What's wrong with this picture?’ tools and simplified visual images aid in recognition of unsafe home environments. Fire and burn prevention education through multi- and non-traditional methods for older adults, migrant and refugee populations, or those having low literacy increases the possibility of change.

Transparency document

Transparency document.

Conflict of interest statement

All authors declare no conflict of interest.