Safe Food Management and Smartphone Technology: Investigating the Impact of an App on Consumer Knowledge Retention.

OBJECTIVES: Diffusion of smartphones has normalised consumers' use of mobile applications (apps). But how do app designs and contexts of use interact with differential consumer attributes to impact on their effectiveness, usability and value over time? For consumer food safety, answering these questions is of importance as numerous food choices increase challenges in safe food management (SFM). This research reports on results of a randomised field experiment with Australian consumers using an SFM mobile app developed by the researchers.
METHOD: The SFM app development employed insights from the Health Literacy Online Heuristics framework and the experiment involved evaluation of information and/or knowledge acquisition from the app versus from a paper-based version. The experiment spanned four weeks and involved eight participants (experimental group n=4; control group n=4).
RESULTS: The results highlight differentials in cognitive burden between paper and the app; beneficial affordances from the app for refreshing consumer knowledge; and longer knowledge retention on safe food management from app use over-time. DISCUSSION: We identified two key impacts of the app on consumer knowledge acquisition and knowledge retention. First, the SFM app takes longer to achieve knowledge acquisition but results in longer knowledge retention than the control. Second, the SFM app induces some level of cognitive load in adoption however; the affordance of its reuse for quick but infrequent revisitations facilitates knowledge retention. Although the study is limited by the small sample size, it however highlights the need for a large scale and purely quantitative investigation that are generalisable to the Australian population.
CONCLUSION: It is anticipated that the insights gained from this study can be used to develop nationwide interventions for addressing consumer SFM knowledge gaps in the home; thus, moving a step closer towards addressing SFM behaviours of Australian consumers.

Introduction

Alleviating food safety risks is a major source of concern for government authorities, the food industry and increasingly many consumers. Diverse mechanisms focused on monitoring and controlling production processes across supply chains, such as ISO22000 [1], HACCP control systems [2], HARPC control systems [3], traceability systems [4], have been employed to address many of these risks. However, most of these food safety mechanisms are largely focused on supply chain activities from ‘paddock-to-purchase’ (pre-purchase) as the legal obligations of supply chain partners on food safety tends to be completed once consumers purchase the products [5]. Thus, mitigating food safety risks during the process of consumption (post-purchase), that entails domestic food management from the point of purchase (purchase) to the point of actual consumption (plate), is largely the responsibility of consumers.

Although the unsettling level of food poisoning outbreaks through domestic food mismanagement from ‘purchase-to-plate’ is not a new phenomenon, the complexity and dynamism of the characteristics of available foods and diversity of consumers has made it very difficult to address. For example; the varying degree of food safety knowledge has facilitated many public food safety information campaigns, education and awareness programs [6]. Despite these efforts, many consumers remain inadequately informed about food safety and continue to engage in unsafe food handling practices.

There are a range of approaches to support consumers in safe food management including using information and communication technologies (ICTs), insights from consumer behaviour theory, knowledge management practices and food safety management guidelines. The widespread diffusion of smartphones has now normalised the adoption and use of mobile applications [7]. The highly personalised nature of smartphones embody a potential user-empowering characteristic [8], thus providing users with an array of capabilities and experiences that can be tailored to their interests. Downloading apps onto their smartphones [9] affords consumers the opportunity to inform themselves about specific areas of interests [10] including safe food management (SFM). Consumers can inform themselves about food in terms of tasks such as personalized grocery shopping apps [11], food cooking apps [12] and food storage or wastage apps [13]. This stated, a key question that arises is how do app designs and contexts of use interact with differential consumer attributes to impact on their effectiveness, usability and value over time? These issues can be examined in three ways.

First, in terms of the context of use, there is evidence that existing apps provide siloed information about the various aspects (safe shopping, transportation, storage and preparation of perishable food items and appropriate kitchen hygiene practices) of domestic SFM for Australian consumers [14]. Second, in terms of user experiences, there is insufficient evidence that existing apps have drawn upon information modality studies that highlight differences arising from use of textual [15], visual [16], verbal [17] or integrated information modalities on consumer behaviours pre-purchase. Aligned to these studies is the principle of modality effect [18], which argues that materials presented in a format that simultaneously uses the auditory and the visual sensory modality is better than by a format that uses only the visual modality [19]. However, available evidence suggests the use of this principle only within pedagogical frameworks [20] thus, it is unclear if this principle is applicable to adult consumers and whether it will improve user experience during the use of SFM apps. Third, there is insufficient evidence to suggest that existing apps in SFM have been comprehensively evaluated [21] or that they were developed based on frameworks guiding mobile health consumer apps. This lack of evidence raises questions about whether best practice guidelines were adhered to. Significantly, there is a dearth of research that assesses how well the content of the app has been designed for consumers with considerations for both usability and health literacy. The few evaluations that have been conducted have been restricted to usability assessments and marginalised contexts of use and consumer attributes and behaviours.

This research reports on results of a randomised field experiment with Australian consumers using a safe food management (SFM) mobile app developed by the researchers to explore these issues more comprehensively. The next section describes the method used in the conduct of this research.

Method

This research adopted an interpretive research philosophy and deployed a mixed-method design structured in three overlapping phases. Phase 1 (Consumer Understanding) involved the conduct of a nationwide survey to identify problems with the current food handling practices of Australian consumers and their information and communication preferences (both pre-purchase and post-purchase). The findings of this survey have been previously published [5]. This led to the selection of three existing apps (text-based, graphics/picture-based and integrated) that most clearly address the SFM practice being targeted to provide insight into consumers preferred styles of design.

Phase 2 (Design) involved the heuristic evaluation of the three existing apps based on Monkman and Kushniruk [22] Health Literacy Online Heuristics (HLOH) framework to identify problems with the apps from an expert’s perspective. Following this, a second usability evaluation from the consumers’ perspective, using the apps as a high-fidelity prototype in scenario-based focus group sessions, was conducted. This research activity aimed to identify the impact of the three information modalities on consumer understanding and to generate user requirements for a new app. The outcome of this phase, which has been accepted for publication elsewhere, provided rich insights into consumer requirements for a safe food management app. This led to the design of a single smartphone application (shown in Figure 1) for educating and assisting consumers on the SFM practices.

Figure 1

An enhanced user-centred design approach Source: Authors

Phase 3 (Implementation and Evaluation) involved the actual implementation and evaluation of the app designed in Phase 2. Phase 3 is the focus of this research paper. After the SFM app was developed, it was evaluated by conducting a randomised field experiment, within a 4-week period. The aim of this research activity was to evaluate the impact of the design on the retention of knowledge on SFM practices over time.

Research Design

Randomised field experiments “allow researchers to scientifically measure the impact of an intervention on a particular outcome of interest through random assignment of study subjects” [23]. It has been argued that randomised field experiments are the ‘gold standard’ as they yield the most accurate analysis of the effect of an intervention [23]. Of these techniques, stratified randomization was deemed most appropriate for this research because it addressed the need to balance and control the influence of co-variates in order to avoid any risk to the conclusions of the study [24]. Whilst this method is difficult to implement for larger studies, it is deemed more appropriate and simple for smaller studies with limited sample sizes [25]. Moreover, it is also appropriate for this study because all the participants were identified through the recruitment process before group assignment [24]. Therefore, like Skarphedinsson, Weidle [26], the authors chose to incorporate stratified randomization.

The two key co-variates that might influence the research are gender and age group. In this context an inclusion criterion for each potential participant to fulfil is their ability to purchase and cook meat in their own homes. There is evidence to support the argument that food preparation is still a strongly gendered household task [27]. In agreement, Worsley, Wang [28] have argued that cooking remains a female responsibility in Australia, thus portraying the importance of gender as a co-variate in this study. Second, the other criterion is the ownership and use of smartphones. There is also evidence to support the argument that electronic channel preferences through the use of smartphones is a higher preference amongst younger Australians [28], thus portraying the importance of age group as a co-variate in this study.

To randomize participants, a stratified randomization procedure was applied using gender and age group as stratification variables, to provide a total number of strata of six as much as it was possible based on the available participants. Following this, each participant was selected through simple randomization. To ensure that randomization could not be predicted in advance, the randomization procedure utilised tags only.

Participant Recruitment

Participants met the study inclusion criteria if they purchase and cook red meat; if they have a smartphone (specifically an android phone 4.0.3 or an iPhone 4 and newer versions) and if they are Australian adults (18 years and above). The recruitment efforts spanned across three months and delivered a final group of 12 participants out of which 4 dropped out due to family related issues.

Procedure and Research Instrument

Knowledge optimisation involves ensuring that knowledge is acquired, retained and can be applied. Drawing upon the revised version of Bloom’s Taxonomy of meaningful learning [29], achieving knowledge optimisation requires three of the six cognitive processes; remember, understand and apply. In this study, ‘remembrance’ demonstrated the level of knowledge acquired, ‘understanding’ demonstrated the level of knowledge retained and ‘application’ demonstrated the level of knowledge applied. These were assessed using multiple choice questions for knowledge acquisition; a problem based learning approach using scenario-based questions for the knowledge retention; and knowledge application process.

Following on from similar studies [30], ‘remembering’ has been evaluated after the use of a mobile app for knowledge acquisition [31]. In a study by Ahmed and Parsons [31], their method involved quantitative assessment through a post-test that was delayed for two-months after the instructional period. They also used questionnaires for the pre and post-tests. Furthermore, ‘understanding’ has been evaluated after the use of a mobile app for knowledge acquisition in many studies [32]. What these studies have in common is their use of pre- and post-test format and multiple choice or short answer questions to assess conceptual understanding [30]. Their questions are typically derived from a curriculum, a standardized test, or created by experienced teachers or researchers. In addition, ‘applying’, which is also known as ‘knowledge application’ has been evaluated after the use of a mobile app for knowledge acquisition [33]. In a study by Hwang, Tsai [33] their method also involved the use of questionnaires for pre and post-tests. It is however worthy to note here that the aforementioned studies on ‘remembering’, ‘understanding’ and ‘applying’ have been conducted based on pedagogical frameworks, as none of those studies have been conducted based on adult learning frameworks situated within the SFM space focused on consumers.

On Day 1 (Pre-Test), the 2-hour session started with briefing the participants, providing them with the information sheet and consent form. Next, they were provided a 20-item baseline questionnaire which was collected from them after it was answered. Following this, those in the experimental group were separately asked to download and install the SFM app on their phone while those in the control group were given a paper-based document. They were asked to use the learning material (app or document) to answer a copy of the unanswered baseline questionnaire given to them. After completing this activity, they were asked to brainstorm on the facts learnt from the learning material within their group. At the end of this session the baseline questionnaire was collected from them, they were allowed to take the learning material home and they were de-briefed. Care was taken to ensure the participants in the control group did not have access to the app while those in the experimental group did not have access to the paper-based tool. On Day 8 (Post-test 1), the same baseline was presented to the participants but with re-ordered questions and answer options. They were not allowed to refer to any learning material. On Day 16 (Post-Test 2), open-ended scenario-based questions that are directly related to the base-line questionnaire were presented to the participants and they were asked to provide short answers to each of the 4 questions. On Day 24 (Post-Test 3), participants were presented with open-ended scenario-based questions that are directly related to the base-line questionnaire but based on raw food products in a kitchen environment. They were asked to provide short answers to each of the 4 questions. This ended with a de-brief. Details of the questions are not provided due to space constraints.

After the data collection, the data was exported to Microsoft Excel 2010 for initial formatting and then imported into IBM SPSS software version 22.0 for better analysis. The data for Weeks 1 and 2 were mainly analysed using descriptive statistics while the data for Weeks 3 and 4 were first analysed manually based on the correctness of the answers before importing the scores to SPSS for descriptive analysis.

Results

Demography of the Participants

All the respondents live in Hobart, Tasmania, Australia and they are above 18 years of age. The eight participants (4 males and 4 females) were divided into two groups of 4 persons each for the experimental group (app users) and the control group (paper-based tool users). In the experimental group, the highest educational qualification of three of the participants is Bachelor or higher, while the fourth participant has a Diploma or Advanced Diploma. All candidates within the control group have a Bachelor or higher educational qualification. This is important as it suggests that the participants are learned, and they can easily access, read and understand text presented to them in the experiment.

Smartphone Usage

For the experimental group, all participants within the experiment group own and use a smartphone. 50% of the participants are Android phone users while the others use iOS-based phones. 50% of the participants have been using a smartphone for more than 4 years while the others have been using smartphones for more than 2 years but less than 4 years. 50% of the participants consider themselves medium smartphone users, 25% regard themselves as light users while 25% believe they are heavy users. For the control group, all participants within the control group own and use a smartphone. 75% of them are iOS-based phone users while the others use Android phones. 75% of them have been using a smart phone for more than 4 years while the others have been using smartphones for more than 2 years but less than 4 years. 50% of the participants consider themselves medium smartphone users, 25% regard themselves as light users while 25% believe they are very heavy users.

Therefore, it is reasonable to state that these participants are familiar with the use of smartphones and mobile phone apps. Thus, suggesting that they will be able to easily access an app presented to them in the course of the experiment.

Food Handling

For the experimental group, all participants have mixed diet which includes red meat and white meat, which shows that none of them are vegetarians or vegans. 50% of them purchase their meat products from supermarkets, 25% from fresh food markets and 25% from delicatessens. All participants within this group cook raw meat products at least once a week. Therefore, this shows that the participants within this group are food handlers in their homes.

For the control group, 75% of the participants have mixed diet which includes red meat and white meat, but 25% have mixed diet which includes only red meat. This shows that none of them are vegetarians or vegans. 75% of them purchase their meat products from supermarkets and 25% from fresh food markets. 75% of the participants within this group cook raw meat products at least once a week while others cook raw meat products at least once a fortnight. Therefore, this shows that the participants within this group are food handlers in their homes.

Experiment Findings

This section presents the findings for each week of the field experiment which was conducted to evaluate participants’ level of knowledge acquisition, knowledge retention and knowledge application.

Pre-Test – Week One

For all 20 questions all participants were told to select the correct answer based on their current knowledge. Each question represents one point. The mean score of the experimental group was 13.25, while the mean score for the control group was 14.75. This reveals that participants in the control group had a better pre-existing knowledge of safe food handling in the home when compared to the experimental group.

Post-Test 1 – Week Two

The follow up questionnaire (post-test 1) is the same as the baseline questionnaire but the only difference is that the questions and answer options are re-ordered. Therefore, there were 20 questions and each participant in both groups was told to select the correct answer based on their current knowledge. Each question represents one point. The mean score of the experimental group was 17.5, while the mean score for the control group was 19.75. This reveals that participants in the control group were able to remember what was learnt in the previous week better than the experimental group.

Post Test 2 – Week Three

In week three, the participants were presented with scenario-based questions that were drawn from, and strongly aligned to, the baseline questionnaire in Week One. The focus of this week was for the participants to demonstrate their understanding of the acquired information in the previous weeks. The format of the scenarios would appear familiar to them. There are 4 scenarios, with one scenario for each question. Each question is assigned 5 points and points are allocated to each participant based on the correctness of their response. The answers for each question are drawn from the smartphone app or paper-based tool which was provided to the participants in the previous weeks. The mean score of the experimental group was 16.875, while the mean score for the control group was 16.25. This reveals that participants in the experimental group were able to demonstrate a slightly better understanding of what was learnt in the previous weeks better than the control group.

Post-Test 3 – Week Four

In week four, the participants were presented with open-ended questions based on real scenarios presented to the participants in a kitchen environment. The questions were drawn from, and strongly aligned to, the baseline questionnaire in week one. The focus of this week was for the participants to apply the knowledge they had acquired in the previous weeks. There were four scenarios, with one scenario for each question. For each scenario, a table is presented to each participant with a certain arrangement of food products to support the question being posed. The mean score of the experimental group was 16.375, while the mean score for the control group was 14.875. This reveals that participants in the experimental group were better in applying the knowledge gained within the previous weeks than the control group.

Discussion

As the overarching aim of the research is to provide insights into how best to share information to facilitate knowledge retention through the use of technology in an attempt to improve the food management behaviour of Australian consumers, it was imperative to investigate the impact of the SFM app on consumers. Whilst two tools (paper-based and app) were involved in the study, the focus was on the app as the paper-based tool was used as a baseline, which contained only textual information modality but the app contained multiple information modalities (text, pictures and videos). Therefore, there was a need to understand the impact of the SFM app on consumer knowledge acquisition and knowledge retention on SFM.

Impact 1: The safe food management (SFM) app requires more time to be spent to achieve knowledge acquisition which resulted in retaining the knowledge for a longer period of time than the traditional information delivery techniques.

The authors draw on the cognitive load theory as the tasks and learning activities in the study required simultaneous integration of multiple and various sets of knowledge, skills and behaviours at a specific time and place [34]. The cognitive load theory (CLT) integrates three key components of the cognitive architecture: memory systems (sensory, working and long-term memory (LTM)), learning processes and types of cognitive load (intrinsic, extraneous and germane) imposed on working memory (WM) [35]. Extraneous cognitive load refers to the burden imposed on the working memory of the learner which is not essential to the task [34]. This load tends to arise when learners use an app at first sight which leads to a distraction that is not related to the knowledge acquisition task.

As the initial use of a smartphone app induces a higher level of extraneous cognitive load, this places a level of demand on the working memory and reduces the rate at which knowledge acquisition occurs. According to Brunken, Plass [18], extraneous cognitive load occurs due to the format and manner of information presentation and the requirements of the instructional activities on the working memory. However, this type and level of cognitive load does not occur when a traditional information delivery technique is used, as evidenced by this study.

It was however discovered that the app users demonstrated a higher level of knowledge retention over time when compared to the document users. This could be explained by the split-attention effect in relation to the cognitive load theory. This effect involves the phenomenon whereby the physical integration, rather than physical separation, of verbal and pictorial information sources enhances learning [36]. However, when split attention occurs, it increases demands on the learner’s working memory (WM) and has the tendency to impact learning negatively [37]. One way to avoid the split attention effect is by externally integrating the different sources of information together into a single integrated source of information [37] as was achieved with the SFM app. It is believed that this strategy was instrumental to the successful outcome of the level of knowledge retention emanated by the participants.

The app contained videos of SFM practices that incorporated the modality effect as the visual figures are linked with auditory (spoken) rather than visual (written) elements [38]. Mayer [38] has argued that the modality effect can only occur under the condition in which the multiple sources of information are unintelligible in isolation and rely on each other for intelligibility to avoid the redundancy effect. This condition was met by the videos included in several pages of the app as they comprised of picture frames (visual elements) and spoken elements that rely on each other for intelligibility; thus complementing the features portrayed by one another [39].

Initially, more time was spent on the app used in this study but the rate at which information and/or knowledge was acquired was lower than that of document users. However, more in-depth details revealed that the app users acquired the knowledge slowly but retained it longer in contrast to the document users. These findings are in line with the study conducted by Herrlinger, Höffler [40] and Leahy and Sweller [41] who have argued that pictures and spoken text enhanced learning better than written text. Similar to this finding is the study conducted by Wang, Tsai [42] which revealed that when more attention was paid to the video and less attention paid to the text there was better retention of the learning outcomes. However, the findings in this study differ from those of Chandler and Sweller [43] who found that students viewing integrated instruction spent less time processing the materials as the app users in this study spent more time acquiring the knowledge due to the extraneous cognitive load which occurred as a result of the additional learning that was required for the initial use of an app. Nonetheless, Chandler and Sweller [43] also agreed that students viewing integrated instruction outperformed those with split attention condition. On the other hand, the findings are in line with the study conducted by Schmidt‐Weigand, Kohnert [44] who also revealed that participants showed a better learning performance the more time they spent looking at visualizations when text was spoken and integrated.

Therefore, in consonance with Schmidt‐Weigand, Kohnert [44], it can be argued that the time devoted to process visualizations with spoken and integrated text such as videos may be an indicator of the quality of processing this information. From this perspective, this study suggests that the time a learner spends in using an app containing visualizations with spoken and integrated text such as it is featured in the safe food management (SFM) app, during the information and/or knowledge acquisition phase, may be advantageous in facilitating knowledge retention for a longer period of time than traditional information delivery techniques.

Impact 2: The SFM app induces some level of cognitive load in adoption however; the affordance of its reuse for quick but infrequent revisitations facilitates knowledge retention.

This research has found that the initial use of the smartphone app which was developed for information and/or knowledge acquisition purposes induces a higher level of extraneous cognitive load; thus reducing the rate at which knowledge is acquired during the first use. According to Brunken, Plass [18], extraneous cognitive load occurs due to the format and manner of information presentation and the requirements of the instructional activities on the working memory. Cognitive load was discovered in this study as evidence suggests that participants using the app experienced a level of demand on the working memory. Based on arguments from Moreno and Mayer [45], that the principle of modality effect can indeed reduce extraneous cognitive load for knowledge acquisition tools developed on mobile devices, this study incorporated the principle. Yet, the results indicate that some level of cognitive load was induced. Although the evidence is lacking, it appears to the authors, that the HLOH framework seem to have minimized the cognitive burden. Thus, there was a better demonstration of knowledge retention after the app has been reused over a short period of time. When participants spent less time on the smartphone app after the initial use, they demonstrated better retention of knowledge whereas spending more time on the paper-based tool resulted in poorer retention of information and/or knowledge.

This finding is in line with the temporal patterns that have been identified in the usage of smartphones and their applications which suggests short bursts of smartphone interactions [46]. For instance, Yan, Chu [47] found that mobile phone usage is brief as half of mobile phone engagement (time between unlocking and relocking) lasts less than 30 seconds. Similarly, Ferreira, Goncalves [48] found that some apps are used in short bursts of less than 15 seconds. Also, a large scale study by Böhmer, Hecht [49] revealed that smartphone devices are used for an average of 59 minutes daily while an average application session lasts 72 seconds. With a focus on overall smartphone users’ habits, Oulasvirta, Rattenbury [50] suggest that smartphones are “habit-forming” devices as users emanate the “checking habit” through brief inspection of content quickly accessible on their smartphones. A follow up study by Ferreira, Goncalves [48] revealed that this habit is one of the behavioural characteristics that leads to short bursts of interactions with applications. In addition, this habit has largely been focused on users making quick revisits to applications that contain fast changing content [48,50]. However, Jones, Ferreira [46] has argued that apps that relate to personal activities such as food handling and food management follow a slow revisitation pattern. As such, this explains the slow revisitation pattern and the little time spent on the SFM app during its subsequent use in this study. Thus, as this facilitated a better demonstration of knowledge retention on safe food management, it suggests that the affordance of re-use for quick but infrequent revisitations facilitates knowledge retention.

Therefore, as it has been earlier argued that multiple information channels enhance food safety information dissemination [51], it can be further argued that other information channels such as TV adverts, brochures, pamphlets and other media can be useful in drawing attention to the reuse or revisitation of such smartphone apps to reinforce and support the retention of consumer knowledge. This indicates that optimising consumers’ safe food management knowledge cannot be a one-off activity as they require cues that prompt them into revising the app so as to maintain adequate knowledge level from time to time.

Limitations

Due to the difficulty in recruiting a sample that was representative of the Australian population, participants were limited to consumers in Hobart, Tasmania; thus, the outcome of the research may be skewed. Based on this small number of participants, the findings of this study cannot be generalised to the Australian population and it may lead to a possibility of potential alternative explanations for the findings which favoured the use of the app rather than the document for knowledge retention. As such further large-scale studies would need to be conducted based on a sample that is representative of the Australian population.

Conclusion

This research was focused on investigating how the affordances of smartphone technology can be leveraged to enhance the provision of information and facilitate knowledge retention as a step towards improving the SFM behaviour of Australian consumers. This paper has presented findings from a randomised field experiment using a developed SFM app for information and/or knowledge acquisition as the intervention and a paper-based document as control with assessments conducted at baseline, Week 2, 3 and 4. We identified 2 key impacts of the app on consumer knowledge acquisition and knowledge retention. First, we discovered that the safe food management (SFM) app requires more time to be spent to achieve knowledge acquisition which resulted in retaining the knowledge for a longer time than the traditional information delivery techniques. Second, we found that the SFM app induces some level of cognitive load in adoption however; the affordance of its reuse for quick but infrequent revisitations facilitates knowledge retention. It is anticipated that the insights gained from this study can be used to develop nationwide interventions for addressing consumer SFM knowledge gaps in the home; thus moving a step closer towards addressing SFM behaviours of Australian consumers.