Design and development of an immersive virtual reality news application: a case study of the SARS event.

In recent years, virtual reality (VR) technologies have been applied to the field of journalism, where the concept of immersive VR news has been proposed. However, despite the fanfare, strong response, and sensational effect caused by its advent, immersive VR news remains a novel journalism paradigm that faces new challenges in its production process. Currently, there is a lack of a unified design framework, and, since most studies in this area have focused on non-interactive VR news, the understanding of the effects of more interactive VR technologies on the news consumer remains inadequate. In this study, we propose a more practical design framework for immersive VR news products. Following this framework, we designed a VR news application and conducted user evaluation in terms of media effects and user experience. Based on the experimental findings, which demonstrated that non-interactive VR news products resulted in a distracting user experience and less immersion, while interactive VR news offered improved media effects and user experience, we then derived concrete design guidelines for immersive VR news design. Finally, we highlight that this study provides a theoretical and practical reference framework for the further study of VR news. © Springer Science+Business Media, LLC, part of Springer Nature 2020.

Introduction

Immersive journalism refers to news that aims to establish a connection between the audience and the news story by offering first-person experiences of events or situations [35]. Generally, the sense and illusion of embodiment is created through spatial perception as digital avatars, which is the basis of immersive journalism [35, 41, 43]. “Immersion” in journalism has evolved from the narrative strategies used by journalists to arouse audiences’ empathy in the early practice into a multi-channel experience utilizing the technology changes of recent years [8, 10, 42].

Currently, immersive journalism is mainly presented in two forms [25, 28, 32]. One is 360° video, which can immerse audiences in real news scenes captured by photographic equipment. Since this new product form is easy to develop, it has been widely used in the media industry; however, the lack of interaction and embodiment results in lower user engagement in traditional 360° video news. The other is known as VR news, which generates a digital experience that closely simulates the real world using virtual reality technologies. The interaction between users and virtual environments in VR news ensures a stronger sense of immersion [14].

While immersive technologies are advancing, there is still insufficient clarity on the effects these technologies have for the news consumer. As a new form of news communication paradigm, there are still many open questions in the design and development of immersive VR news. For example, virtual reality technology has raised new requirements for the traditional news production process [14, 28, 45], and the change of perspective caused by the disappearance of the physical screen and the appearance of interactive elements brings a new challenge to the narrative frame of journalism [13]. At the same time, academia is also exploring whether the application of VR in news may impair professionalism in journalism. For example, a deliberately misleading or fabricated perspective can impair the ability to convey accurate and true facts to the audience during an immersive experience [25]. Therefore, a set of production process standards and design specifications is required for the further application of immersive VR news.

In this study, we explore the immersive VR news production process and propose concrete design guidelines through a practice case study. Our work includes 1) the production of immersive VR news investigated under different conditions (VR news with and without interaction); 2) a deeper understanding of user experience and media effects in immersive VR news, and 3) design guidelines for immersive VR news. We hope that our research findings will enable practitioners to design and develop more effective applications and consequently create a better interactive experience for immersive VR news products.

Related work

Our work primarily concerns research related to the evaluation of news production and systems; thus, our literature review focuses on work in these areas, including traditional news production and existing design practices in VR news products.

Gatekeeping theory

Currently, there is no general production process for traditional news. In general, the production process is flexible and variable to meet the needs of different news topics. Paterson and Domingo [34] divided the online news production process into five stages based on ethnography, namely 1) interview and observation; 2) selection and filtering; 3) processing and editing; 4) distribution, and 5) interpretation. In such a process, the stages of news selection, filtering, processing, and editing are guided by the gatekeeping theory proposed by White [49]. Gatekeeping refers to the system through which countless pieces of information are screened and processed into a limited number of messages and subsequently disseminated to the audience [44]. White [49] claimed that news stories must be strictly selected and filtered by gatekeepers in the news selection and filtering stage before they move on to processing. Bass [2] further emphasized that news information processing should also be guided by this gatekeeping theory. More recently, gatekeeping theory was suggested to be involved in the news distribution and interpretation stages as well [38], and even any decision-making process about the encoding of the information [9].

Virtual reality application in journalism

Many immersive VR news products have emerged in recent years. We collected previous applications and analyzed the component factors of news products, including the themes, narrative frame, and mode of user experience and media effects. As for news themes, current VR news products mainly focus on events that reflect social problems, e.g., Hunger in LA1; important historical events, e.g. Aftermath VR: Euromaidan,2 and some subjects with popular science value that are difficult for ordinary people to access or experience, e.g., Home: A VR Spacewalk.3 Regarding narrative frame, the production of VR news is moving away from the linear and planned storytelling structure adopted by traditional news: In immersive VR news, the audience can not only understand the whole news event from an omniscient perspective (e.g., Kiya)4 but also participate in the news story as bystanders (e.g., Use of Force)5 or even become the protagonist to experience the feelings of the parties involved (e.g., Across the Line).6 In terms of the mode of user experience and the effects of this media, the degree of user participation in the news varies in different cases. Current mainstream immersive VR news mostly focuses on non-interactive products that only allow users to “visit” or experience news events from a bystander’s perspective, although some recent products enable the users to interact with 3D objects in the virtual environment (e.g., Remembering Pearl Harbor).7

Evaluation of immersive VR news

Existing evaluations of immersive VR news have primarily focused on 1) immersion and empathy from the perspective of the user experience, and 2) accuracy, credibility, and audiences’ emotional attitude from the perspective of the media’s effects.

Immersion

Although VR technologies have been recently used to convey news stories in immersive journalism, how the audience experiences these stories remains unclear. Implementing an immersive experience is an important goal in a computer-mediated environment [29] and immersion can be used to characterize the technical support of the VR environment [41]. Slater et al. [46] claimed that walking in interactive space can enhance participants’ experience in terms of presence and immersion. Moreover, increased audience satisfaction, greater credibility, and a superior interactive experience can be achieved using VR interactive devices capable of triggering a high degree of immersion. This immersion largely depends on the user’s characteristics, intent, and context rather than the definition by the media. It also affects the user’s perception and view intention regarding the news story and can be defined both qualitatively and quantitatively. Jennett et al. [16], for example, proposed a scale to measure the sense of immersion in immersive VR news.

Empathy

The term “empathy” originated from the field of art and has been widely used in psychology. At first, it referred to the emotional connection between human beings and nature but was later used to describe a person’s reaction to the feelings of others [3]: Wondra and Ellsworth [51] stated that empathy involves caring, identifying, and understanding others. Rath et al. [37] claimed that one of the important factors of the virtual reality technique is to provide a sense of empathy that is similar to a person’s real life. More specifically, Laws and Luisa [24] concluded that some immersive news is indeed approaching a form that can produce and enhance the user’s sense of empathy.

Credibility

Credibility refers to the degree of trust that news audiences have in media, news organizations, or news sources [11]. In addition to the sources and content of the news, credibility may also be affected by media technology, and there are concerns that immersive VR news may damage the credibility of news and weaken its ability to present facts to the audience [6, 40]. In comparison, Peña et al. [35] suggested that the purpose of immersive journalism is not merely to present the facts but rather to offer an opportunity to experience them. Their experimental results showed that immersive VR news with a high level of telepresence has higher credibility, and the experience feedback of the audience that used VR interaction devices capable of triggering a high level of immersion indicated a higher level of trust in the news content. Similarly, Nielsen and Sheets [32] claimed that virtual reality can add considerable value to traditional news products, potentially boosting engagement and trust. This credibility can be defined quantitatively; for example, Meyer [30], proposed a credibility scale to measure this aspect of the immersive VR news product.

Accuracy

Regarding accuracy, Tse et al. [47] found that 35% of participants mentioned experiencing the fear of missing information in immersive VR environments, while 33% of participants noted that immersive media distract them from the content of the story itself. According to Neisser [31] and Perry [36], the audiences’ cognition should be identified as an important factor when considering the influence of media effects. Based on the famous “Stimulus-Response” model, cognition reflects human psychological processes, such as attention, knowledge, memory, and perception. To measure the cognitive accuracy of immersive VR news, the six elements of news (i.e., 5W1H: who/when/where/what/why/how) has been popularly used in previous studies to measure users’ perceptions of news information and its accuracy.

Emotional attitude

Audiences’ attitude has always been an important dependent variable in news media effects research [36] and has been extensively studied. Attitude is typically divided into three independent aspects [7], that is, cognitive, affective, and conative. This “cognitive-affective-conative” (CAC) framework has been widely used in communication studies [5, 18, 19, 26, 27, 33] to measure audiences’ attitudes; therefore, this study also measured the media effects of immersive VR news using this framework.

In summary, while immersive technologies are advancing, the understanding of their effects on the news consumer remains inadequate. Currently, there is a trend for audience identity diversification in the current VR news applications; however, our knowledge about whether user experience and media effects of VR news can be influenced by different audience identities is still limited. Moreover, the current mainstream immersive VR news continues to focus on non-interactive products that only allow users to “visit” or experience news events from a bystander’s perspective, and the user experience and media effects of interactive immersive VR news have rarely been investigated. We found that the narrative frameworks adopted in different studies differ markedly from those of traditional news coverage: The mode of viewing an event from the perspective of an omniscient narrator is not suitable for all news topics, and the viewing perspectives are beginning to diversify, such as making eye contact with the protagonist and even experiencing the feelings of other parties as the protagonist themselves. The degrees of participation and interaction of viewers in different cases vary. In interactive cases, specific interactions include those with the scene, object, and other characters through, for example, eye contact, and this degree of interaction is deepening.

To address these issues, we proposed a design process for immersive VR news. By employing an iterative design method, this process provides a reference framework for designers to develop more practical and user-friendly VR news applications. Based on this process, we designed and developed a VR news product, and, from the experimental results and findings, we derived concrete design guidelines for immersive VR news design.

Research motivation

Unlike the process of traditional news production, immersive VR news faces new production challenges. According to Sirkkunen et al. [45], the technical production process of immersive VR news can generally be divided into three stages, that is, 1) capturing new content with a camera and/or voice recorder; 2) post-production comprising image processing, motion graphics, and 3D-modeling techniques, and 3) distribution with headset technologies and the associated content stores. This production process requires specialized knowledge in image processing, 3D modeling, and computer animation, which are difficult and beyond the reach of average journalists; therefore, traditional journalists must work with those who are proficient in HCI or VR technologies to better understand the capabilities and limitations of VR technologies in immersive news storytelling [14, 28, 45]. As a result, more studies are needed in journalism production to better understand how VR technologies work within the fields of news production and communication.

One of the challenges of immersive VR news is the design of a reasonable storytelling framework, in which the time, space, and causality of the news story are more credible [13]. Another challenge is providing a vivid representation of the current news story rather than separating facts from opinions in the usual way adopted by traditional reporters or narrators. This kind of storytelling must be dealt with carefully otherwise it may degrade the capabilities of telling the truth to the public and, consequently, lead to ethical dilemmas [13, 20, 25, 28, 35, 39]. Designers, thus, inevitably face the challenge of balancing news reporting and conveying the facts accurately in immersive VR news.

Furthermore, the immersive VR news audience no longer comprises passive recipients of content but rather those with autonomy who can actively explore news content to a certain extent [10, 17]. In such a situation, users’ perception and intention can significantly impact their experience of immersive VR news [41]; for example, they may ignore the important information that the news reporter wants to convey while focusing on less relevant information in the VR scenario [47], which consequently leads to the suspension of the storytelling. Therefore, the designers should pay attention to the user’s behavior and experience when designing immersive VR news products.

In summary, although the application of VR technology in the field of journalism provides a brand-new immersive experience as compared to traditional news, it also brings new challenges to the news production process, narrative framework, and journalistic professionalism. Currently, there is a lack of unified framework and design guidelines that can be followed by the designers to create news products that can provide better user experience and media effects.

Methodology

To address the abovementioned issues, we proposed a practical method for developing more reliable immersive VR news products than the traditionally offered news products. We applied this method to an immersive VR news report—“Beijing: Preventative action for severe acute respiratory syndrome (SARS)”—and investigated its user experience and media effects through user evaluation in two different storytelling frameworks: VR news without interaction (VR-I) in Study 1 and VR news with interaction (VR + I) in Study 2.

Study 1: VR news without interaction

Given that the current mainstream immersive VR news focuses on non-interactive products, we first explored the production process for VR-I in this study. Figure 1 shows the multi-stage design and development process we proposed for immersive VR-I products. Unlike traditional media, such as text, images, and video, VR news production has both the attributes of journalistic professionalism and the characteristics of software product iteration; therefore, gatekeeping theory and the principles of journalistic professionalism were integrated into the production process to guide the news product design.

Fig. 1

 Immersive VR-I product process

In the first stage of our proposed process, specific news topics are selected, and related information is gathered according to the requirements of news production organizations. In the second stage, the news information obtained from multiple channels is sorted and processed based on the news topics determined in the previous stage. The third stage is the frame design, in which the structure of the content and the function modules are determined and designed. The fourth stage focuses on visual content design, the output of which will be fed into the fifth stage to develop the system. In the sixth stage, the whole system is evaluated to ensure that the product meets the requirements of both news products (e.g., media effects) and VR systems (e.g., user experience). Throughout the design and production process, journalistic professionalism was considered as the core principle and guided the design practice.

Principles of journalistic professionalism

The principles of journalistic professionalism are widely used as the professional belief system and moral code by numerous news practitioners. Throughout the design process of immersive VR news, we applied the five guidelines summarized by White [50] and based on the Global Charter of Ethics for Journalists [12]:

Fairness and impartiality. This principle is mainly embodied in the first three stages of the production of immersive VR news, including newsgathering, news processing, and frame design. The designer should consider the content design based on multiple sources to balance reporting. More specifically, the balance between different sources and facts should be considered in terms of the immersive differences that virtual reality technology brings.

Independence. This principle should be carefully considered in the frame design stage for VR news. Given the fact that the user is involved in the VR news event as an actor may conflict with this principle, the designer should distinguish between opinions and facts during frame design to ensure that the user remains a participant or observer with a neutral attitude and is not arbitrarily involved in the news event.

Truth and accuracy. This principle is applied throughout the whole production process, including scenes, characters, animations, and interactions. Details in all these design stages are required to be consistent with the news facts, allowing users to access accurate information through multiple channels.

Humanity. This principle reminds us that those news materials that are inclined to cause negative effects, such as excessive violence and bloody scenes, should be avoided in the newsgathering and processing stage. The user’s physical experience should be carefully considered during the system development process to avoid adverse reactions.

Accountability. This principle is consistent with the idea of user evaluation for virtual reality systems. All immersive VR news products should be carefully evaluated and refined iteratively to ensure the system is user-friendly and socially acceptable.

Newsgathering

By collecting and analyzing the results of previous studies, we learned that immersive VR news has certain requirements for news subjects and is unsuitable for breaking or time-sensitive news; it is more appropriate for social and disaster news, which have distinct characters and complete story timelines and are not commonly experienced by ordinary people. As such works can be used repeatedly for a relatively long period and are not prone to becoming outdated, they are focused on human nature and life experience in a more general sense, whereas in the case of ordinary news events, the application value of immersive VR news is insignificant.

Based on this analysis, we chose “Beijing: Preventative action for severe acute respiratory syndrome (SARS),” an investigative news report broadcasted on the China Central Television (CCTV) news channel on April 26, 2003, as the news material for this study. We chose this news material because 1) it has high social and humanistic values and represents the news type we have previously targeted; 2) the video was released by a national Chinese TV station whose news reports have been influential and authoritative, earning significant credibility and, thus, reflecting the typical news narrative and traditional mainstream media; 3) the news event happened in the SARS epidemic isolation ward of Beijing You’an Hospital, in which a reporter visits a patient, representing the kind of event that ordinary people do not often experience, and 4) the epidemic is now fading into the distant past. Most people today only hold vague ideas and limited knowledge about this event, which is conducive to a more accurate measurement of the media effects (e.g., accuracy of information reception) of immersive VR news.

News processing

In this stage, we needed to process and screen the information collected in the newsgathering stage, based on gatekeeping theory. Unlike traditional text and/or video news, the carrier of an immersive VR news report is a computer-generated virtual reality application. The development of this application involves a variety of different functional modules and design elements, including the scenarios, medical items, characters, and synopsis in the SARS case. The outcome of this stage is a script design document (Fig. 2).

Fig. 2

Newsgathering and processing

Frame design

Newsgathering and processing were followed by frame design, in which the news content, structure, and system functionalities were considered and designed as a whole. In Study 1, we adopted a linear, non-interactive narrative framework, which has been widely used by existing research on immersive VR news products (e.g., Use of Force, Kiya, Across the Line). This news mainly takes the form of reporters’ dialogues with doctors and patients while conducting interviews. The script of the entire news story encompasses three scenes (Fig. 3), a clean area (disinfection room), a partially contaminated area (corridor), and a contaminated area (ward). In the disinfection room, a doctor introduces and demonstrates the cleaning and preparation procedures before entering the partially contaminated area, including wearing protective clothing, headgear, protective glasses, and shoe covers, and explains the differences among the three areas. In the corridor, the doctor explains the functions of the protective shoe cover and disinfectant and then introduces the specific procedures for treating SARS patients. In the ward, the reporter interviews a SARS patient to understand her condition and treatment plan and talks with a doctor, asking why the doctor is risking his life to treat these patients.

Fig. 3

Three scenes in the “SARS” case

Visual content design

In this stage, we first made the 2D floor plan where the news story happened (Fig. 4a) using AutoCAD software and according to the video content we collected in the first stage. Thereafter, the 3D scenes were modeled using Unity3D to match the 2D floor plan of the story (Fig. 4b).

Fig. 4

2D and 3D modeling of the news story scenarios. a 2D floor plan of the news story, b 3D scenes of the news story

Next, we used 3D modeling software (e.g., 3D Max, Maya) to create the characters (e.g., the reporter, doctors, and nurses) and adjust the details of the characters such as skin and the five sense organs to make them look more like the real people that were represented in the original news video. We then created character animation for the existing characters in a process that involved operation steps such as bone-binding, skin-covering, and controller building and based on the reporters’ dialogues and movements in their interactions with doctors and patients in the hospital, following the truth and accuracy principles required by journalistic professionalism (Fig. 5).

Fig. 5

Virtual scene and animation modeling. a Character modeling, b Character and 3D object modeling

System development

In this stage, we used the Oculus Rift VR headset to enable the user to freely explore the virtual environment. The monocular resolution of the Oculus Rift VR headset is 640 × 800 while the binocular resolution is 1280 × 800. The refresh rate is 90 Hz.

There are many development engines available in the VR field, such as Unity3D, Unreal Engine4, CryEngine3, and Cocos 3D. In this study, we chose Unity3D, which has been well integrated with the Oculus SDK for the development of virtual reality applications. We used scripting languages to implement relevant interactive functions.

Evaluative testing

In this stage, the proposed immersive VR application was evaluated in terms of user experience and media effects.

Participants and apparatus

We recruited 88 participants (36 males and 52 females) from a university via advertisements, Webchat, and microblogs. They were aged between 18 and 33 (M = 26.5, SD = 3.297) and were pursuing different majors, including economics, law, literature, science, engineering, and management. Based on the information collected in the pretest questionnaire survey, individuals who had full knowledge of the content of the news material used in this experiment or who exhibited strong personal opinions and emotional attitudes were excluded during the recruitment process to avoid any influence of participants’ biases on the accuracy and credibility of the experimental results. In addition to the Oculus Rift headset, we also used a web camera and a voice recorder to record the participants’ behaviors and soliloquies for later data analysis.

Procedure

During the experiment, participants were randomly divided into two groups: Group A (44 participants) and Group B (44 participants). Participants in Group A were required to view traditional 2D video news on a computer and those in Group B were asked to wear the provided Oculus VR headset to watch the VR news that was reproduced based on the traditional 2D video news segment.

After the participants completed the experiment, they were asked to complete a questionnaire that contained items regarding media effects and user experience. To assess media effects, we generated six questions on the six elements of news (i.e., 5W1H: who/when/where/what/why/how) to measure the subjects’ perception of the news information and its accuracy. Six items were selected from the original 20-item PANAS adjective checklist [48] to measure emotion, including “scared,” “interested,” “nervous,” “inspired,” “distressed,” and “excited.” We also used the five major credibility indicators [30] to measure credibility. To evaluate user experience, we adopted Jennett et al.’s [16] immersion scale to measure immersion and the Toronto Empathy Questionnaire of Kourmousi et al. [21] to measure empathy. After all participants had completed the questionnaire, our laboratory personnel conducted one-on-one semi-structured interviews with each participant to understand their views and suggestions on the entire system. The experiment lasted 30–40 min.

Results and user feedback

Media effects. As for accuracy, the average score of traditional video news and immersive VR-I was 5.39 (SD = .655) and 4.32 (SD = 1.029), respectively. The Mann-Whitney U test indicated that there was a significant difference between the two treatments (Z = −4.920, p = .000).

In terms of emotion, traditional video news and immersive VR-I averaged 3.61 (SD = 1.351) and 3.11 (SD = 1.543) for “scared,” 3.79 (SD = 1.807) and 3.29 (SD = 1.352) for “nervous,” and 4.56 (SD = 1.591) and 4.19 (SD = 1.341) for “distressed.” The Mann-Whitney U test indicated that the traditional video news was superior to immersive VR-I news in terms of fear (p = .001), nervousness (p = .005), and distress (p = .024).

As for credibility, the average score of traditional video news and immersive VR-I news was 4.93 (SD = 0.934) and 4.49 (SD = .915), respectively. The Mann-Whitney U test indicated that there was a significant difference between the two treatments (Z = −1.920, p = .047).

Through experimental observations and interviews with the participants after the experiment, we found that they were easily distracted by irrelevant things in the VR-I report. Traditional video news has a limited visual boundary, i.e., a frame, in which the reporter can precisely convey the content and the focus of the information they want to convey to the audience. In contrast, immersive VR news allows a 360-degree field of vision and all information is presented to the audience in parallel. Moreover, compared with traditional third-person perspective video news, even though VR-I news provides a first-person perspective, it does not change the one-way dissemination mode of news information. In the news narrative, many participants were attracted by the new form of VR and looked around while wearing the HMD headset, thus distracting them from the main message that the news story aimed to convey. They stated:

“The red hot water bottle on the bedside table attracted me” [P22 in Group B]

“My attention focused on the old cell phone by the patient’s pillow, so I wasn’t listening to the conversation between the reporter and the doctor and consequently missed a lot of important information.” [P42 in Group B]

“If I had known in advance that I would be asked questions whose answers could be found in the conversation between the reporter and the doctor, I would not have paid attention to the scalpels during the experiment.” [P44 in Group B]

User experience. As for immersion, the average score of traditional video news and immersive VR-I without interaction was 5.17 (SD = 0.920) and 5.11 (SD = 0.786), respectively. The Mann-Whitney U test indicated that there was no significant difference between the two treatments (Z = −0.494, p = .621).

Regarding empathy, the average score of traditional video news and immersive VR-I was 5.60 (SD = 0.733) and 5.20 (SD = 1.178), respectively. The Mann-Whitney U test indicated that there was a significant difference between the two treatments (Z = −3.407, p = .016).

Through the interviews, we learned that in watching the traditional video news, participants believed the characters and scenes were all real. As the news story unfolded and continued, the participants were unconsciously immersed in a depressing atmosphere and, thus, felt strong empathy. In contrast, the VR news application is an artificial virtual environment. Although we tried to faithfully reproduce the story from the news video, we were unable to fully replicate material shot through a camera lens in various ways, such as the character model, scene model, scene tone, lighting, and shadows due to the limitations of the existing modeling capabilities. Therefore, participants could not be fully immersed in the scene and, thus, could not feel such a strong sense of empathy. Furthermore, participants also described the problem of “bystander identity” in the VR news report and described a reduced sense of immersion and empathy in this identity that they were assigned in the immersive VR environment:

“I feel that it is boring to watch the conversation between the reporter and the doctor. I had nothing to do but watch the conversation foolishly in the virtual environment” [P18 in Group B]

“As a product experiencer, I prefer to be assigned to a specific identify (e.g., the reporter), so that I can have a deeper sense of telepresence” [P25 in Group B]

Discussion of study 1

Overall, traditional video news was rated superior to immersive VR news without interaction in this study. The primary reason for this may lie in that the opportunity for participants to explore freely in the virtual environment facilitates the participants in focusing on different things according to their personal interests or habits. Traditional immersive VR news without interaction, including the SARS case in this study, did not precisely gatekeep the key information that needed to be conveyed to the audience, in particular, how the news information should be shaped, encoded, and highlighted in an immersive virtual environment with multiple degrees of freedom. This result was consistent with the findings reported by Krugman [23] that media effects in different modes are prone to be influenced by the motivation of watching and the degree of audience participation.

The experimental results and findings in this study informed us of three important requirements that should be carefully considered in the design of immersive VR news products. The first is concentration, in other words, directing the audience’s attention to the information that the news reporters want to convey. The second is information logic, which means information should be effectively inducted, summarized, and expressed in a more logical order to improve media effects. Finally, engagement should be considered since emotions and a sense of empathy can be affected by the user’s viewing perspective. Participants should, therefore, be allowed to immerse into the news story in a more active approach to enhance the sense of engagement.

VR news with interaction

Compared to Study 1, participants acted as protagonists (e.g., the reporter) rather than bystanders in this study and could actively interact with other characters (e.g., the doctors or the patients) rather than passively watching the conversation in the VR environment. Thus, we extended the design method proposed in Study 1: As shown in Fig. 6, news content was rearranged and key information was refined in the news processing stage to lay the foundation for the selection of corresponding system functionalities and interaction techniques in the subsequent stages. In addition, in the frame design stage, a newly added interaction design module was integrated to determine the functional set according to specific requirements, design-appropriate interaction techniques according to the key information, and design the whole user experience process. Similarly, a new user interface design module was integrated into the visual content design stage. In the system development stage, the news scripts and content logic structure were updated and modified to meet the requirements of corresponding system functions determined in the previous stages. Finally, the interactive product was evaluated in terms of user experience and media effects.

Fig. 6

 Refined immersive VR news product process

News processing

As mentioned above, information gatekeeping theory has generally been used in the realm of traditional media. In the new media era, it has been suggested that gatekeeping theory should be replaced by gatewatching theory [4], which emphasizes the shift from simply eliminating less important information to purposefully highlighting more important information [1]. Therefore, we applied gatewatching theory in this stage, emphasizing key information by re-designing the information structure and interactions to convey news more effectively rather than simply eliminating the less important content.

Frame design

According to the results of Study 1, we identified three main design requirements, including concentration, information logic, and engagement (Fig. 7).

Fig. 7

 Three design guidelines for immersive VR news design

We embodied the three design requirements and proposed the commonly used function set for the immersive VR news. Different functions were identified to match different types of information, for example, the point of information, “wearing the protective clothing,” in the conversation between the reporter (the user in this study) and the doctor contains a piece of an important knowledge segment with a strong attribute of operability in the virtual environment, so we designed an interaction technique that enables the participant to practice wearing protective clothing in the news story. Finally, four function types were identified throughout the three scenes, including event handling, dialogue, viewing item details, and scene transition (Table 1).

Table 1

Four types of interactive functionalities

Scenes	Functionality	Description
Scene 1	Dialogue	Dialogue between the reporter and the doctor
	Event handling	Wear protective equipment
Scene 2	Dialogue	Dialogue between the reporter and the doctor
	Viewing item details	Check the facilities along the corridor
Scene 3	Dialogue	Dialogue between the reporter and the patient
	Dialogue	Dialogue between the reporter and the doctor
	Viewing item details	Check the respirator
Between Scenes	Scene transition	Transitions between different scenes

The user experience process in this study was totally different from that of the previous study. In Study 1, participants experienced the news event linearly as bystanders without clear roles or identities along a timeline predefined by the system. Such an experience design was consistent with the concept of “public journalism” or “civic journalism” [22], in which the audience can obtain information without any restrictions. Although this design idea represents the concept of “everyone as a gatekeeper,” it is unrealistic to expect the average audience to play the role of a professional gatekeeper when obtaining news presented in a disorganized way. Compared to the previous study, we proposed a new user experience mode based on gatekeeping theory, namely “two freedoms under two constraints.” Under this experience mode, the user is required to participate in the news events in a fixed flow of interactions and an organized way to obtain the necessary key information. Once the users had completed the specified interaction tasks required by the system, they were allowed to explore the VR scenario freely within each scene (Fig. 8).

Fig. 8

“Two freedoms under two constraints” user experience mode

The “two constraints” refers to the constraints between different scenes, requiring that users could not redirect to the next scene unless they had completed all the given interactive tasks in the current scene. In addition, the constraints were also predefined between different interactive tasks, i.e., interactions could only be triggered in the proper stage of a specific scene. Meanwhile, the “two freedoms” refers to 1) the freedom to explore different things in the current scene, i.e., the user is free to explore the scene without the limit of time and space, and they can decide when to move on to the next scene according to their personal willingness, and 2) the freedom to manipulate different objects involved in the same interactive task without an order constraint, for example, the user can pick up the medical record and then talk to the doctor, or they can pick up the medical record during the conversation with the doctor.

Evaluative testing

Participants and apparatus

We used the same apparatus as Study 1. Here, participants were required to use the two Oculus touch controllers to perform the given interactive tasks, including grabbing an object in the ward and viewing details of this object, wearing disinfection gloves and goggles, and checking the medical records.

Forty-three college students (19 males and 24 females) participated in this study, aged between 18 and 28 (M = 25.7, SD = 2.981). Participants were pursuing different majors, including law, literature, computer science, medical engineering, management, and history. Similar to study 1, none of the participants had ever watched the original video news before this study.

Procedure

Participants in this study were assigned to Group C. The experimental procedure was the same as Study 1. During the experiment, participants were required to wear the provided Oculus VR headset to view the VR news and interact with the system with the Oculus Rift handle controllers. After the experiment, all participants were asked to complete a questionnaire that has been used in Study 1. We also conducted one-on-one semi-structured interviews with each participant to collect further user feedback from the participants. The experiment lasted approximately 40 min.

Results and user feedback

In this study, experimental results showed that the immersive VR news with interaction (VR + I) effectively improved media effects and user experience as compared to immersive VR news without interaction (VR-I).

Media effects

The average accuracy scores for traditional video news, VR-I, and VR + I were 5.39 (SD = 0.655), 4.32 (SD = 1.029), and 5.05 (SD = 1.045), respectively. The Kruskal-Wallis test results showed statistically significant differences among traditional video news, immersive VR-I, and immersive VR + I (χ 2 (2, n = 131) = 24.729, p = .000). Post-hoc testing indicated that VR + I was perceived to be significantly more accurate than VR-I (p = .001), and traditional video news was also perceived to be significantly more accurate than VR-I (p = .000), while VR + I exhibited no significant difference from traditional video news (p = .523).

Participants mentioned in the interviews that the news information conveyed in the interactive experience existed as “memory points,” which were more impressive than coherent yet non-interactive stories. Such findings verified the importance of the interactivity as gatewatching [1] in emphasizing information. For example, participants stated:

For emotional attitude, the average scores for traditional video news, VR-I, and VR + I were 4.66 (SD = 1.180), 5.07 (SD = 1.129), and 5.26 (SD = 1.136), respectively in terms of interest, and 3.79 (SD = 1.807), 3.29 (SD = 1.352), and 4.45 (SD = 1.372), respectively in terms of nervousness. The Kruskal-Wallis test results showed statistically significant differences among traditional video news, immersive VR-I, and immersive VR + I regarding interest (χ 2 (2, n = 131) = 6.363, p = .042) and nervousness (χ 2 (2, n = 131) = 7.388, p = .025).

“Before I entered the ward, I was required to wear the protective equipment. This made me feel that the outbreak (SARS) is very serious. I personally agree with this form of interaction technique because it enhances the sense of substitution and is quite impressive.” [P10 in Group C]

“The interactive experiences, such as wearing protective clothing before entering the ward, were impressive. If you just sit on the sidelines and watch other people wear the protective clothing, you won’t have a strong sense of substitution and you won’t try to remember the detailed information conveyed by the news.” [P11 in Group C]

Participants explained that they experienced a higher degree of emotion due to the increased freedom and autonomy in VR + I. Under the “two freedoms under two constraints” user experience procedure, participants can freely explore the news event without worrying about missing important information; therefore, they are more aware of the interest and nervousness conveyed by the news itself. One participant noted:In terms of credibility, the average scores for traditional video news, VR-I, VR + I were 4.93 (SD = 0.934), 4.49 (SD = 0.915), and 5.27 (SD = 0.986), respectively. The Kruskal-Wallis test results showed statistically significant differences among traditional video news, immersive VR-I, and immersive VR + I (χ 2 (2, n = 131) = 6.578, p = .032). Post-hoc testing indicated that VR + I was seen to be significantly more credible than VR-I (p = .019), and traditional video news was also significantly more credible than VR-I (p = .020), while VR + I exhibited no significant difference from traditional video news (p = .583). Post-hoc interviews revealed that the higher news credibility may be attributed to both the enhanced subjective experience of “being there” and the objective fact that the VR + I conveys much more detailed information in a more effective way. Participants observed:

“This is a relatively open-ended scenario, in which I can explore important news information in a free mode or make decisions to help the patient under the guidance of the doctor in a constrained norm way when necessary.” [P1 in Group C]

“The VR+I requirements for more real and richly detailed information prompt the news producers to review and present information in a more effective way, which consequently leads to an increase in authenticity and credibility.” [P23 in Group C]

User experience

The average scores for immersion for traditional video news, VR-I, and VR + I were 5.17 (SD = 0.920), 5.11 (SD = 0.786), and 5.47 (SD = 0.706), respectively. The Kruskal-Wallis test results showed statistically significant differences among traditional video news, immersive VR-I, and immersive VR + I (χ 2 (2, n = 131) = 9.493, p = .006).

As for empathy, the average scores for traditional video news, VR-I, and VR + I were 5.60 (SD = 0.733), 5.20 (SD = 1.178), and 5.36 (SD = 0.778), respectively. The Kruskal-Wallis test results showed that no statistically significant differences were found among traditional video news, immersive VR-I, and immersive VR + I (χ 2 (2, n = 131) = 2.463, p = .292).

Participants showed a more positive attitude regarding immersion and empathy for VR + I as compared to traditional video news and VR-I, explaining that the scene transition and interaction design in VR + I made the news story clearer and conveyed key information more accurately to them:

“The news is well organized. Different from 360 news or traditional VR news without interaction, I can learn what to do next and how to do it by touching the highlighted interactive widgets or the pop-out dialogue to obtain the necessary information or guidelines provided by the system when I was lost or confused.” [P5 in Group C]

“The story proceeds in a good order and the interaction is impressive.” [P26 in Group C]

Discussion of study 2

Based on the experimental results, we conclude that VR + I can effectively improve the media effects and user experience of virtual reality news. In the previous study, some participants walked around in the virtual space while others were attracted to some small widgets (e.g., the old-fashioned cell phone next to the patient’s pillow). This kind of unrestricted user experience mode greatly affected the communication of key news information and, consequently, lead to the poor performance in media effects and user experience in Study 1.

In comparison, the advantages of immersive VR + I lie in 1) capturing the user’s attention exactly through different interactive techniques; 2) enabling the user to explore key news information as an active seeker in the virtual environment; 3) prompting the user’s sense of social identity and duty, and 4) embodying the application of gatekeeping theory [15] effectively. For example, participants stated:Despite the positive feedback, some participants raised concerns about the application of immersive VR news in practice:

“Without interaction, VR news is just another form of video news. It's easy to get distracted if you just watch the news story on the sidelines.” [P7 in Group C]

“The interaction and first-person substitution prompted me to pay attention to the doctor’s advice. Also, I felt that I must remember the important information, for example, how to wear the protective clothing correctly and wash my hands before and after meals in my daily life, because the outbreak (SARS) was happening right next to me.” [P24 in Group C]

“The production techniques of immersive VR news are in an early stage of development, with high production difficulty and cost, making them difficult to design and develop.” [P3 in Group C]

“For most people, particularly businessmen or white-collar workers, VR news is time-consuming and troublesome to watch (e.g., the users must wear heavy head-mounted displays). In contrast, it is more efficient and convenient to watch traditional video news on personal mobile phones.” [P14 in Group C]

“Although VR news can provide a free navigation space, it usually requires expensive equipment (e.g., the head-mounted display and the handle controller) and a large space in the physical world to allow the user to move around.” [P26 in Group C]

“It made me feel like I was really in a virtual world, with no way to embody myself in the virtual world.” [P33 in Group C]

Implications for immersive VR news design

Based on the results we obtained from these two user studies, we derived concrete guidelines for immersive VR news design.

Selection of news topics

Although immersive VR + I was rated superior to traditional video news and VR-I, many problems still exist, for example, the high cost of VR news production and consumption determines that VR news must be selectively applied to certain news types. Currently, it seems unfeasible for media organizations to pursue VR news production on a large scale. However, scenes of disasters and major historical events and scenes that are difficult for ordinary people to access or experience are not prone to expiration, can be used repeatedly for quite a long time, and cover topics related to human nature and life experience, and, thus, are more conducive to VR production. It would not be meaningful to heavily invest in immersive VR news production for other common news events.

Wizard design for immersive VR news

The application of immersive VR news also brings new challenges for designers. For example, in an immersive VR environment, users rely on the vision channel to search for objects (e.g., the protective clothing). Such eye-engaged interactive techniques may significantly degrade the interaction efficiency and user experience, particularly when users have to turn their head frequently to search for a target object in the limited field of view of a head-mounted display. Such interactive techniques may increase the user’s burden and even make them feel confused or frustrated. Therefore, designers should provide suitable guidance in the VR environment, for example, indicators to guide the users’ visual search path or interactive objects and eye-catching slogans to attract the user’s attention and prompt them to continue the exploration and performance of the specified tasks that the system has pre-defined.

The balance between freedom and constraint

Designers should also consider the balance between freedom and constraint in immersive VR news. On the one hand, immersive VR news should allow the users to move around while navigating in a virtual environment, offering them greater freedom to explore news information. On the other hand, it is necessary for the information to be reasonably arranged under the framework of gatekeeping theory in VR news reporting. Interaction design can balance the requirements of these two factors. Through reasonably designed interaction technologies and guiding the user’s attention in a reasonable way, the system can support free exploration under a certain degree of constraint (e.g., Fig. 7).

Conclusion and future work

While immersive VR technologies are advancing in the field of journalism, there is still insufficient clarity on the effects these technologies have for the news consumer. This study focused on the user perspective, which we hope can advance both academia and the industry.

This study’s contribution is fourfold. First, we propose a more practical design process for immersive VR news products that combines the process of news products from the perspective of news professionalism with the development of virtual reality applications from the perspective of iterative software development. Second, we investigated the production of immersive VR news under different conditions (VR-I and VR + I). Third, we provide a deeper understanding of media effects and user experience in immersive VR news, and fourth, we provide empirical evidence on the benefits of practices involving interactivity in immersive VR news. Our results show that interactivity can significantly guide the users’ attention on key information that a news reporter wants to convey, which has a significant positive impact on the media effects of VR news. Meanwhile, interactivity plays a gatekeeper role, which is important to address issues such as information overload, parallel expression, and information interference that current mainstream VR-I faces. In addition, interactivity can enable the user to change from a passive bystander who watches news from an omniscient perspective to a protagonist who can experience the feelings of the parties involved in the news story. From the perspective of application, VR + I is similar to news games and is a valuable expansion of and supplement to traditional news. However, different from news games, VR + I cannot replace news and retains the basic connotations of news, so it must comply with the news reporting principles of truthfulness and fairness.

Our future work can be extended in various directions. For example, the proposed method was currently validated based on a single news case. In future, the application scenarios of immersive VR news can be expanded to further generalize the conclusions of this study. Additionally, a larger-scale test regarding the effects of VR news should be conducted with more participants.