The impact of culture and product on the subjective importance of user experience aspects.

Many software products are created by considering targeted users from different countries. For such products, it is significant to investigate users' expectations related to cultural aspects to achieve user acceptance as broad as possible in all relevant countries. The researchers of the current study examined whether the cultural background of an individual has an influence on the subjective importance of user experience (UX) aspects for several common product categories. The researchers compared findings from a previous study conducted in Germany and a replication study carried out in Indonesia. Results show significant differences concerning the rated importance of UX aspects for many product categories. However, a detailed analysis of the results also shows that the impact of culture is considerably lower than the impact of interindividual differences between persons of the same culture. In addition, both samples show quite similar rankings of the importance of UX aspects. Thus, the product type has a much bigger impact than cultural differences.

Introduction

The Internet has made it possible for companies to globalize their products. This opportunity may become a competitive edge, and even a good feat against competitors. Endeavors in shaping a product into something which could be accepted by the market have been done in order to achieve a broad user acceptance (Clemmensen et al., 2010; Reinecke et al., 2010). Such endeavors require careful consideration of design decisions. It is because user acceptance is a result of a good user experience (UX), which is largerly affected by many factors (Konstantakis et al., 2017; Zarour and Alharbi, 2017).

One factor that has been a prominent topic in UX is culture (Gallivan and Srite, 2005; Ford and Kotze, 2005; Wallace et al., 2012; Konstantakis et al., 2017). Broader market means more audience diversity. For example, if a Western finance company, for instance from the US, builds a financial software and targeting Southeast Asia market, that software will naturally consider the targeted country's tax laws. However, as Western and Eastern societies generally have different cultures, some adjustments to the local culture might be needed. Finance is quite a sensitive aspect and some people prefer it to be delivered in their own language in order to reduce risks. Yet, adaptations of software products to different countries or cultures are often limited to the translation of texts of changes in the user interface, the documentation, or the functionality in accommodating local laws or regulations.

Previous studies have tried to enrich the comprehension of how culture relates to UX. In e-Learning, Thowfeek and Jaafar (2012) found that cultural factors could explain the variation of instructors' behavior in adopting e-Learning system. Bhuasiri et al. (2011) discovered six dimensions and 20 critical factors for e-Learning system in developing countries. Aparicio et al. (2016) found that students who are influenced by collective culture perceive more individual and organizational impacts than students with individualistic culture. In e-Government, it is revealed that some national cultural characteristics have directly impacted the diffusion of e-Government. Zhao et al. (2014) and Zhao (2011) realized that the cultural dimensions defined by Hofstede (2001) have dramatic correlations to the development of e-Government. To put it simply, it is evident that culture could drive the way people experience a product.

On the other hand, studies have uncovered several UX aspects which have to be put into consideration for an efficient design process. Referring to Winter et al. (2017), there are 16 important UX aspects empirically measured for different products. However, a further study (Schrepp, 2018) found that these aspects are not all applicable for certain type of products. In other words, a different type of product calls for a different design decision.

Since teams responsible for a product design are usually small and, in many cases, also quite homogeneous concerning their cultural background, it is natural to question whether that product design will be equally acceptable in different cultures (see, for example, Nielsen, 1994; Marcus and Gould, 2001 or Heimgärtner, 2017). Thus, the question if the cultural background of a user influences his or her expectation concerning the user experience (short UX) of a product is of theoretical, but also of practical relevance.

The aim of the study is to investigate the impact of culture on the perception of UX compared to the impact of such interindividual differences. Knowing that culture might affect user experience and types of products might determine design decisions, it is necessary to understand how culture might also affect a design decision. Especially in the context of gaining considerable acceptance in a diverse market, designers need to accommodate relevant aspects and factors. This study also contributes to a culture impact on user experience in Indonesia.

Background

How do we conceptualize user experience?

The term user experience (UX) is used by different authors with slightly different meanings and intentions. There is no commonly accepted definition of this term in research literature, so we have to clarify how we understand this concept within this paper.

An attempt to define UX can be found in the norm DIN EN ISO 9241-210 (ISO, 2019), where UX is defined as "a person's perceptions and responses that result from the use and/or anticipated use of a product, system or service". This definition clearly states that user experience is a subjective impression which is not restricted to the time span during which a person interacts with a product (in contrast to the usual definition of usability).

The problem of this definition is that it is quite abstract and does not help designers to find out which aspects of a product design they should focus on to improve it. In addition, this definition does not provide any hint on how UX can be measured, for example by a questionnaire.

To overcome these restrictions, another conceptualization of user experience as a set of quality criteria that exceed the well-known usability criteria was established and is quite common in research. This is based on Preece et al. (1994) who define two sets of quality goals for an interactive product. Usability goals represent the classical usability criteria or dialogue principles, as defined in the ISO 9241 110 (ISO, 2006) (suitability for the task, suitability for learning, controllability, etc.). UX goals are quality criteria that are not related to concrete tasks the users perform with a product, but concentrate on the subjective feelings of the users during the interaction, such as the fun of use or the beauty of the design. In this view, UX is seen as a set of quality criteria that are separate from the usability criteria.

Another similar conceptualization of UX was presented by Hassenzahl (2004). Hassenzahl distinguishes between pragmatic qualities, for example efficiency or learnability, and hedonic qualities, for example stimulation or fun of use. The pragmatic qualities are strongly related to the tasks a user performs with a product and correspond to the usability goals of Preece et al. (1994). The hedonic qualities are not related to tasks and correspond to the UX goals of Preece et al. (1994). UX in Hassenzahls' (2004) view is defined by the union of the pragmatic and hedonic qualities, i.e. a concept that extends the classical usability concept with some quality criteria that are not task related. We adopt this view in our paper.

How to define culture?

In investigating the cultural impact on UX, a clarification of how culture is defined is far from being trivial. There are several theoretical attempts to define cultural dimensions that can be used to describe differences between cultural groups (Aparicio et al., 2016; Cronje, 2010; Ertl and Helling, 2014; Tarhini et al., 2013; Zhao et al., 2014; Zhu et al., 2017). Probably the most popular is the cultural dimensions of Hofstede (2001). This model is based on extensive empirical research. The basic idea of Hofstede's model is the assumption that culture is a set of learned traits, which cause behaviors or reactions to specific situations to be more likely in some cultures than in others. Thus, differences between cultures result from differences in basic values shared by the members of these cultures. This model contains six distinct cultural dimensions as can be seen in Fig. 1.

Fig. 1

Hofstede's six cultural dimensions.

Hofstede provides concrete scores for these dimensions for several cultures. Based on these scores, Germany and Indonesia show a quite different profile (Santoso et al., 2017). Indonesia is described as strongly collectivistic, while Germany is strongly individualistic. Indonesia shows a high level of power distance, while this value is much lower for Germany. In regards to Indulgence vs. Restraint, both countries show nearly the same value. For the rest of the dimensions, the German scores are moderately higher than the Indonesian scores, i.e. Germans live in a more masculine culture and show a higher uncertainty avoidance and long-term orientation. Thus, according to these scores, both countries are quite different in terms of their cultural values and are deemed to be good candidates for research in the context of the impact of culture on UX.

Hofstede's model is the most cited model when it comes to an investigation of the impact of culture on usability. The model has a sound empirical basis. However, there is also some criticism concerning the model; for example, it is mentioned that it is too stereotypical (Bourges-Waldegg and Scrivener, 1998).

The connection of the Hofstede dimensions to user interface design elements of web sites is discussed in the research by Marcus and Baumgartner (2004) and the research by Marcus and Gould (2001). The Hofstede model is the best investigated cultural model with respect to usability and UX.

There are a huge number of different cultural models, for example, Trompenars (1995), Victor (1997) or Hall (1989). An extensive overview on existing cultural dimensions is provided by Baumgartner (2003).

The problem with these cultural dimensions is that they mainly describe how people in a specific culture interact with other people. It is therefore difficult and invariably a bit speculative to derive a concrete conclusion about how people in a specific culture deal with interactive products and what their expectations concerning such products are.

Furthermore, cultural dimensions uncover patterns of behaviors or learned traits that are more frequent in some cultures than in others. However, there are also undeniable interindividual differences between persons sharing the same cultural background. The subjective impression of a person concerning the overall UX of a product depends also on the personal history of the person (such as existing experiences with similar products increasing the perception that a newly adopted product is easy to use) and on personal preferences.

Thus, it is natural to ask how big the impact of culture on the perception of UX is compared to the impact of such interindividual differences. This is one of the research questions of this paper.

Research concerning the impact of culture on UX

The impact of culture on various aspects of UX is already investigated in many papers. A study by Forer and Ford (2003) provided some evidence that users perform better if the user interface of an application is designed to match their cultural profile. In this study, the cultural profile is defined in the sense of Hofstede's (2001) cultural model. Straub et al. (1997) showed that the users' subjective cultural profile had an impact on the acceptance of specific technologies for some typical tasks. Nantel and Glaser (2008) provided evidence that the perceived usability of a web site increased when it was originally conceived in the user's native language.

Conversely, there are also authors who claim that the purpose or goal of a user when he or she uses an interactive product, or a web site is the factor that mainly influences the user experience and that culture plays only a minor role in the context (Fitzgerald, 2004). In addition, due to the increasing globalization people get used to reacting on interactive products designed from designers with a different cultural background; thus, patterns of interaction will become more and more familiar and the expectations from the cultural background will decrease over time (Bryan et al., 1994). Hence, there is obviously no clear picture concerning the impact of culture compared to the impact of other relevant factors for the subjective perception of UX.

There is also some research which deals with the influence of culture on concrete elements of the user interface design. Duncker (2002) investigated the cross-cultural use of computing metaphors. Such metaphors are an important part of the design of modern information systems but are often deeply rooted in culture. Metaphors are also quite an important aspect for the design of icons. Since such metaphors heavily depend on culture, it is clear that icons representing a metaphor not known in the culture of a user are hard to understand.

There are many other results which deal with the impact of culture on specific design elements. Some of the examples are the usage of colors (Noiwan and Norcio, 2006) and the usage and design of icons (Choong and Salvendy, 1998; Kim and Lee, 2005; or Pappachan and Ziefle, 2008).

Aspects of UX quality

UX is a multi-dimensional construct. The overall impression concerning the UX of a product is based on the impression concerning several distinct UX qualities, such as the efficiency of the interaction, the ease of learning the interaction, the fun of using the product, or the beauty of the user interface (for example, Preece et al., 1994; Hassenzahl et al., 2000; or Schrepp, 2018). Such distinct UX qualities hereinafter will be referred to as UX aspects.

Some current empirical research (Winter et al., 2017 or Mayer et al., 2018) shows that the importance of different UX aspects for the overall UX impression depends heavily on the product type. Intuitively this is not surprising. For a software used to perform important tasks of work, we would be somewhat sensitive even for minor issues affecting the efficiency of the product, such as unnecessary clicks forced by a bad interaction design or long latencies between our commands and actions and the reaction of the product. For a tool that is used once a year, such as a tax reporting software, we would be more relaxed concerning efficiency. Other aspects, intuitive usage and usefulness being some examples, are much more important in the context of usage.

For some aspects and products, it is relatively evident how important those aspects are for users of the product. For others, empirical research is required to get a clearer picture. A list of relevant UX aspects was generated by Winter et al. (2015) and later refined by the same authors as a result of extensive empirical research (see Winter et al., 2017 or Schrepp, 2018). The current version of this list contains the following UX aspects (Fig. 2).

Fig. 2

UX aspects.

These UX aspects were extracted from the analysis of more than 20 established UX questionnaires (see Schrepp, 2018), a broad literature review and by interviewing some UX experts (see the description in Winter et al., 2015, 2017). Thus, these UX aspects represent established facets of UX that are considered as important in the UX research community. The original German descriptions of the UX aspects were carefully developed and reviewed by several UX experts to make sure that they transfer the intended meaning of the UX aspects accurately and can be understood by study participants, i.e. by typical users of products and not only by UX experts.

Based on some current research (see Hassenzahl, 2001; Thüring and Mahlke, 2007; or Schrepp, 2018) these UX aspects can be grouped into several categories. Customization, Perspicuity, Efficiency, Intuitive Usage, Usefulness, and Controllability are pragmatic quality aspects, i.e. the quality aspects of a product that are related to the ease or difficulty to solve certain tasks with the product. Immersion, Novelty, Beauty, Identity, Value and Stimulation are hedonic quality aspects, i.e. the description of non-task related qualities of an interaction that are related to fun of use or emotional states of a user. Content Quality is not clearly pragmatic or hedonic, so it is somehow in-between. Loyality and Trust result from a high level of product quality over time, i.e. those aspects which are not so much related to concrete product features or product quality, but are in general over time based on positive or negative perceptions of users interaction with the product.

Obviously, this list of UX aspects cannot claim to be complete. Firstly, additional aspects are important for special products, for example acoustic quality and haptic feedback for household appliances (Boos and Brau, 2017). Second, new products will create new interaction paradigms and therefore new relevant UX aspects. For example, Trust was not considered as a relevant UX quality until e-Commerce and online banking became popular scenarios. Nevertheless, the list covers a wide range of UX aspects relevant for a broad range of products and is therefore sufficient for the interests of this research.

Practical relevance of the importance of UX aspects

What is the importance of such UX aspects of interest to designers? Let us illustrate this with a short example. Assume that we have a business application that is quite frequently used during a workday. Naturally, efficiency is quite an important UX aspect here. If a user is forced to do some unnecessary interactions for a task and this task must be performed 50 times a day that really hurts. That the application can be used intuitively is in this case nice, but clearly less important than efficiency (Mayer et al., 2018). Assume now that one of the planned features would increase intuitive usage, but decrease efficiency (for example, if a wizard replaces some more flexible screen flows). Given the higher importance of efficiency compared to intuitive usage, it would be wise to decide against this feature for this application.

Now let us assume we have a similar decision for a self-service application (for example, create a leave request). Such self-service applications are used rarely (once a month or less); thus, intuitive usage is quite important here, since we cannot assume that the user remembers how to use it between two usage points. Efficiency is of much less importance here. In this case we would draw a different decision based on the changed order of importance.

In the example above, it is intuitively clear how the importance of the UX aspects varies between the two different product categories. However, in many cases, this is less evident, and more empirical investigations are required to determine how important the different UX aspects are seen by the average users. In other words, if a product is designed to be used by users with different cultural background, it is good to know that the importance of UX aspects varies heavily between cultures. It would be problematic if users of two different cultures have distinctive views on the importance of UX aspect design decisions based on the assumptions that an UX aspect is more important than the other. In a previous study (Santoso et al., 2017) German and Indonesian participants evaluated Amazon and Skype with an extension of the User Experience Questionnaire (UEQ) (see Laugwitz et al., 2008 for the original version and Santoso et al., 2014 for the Indonesian translation). The results showed differences in the UEQ scale means and perceived the importance of UX aspects represented by the UEQ scales on the overall UX impression. However, the UEQ contains only 6 scales (each scale describes a distinct UX aspect), resulting in a lack of clarity of which findings should be generalized to other UX aspects. A second limitation of the study was that only two products were investigated. Our paper attempted to overcome these limitations by considering more UX aspects and a wider range of product categories.

In a study with German participants (Winter et al., 2017) the importance of several well-known UX aspects or UX dimensions was investigated for a wide range of different products. We replicated this study with Indonesian participants and compared the data from the two samples in this research.

Materials and methods

Our study is a replication of a study with German students described in Winter et al. (2017). We attempted to be as close as possible to the experimental procedure used in this study. Consequently, the textual information was translated from German to Indonesian.

Participants

One hundred and fourteen students of the Faculty of Computer Science at a large public university (64 males, 50 females) participated in the study. The students enrolled in the Human-Computer Interaction course and got some credit points for their participation in the study. The mean age was 21.34 years.

Compared to German version (58 participants; for details, see Winter et al., 2017), the data set from the Indonesian version is a bit larger.

Material

The participants were asked to provide their judgements regarding the importance of the UX aspects described in the previous section for several software product categories. Each category has a name and several examples. Several examples are added and adapted to Indonesian participants. Below is the list of the product categories and the examples used (if the example is only used in one of the two versions, it is followed by a German flag picture for the German version or an Indonesian flag picture for the Indonesian version) (Fig. 3).

Fig. 3

Product categories.

Procedure

The researchers sent an MS Excel list which contained the UX dimensions as rows and the product categories as column headers per mail. The participants were asked to fill out the Excel and send it back within one week. The same instructional text (all texts were carefully translated to Indonesian) as in Winter et al. (2017) was used. In both studies the students took some course credits for their participation.

Each cell could be filled with the answer categories Meaningless (this should be used if the UX dimension did not make sense at all for the product category), Extremely unimportant (1), Somewhat unimportant (2), Slightly unimportant (3), Neutral (4), Slightly important (5), Somewhat important (6), Extremely important (7). The instrument consists of 16 UX dimensions and 15 software product categories. The participants had to provide their judgments by filling 240 cells or making 240 decisions.

Results

Findings of the current study are presented using mean importance ratings per software product category and UX aspects. The results show differences between the German and Indonesian samples. If the difference is significant (two-sample-t-test, two sided) on the .05 level, the values are formatted in a bold font. The value in the last column describes the product moment correlation between the mean importance ratings of both samples (see Table 1).

Table 1

Means of the importance ratings for Perspicuity (PE), Efficiency (EF), Controllability (CO), Intuitive Use (IN), Usefulness (US), Customization (CU), Clarity (CL), Novelty (NO), Beauty (BE), Identity (ID), Stimulation (ST), Immersion (IM), Value (VA), Loyality (LO), Trust (TR), and Content Quality (CQ). Scale ranges from 1 to 7.

		PE	EF	CO	IN	US	CU	CL	NO	BE	ID	ST	IM	VA	LO	TR	CQ
Word Processing	D	5.91	6.31	6.60	5.60	6.45	5.07	6.00	2.46	3.33	1.52	3.24	2.37	4.84	3.84	5.26	4.10	0.94
	I	6.62	6.48	6.03	5.91	6.55	5.24	6.27	4.22	4.35	4.02	4.42	3.75	5.98	5.30	5.75	5.64
Spreadsheet	D	6.00	6.33	6.53	5.29	6.53	4.91	5.98	2.39	3.19	1.48	3.24	2.25	4.88	3.84	5.23	3.95	0.95
	I	6.51	6.54	6.02	5.75	6.58	5.12	6.27	4.21	4.28	3.95	4.39	3.87	5.96	5.15	5.80	5.67
Programming Tools	D	5.83	6.41	6.69	5.00	6.52	6.38	5.86	2.91	3.74	1.89	4.38	4.35	5.21	4.50	5.31	5.11	0.92
	I	6.34	6.23	6.14	5.66	6.25	5.82	6.02	4.79	4.58	4.15	5.13	4.65	5.99	5.04	5.44	5.70
Image Processing	D	5.79	6.22	6.43	5.14	6.34	6.00	5.88	3.11	4.31	1.78	4.70	4.41	5.33	4.63	5.02	4.36	0.91
	I	5.96	5.98	5.93	5.46	6.03	5.72	6.06	5.06	5.19	4.09	5.14	4.79	5.79	5.19	5.12	5.45
Booking system	D	5.97	6.14	6.47	5.74	5.55	3.76	6.03	3.00	3.93	1.56	3.22	2.38	5.71	3.40	6.70	6.44	0.89
	I	6.35	6.51	6.06	6.13	6.12	4.67	6.33	5.61	5.84	4.33	5.06	4.28	6.04	5.75	6.40	6.53
Online banking	D	5.79	5.84	6.83	5.83	5.59	3.14	6.02	2.29	3.21	1.38	2.39	1.98	6.10	4.00	6.98	6.68	0.94
	I	6.36	6.38	5.98	6.03	6.38	3.93	6.03	4.50	4.61	4.26	4.36	3.87	6.22	4.88	6.65	6.24
Web-Shop	D	5.97	5.47	6.17	5.93	4.96	3.31	6.07	4.60	5.59	2.00	4.70	4.40	5.98	4.45	6.62	6.42	0.91
	I	6.24	6.29	5.82	5.92	5.86	4.47	6.44	5.72	5.94	4.50	5.33	4.86	6.00	5.56	6.46	6.36
Messenger	D	5.95	5.98	6.09	6.28	4.60	4.03	5.45	3.98	4.50	5.75	4.51	3.35	4.60	4.58	6.67	4.93	0.76
	I	6.22	6.25	5.98	6.19	5.88	4.77	6.37	5.71	5.71	5.38	5.46	5.26	5.51	5.61	6.58	5.67
Social Network	D	5.55	5.00	5.91	5.66	3.73	4.69	5.26	4.84	5.31	6.39	5.65	4.58	4.90	5.19	6.66	5.67	0.65
	I	6.14	5.83	5.89	5.95	5.39	5.38	6.28	5.91	5.98	5.78	5.77	5.57	5.61	5.48	6.39	5.84
Video Conferencing	D	5.48	5.86	6.07	5.67	5.64	4.16	5.36	3.22	3.95	5.04	3.86	3.24	4.66	3.87	6.24	5.05	0.83
	I	5.83	6.12	5.68	5.74	5.91	4.65	6.06	5.03	5.10	4.92	5.19	4.88	5.50	5.11	6.19	5.44
Learning Platforms	D	5.95	5.78	5.53	5.55	6.05	5.05	5.97	4.38	4.90	4.08	5.43	4.70	5.36	3.96	5.56	6.53	0.84
	I	6.35	6.12	5.91	5.89	6.44	5.12	6.26	5.13	5.25	4.99	5.39	4.86	6.04	4.90	5.91	6.56
Video Portals	D	5.00	4.65	4.72	5.28	3.57	3.67	5.16	4.53	4.93	3.67	5.14	5.22	4.50	4.02	5.07	5.12	0.84
	I	5.57	5.46	5.54	5.47	5.22	4.63	6.11	5.50	5.54	4.99	5.92	5.74	5.38	5.33	5.68	5.75
News Portal	D	4.98	4.66	4.43	5.23	4.30	3.54	5.50	4.12	4.66	2.15	4.42	4.27	5.14	3.93	4.39	6.88	0.91
	I	5.26	5.26	5.09	5.51	5.14	3.78	5.90	5.29	5.39	3.99	4.85	4.74	5.68	4.76	5.38	6.53
Info Web Pages	D	4.97	4.79	4.41	4.86	4.56	2.67	5.60	3.93	4.64	2.63	4.11	3.64	4.60	3.25	4.35	6.41	0.93
	I	5.57	5.26	5.15	5.45	5.50	3.92	5.96	5.23	5.31	4.12	4.61	4.08	5.71	4.31	5.61	6.45
Games	D	5.53	5.02	5.82	5.48	2.06	4.47	4.81	6.33	6.38	4.50	6.84	6.84	4.67	5.06	4.56	4.04	0.89
	I	5.64	5.14	5.96	5.67	4.22	5.17	5.95	6.03	6.22	4.70	6.03	5.92	5.15	5.30	5.08	5.15
Average	D	5.64	5.63	5.91	5.50	5.10	4.32	5.66	3.74	4.44	3.05	4.39	3.87	5.10	4.17	5.64	5.45	0.93
	I	6.06	5.99	5.81	5.78	5.83	4.83	6.15	5.20	5.29	4.55	5.14	4.74	5.77	5.18	5.90	5.93

Looking at the average rating of the dimensions over all product categories it can be seen that the six highest differences between our two samples are found for Identity, Novelty, Loyalty, Immersion, Beauty and Stimulation. These are, except for Loyalty, all hedonic UX aspects. Fig. 4 shows the mean importance ratings as bar charts.

Fig. 4

Blue bars (left) represent the German sample, green bars (right) the Indonesian sample.

From Fig. 1, it can be seen that each product category has a specific pattern of importance ratings. Related product categories (for example, Word Processing and Spread Sheet, Programming Tools and Image Processing or Messenger and Video Conferencing) show quite similar patterns in both samples. In contrast, the other product categories show different patterns (for example, Learning Platforms and News Portal).

Hedonic UX aspects are relatively more important for Indonesian students than for German students. There is a clear discrepancy between both samples, especially for products that are mainly used for work or administrative personal tasks (Word Processing, Spread-Sheet, Programming Tools, Image Processing, Booking System, and Online Banking). In the German sample, the pragmatic qualities show very high ratings compared to hedonic quality aspects. Compared to the German sample, the pragmatic aspects are also rated higher than the hedonic aspects in the Indonesian sample, but the difference in the ratings is much lower.

However, it can also be seen that the correlations between the mean ratings are quite substantial between the samples. Thus, participants in both samples seem to have a similar opinion about the relative importance of the UX aspects in different product categories.

In order to achieve concrete design decisions based on the importance of UX aspects, for example if a design conflict between competing requirements should be solved, it is of interest to discover which the most important UX aspects are per product category. The rank order of the UX aspects is shown in Table 2. The value in the last column is also the rank correlation of the values from the German and Indonesian samples.

Table 2

Importance ranks for the UX aspects and the product categories.

		PE	EF	CO	IN	US	CU	CL	NO	BE	ID	ST	IM	VA	LO	TR	CQ
Word Processing	D	5	3	1	6	2	8	4	14	12	16	13	15	9	11	7	10	0.91
	I	1	3	5	7	2	11	4	14	13	15	12	16	6	10	8	9
Spreadsheet	D	4	3	1	6	1	8	5	14	13	16	12	15	9	11	7	10	0.94
	I	3	2	5	8	1	11	4	14	13	15	12	16	6	10	7	9
Programming Tools	D	6	3	1	10	2	4	5	15	14	16	12	13	8	11	7	9	0.90
	I	1	3	4	9	2	7	5	13	15	16	11	14	6	12	10	8
Image Processing	D	6	3	1	8	2	4	5	15	14	16	10	12	7	11	9	13	0.85
	I	4	3	5	8	2	7	1	14	11	16	12	15	6	10	13	9
Booking system	D	6	4	2	7	9	11	5	14	10	16	13	15	8	12	1	3	0.89
	I	4	2	8	6	7	14	5	12	10	15	13	16	9	11	3	1
Online banking	D	8	6	2	7	9	12	5	14	11	16	13	15	4	10	1	3	0.76
	I	4	2	9	7	2	15	8	12	11	14	13	16	6	10	1	5
Web-Shop	D	6	9	3	7	10	15	4	12	8	16	11	14	5	13	1	2	0.86
	I	5	4	10	8	9	16	2	11	7	15	13	14	6	12	1	3
Messenger	D	5	4	3	2	10	14	7	15	13	6	12	16	9	11	1	8	0.71
	I	4	3	6	5	7	16	2	9	8	14	13	15	12	11	1	10
Social Network	D	7	11	3	5	16	14	9	13	8	2	6	15	12	10	1	4	0.59
	I	3	9	7	5	15	16	2	6	4	10	11	13	12	14	1	8
Video Conferencing	D	6	3	2	4	5	11	7	16	12	9	14	15	10	13	1	8	0.81
	I	5	2	7	6	4	16	3	13	12	14	10	15	8	11	1	9
Learning Platforms	D	4	5	8	7	2	11	3	14	12	15	9	13	10	16	6	1	0.93
	I	3	5	7	9	2	13	4	12	11	14	10	16	6	15	7	1
Video Portals	D	7	10	9	1	16	14	3	11	8	15	4	2	12	13	6	5	0.83
	I	6	11	7	10	14	16	1	9	7	15	2	4	12	13	5	3
News Portal	D	5	6	8	3	11	15	2	13	6	16	9	12	4	14	10	1	0.85
	I	9	8	11	4	10	16	2	7	5	15	12	14	3	13	6	1
Info Web Pages	D	3	5	9	4	8	15	2	12	6	16	11	13	7	14	10	1	0.84
	I	5	9	11	7	6	16	2	10	8	14	12	15	3	13	4	1
Games	D	6	9	5	7	16	14	10	4	3	13	1	1	11	8	12	15	0.82
	I	8	13	4	7	16	10	5	2	1	15	3	6	12	9	14	11
Mean rank	D	5.60	5.60	3.87	5.60	7.93	11.33	5.07	13.07	10.00	13.06	10.00	12.40	8.33	11.87	5.33	6.20	0.90
	I	4.33	5.27	7.07	7.07	6.60	13.33	3.33	10.53	9.07	14.47	10.60	13.67	7.53	11.60	5.47	5.87

The order of the mean importance ratings of the different UX aspects in both samples is quite similar. Thus, the importance seems to be mainly determined by the product category and not so much by culture. Again, the high rank correlations indicate that the ranking of the UX aspects in both samples is highly similar, i.e. there is a high impact of the product type.

For designers it is interesting to understand which UX aspects are important and which can be neglected per product type. Surely, one could set a threshold and consider all UX aspects as important, which show ratings above this threshold. Still, the definition of such threshold must be done separately by sample and by far, as there is no method to do it objectively.

Therefore, a cluster analysis was done per product category and per sample (K-Means clustering) to split the UX aspects in two groups, corresponding semantically to important and unimportant UX aspects. K-Means clustering of a set of objects into n clusters was attempted to define n groups in a way that the members within a group are as homogeneous as possible, and the members of different groups showed a distinguished difference concerning the metric used for clustering. The results of this analysis are shown in Table 3.

Table 3

Results of a kmeans clustering into important (+) and unimportant (-) UX aspects.

		PE	EF	CO	IN	US	CU	CL	NO	BE	ID	ST	IM	VA	LO	TR	CQ
Word Processing	D	+	+	+	+	+	+	+	-	-	-	-	-	+	-	+	-
	I	+	+	+	+	+	+	+	-	-	-	-	-	+	+	+	-
Spreadsheet	D	+	+	+	+	+	+	+	-	-	-	-	-	+	-	+	-
	I	+	+	+	+	+	-	+	-	-	-	-	-	+	-	+	+
Programming Tools	D	+	+	+	+	+	+	+	-	-	-	-	-	+	-	+	+
	I	+	+	+	+	+	+	+	-	-	-	-	-	+	-	+	+
Image Processing	D	+	+	+	+	+	+	+	-	-	-	-	-	+	-	+	-
	I	+	+	+	-	+	+	+	-	-	-	-	-	+	-	-	-
Booking system	D	+	+	+	+	+	-	+	-	-	-	-	-	+	-	+	+
	I	+	+	+	+	+	-	+	+	+	-	-	-	+	+	+	+
Online banking	D	+	+	+	+	+	-	+	-	-	-	-	-	+	-	+	+
	I	+	+	+	+	+	-	+	-	-	-	-	-	+	-	+	+
Web-Shop	D	+	+	+	+	-	-	+	-	+	-	-	-	+	-	+	+
	I	+	+	+	+	+	-	+	+	+	-	-	-	+	+	+	+
Messenger	D	+	+	+	+	-	-	+	-	-	+	-	-	-	-	+	-
	I	+	+	+	+	+	-	+	-	-	-	-	-	-	-	+	-
Social Network	D	+	-	+	+	-	-	-	-	-	+	+	-	-	-	+	+
	I	+	+	+	+	-	-	+	+	+	-	-	-	-	-	+	+
Video Conferencing	D	+	+	+	+	+	-	+	-	-	+	-	-	-	-	+	+
	I	+	+	+	+	+	-	+	-	-	-	-	-	+	-	+	+
Learning Platforms	D	+	+	+	+	+	-	+	-	-	-	+	-	+	-	+	+
	I	+	+	+	+	+	-	+	-	-	-	-	-	+	-	+	+
Video Portals	D	+	+	+	+	-	-	+	+	+	-	+	+	+	-	+	+
	I	+	+	+	+	-	-	+	+	+	-	+	+	+	+	+	+
News Portal	D	+	+	+	+	+	-	+	+	+	-	+	+	+	+	+	+
	I	+	+	+	+	+	-	+	+	+	-	-	-	+	-	+	+
Info Web Pages	D	+	+	+	+	+	-	+	-	+	-	+	-	+	-	+	+
	I	+	+	+	+	+	-	+	+	+	-	-	-	+	-	+	+
Games	D	+	-	+	+	-	-	-	+	+	-	+	+	-	-	-	-
	I	+	-	+	+	-	-	+	+	+	-	+	+	-	-	-	-

Again, from Table 3, it is confirmed how similar the perceived importance of the UX aspects per product type is in our two samples. With a few exceptions UX aspects that are classified as important in Germany are also classified as important in Indonesia and vice versa. Differences seem to be lower for tools (for example, Word Processing or Programming Tools) that are clearly associated to clear defined work tasks. The biggest deviations are seen for applications (for example, Social Network, Web Shop, or News Portal) that are more related to spare time activities.

Now we turn our attention to the question of how big the impact of culture is compared to the impact of other interindividual differences between persons. Table 4 shows the variances of the observed importance ratings in both samples.

Table 4

Variances for the importance ratings.

		PE	EF	CO	IN	US	CU	CL	NO	BE	ID	ST	IM	VA	LO	TR	CQ
Word Processing	D	0.98	1.08	0.34	1.48	0.49	2.37	0.97	2.18	1.94	1.85	2.10	2.13	2.06	3.51	2.55	2.00
	I	0.46	0.41	1.17	1.27	0.48	2.66	0.92	2.47	1.93	2.16	2.17	1.91	1.23	2.11	1.97	1.63
Spreadsheet	D	0.93	1.12	0.39	2.48	0.46	2.25	0.98	2.15	2.02	1.91	2.17	2.07	2.18	3.65	2.49	2.24
	I	0.67	0.44	1.16	1.59	0.63	2.80	0.99	2.27	2.17	2.01	2.06	2.03	1.34	2.46	1.91	1.70
Programming Tools	D	1.49	1.07	0.28	2.62	0.59	0.44	1.46	2.26	2.09	2.12	2.27	3.25	1.75	2.49	2.50	2.06
	I	1.17	1.02	1.14	1.68	1.29	1.50	1.19	2.24	2.00	2.54	2.31	2.69	1.24	2.18	2.61	1.67
Image Processing	D	1.27	1.04	0.62	2.64	0.54	0.72	1.45	2.73	2.11	2.07	2.15	2.68	1.53	2.65	2.37	1.61
	I	1.87	1.19	1.47	1.98	1.62	2.03	1.02	1.69	1.50	2.41	1.91	2.67	1.36	2.01	2.82	1.83
Booking system	D	0.93	0.74	0.59	1.50	1.42	2.62	0.86	2.19	1.55	1.91	2.34	1.56	1.21	3.52	0.61	1.45
	I	0.72	0.60	1.29	0.87	1.53	2.97	0.87	1.41	1.13	2.60	2.19	2.88	1.48	1.82	1.22	0.58
Online banking	D	2.06	1.30	0.28	1.87	1.32	2.07	1.19	1.92	2.00	1.81	1.66	1.80	1.09	2.86	0.02	0.71
	I	0.98	0.72	1.82	1.27	0.88	3.09	1.34	2.46	2.15	2.39	2.52	2.50	1.27	2.81	0.69	1.50
Web-Shop	D	1.17	1.73	1.07	1.03	2.02	1.92	0.82	2.55	0.66	1.76	1.73	3.06	0.74	3.31	0.65	0.58
	I	0.92	0.84	1.41	1.11	1.85	2.90	0.49	1.24	1.04	2.72	1.79	2.45	1.61	2.47	0.83	1.00
Messenger	D	1.39	1.15	0.98	0.79	2.80	2.38	1.18	2.84	1.84	1.72	1.53	2.27	2.20	2.73	0.39	2.75
	I	1.15	0.93	1.24	1.10	1.77	2.84	0.74	1.14	1.22	1.88	1.62	2.49	1.95	1.75	0.77	1.88
Social Network	D	1.70	1.91	1.11	0.92	2.67	2.49	1.64	1.90	0.97	1.23	0.96	3.14	1.64	2.76	0.43	1.20
	I	0.88	1.48	1.48	1.10	1.87	1.83	0.83	1.09	1.05	1.58	1.47	2.49	1.92	1.88	1.20	1.48
Video Conferencing	D	1.42	1.57	0.96	0.98	1.44	2.20	1.37	2.17	1.53	2.32	1.39	2.69	1.74	2.62	1.08	1.94
	I	1.16	1.14	1.50	1.53	1.23	2.20	0.94	1.91	2.13	2.38	1.95	2.21	1.81	2.01	1.07	1.85
Learning Platforms	D	0.81	1.24	1.32	1.25	1.33	2.12	0.62	3.37	1.27	2.03	1.83	3.28	1.47	3.38	2.49	0.52
	I	0.96	1.07	1.16	1.30	0.47	2.28	1.09	1.60	1.40	2.24	2.34	2.19	1.58	2.50	1.81	0.70
Video Portals	D	1.55	2.20	1.99	1.51	2.31	2.15	1.34	2.39	1.37	2.54	2.11	2.34	1.01	3.12	2.10	1.44
	I	1.63	1.79	1.76	2.09	2.20	2.56	0.89	1.39	1.39	2.17	1.44	2.09	1.82	2.14	1.81	1.47
News Portal	D	2.00	2.85	2.18	1.92	2.23	2.80	1.77	2.62	1.12	2.12	1.61	2.33	1.98	3.10	3.08	0.21
	I	2.11	2.16	2.34	1.65	2.16	2.34	1.31	1.68	1.50	2.55	2.24	2.43	1.79	2.71	2.29	0.72
Info Web PaGamesges	D	1.79	2.51	1.97	1.86	2.04	2.36	1.27	2.65	1.33	2.35	2.37	2.53	1.79	3.41	4.00	0.62
	I	1.76	2.17	2.79	1.93	1.95	2.22	1.40	1.40	1.35	2.53	2.33	2.41	1.90	2.54	2.78	0.65
Games	D	1.46	3.83	1.68	1.84	2.13	2.49	2.08	0.63	0.86	2.42	0.23	0.17	2.64	3.14	3.32	3.07
	I	1.81	2.73	1.63	2.01	2.71	2.46	1.55	1.78	1.47	3.02	1.94	2.57	2.56	3.18	3.25	2.69

For a rating scale with just 7 points the variances are quite substantial for most combinations of product types and UX aspects. This is the first indication that there is a massive impact of interindividual differences even for people with a similar cultural background.

To get a clearer picture concerning the variance of the two culturally homogeneous samples, an analysis of variance was performed. Thus, for each combination of product types and UX aspects the total variance VAR over the complete data set (German and Indonesian participants) was calculated. The variance explained by the two groups VAR(G,I) was calculated using the formula as follows:where n,n are the numbers of persons in the German and Indonesian sample, and are the mean values in both samples and the complete data set. This is the variance we would observe if all persons inside one cultural sample would show the same importance rating, i.e. if the importance ratings would be completely dependent on culture.

Table 5 shows the value of VAR(G,I)/VAR, i.e. the relative amount of variance explained by the cultural groups compared to the complete variance. The impact of culture is quite small compared to the impact of differences on individual level.

Table 5

Proportion of the variance explained by the cultural differences.

	PE	EF	CO	IN	US	CU	CL	NO	BE	ID	ST	IM	VA	LO	TR	CQ
Word Processing	0.15	0.01	0.08	0.02	0.01	0.00	0.02	0.20	0.11	0.08	0.12	0.13	0.16	0.15	0.06	0.23
Spreadsheet	0.07	0.01	0.06	0.02	0.00	0.00	0.02	0.23	0.11	0.08	0.11	0.16	0.14	0.12	0.07	0.24
Programming Tools	0.04	0.01	0.08	0.05	0.02	0.06	0.00	0.25	0.07	0.10	0.05	0.01	0.09	0.03	0.01	0.07
Image Processing	0.00	0.01	0.05	0.01	0.02	0.01	0.01	0.29	0.09	0.11	0.02	0.01	0.03	0.03	0.00	0.14
Booking system	0.04	0.05	0.03	0.03	0.05	0.05	0.02	0.47	0.39	0.16	0.24	0.19	0.02	0.33	0.01	0.00
Online banking	0.05	0.06	0.11	0.00	0.13	0.05	0.00	0.31	0.17	0.12	0.23	0.17	0.00	0.05	0.06	0.03
Web-Shop	0.02	0.12	0.02	0.00	0.09	0.09	0.05	0.14	0.03	0.20	0.04	0.02	0.00	0.09	0.01	0.00
Messenger	0.01	0.01	0.00	0.00	0.15	0.04	0.17	0.28	0.19	0.02	0.12	0.25	0.08	0.11	0.00	0.09
Social Network	0.06	0.09	0.00	0.01	0.21	0.05	0.17	0.15	0.09	0.05	0.00	0.07	0.06	0.01	0.02	0.01
Video Conferencing	0.02	0.01	0.03	0.00	0.01	0.02	0.09	0.26	0.13	0.00	0.18	0.19	0.08	0.13	0.00	0.06
Learning Platforms	0.04	0.02	0.02	0.02	0.05	0.00	0.02	0.05	0.02	0.07	0.00	0.00	0.06	0.06	0.02	0.00
Video Portals	0.04	0.07	0.07	0.00	0.19	0.07	0.16	0.10	0.05	0.12	0.08	0.03	0.09	0.13	0.04	0.06
News Portal	0.01	0.03	0.04	0.01	0.06	0.01	0.02	0.13	0.08	0.09	0.02	0.02	0.03	0.05	0.07	0.05
Info Web Pages	0.04	0.02	0.04	0.04	0.09	0.11	0.02	0.16	0.07	0.07	0.02	0.01	0.12	0.07	0.10	0.00
Games	0.00	0.00	0.00	0.00	0.15	0.04	0.13	0.02	0.01	0.00	0.11	0.11	0.02	0.00	0.02	0.09

Again, there are differences for the different product categories and UX aspects. The proportion of variance explained by the cultural differences is in general higher for the hedonic UX values than for the pragmatic ones. However, the overall values are quite low compared to the interindividual differences inside the culture.

Then, some additional analyses, which are a bit more explorative in nature, were conducted to the data set. They may help to generate some more hypothesis for future research. Firstly, the similarity of product categories was identified. A multi-dimensional scaling (MDS) was used to visualize these data (Torgerson, 1958). An MDS is based on a set of objects (in the case of this research, it was the product categories) and a matrix that shows the similarity indeks for each pair of objects. The similarity index of product categories was calculated based on the correlations of the importance ratings for all UX aspects. Thus, two products are deemed similar if they show similar importance ratings for the UX aspects. The MDS then illustrates the objects as points in a two-dimensional space, so that the Euclidean distance between the points reflects the similarity of the objects as close as possible. Thus, it is mainly a visualization technique (Fig. 5).

Fig. 5

MDS results for the product categories.

Games, Video Portals and Social Networks are seen in both samples as product categories t which are quite distinct from other product categories concerning the importance of the UX qualities. In the German sample, a group of communication tools (Video Conferencing, Messenger) is separated from the rest. In the Indonesian sample, these communication tools are not seen as a group of products with special requirement, whereas Programming Tools and Image Processing are somehow separated.

Fig. 6 depicts an MDS based on the similarity of UX aspects in both samples. Two UX aspects are similar if they show similar importance ratings over all product categories.

Fig. 6

MDS results for the UX quality aspects.

Again, there are some similarities between the two cultures, but there are also some remarkable differences. The two groups, the hedonic (orange circles) and the pragmatic quality (blue circles) can clearly be seen. Intuitive usage is in both cultures between hedonic and pragmatic qualities (which are interesting, since typically this is seen as a pragmatic quality aspect). Clarity in Indonesia is close to Intuitive usage, while in Germany clearly it is close to the pragmatic qualities. Value in both cultures is close to pragmatic quality, while Content Quality is quite distinct from other quality aspects.

Discussion

A study of Winter et al. (2017) was replicated with the involvement of Indonesian students as the participants to determine if there is an impact of cultural differences on the importance of UX aspects for product categories. Results show significant differences regarding the rated importance of UX aspects for many product categories. To be more specific, hedonic UX aspects seem to be of relative higher importance for Indonesians than for Germans. This result is also in line with some previous research (Santoso et al., 2017).

For concrete design decisions however only the most important UX aspects play a role. The order of importance is fairly similar in both samples, since the correlations between the mean importance ratings are very high. This is also true for the ranks concerning importance. Thus, in cases where decisions between competing features of a product on the basis of the importance of UX aspects should be made, it could be assured that these decisions would be valid for both cultures. It seems therefore unlikely that a completely different approach is needed for the interaction design to be acceptable in both cultures. Inevitably, there may be details, such as the icons used in a user interface, which need to be considered for cultural differences. However, the general impression of which qualities are important seems to be not much influenced.

In addition, the impact of cultural differences seems to be rather small compared to interindividual differences of persons inside a culture or compared to the product types. Thus, personal preferences seem to be more important for the subjective importance of UX quality aspects than the impact of culture.

There are some obvious limitations in our research. Firstly, it only investigated two cultures. These show reasonably distinct patterns on the cultural dimensions of Hofstede (2001) model, so there are certainly good candidates to find some cultural impacts on the UX perception. Still, there could definitely be considerable differences of effects in other cultures, requiring a collection of more data in some other countries to get a clearer picture here. Secondly, there has always been a problem in investigating cultural differences because instructional materials must be translated. Therefore, it is always possible that a not optimal translation influences the result. In the case of the research, the material of the German study was first translated to English, and the Indonesian version based on the English translation was constructed after. Despite being as careful as possible in the process, it is undoubtedly possible that the double translation might have altered the meaning. However, these effects would most likely influence only some of the combinations of UX aspects and product categories; hence, the general pattern of findings would most likely be not much influenced. The same applies to what impact of the different examples in the product categories (forced by the fact that some of the original examples from the German study were not known in Indonesia) might have on the result. Thirdly, the participants of our study rated the importance of the UX aspects for product categories and not for concrete products. Thus, different participants had maybe different products in mind when they did their evaluation. In addition, we could not guarantee that each participant has practical experience with products from each of the product categories. On the other hand, the used product categories represent quite common types of interactive products, so that we can assume that the students came in touch with most of them. In addition, the participants were not forced to rate a product category completely unfamiliar to them. We assumed that the importance of an UX aspect depends mainly on the structure of the tasks performed with typical products of this category. Given this assumption, the concrete products participants have in mind when they rated the importance of UX aspects for a product category should have a limited impact on the rating. Further studies should be conducted if this quite natural assumption is justified.

Conclusion

This study compared a previous study involving German participants and a replication study involving Indonesian participants to capture whether the cultural background has an influence on the subjective importance of UX. The participants were invited to make judgments upon software product categories across different UX dimensions. Results revealed similar and different patterns of importance ratings between German and Indonesian samples. Participants provided different judgments upon software products used for administrative tasks, learning activities or for fun. These findings provide additional insights into how users of products from different cultures perceive software products based on the importance of UX. The insights gathered in the current study can be used as considerations for developing products targeting users from different cultural background.

Declarations

Author contribution statement

Harry B. Santoso, Martin Schrepp: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

The researcher received funding from the Directorate of Research and Community Engagements Universitas Indonesia, Indonesia, to cover the publication fee of the article. The contract number is NKB-0208/UN2.R3.1/HKP.05.00/2019.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.