Salient knowledge that others are also evaluating reduces judgment extremity.

As companies increasingly conduct marketing research online (e.g., through social networking sites or their brand community platforms), the knowledge that others are also filling out the same surveys becomes increasingly salient to respondents. This research examines how the salience of this knowledge influences consumer judgments. Two important characteristics of our research paradigm are especially relevant to digital contexts: (1) judgements made by the consumers are neither observable nor subject to others' disapproval; and (2) consensus is not observable or verifiable. Nevertheless, in six main studies and one auxiliary study (Web Appendix), we found that high knowledge salience of others also evaluating reduced judgment extremity. Judgment extremity is quantified by the degree or strength of an evaluation or numeric estimate about a judgment target. This effect was driven by consumers' tendency to predict a moderate consensus and to conform to this perception. Implications for marketing research and crowdsourcing are discussed. Supplementary Information: The online version contains supplementary material available at 10.1007/s11747-021-00807-w. © Academy of Marketing Science 2021.

It has been argued that the success of many, if not all, marketing initiatives hinges upon the insights learned from the marketing research supporting it (Dolan & John, 2015). In 2018, companies across the globe spent up to $80 billion collecting consumers’ opinions through marketing research (ESOMAR, 2019). Firms can research consumers by conducting personal surveys or interviews via telephone, in person, or by email. In this digital era, marketers have been increasingly relying on online platforms and social networking sites to gather market intelligence on myriads of topics. A recent industry report suggests that 17% of all research spends worldwide (approximately $14 billion) are on online surveys, and that this number is growing annually (Statista, 2021). Indeed, in Germany, it has been estimated that about 50% of the spending on surveys are for online research. These topics of these online research surveys range from basic research on brand positioning to more complex research, such as testing new products via digitally simulated shopping with virtual others (Baron & Siepmann, 2000; Dolan & John, 2015).

While online platforms aid marketing research, could answering surveys in these digital (vs. traditional) contexts inadvertently cause respondents to become increasingly aware of others taking the same survey? One of us, for example, was recently invited to a survey via an online social network, and the invitation came with small thumbnail photographs of many others who were also participating. Firms that gather marketing intelligence online routinely use social media platforms (e.g., Facebook, Twitter) or streaming sites (e.g., YouTube) to highlight how many people have responded to their surveys or polls (see Appendix for examples). Similarly, online networking sites (e.g., LinkedIn) provide the number of users who are interested in a particular topic or news story. Crowdsourcing sites often highlight their online traffic for each task or project. While these features can increase the participation rates or realism of the studies, they might inadvertently make the knowledge that others are also evaluating more salient to the respondents at the time of judgment. For example, although participants know they are not the only one responding to the survey, this knowledge might not be salient unless they see a recruitment message saying, “Other people like you have filled out the online survey.” How will the salient knowledge that others are also evaluating impact respondents’ expressed opinions about a product, service, or program in marketing surveys? Will this salient knowledge influence judgments even though respondents know their opinions are unobservable to others and are thus not subject to disapproval? We draw from prior research on social influence and need for conformity (Cialdini, 2009) to propose that there will be an effect.

It has been well established that the physical presence of others increases consumers’ need to conform to a group norm or consensus by creating a desire to manage impressions, leading people to act in socially desirable ways (Cialdini, 2009; Leonardelli et al., 2010). The presence of others can also cause consumers to adopt an accuracy heuristic favoring the group consensus (Cialdini, 2009; Puntoni & Tavassoli, 2007; Zhang & Liu, 2012). An important premise undergirding the effect of conformity is that both the consensus and consumers’ own judgment(s) are observable in the context of these studies. However, would the need for conformity still influence judgments when consensus is unobservable and judgments are not subject to others’ disapproval? To answer this question, we conducted seven experiments (six main studies and one auxiliary study) and found that when it is salient in their mind that others are also evaluating, people conform to predicted consensus and report less extreme judgments, despite the anonymity and confidentiality of their responses, a counterintuitive finding that prior research does not predict. Our new paradigm augments the studies on social influence and need for conformity and shows that both physical and cognitive presence of others can influence anonymous consumer judgments.

In the rest of the paper, we will first review the literature on need for conformity, followed by a discussion of the relationship between conformity, knowledge salience of others, and predicted consensus. We will then present seven studies and discuss the theoretical and practical implications, limitations, and future directions of our research.

Conceptual framework

Need for conformity in group settings

Individual behaviors in a group setting are influenced by the opposing forces of the need for conformity and the need for uniqueness. Conformity refers to the act of changing one’s behavior or opinion to match the response of others (Cialdini, 2009). Uniqueness refers to the act of changing one’s behavior or opinion to deviate from the norms (Simonson & Nowlis, 2000). Conformity mitigates the risks of rejection, while uniqueness gives people intrinsic satisfaction (Simonson & Nowlis, 2000; Snyder & Fromkin, 1977). According to this literature, conformity plays a larger role in judgment and decision-making than uniqueness; people tend to adhere to, rather than deviate from, social norms in the presence of others to ensure the appropriateness of their behaviors or opinions (Asch, 1951; Cialdini, 2009).

The literature is replete with studies showing the effects of conformity in various contexts. In an early demonstration, Asch (1951) showed that participants who could correctly estimate the length of a line made errors when surrounded by a group of confederates all voicing an incorrect answer. Likewise, people display greater variety-seeking tendency (a publicly desirable behavior) in public than in private settings (Ratner & Kahn, 2002). When people know the opinions of the audience, they express more liberal attitudes to a liberal audience than to a conservative audience (Tetlock et al., 1989). Making people aware of others’ positive behavior can also nudge them toward desirable behavioral changes. For example, greater number of small business owners pay tax on time when knowing that majority others do (Cabinet Office UK Behavioural Insights Team, 2012), and people are more likely to give to a charity if it is viewed as the social norm (Smith et al., 2015). Neuroscience research in the last decade provides additional evidence for how the opinion of others affects the valuation and perception of simple stimuli (e.g., estimating number of dots in an image; Stallen et al., 2013), changes in attitude (vs. mere conformity; Zaki et al., 2011), and changes in risk-taking (Chein et al., 2011; for reviews, see Stallen & Sanfey, 2015; Schnuerch & Gibbons, 2014).

As seen in existing studies on conformity (Cialdini, 2009), the standard paradigm is to publicly compare individual’s judgments with varying group consensus information or creating an anticipation of such comparison subsequently. Some studies were conducted in private settings, and participants were provided with explicit consensus information, which evokes social comparison and the need to conform. For example, people decrease their energy use when informed that they use more power than their neighbors (Schultz et al., 2007). Would the need for conformity still play a role when consumers, being keenly aware of others also evaluating, fill out confidential surveys? In this sense, our research tests the role of the need for conformity under a combination of two contexts that have not been examined in the literature: (1) when consumers’ opinions are not viewed or judged by others; and (2) when the group consensus is unobservable and unverifiable. Our research also addresses a common marketing research procedure whereby consumers answer surveys without access to consensus information.

The knowledge salience of others and need for conformity

The malleability of knowledge salience of others

Most theories of human judgment assume that people base their judgments on declarative information (facts and memories that are consciously recalled) that is relevant to the judgment target and happens to come to mind at the time of judgment (Higgins, 1996; Schwarz, 2004; Wyer, 2008). In the context of social influence, calling attention to people’s cultural identity can increase conformity to certain cultural norms in choices and behaviors, but the same identity has little effect when it is not salient at the moment of decision making (Briley, 2005; Briley & Wyer, 2002; Briley et al., 2000). In our study, although most people in any survey recognize that they are not the only participants, this knowledge is only incorporated into judgement and decision making when it is salient in people’s mind. Presumably, the salient knowledge of others also evaluating can vary due to its accessibility in working memory. Calling attention to the fact that others are also evaluating should enhance the salience of this knowledge and thus increase people’s tendency to conform to group consensus in their judgments.

Note that knowledge salience of others can fluctuate in both a cognitive and physical sense. When a consumer fills out a survey by themselves, the knowledge of others also evaluating is probably not salient. Yet, a messaging prompt as shown in Studies 1–5 can increase the salience of this knowledge and the cognitive presence of others. Alternatively, when the consumer fills out a survey in a group setting, the physical presence of others in Study 6 will automatically increase the salience of others at the moment of making judgments. In this research, we devote our attention primarily to the cognitive presence of others for three reasons: (1) digital marketing research is becoming increasingly important; (2) prior research on need for conformity in physical group settings has not been tested extensively in online settings where consumers complete surveys alone; and (3) online settings provide a natural context that ensures the anonymity and confidentiality of survey responses.

Need for conformity

The literature suggests that conformity is driven by normative and/or informational conformity motivations (Deutsch & Gerard, 1955; Spears, 2021; Spears et al., 2001). The normative account is based on the desire to obtain social approval from others. The normative account alone implies that high knowledge salience of others should have little effect on judgments when survey responses are anonymous and confidential. On the other hand, the informational account is based on the desire to form an adequate interpretation of reality and behave accordingly. When people are unclear about the appropriateness of their judgments, they can mitigate it by agreeing with others with whom they identify (Festinger, 1954; Hughes et al., 2019; Orive, 1988; Petrocelli et al., 2007; Visser & Mirabile, 2004). Indeed, when people are unsure whether their judgments are sound, justified, or valid, they tend to align their opinion with the majority’s opinion and gravitate toward that consensus, which serves as an accuracy heuristic (Erb et al., 2002; Mackie, 1987). In summary, the salient knowledge of others is unlikely to trigger the need for normative conformity. However, the informational account suggests that salient knowledge of others should influence judgments even when: (1) one’s own judgments are not the target of others’ disapproval; and (2) the actual norm (others’ opinions) cannot be observed or verified. Just as the physical presence of others can increase informational motivation, the cognitive presence of others also evaluating may increase individuals’ motivation to align their judgments with group norms or group consensus. Therefore, we propose that the salient knowledge of others also evaluating could increase people’s conformity to consensus.

Predicted consensus

How might consumers conform to the consensus of others who are also evaluating when the consensus is unobservable and unverifiable? They will need to make predictions about the consensus first. Prior research suggests that when the appropriate judgment about an object, event, or person is unclear, people tend to express moderate opinions. Indeed, when consumers are uncertain about the preferences of others, they tend to avoid extremes and “choose the middle alternative, which is likely to be the safe choice with the smallest maximum error” (Simonson, 1989, p. 162). Similarly, when others’ preferences are unclear, people tend to consider opinions from different perspectives and integrate arguments for both pros and cons about an issue, resulting in moderate opinions (Tetlock et al., 1989). Although the paradigms in these studies are different from ours, these studies suggest that people tend to choose middle-ground options or express moderate opinions when the appropriate judgment is unclear due to various contextual factors. Applying this principle to our study where the group consensus is unclear, unobservable, and unverifiable, we conjecture that people will also avoid extreme values when predicting the group consensus, resulting in a moderate prediction. When consumers conform to their moderate predictions about the group consensus, the extremity of their own judgments will decrease accordingly. More formally,

H1

The high salience of the knowledge that others are also evaluating reduces judgment extremity.

H2

Salient knowledge of others reduces judgment extremity by increasing the need to conform.

Boundary condition

One key determinant of the need to conform is the perceived similarity between the judges and their group members. People have a need to conform to a group norm or consensus when they perceive other group members as similar to them. However, this need lessens when group members are perceived as dissimilar (an out-group; Turner, 1991). In fact, people avoid and disparage out-groups (e.g., Brewer, 1979; Marques et al., 1998; Turner, 1975). For example, they exhibit negative attitudes toward lifestyles (e.g., skaters vs. jocks) they wish to avoid (Lowrey et al., 2001). Therefore, the physical or cognitive presence of dissimilar others (i.e., out-group) should reduce conformity to dissimilar others’ norm or consensus. Furthermore, consumers may wish to dissociate themselves from certain reference groups that they do not identify themselves with (Englis & Solomon, 1995; Turner, 1991). For example, males had more negative evaluations of, and were less inclined to choose, a product associated with a dissociative (i.e., female) reference group than a neutral product (White & Dahl, 2006). Drawing on this literature, we propose a theory-driven boundary condition where lower perceived similarity of others reduces the need to conform, thereby attenuating the effect of salient knowledge of others on judgment extremity. More formally,

H3

When similarity of others is low (which reduces need for conformity), the effect of salient knowledge of others will be weakened.

Overview of studies

Study plan

As shown in Table 1, we conducted six main studies and one auxiliary study (see Web Appendix, WA) to test our hypotheses. We first showed the core effect (H1) that the salient knowledge of others also evaluating reduced the extremity of judgments about business events in Study 1 and hedonic products with negative or positive valance in Study 2. To demonstrate people’s tendency to align their judgments with predicted consensus, we included a consensus-prediction condition in Studies 1 and 3 where participants were asked to predict the consensus of other students who also filled out the survey and confirmed that the (low) extremity of their prediction was similar to that in the high-salience condition. Moreover, we manipulated perceived consensus by providing consensus information explicitly to participants. In Study 3, we replicated the effect of H1 by providing moderate consensus information, and in its follow-up study, we attenuated the effect by providing extreme consensus information. Next, we obtained process evidence (H2) by directly manipulating participants’ belief in the need to conform in Study 4. In Studies 5 and 6 we provided additional support for the conformity account by testing the boundary condition where similarity of others also being surveyed was lowered (H3).

Table 1

Overview of Studies (All Between-Subjects Design)

Study Purpose	Study Design	DV: Judgment Targets	Manipulation of KS	Potential Confound Checks
Demonstrate core effect of knowledge salience (KS) (H1) Test the role of perceived consensus in S1, S3, and the follows-up of S3	Study 1 (3-level): High KSLow KS/controlConsensus prediction	Financial product (stock, foreign currency) Entertainment (movies) Event (job placement) Converted into z-score	Messaging prompt of othersPhoto task	Age, Gender, Expertise, Affect
	Study 2: 2 (KS) × 2 (stimulus valance)	Consumer product (poster)	Messaging prompt of othersPhoto task	Age, Gender, Self-construal
	Study 3 (4-level): High KSLow KS/controlModerate consensusConsensus prediction	Consumer product (poster)	Messaging prompt of others	Age, Gender, Competencea
Test the role of need for conformity (H1 & H2)	Study 4: 2 (KS) × 2 (need for conformity)	Financial product (four stocks)	Messaging prompt of othersPhoto task	Age, Gender, Expertise, Affect
Further test the role of need for conformity via a boundary condition on similarity (H2 & H3)	Study 5: 2 (KS) × 2 (similarity of others)	Financial product (stock, purchase intention) Consumer product (wearable health device SIMBAND) Converted into z-score	Messaging prompt of othersPhoto task	Age, Gender, Expertise, Self-construal
	Study 6 (field study): 2 (KS) × 2 (similarity of others through course type)	MBA course evaluation	In class vs. outside of class	NA

aIn the follow-up study of Study 3 (see WA-D), we attenuated the effect of knowledge salience by manipulating the extremity of consensus and measured potential confounding variables including age, gender, expertise, NFU, self-confidence, and involvement.

Importantly, we do not compare judgment extremity between those who have the knowledge of others also evaluating versus those who do not have this knowledge. Most people, if not all, know that they are not the only participant of any survey. Our key construct is the salience of this knowledge during judgment, which we manipulated using a message prompt to remind participants that others are also evaluating, accompanied by photos or descriptions of other participants (see Table 1).

To cleanly test our hypotheses, our manipulations have no bearing on how participants would be judged by others. We accomplished this in Studies 1–5 by conducting the studies entirely online without interaction between participants and human experimenters. In Study 6 (a field study) where others were physically present in the high-salience condition, the MBA students knew their opinions would not be subject to others’ disapproval. Although some students were surrounded by their classmates when asked to complete the survey in class and other students did it outside without their classmates around, they all filled out the confidential survey online anonymously.

Dependent measures

Across different studies, our dependent measures include predictions about future events (e.g., financial forecasting) and evaluations of products and services (e.g., consumer products, education program). It is important to include predictions about future events because they are highly influential on consumers’ subsequent purchase decisions. For example, a consumer’s prediction of stock price changes will determine important financial and purchase decisions (e.g., Will Apple’s stock continue to go up? Should I invest in this stock now? Will this allow me to splurge on my 25th wedding anniversary?).

To capture participants’ judgment extremity and ensure the robustness of the findings, we tested our hypotheses using different judgment tasks. In accordance with extant literature (Soll & Klayman, 2004; Tannenbaum et al., 2017; Tsai et al., 2008), we quantified judgment extremity by the degree or strength of an evaluation or the numeric estimate about a judgment target. As in Tannenbaum et al. (2017), when judgment targets are rated on bipolar scales with the midpoint being the explicit status-quo prediction or neutral evaluation (Studies 1, 3, 4, and the stock prediction task in Study 5), we measured judgment extremity by taking the mean absolute distance (MAD) from the midpoint of the scale. In Study 2 where a bipolar scale was also used, we analyzed judgment extremity using raw ratings to clearly demonstrate the effect of high knowledge salience on positive and negative stimuli.1 To expand our findings, we used unipolar scales without explicit neutral evaluation point in Study 5 (purchase intention and product evaluation) and Study 6 (course evaluation). As in Skowronski and Carlston (1987), we measured judgment extremity using raw ratings when judgment targets are rated on unipolar scales.

Finally, in Studies 1 and 5 where different judgment tasks and scales were used, we (1) computed MAD for items rated on bipolar scales for each participant, (2) standardized MAD and raw ratings (for items rated on unipolar scales) for each task into z-scores, and then (3) created an extremity index by averaging the z-scores across judgment tasks. The consistent results across different judgment tasks and measurements of extremity attest to the robustness of the findings.2 (see Table 2 for a summary of results in each study). See WA for details about the study stimuli and data exclusion criterion.

Table 2

Summary of Study Results

Study 1	High salience		Control			Consensus
Overall extremity z-score index	–.09		.12			–.04
MAD for predictions about stock price change and box office revenue (-60% to 60%)	17.68%		20.94%			18.91%
MAD for predictions about currency and job placement (-18% to 18%)	4.14%		4.56%			4.32%
Study 2	Attractive poster				Unattractive poster
	High salience	Control			High salience			Control
Poster evaluation (Likert scale)	2.95	3.50			–2.04			–2.56
Study 3	High salience	Control			Moderate consensus			Consensus prediction
MAD for poster evaluation (5-star scale)	.92	1.22			.97			.86
Study 4	High NFC				Low NFC
	High salience	Control			High salience			Control
Average MAD for stock price change (%)	11.21%	15.13%			17.32%			16.91%
Study 5	Similar others				Dissimilar others
	High salience		Control		High salience			Control
Overall extremity z-score index	–.21		.08		.19			–.05
MAD for stock price change (-60% to 60%)	11.35%		13.59%		15.31%			13.24%
Purchase intention (1–7 Likert scale)	2.76		3.64		3.47			3.09
SIMBAND evaluation (1–9 Likert scale)	6.36		6.72		6.95			6.52
Study 6 (Field study) MBA course evaluation	Similar others (Electives)				Dissimilar others (Required courses)
	High salience			Control	High salience			Control
Course index (α = .88) (Likert scale)	5.87			6.07	5.81		5.55
Instructor index (α = .89) (Likert scale)	6.05			6.17	5.99		5.76

Related constructs and potential confounds

In addition to the need to conform, the knowledge salience of others might inadvertently affect several other factors. First, although our participants knew that their responses were not subject to disapproval by others, the salient knowledge of others might induce anxiety or other negative affect, increase their involvement with the task, or even reduce their chronic individual difference in self-confidence (Keller, 2010), need for uniqueness (Snyder & Fromkin, 1977), or perceived competence compared with others. Also, people with domain knowledge may have a lower need to conform and be less prone to our proposed effect because they perceive their judgments to be more appropriate compared with an average other. Finally, reminding survey respondents that others are also evaluating might evoke an interdependent self-construal. Self-construal theory distinguishes two ways to think about the self in relation to others: while “independents” tend to view their self as autonomous and separate from others, “interdependents” tend to view their self as connected with others (Agrawal & Maheswaran, 2005; Ma et al., 2014; Markus & Kitayama, 1991; Singelis, 1994). Although our manipulation of knowledge salience of others did not establish any relationship between the survey respondents and the unknown “others” (i.e., they did not interact with others in Studies 1–5), it may still have caused some participants to adopt an interdependent perspective and view their responses as being connected to others’ opinions at a conceptual level. We measured these variables to explore their potential role and ruled them out as alternative explanations. We summarize the results in Table 3 and do not report them in each study.

Table 3

Summary of potential confounding variables in Studies 1–5

Design of each study (all studies employed between-subjects design) Study 1: 3 conditions (knowledge salience: high vs. control/low vs. consensus prediction) Study 2: 2 (knowledge salience) × 2 (stimulus valence) Study 3: 4 conditions (knowledge salience: high vs. control/low vs. moderate consensus vs. consensus prediction); S3 Follow-up (WA-D): 3 conditions (knowledge salience: high vs. control/low vs. extreme consensus) Study 4: 2 (knowledge salience) × 2 (need for conformity: high vs. low) Study 5: 2 (knowledge salience) × 2 (need for conformity/similarity: high/similar vs. low/dissimilar)

Age: Controlling for age as a covariate in the analyses for S1-5 confirmed that age did not have a reliable effect (p < .05 in only S2; p ≥ .12 in S1, S3-5, S3 Follow-up). Importantly, our core effect in each study remained significant after controlling for age, confirming our effect operates independently of age

Gender: Controlling for gender as a covariate in the analyses for S1-5 confirmed that gender did not have an effect in all the studies (all p > .12), except for S3 Follow-up (p < .05). Importantly, our core effect in each study remained significant after controlling for gender, confirming gender had no effect on our DVs

Expertise: We measured expertise using objective measurements (participants’ major in S1, S4-5; quiz in S3 Follow-up, S5), their perceived expertise about investment (S4-5), and perceived expertise for high-tech products (S5). Controlling for these various expertise measures as covariates in the analyses of each study confirmed that expertise did not have an effect on our DVs (p-value range = .06 – .99), except for the quiz in S3 Follow-up, p < .05). Importantly, our core effect in each study either remained significant or became even more significant after controlling for expertise, confirming our effect operates independently of expertise

Affect: Anxietya and moodb were measured in S1 and S4. None of our IVs had an effect on anxiety or mood (p-value range: .11-.66), except for a marginal main effect of need for conformity in S4 (p = .08). However, this marginal main effect cannot explain the 2-way interaction in S4. Controlling for affect and its interaction with knowledge salience did not change the results

Self-construal: Administered the 13-item independence scale and 13-item interdependence scale (Singelis, 1994) in S2 and S5. None of the IVs had an effect on either scale (p-value range: .17–1.00), except for a 2-way interaction on independence (pinde = .043) in S5. Controlling for each self-construal as a covariate in ANCOVAs showed that our core effect remained similar to models without these covariates. We also conducted two post-tests where self-construal was manipulated and the stimuli were similar to S2. Self-construal and its interaction with valence had no effect on our DVs, all p > .39

Perceived Competencec: Measured in S3. The IV did not have an effect on competence (p = .29)

Need for uniqueness (NFU)d: The IV did not have an effect on NFU (p > .11; S3 Follow-up)

Self-confidencee: The IV did not have an effect on self-confidence (p > .22; S3 Follow-up)

Involvementf: The IV did not have an effect on involvement (p > .84; S3 Follow-up)

a“How anxious/nervous/worried did you feel when you were making the judgments”, 7-point scales (both α > .85)

b“How did you feel when you were making the predictions” (1 = happy, 7 = sad)

c “Do you feel other participants of this study are more or less competent in making evaluations in this survey, compared with yourself?” (1 = Much less competence than me, 4 = Similar to me, 7 = Much more competence than me)

d32-item NFU scale (Snyder & Fromkin, 1977)

e6-item self-confidence scale (Keller, 2010)

fHow involved/engaged were you when you answered the questions earlier? How completely did you read the questions? 6-point scales with appropriate anchor labels; involvement index: average of 3 items, α = 73

Detailed measures are presented in WA-H

Study 1

We conducted this experiment by recruiting participants on Facebook and tested the hypothesis that high knowledge salience of others also evaluating reduces judgment extremity (H1). Online social networking sites like Facebook provide a natural and appropriate context for testing our hypothesis because users of these sites are likely to be sensitized to others’ presence when making judgments (e.g., “100 of your friends are taking this survey”). Content is shared rapidly on Facebook and similar social networks with most content consumption activities accompanied by information about who else and how many others are doing the same activities (e.g., reading a newspaper article, answering a poll, watching a video; Buss & Strauss, 2009; Howard, 2010).

Method

Participants and design

One hundred fifty-six university students (107 females; Mage = 20.73) from a North American metropolitan area participated in this experiment. Participants were entered in a raffle for their participation; one in every three participants was randomly drawn to win a $25 cash prize. Study 1 had three between-subjects conditions (knowledge salience of others: high vs. low/control, vs. consensus prediction). Participants were randomly assigned to one of the three conditions. The experiment consisted of two tasks. The first task began with a photo evaluation task, which was used to strengthen the manipulation of the knowledge salience of others for the main task. The second task was the main task, which measured the extremity of participants’ judgments about various domains. Between these two tasks, participants completed a 10-min filler reading task to mitigate any potential priming effect arising from the photos seen in the first task. The reading task was also used as an attention check for the study. Participants were told that they would be entered into a raffle with a $10 cash prize if they completed the reading task adequately.

Procedure

We first formed an online group on Facebook called the Guesstimatum group that ostensibly hosted trivia contests. The experimenter served as the online group manager and recruited participants by advertising the group to undergraduate student participants on Facebook. All participants first evaluated the quality of ten photos of university students from 1 (very poor quality) to 10 (very good quality). They viewed and rated one photo at a time (see WA-A for sample photos). This photo rating task served as a cover story to embed the manipulation of knowledge salience later. To ensure that all participants (regardless of knowledge salience) were exposed to the same content before the main study, everyone was asked to view and rate the photos. After the photo evaluation task, participants did a 10-min filler task. The filler task was used to eliminate any potential priming effect due to the exposure to images of other students in the earlier photo evaluation task and served as an attention check.

After the filler task, all participants were asked to estimate the average age of the individuals from the earlier photo task (multiple choice: 1 = 18–22, undergraduate students; 2 = 30–40, working adults; 3 = 60–70, senior citizens). Most participants chose the age group correctly. Thus, any conditional effect cannot be attributed to differences in participants’ processing during the photo task.

Then, participants assigned to the high-salience condition received a message that the people they saw in the photos earlier were members of the Guesstimatum group who also filled out the next survey. Participants assigned to the other conditions did not receive this information and proceeded to the main task immediately after predicting the age of the people in the photos earlier. In the high-salience and control conditions, participants expressed their own opinions about the judgment targets. In the consensus-prediction condition, participants predicted the opinions of the Guesstimatum group. This condition allowed us to compare the extent to which the high-salience group and the control group conformed to the predicted consensus in their judgments. Having a separate group predict the consensus is important. If we had asked participants to first express their opinions and then predicted the consensus, these judgments might be inadvertently correlated.

Main task and dependent measure

Next, participants began the main task and made judgments about a variety of products and consumer events including investment, movies, currency exchange, and job placement, one at a time. Specifically, they first viewed some financial data for a company without its name (see WA-B). They then predicted the change in the closing price of the stock in three months’ time on a 13-point scale (left anchor: –60% or less; right anchor: 60% or more; in 10% increments with 0% change as the midpoint). This type of prediction was repeated for another company. We removed the names of the companies so that participants had to make their predictions solely based on the information provided in the study rather than their prior knowledge about the stocks. Subsequently, they viewed information about a classic, blockbuster comedy “The Hangover” (see WA-B). They were asked to compare it with a sci-fi film and estimate the difference between the two movies’ box office revenue on their respective opening weekends on a similar 13-point scale (left anchor: –60% or less than the Hangover; right anchor: 60% or more than the Hangover; in 10% increments with 0% as the midpoint). Participants then viewed information about an animated film and estimated the difference in the box-office revenue between “The Hangover” and the animated film. Lastly, participants predicted shifts in the currency exchange rate between US and Canadian dollars over the next three months and the difference in job placement rates of university students from their city between the current year and the next year on a 13-point scale (left anchor: –18% or less; right anchor: 18% or more; in 3% increments with 0% change as the midpoint). We judged that the narrower range of –18% to 18% for the last two questions would be more appropriate than the wider range of –60% to 60% because the range of currency exchange rates and job placement rates are generally smaller than stock prices or box-office revenues. To help participants become familiarized with the prediction task, we gave them a sample trial where they practiced predicting changes of stock prices first, and then moved on to the main task. After the main task, participants reported their age and gender. Similar demographic information was also collected for Studies 2–5.

Pretest

A pretest (N = 159) was conducted to validate our manipulation of knowledge salience of others where we used a message to remind participants that other people like them also filled out the same survey. (This message was used to manipulate knowledge salience of others in Studies 1–5.) The manipulation check for knowledge salience of others included four items (1 = not at all, 7 = very; α = 0.92): how aware were they/how obvious/noticeable/clear was it that other participants like them also filled out the same survey. The results showed that the knowledge that others were also evaluating was more salient in the high-salience condition than the control condition (Mhigh-salience = 5.82 vs. Mcontrol = 4.11; t(157) = 7.03, p < 0.001), confirming our messaging increased knowledge salience of others as intended.

Results

Participants who failed the attention check (11.54%) were removed from the analysis, yielding 138 valid responses.

Judgment extremity

Extremity of judgment was measured by MAD, the mean absolute distance from the midpoint of the scale (0% change from the price at the time of judgment). For example, if a participant predicted that the change of the stock price to be –50% in three months, his/her MAD would be coded as a difference of 50% from the current price. Due to the different ranges of the scales used across judgment tasks, we standardized MAD for each task and each participant and averaged the standardized MAD to form an extremity index (z-score). Higher extremity z-scores indicate more extreme predictions.

We submitted the extremity index (average z-score) to a Generalized Linear Model (GLM; Madsen & Thyregod, 2010) with knowledge salience as a categorical variable (0 = high knowledge salience, 1 = control, 2 = consensus). The analysis revealed that the extremity index was marginally different across conditions (Mhigh-salience = –0.09, Mcontrol = 0.12, vs. Mconsensus = –0.04; p = 0.089; Fig. 1).3 For our key comparisons, planned contrasts showed that judgments in the high-salience condition were less extreme than those in the control group (p = 0.037) but were similar to those in the consensus-prediction condition (p > 0.59), suggesting that people in the high-salience condition conformed to predicted consensus and that people predicted moderate group consensus (see Table 2 for non-standardized means for judgments on each scale).

Fig. 1

Results of Study 1

Discussion

The results of this experiment provide initial support for H1 that judgments become less extreme when the knowledge that others are also evaluating is highly salient at the time of judgment formation. In the next study, we further tested this hypothesis by examining its effect on consumer product evaluation in both positive and negative domains.

Study 2

Study 2 expanded our findings in two ways. First, it served as a more-conservative test for our hypothesis in the domain of consumer products—a poster. Evaluation of posters and many other consumer products (e.g., garments, furniture, or wine) is often perceived as a matter of personal taste. Thus, people may be less likely to conform to group consensus when evaluating this type of product. However, if the effect of knowledge salience of others also evaluating is sufficiently robust, it would still influence the evaluation of this type of product (“I don’t like that poster, but what if everyone else likes it?”).

Second, one might argue that the salient knowledge of others also evaluating simply lowers evaluation rather than reduces judgment extremity. Study 2 examined this alternative interpretation by manipulating stimulus valence. Our theoretical model predicts an interaction effect of salient knowledge of others and valence on judgment extremity. That is, when the knowledge of others also evaluating is highly salient, an attractive product will be rated as less positive and an unattractive product less negative. However, if salient knowledge of others simply lowers people’s ratings, then we should observe a negative main effect of this factor regardless of stimulus valence, rather than our predicted two-way interaction.

Five hundred and forty-four students (314 females; Mage = 19.78) from a North American university participated in this web-based study in exchange of course credit. The experiment employed a 2 (knowledge salience of others: high vs. low/control) × 2 (stimulus valence: positive vs. negative) between-subjects design. To manipulate stimulus valence, we adopted a procedure from Tsai and McGill (2011, p. 815). Specifically, we either kept the attractive original color of the poster (positive condition) or changed the hue to an unattractive bright green (negative condition) while holding everything else constant (WA-C). A pretest using the same dependent measure as the main task (-5 = don’t like it at all, 0 = neither dislike it nor like it, + 5 = like it very much) confirmed that the attractive poster was rated higher than the unattractive poster (Mattractive = 3.34 vs. Munattractive = –2.70; t(1, 185) = 28.89, p < 0.001).

At the beginning of the study, participants did the same photo rating task and the filler task as the attention check from Study 1. Then only those in the high-salience condition encountered the same manipulation of knowledge salience from Study 1 with a messaging prompt that reminded them others were also filling out the survey. Next, to help participants become familiarized with the main task, they were given a trial-run at rating a poster. Then, depending on their assigned condition, participants viewed and evaluated either an attractive or unattractive poster on an 11-point scale (–5 = don’t like it at all, 0 = neither dislike it nor like it, + 5 = like it very much).

Participants who failed the attention check (26.84%) were removed from the analysis, yielding 398 valid data points. This attention failure rate is higher than Study 1 but was still in line with that of published studies in psychology (failure rates for unsupervised studies using university subject pool in North America ranged from 10 to 61% or higher; Hauser and Schwarz 2016; Klein et al. 2014).4

We submitted the raw ratings of the poster to a two-way ANOVA. Unlike Study 1, the dependent measure was not standardized because there was only one judgment target. Importantly, the primary goal of Study 2 was to use the negative ratings to show the effect of valence and its interaction with knowledge salience. (The effect of high knowledge salience on MAD as the dependent measure was also significant, p = 0.007, as mentioned in the Overview of Studies). The analysis on the raw ratings revealed a significant main effect of valence (Mattractive = 3.22 vs. Munattractive = –2.32; F(1, 394) = 804.09, p < 0.001) and the predicted two-way interaction (F(1, 394) = 7.42, p = 0.007; Fig. 2). The main effect of knowledge salience was not significant (p = 0.95). Planned contrasts showed that the evaluation for the attractive poster was less positive in the high-salience condition than the control condition (Mhigh-salience = 2.95 vs. Mcontrol = 3.50; F(1, 394) = 4.04, p = 0.045) but less negative in the high-salience condition than the control condition for the unattractive poster (Mhigh-salience = –2.04 vs. Mcontrol = –2.56; F(1, 394) = 3.41, p = 0.065), confirming that high knowledge salience of others reduced judgement extremity rather than lowering evaluations per se.

Fig. 2

Results of Study 2. Error bars denote standard errors

Study 2 provided further support for H1 and showed that consumers are less positive about an attractive product but less negative about an unattractive product, as opposed to simply lowering their evaluations, when it is salient in their mind that others are also asked to express their opinions. Thus, we conceptually replicated Study 1 on consumer product evaluations in both the positive and negative domains.

Study 3

Study 3 further tested our core effect (H1). We had argued that knowledge salience of others reduces judgment extremity by increasing consumers’ need to conform to their predicted consensus, which is likely to be moderate, rather than extreme. To this end, we designed a condition where participants were explicitly provided with moderate, less extreme consensus information (moderate-consensus condition). According to our theory, participants’ judgement extremity in the high-salience condition would resemble that of participants who were led to think the consensus was moderate. As in Study 1, we also included the consensus-prediction condition to further demonstrate that the high-salience group conformed to perceived consensus. To increase the realism of the study, we used a 5-star scale that is utilized by many e-commerce and review websites.

Two hundred twenty-one undergraduate students (131 females, Mage = 20.40) in a North American university participated in this web-based study in exchange for course credit. The study had four between-subjects conditions (high salience vs. low salience/control vs. moderate consensus, vs. consensus prediction). There were two ostensibly unrelated tasks: a survey for the manipulation of knowledge salience of others and the main task for measuring judgment extremity of a consumer product (poster, WA-C).

As illustrated in Fig. 3, all the participants first answered a short survey which contained questions about trivial knowledge, preference of charitable organizations, and their year at the university. This survey served as a cover story for the manipulation of knowledge salience of others later. Next, participants were taken to another survey which contained our main task. The control group proceeded with the main task directly and were told that they would answer questions about a consumer product. Participants in the high-salience and moderate-consensus conditions were told on the landing page of the main task that other participants of the study were from the same year at their university and share similar values as them based on their responses to the short survey at the beginning of the study. A small visual with some college students and the mascot of the university was displayed on the upper right corner of the screen to strengthen the manipulation of knowledge salience. In the consensus-prediction condition, participants were told that they would predict other people’s opinion about a consumer product and that they should think about how similar others (e.g., students from the same year and share similar value) would evaluate the product. The same visual with some college students and the university mascot was also displayed in the same position of the screen as in the high-salience condition.

Fig. 3

Study 3 procedure

In the main task, all participants except the consensus-prediction condition were asked to rate a poster for interior decoration (randomly selected from a set of 15 by computer) using a 5-star scale where half star was allowed. In the moderate-consensus condition, participants received additional instructions prior to their evaluation, “The average rating of the poster from similar participants was 3.02 out of 5, which was quite neutral.” Participants in the consensus-prediction condition were asked to predict other similar participants’ evaluation of the poster using the same 5-star scale. To help participants become familiarized with this scale (common in the real world but less common for lab studies), we gave them a practice trial where they tried this scale. Next, they evaluated the target poster using two items: (1) “What’s your overall evaluation of this poster?” (1 star = not good, 3 stars = neutral, 5 stars = good); (2) “How would you rate this poster in terms of its quality?” (1 star = low quality, 3 stars = average, 5 stars = high quality). The procedure of each condition is summarized in a visual as follows. Finally, participants indicated their perceived competence relative to other participants.

We averaged the two evaluation items (overall evaluation and quality rating; α = 0.78) and computed MAD from the midpoint of the scale as Study 1 (midpoint = 3, labelled as “Neutral” and “Average” on each 5-star scale). A one-way ANOVA showed a significant main effect of knowledge salience on MAD across the four conditions (F(1, 217) = 2.87, p = 0.038; Fig. 4). Planned contrasts showed that compared with the control group (Mcontrol = 1.22), participants’ evaluation for the poster was less extreme (lower MAD) when others’ salience was highlighted (Mhigh-salience = 0.92; p = 0.024), supporting H1. Importantly, judgment extremity in the high-salience condition did not differ significantly from that in the moderate-consensus condition (Mhigh-salience = 0.92 vs. Mmoderate-consensus = 0.97; p = 0.69) or the consensus-prediction condition (Mhigh-salience = 0.92 vs. Mconsensus = 0.86; p = 0.66), further suggesting that participants conformed to the group consensus in the high-salience condition.

Fig. 4

Results of Study 3

The results of Study 3 replicated our core effect between the high-salience and control conditions (H1). The fact that judgment extremity did not differ between the high-salience condition, moderate-consensus condition, and consensus-prediction condition provided additional evidence that high knowledge salience of others causes people to conform to perceived consensus and reduces judgment extremity. People conform to group consensus according to their prediction or when they are provided with the consensus information directly in the survey. We further demonstrated the role of consensus in a follow-up study (N = 301 Amazon Mechanical Turkers) where we used a three-level between-subjects design (high salience vs. low salience/control, vs. extreme consensus). The results showed that when extreme consensus information was explicitly provided, conforming to group consensus increased judgment extremity, thereby attenuating the effect of knowledge salience (see WA-D for details).

Thus far we have obtained consistent evidence for our hypothesis that high knowledge salience of others also evaluating reduces judgment extremity (H1). Next, we tested the critical role of need for conformity as the underlying mechanism (H2) in three studies where we reduced the need to conform directly (Study 4) and tested a theory-driven boundary condition where the similarity of group members was lowered (Studies 5 and 6; H3).

Study 4

Study 4 tested the key premise in H2 that knowledge salience of others reduces judgment extremity because of the increased need to conform. To do so, we directly manipulated participants’ beliefs about the need to conform in this experiment. We predicted that when participants were led to experience a lower need to conform to others’ judgments, the effect of knowledge salience would be attenuated such that they would be less likely to align their judgment with perceived consensus. We expected to replicate the effect of knowledge salience when participants were led to experience a higher need to conform to others’ judgments.

One hundred and fifty four university students (120 females; Mage = 20.30) participated in this web-based study. Study 4 employed a 2 (knowledge salience of others: high vs. low/control) × 2 (need for conformity: high vs. low) between-subjects design. Participants were entered into a raffle with a one-in-three chance of winning a cash prize of $25 for participation as in Study 1.

Study 4 consisted of three ostensibly unrelated tasks: Participants first completed the photo evaluation task (10 photos of university students) from Study 1. Then they completed another task that was used to manipulate their beliefs about the need to conform. Following Simonson and Nowlis (2000), we described the importance of either being similar to or different from others. To reinforce the manipulation, we asked participants to write down a past experience that supported their (randomly) assigned position (WA-E). Then participants in the high-salience condition encountered the knowledge salience manipulation from Study 1 (that those individuals in the photos were invited to participate in the subsequent study) but not the control group. The filler reading task between the photo task and main task was not needed here because participants were asked to complete the task designed to manipulate of need-for-conformity between the photo task and main task. Next, participants proceeded to the main task, which measured judgment extremity. The main task involved predictions of the price change of four anonymous stocks after a sample trial, using the same scale from Study 1 (left anchor: -60% or less, midpoint: 0% change, right anchor: 60% or more; 10% increments).

In a pretest (N = 149), we gave participants the same task that manipulated their need for conformity (high need vs. low need; see WA-E for instructions). Thereafter, participants responded to a 3-item alignment scale (1 = strongly disagree, 6 = strongly agree): (1) “I tried to align my opinion to others’ opinions.”; (2) “I tried to stay close to others’ opinions.”; and (3) “It was important for my opinion to be similar to others’ opinions.” We averaged the ratings of the three items (α = 0.89) to form an alignment index and confirmed that our manipulation of need for conformity indeed altered participants’ tendency to align their opinions with others (Mhigh-conformity = 4.41 vs. Mlow-conformity = 3.52; t(147) = 4.44, p < 0.001).

Results and discussion

As in Study 1, judgment extremity was measured by MAD from the midpoint of the scale (0% change from the price at the time of judgment). We created an extremity index by averaging MAD across the four stocks for each participant. We did not standardize MAD because the same 13-point scale was used for each judgment. A two-way ANOVA on the extremity index revealed a significant main effect of need for conformity (Mhigh-conformity = 13.42% vs. Mlow-conformity = 17.11%; F(1, 150) = 10.08, p < 0.01), and importantly, a marginally significant two-way interaction (F(1, 150) = 3.04, p = 0.08; Fig. 5). As expected, the main effect of knowledge salience was not significant (F(1, 150) = 1.99, p = 0.16, NS),

Fig. 5

Results of Study 4. * Error bars denote standard errors

Planned contrasts further supported our hypotheses. We first replicated the results from previous studies under high need for conformity: participants’ prediction of stock price changes was less extreme (lower MAD) in the high-salience condition than that in the control condition (Mhigh-conformity/high-salience = 11.21% vs. M high-conformity/control = 15.13%; F(1, 150) = 4.58, p = 0.03). However, under low need for conformity, the effect of knowledge salience of others was attenuated (Mlow-conformity/high-salience = 17.32% vs. Mlow-conformity/control = 16.91%; F < 1, NS). These results lend support to our H2 regarding the critical role of need to conform underlying the effect of high knowledge salience on reduced judgment extremity. In the next two studies, we examined a theory-driven boundary condition where we manipulated similarity of others.

Study 5

Study 5 further tested the role of need for conformity by varying the perceived similarity of others (H3). Specifically, we manipulated the similarity between our participants and others who were also evaluating—students from the same university (similar condition) versus senior citizens (dissimilar condition). We predicted the effect of salient knowledge of others to be attenuated when others who are also evaluating are perceived as dissimilar (an out-group) to the participants because dissimilar others should reduce the need to conform (H3).

Study 5 also further ruled out priming as an alternative account. In Studies 1, 2 and 4, we showed participants photos of undergraduate students. One might argue that the exposure to a certain group of people can produce a priming effect, which causes participants to behave in the same way as the group would. In those studies, we addressed this potential priming effect using a 10-min reading task as a filler task. If priming is the key driver, then viewing photos of older adults should cause participants to behave like them and become less extreme in their judgments given prior research showing that older adults were associated with reduced risk taking in financial decisions and lower preference for extreme gambles (e.g., extremely low probability with high payoffs) compared to young adults (Deakin et al., 2004). However, if the need to conform is the key driver, given that the social norm among senior citizens is less relevant to young adults, being reminded that older adults (dissimilar others) are also evaluating should not increase the need to conform and the effect of knowledge salience on judgment extremity will be attenuated. This experiment tested these different predictions.

Participants and study design

One hundred eighty-nine students (145 females; Mage = 20.09) from the same North American university participated in this web-based study. Study 5 employed a 2 (knowledge salience of others: high vs. low/control) × 2 (similarity of others: similar/high need for conformity vs. dissimilar/low need for conformity) between-subjects design. The study consisted of two ostensibly unrelated tasks and a 10-min filler task between them as attention check. The first task was used to manipulate knowledge salience of others also evaluating and similarity of others, and the second task was the main task, which measured judgment extremity. Participants were entered into a raffle with a one-in-three chance of winning a cash prize of $10 for participation and an additional bonus of $25 for accurate predictions.

All participants were first asked to rate ten photos that were either of university students or older adults. They then completed the same filler task from Study 1. After the filler task, half of the participants were asked to estimate the average age of the individuals whose photos they had seen (high-salience condition). Almost everyone guessed the age group correctly. These participants were further told that those individuals in the photos were invited to participate in the subsequent study. The other half of the participants did not get this question and were not reminded of the individuals in the photos (control condition). They moved on to the main task directly after the filler task. (Asking this question or not in the control group did not alter the effect, as seen across studies.)

A pretest (N = 199) was conducted to validate our procedure of using similarity to manipulate participants’ need to conform. Participants were exposed to others either from a similar or different age group and responded to the same three alignment-tendency scale from the pretest in Study 4 (α = 0.91; 1 = strongly disagree, 6 = strongly agree; higher number indicates greater alignment). The results confirmed that participants were more likely to align their evaluations with others’ opinion when they perceived others who were also evaluating to be similar to them in age than dissimilar to them (Msimilar = 4.50 vs. Mdissimilar = 4.01; t(197) = 2.15, p = 0.03). These results confirmed that our manipulation of similarity influenced need for conformity as intended.

As in Study 1, all participants in the main task predicted the price change of one stock in the next three months. Then they indicated their purchase intention for the stock (“If you were given $1000 to invest, how likely would you invest in this stock?” 1 = very unlikely, 7 = very likely). Next, participants viewed and rated a new wearable health-monitoring device called SIMBAND (“What is your overall evaluation of the SIMBAND?” 1 = bad, 9 = good; “How would you rate the SIMBAND?” 1 = poor, 9 = excellent; α = 0.90; see WA-F for the stimuli). Because this is a niche product that only appeals to a segment of the general population, we included a covariate about consumers’ attitude toward this technology: “How useful is this product to you?” (1 = not useful at all, 9 = very). The measures and format of product information for SIMBAND were adapted from Zhao et al. (2009).

Excluding a total of 24.87% of the participants who failed to pass the attention check yielded 142 valid data points. We reported the aggregated results across judgment tasks first and then summarized the results for each task (see Table 2 for the means of each task).

As in Study 1, we first computed MAD from the midpoint of the scale (0%) as the measure of judgment extremity for the stock price prediction. Due to different scales used across judgment tasks, we followed Study 1 to create an extremity index. Specifically, we first standardized MAD for the stock price predictions for each participant. For the other two judgment targets rated on unipolar scales, we standardized the raw ratings. We then averaged the z-scores for each judgment task to form the extremity index (α = 0.52). Again, higher z-scores indicate greater extremity. A two-way ANOVA on the extremity index revealed a predicted significant interaction of knowledge salience of others and similarity only (F(1, 138) = 6.37, p = 0.01).5 Neither similarity nor knowledge salience had a significant effect on judgment extremity (both p > 0.21). Planned contrasts showed that when participants perceived other survey respondents as similar to themselves, and thus need for conformity was high, knowledge salience reduced judgment extremity (Msimilar-high-salience = –0.21 vs. Msimilar-control = 0.08; F(1, 138) = 4.00, p = 0.047), replicating our core effect again. However, when participants perceived others as dissimilar, and thus need for conformity was low, the effect of knowledge salience was attenuated (Mdissimilar-high-salience = 0.19 vs. Mdissimilar-control = –0.05; F(1, 138) = 2.52, p = 0.12). Because the results were moderately correlated across judgment tasks, we repeated the analyses for each judgment task. Separate analyses on individual evaluations (MAD for stock prediction; raw ratings for purchase intention for the stock and evaluation of SIMBAND) showed similar two-way interactions (pstock = 0.13, ppurchase = 0.009, pSIMBAND = 0.03; see Table 2 for the cell means).6

Results of Study 5 provided further support for our hypotheses. When participants perceived other survey respondents as similar to themselves, the salient knowledge of others also evaluating increased the need to conform and caused their judgments to become less extreme (H1 and H2). By contrast, when other respondents appeared to be dissimilar, participants’ need to conform was lowered, and the effect of knowledge salience was attenuated (H3). In addition, results of Study 5 ruled out priming as an alternative explanation. If priming was the key driver, there should be a main effect such that viewing photos of older adults should have reduced judgment extremity, regardless of our knowledge salience manipulation. However, we found a two-way interaction of knowledge salience and similarity of others, providing further evidence for our theory.

Study 6

Study 6 investigates two questions: (1) Will the knowledge salience of others reduce judgment extremity in natural field settings when others are physically, rather than cognitively, present? (2) Will dissimilarity of other participants reduce the need for conformity in a field setting? To answer these questions, we obtained and analyzed the results of course evaluations completed by full-time MBA students at a business school in North America during 2010–2011. This business school introduced the online evaluation system in 2010, which allowed instructors to administer the course evaluations online either in class or outside of class. Some instructors set aside some time in class for students to complete the evaluation online. In these cases, evaluations were made in the physical presence of others, which should increase students’ need to conform (high-salience condition). In other cases, instructors let students complete the evaluations outside of class and so students’ knowledge of others also evaluating was relatively less salient.

The dataset contains ratings from both elective and required courses. We had two predictions: (1) We expected to replicate the core effect of high knowledge salience of others on the extremity of evaluations in elective courses (H1); and (2) we expected the effect to be moderated in the evaluation of required courses where similarity of other students was lower than that in the electives (H3). This predicted interaction between knowledge salience and course type is attributed to different degree of student similarity between electives and required courses. At this institution, similarity among students tend to be higher in elective courses than required courses. For example, most students in a consumer behavior elective class are interested in the subject, have similar major, and even similar career goals. Given the similar student body in elective courses, need for conformity should be higher when knowledge of others is salient. However, for required courses, students with different majors, interests, and career goals must take these courses to earn their degree. Therefore, similarity among students in required courses is relatively low, which reduces the need to conform and moderates the effect of knowledge salience of others.

The dataset also allows us to further generalize the effect of knowledge salience of others to an older and larger sample population with work experiences (Mage = 27.40; Myears-worked = 4.00) in a field setting. The dependent variables in this study were evaluative judgments, which were similar to customer satisfaction ratings of education programs. Participants knew their responses would be kept strictly anonymous and confidential. Thus, deviating from the norm would not have any repercussions just as in Studies 1–5. Finally, the field setting allows us to show that the need to conform can be activated without explicit instructions or knowledge salience messaging.

The dataset comprised 7603 entries of course evaluations for the course content and instructors of 117 sections of required courses and 206 sections of elective courses for the full-time MBA program during 2010–2011. Data was separated into high-salience and control conditions by survey completion time. Specifically, the web recorded the time that students submitted the evaluations. Data from the classes where course evaluation was conducted during class hours were assigned to the high-salience condition. Data from classes where the evaluation was completed outside of class hours were assigned to the control condition. This coding regime ensured that the control group completed the evaluation outside the classroom and was unlikely to be surrounded by their classmates while they rated the courses. Although students coded as the control group (outside of class) could theoretically contact their classmates before completing the evaluations, they were unlikely to be surrounded by the entire class because they had to attend other classes, events, or appointments afterwards. The percentage of courses assigned to the high-salience (in class) condition versus the control (outside the class) condition was similar between the required and elective courses. As such, Study 6 tested the role of need to conform in a natural 2 (knowledge salience of others: high vs. low/control) × 2 (similarity of others: similar vs. dissimilar) between-subjects design.

Dependent measures

The course evaluation survey contained seven items about the course content and nine items about the instructor(s) (WA-G). Respondents indicated the extent to which they agreed with each statement on 7-point scales (1 = disagree completely, 7 = agree completely).

The web-based course evaluation form required respondents to enter a code that is unique for each respondent and could only be entered once. However, the dataset had 307 respondents who completed the survey more than once (307 code numbers appeared more than once). This is likely an error when data was compiled by the administrative office because the entries and the survey completion time of the first set and the additional sets were identical. We therefore kept the first set of evaluations from these respondents and deleted the additional sets in the file. We created two indices, one for the course content (α = 0.88) and the other for the instructors (α = 0.89), by averaging the raw ratings of their respective scale. These two indices were also highly correlated (α = 0.90).

Judgment extremity of course index We first submitted the course index to a GLM with similarity, knowledge salience, and their interaction term as independent variables. As predicted, the analysis showed a significant two-way interaction (β = –0.46, p < 0.001; Fig. 6). Planned contrast (full model, degrees of freedom = 7292) showed that for elective courses (high similarity with higher need for conformity), the course ratings were lower when students completed the survey with their classmates in class (high-salience condition) than outside of class (control group) (Msimilar-high-salience = 5.87, Msimilar-control = 6.07; p < 0.001), confirming H1 such that the evaluation was less extreme. Although the difference might seem trivial, knowledge salience of others dragged the ratings for electives from the 6-point category down to the 5-point category. The difference signaled a substantial difference to students and could have a large impact on student enrollment when the courses were offered again in subsequent years. In contrast, high knowledge salience of others increased judgment extremity for required courses (low similarity and lower need for conformity) (Mdissimilar-high-salience = 5.81, Mdissimilar-control = 5.55; p < 0.001), further confirming H3. Although not of theoretical interest, the effect of similarity on the evaluation was significant (Msimilar / electives = 5.90; Mdissimilar / required course = 5.75; β = 0.26, p < 0.001).

Fig. 6

Results of Study 6. * Error bars denote standard errors

Judgment extremity of instructor index

Similar patterns of the results were observed in the analyses of the instructor index. A GLM showed a significant two-way interaction (β = –0.35, p < 0.001; Fig. 6). Planned contrasts confirmed that the simple effects for elective and required courses were also significant and in the same direction as the course index (both p < 0.05).

This field study showed that our effect applies to MBA course evaluations, a form of customer satisfaction survey for the education industry and demonstrated that the ratings varied systematically when the survey was conducted in the physical presence of others despite the fact that the average ratings (group consensus) was unavailable and the evaluations were confidential. The interaction effect provides additional support to our H2 on need for conformity and H3 regarding the boundary condition with dissimilar others.

Although the most relevant distinction between the electives and required courses to the present research is the similarity of student interests in class, we acknowledge that there are other idiosyncratic differences between different course types (e.g., instructor, class content, year in the program) that we cannot control for due to the lack of random assignment in the field study. However, these factors imply a main effect of course type or knowledge salience rather than the observed two-way interaction. These issues were further mitigated given the reasonably large number of courses in the dataset and the similar number of courses that had evaluations filled out in class versus after class across course types. Finally, the institution that provided the dataset grouped students with diverse interests and backgrounds into each required course, which resulted in low similarity among students and provided a natural quasi-experimental setting for us to test the boundary condition with dissimilar others. For institutions that group students by their interests, career goals, or expertise in their required courses, similarity among students would be high, and one might expect a main effect of knowledge salience of others.

Although not the main focus of this research, one might ask why perceiving other respondents as dissimilar only attenuated the effect of high knowledge salience in Study 5 but reversed it in Study 6. To understand why this happened, we draw from prior research on consumers’ tendency to deviate from, rather than merely ignore, the norm of a dissociative reference group (White & Dahl, 2007). Although the dissimilar others in Studies 5 should be perceived as an out-group that our participants did not feel that they belonged to or were a part of (Escalas & Bettman, 2003), the dissimilar MBA students in the required courses in Study 6 might even be perceived as an dissociative group (a special type of out-group) that participants go out of their way to be differentiated from given their different career goals and interests. Presumably, MBA students have very practical reasons such as recruiting to express contrarian views about required courses. For example, those who hope to find marketing jobs would be favorable toward marketing courses and even desire to be perceived as so whereas those who want to work in finance might avoid being associated with marketing. This may be why MBA students’ expressed opinion further deviated from their perceived consensus of dissimilar others in their course evaluations. This interpretation is consistent with prior research on the effect of dissociative reference group (White & Dahl, 2007), but worth empirical investigation in future research.

General discussion

Imagine that a consumer alone in a closed room is evaluating a product, issue, advertisement, or a job candidate anonymously. Clearly, whether or not other individuals are also evaluating the same thing should have no bearing on this consumer’s judgment. However, across seven studies (six main studies and one auxiliary study in the WA), we find that increasing the salience of the knowledge that others are also evaluating reduces the extremity of consumers’ judgments (H1). In addition, three studies (Studies 1, 3, and follow-up study of Study 3) tested the important role of (perceived) group consensus. Specifically, Studies 1 and 3 showed directly that judgment extremity in the high-salience condition was similar to that of the predicted consensus. Further, we manipulated extremity of consensus directly in two studies. In Study 3, we had a condition where moderate consensus information was explicitly provided to participants and replicated the effect of knowledge salience of others. In its follow-up study (WA-D), we provided extreme consensus information to participants directly and found that participants’ judgment was more extreme than the high-salience condition and as extreme as the control condition, thereby attenuating the effect of knowledge salience. Together, these studies showed that (1) people conform to perceived consensus when it is salient in their mind that others are also evaluating; and (2) the perceived consensus tends to be moderate in general unless explicitly instructed otherwise.

We further examined the role of need for conformity as the underlying mechanism by directly manipulating it (H2; Study 4), and tested a boundary condition when participants perceived other survey respondents as dissimilar to themselves (H3; Studies 5 and 6). Data ruled out potential confounds such as expertise, affect, competence, self-construal, and involvement in the judgment task (Table 3). Lastly, our results were robust regardless of whether a performance-based bonus was offered or not.

Theoretical contribution

Our work contributes to extant literature on social influence, in particular, need for conformity. Prior research in this area primarily studied judgments in situations where group consensus was observable from the physical presence of other group members or explicitly provided by the experimenters. Our work offers a new paradigm to study conformity to group norms or consensus; regardless of cognitive presence (Studies 1–5) or physical presence (Study 6). Despite the anonymity and confidentiality of individual responses, the salient knowledge of others puts consumers in a “conformity mode.” As such, our work extends the literature in an important way by showing conformity to consensus in a much subtler context; consumers conform to their perceptions of consensus even if their responses are strictly anonymous and confidential, and deviating from group consensus would not have any repercussion.

Also, our paradigm can discern two mechanisms underlying the need to conform—informational versus normative motivations. Although extant literature considers these two accounts to be conceptually independent, it has been difficult to disentangle them empirically (Cialdini & Goldstein, 2004; David & Turner, 2001). Our findings suggest that informational motivations alone can systematically influence judgments because the inherent anonymity and confidentiality in our studies ruled out normative motivations. Further, whereas extant literature has focused on the alignment of participants’ judgments with observed consensus provided by an external source, our work shows that the salient knowledge of others can influence judgments by evoking a need to conform, even when the group consensus is unverifiable.

Future research and limitations

Although we showed that expressed judgments and preferences in anonymous online surveys were systematically influenced by cognitive as well as physical presence of others, might there be other behaviors that would differ for online versus offline groups? It can be argued that the membership of an online group is based on a looser set of criteria due to the inherent anonymity in cyberspace and is more heterogeneous than an offline group. Furthermore, how do the structural features of a group influence the strength of conformity? For instance, is conformity greater for smaller versus larger groups, for groups based on beliefs and causes, professions, or hobbies rather than based on geographic locations or demographics? Relatedly, although our effect does not hinge upon commitment and accountability in the group (Tetlock et al., 1989), these factors are interesting issues to consider for future research.

While our work focuses on conformity, it is also worth examining circumstances where people have a greater need for uniqueness than conformity. For example, uniqueness might be more appreciated and accepted as a norm in a culturally-diverse community than a relatively homogeneous community; or when others are not just different (e.g., people from different age groups like in our Study 5) but also represent a dissociative group (e.g., people with distinctively different academic and career interests like in our Study 6). Relatedly, future research can investigate potential individual differences in self-construal (Laroche et al., 2004; Markus & Kitayama, 1991) or the impact of social isolation driven by factors beyond one’s control, such as the COVID-19 pandemic (Clair et al., 2021). When people feel isolated and perceive that they do not belong to any group, they might be less likely to conform to a group norm. If this is the case, then social isolation could be a potential moderator for the effect of high knowledge salience of others. Indeed, there was some hint from our Study 3 where we observed that knowledge salience did not alter judgment extremity among 41 students who were living off campus and chose remote learning format during the pandemic (p = 0.47). Presumably separation from their peers reduced their need to conform to the perceived norm of this group. Although this null effect can be attributed to a host of other reasons such as the extremely small sample size, negative affect due to remote learning, or lower perceived control during the pandemic, it is worth investigating the role of social isolation in the effect of knowledge salience of others in future research. Furthermore, how might social desirability play a role in online surveys? Although our paradigm ensures the anonymity and confidentiality of participant’s responses and should elicit truthful judgments, some participants might still derive satisfaction from giving socially desirable responses. While this idea is beyond the scope of our research, a careful study on social desirability in a similar setting deserves future research attention.

Another area of interest is crowdsourcing. As mentioned earlier, crowdsourcing has become an important business practice because companies can enlist ever-larger numbers of non-experts to do ever-more complex and creative tasks at significantly reduced costs. However, a common disadvantage of crowdsourcing is the repetition of one or two original ideas posted by the first few participants. According to our research, crowdsourcing sites can encourage independent thinking by reducing the need to conform, using creative ways to foster participants’ desire for uniqueness (e.g., emphasizing group heterogeneity), or choosing appropriate segmentation for more targeted crowdsourcing in order to obtain more representative evaluation among the specific target (e.g., for business problems related to spicy food, they should target different segments that like or dislike spicy food to obtain unique attitude from each segment, rather than survey the mass population which will likely result in moderate evaluation across consumer segments).

Conclusion and practical implications

Consumers are bombarded with requests from companies and public opinion pollsters to complete surveys, product tests, and opinion polls. Prior research in psychology has identified a number of challenges related to interpreting data from such surveys. These challenges primarily arise either from the framing of the questions or structural issues related to the survey itself (its length, sequence of questions, the scales used for responses, etc.). The present research identifies another potential challenge of interpreting marketing research by showing that when consumers are sensitized to the fact that other survey respondents are also filling out the survey, their opinions become less extreme. Given the popular trend of data collection via digital platforms and firms’ heavy reliance on social networking sites to drive respondents to such survey sites, individuals are more likely to find themselves in situations where they are sensitized to other people who are evaluating the same products or events as they are. In some situations, the sensitization might be general (e.g., “100 of your friends are taking this poll”), while in other cases it may be very salient and specific (e.g., when the names and photos of these people are displayed). Our manipulation of knowledge salience of others is even more subtle because it only mentions unknown others are also evaluating the same items. In a world where most consumers, if not all, share their opinions via some form of social media (e.g., Facebook, Instagram, Twitter, WhatsApp groups etc.), the practice of conducting research online will surely proliferate further (Pimentel, 2012). Indeed, a number of start-up web businesses such as GutCheckIt.com, Pollfish.com, and Driveresearch.com, have developed this market by helping managers drive traffic from social media sites to their surveys. Given these trends, we expect our research will become increasingly relevant to academic scholars, policy makers, and practitioners in the years to come.

Our work suggests that managers and policy makers should consider the potential distortion of their data when the knowledge of others is made salient to survey respondents. When conducting research in such high-salience settings, managers and researchers may experience difficulty in capturing potential differences in people’s preferences and choices. Indeed, they are likely to find weak or even null results that are difficult to interpret. As an example, consider online ethnography—a data collection technique that has been used frequently in recent years. This method requires participants to submit their usage photos tagged with consumption stories. This online technique is attractive to companies because it requires less time, manpower, and cost to gather information about consumers’ usage of products than the traditional observational research (e.g., visiting consumers in their homes and observing their consumption). However, submitting usage photos to interactive blogs makes consumers acutely aware that others are also using and evaluating similar products.

Another issue to consider is that consumers are becoming more aware that marketers observe, monitor, and track their activities on social media sites after the disclosure of dubious data practices (e.g., Facebook and Cambridge Analytical scandal) and censorship accusations. Therefore, for products that consumers typically enjoy or dislike individually, new online research methods highlighting the fact that others are monitoring can cause consumers to underreport their true feelings about the products. Related with crowdsourcing discussed before, to address the issue of conformity, marketers should emphasize group heterogeneity or the value of uniqueness.

This research also shed lights on testing marketing initiatives such as new advertising campaigns. There have been numerous massive advertising campaign failures (AdAge, 2017). Examples include Pepsi’s 2017 commercial showing Kendall Jenner magically settling a Black Lives Matter standoff by offering a Pepsi to a police officer. It was ridiculed on social media, parodied on YouTube, and promptly pulled. Given the prevalence of online marketing research, we would not be surprised if these firms used online platforms to pretest the commercials and found favorable testing results before they ran the campaigns. If they did, these inaccurate test results may be driven, in part, by the salient knowledge of others at the time of testing. In other words, their pretest results were likely uninformative because pretesting ads on social networking sites where the knowledge of others was salient and consensus was unobservable could cause respondents to underreport their dislikes of the ads. However, when distasteful commercials go live later, opinion leaders (probably with a high need for uniqueness) will openly excoriate them on social media, thereby causing other viewers to conform to such extreme consensus, which ultimately produces a snowballing effect. These dynamics are likely to have been missed in the pretests. In summary, in the virtual world, consumers’ evaluations of delight or disgust may be dampened and reported as a mild liking or a mild disliking when the knowledge of others also evaluating is salient and the predicted consensus of others is unobservable.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 1204 KB)