Development of the perceived risk of HIV scale.

Past studies have used various methods to assess perceived risk of HIV infection; however, few have included multiple items covering different dimensions of risk perception or have examined the characteristics of individual items. This study describes the use of Item Response Theory (IRT) to develop a short measure of perceived risk of HIV infection scale (PRHS). An item pool was administered by trained interviewers to 771 participants. Participants also completed the risk behavior assessment (RBA) which includes items measuring risky sexual behaviors, and 652 participants completed HIV testing. The final measure consisted of 8 items, including items assessing likelihood estimates, intuitive judgments and salience of risk. Higher scores on the PRHS were positively associated with a greater number of sex partners, episodes of unprotected sex and having sex while high. Participants who tested positive for HIV reported higher perceived risk. The PRHS demonstrated good reliability and concurrent criterion-related validity. Compared to single item measures of risk perception, the PRHS is more robust by examining multiple dimensions of perceived risk. Possible uses of the measure and directions for future research are discussed.

Introduction

Beliefs about personal risk for HIV infection are central to understanding what motivates people to engage in behaviors that reduce or increase their risk of HIV infection. Centers for Disease Control and Prevention guidelines for HIV testing and counseling include HIV risk assessments to enhance self-perception of risk [1], and HIV/AIDS interventions often aim to influence how people perceive their risk of HIV infection [2-5]. Models such as the Protection Motivation Theory [6], Health Belief Model [7], Social Cognitive Theory [8], Extended Parallel Process Model [9] and the AIDS Risk Reduction Model [10], all include perceived risk as an important predictor of risk behaviors. Although perceived risk alone is not sufficient to motivate attitude or behavior change, it is seen as necessary for change.

In a recent review of behavioral interventions to reduce HIV risk behaviors [11], 15 out of 38 studies reported examining or enhancing perceived risk. Although the construct of perceived risk is often measured, other than reporting reliability, few studies report the psychometric properties of the risk perception measures they employ [12, 13]. Accurate measures of risk perception are needed to examine how people think and feel about risk, how perceived risk relates to behavior and actual HIV infection, and how effective interventions are at enhancing perceived risk [12].

Existing measures of perceived risk of HIV infection often rely on single items [14-16]. A single item may not adequately capture how people think and feel about their risk of contracting HIV. A review of past research revealed that perceived risk for HIV infection has largely been operationalized as a cognitive assessment of risk, such as the probability, chance, or likelihood of becoming infected with HIV [17]. However, there is an increasing recognition of the role of affective or intuitive processing in how people think about risk [12, 18–21]. Dual-process theories [22] suggest people think about their risk both in an analytical, deliberative way and in a more intuitive affective-based manner [19]. Acknowledging that the risk of HIV is a possibility without feeling at risk may not be sufficient to motivate behavior change [23]. Another dimension of perceived risk that is important for understanding how people think about their risk is the salience of the risk or how often someone thinks about the risk [20, 21, 24, 25]. There have been calls for broader conceptualization and measures of risk perception [21] and more research to understand how best to measure HIV risk perception [26]. Researchers have begun exploring approaches to measuring risk perception in other domains such as skin cancer [12] and influenza vaccination [27], however, there is a lack of research examining how best to measure perceived risk for HIV infection.

The current study aimed to develop a measure that combines different approaches to measuring perceived risk, including likelihood estimates, intuitive feelings about risk and the salience of the risk of HIV infection. To aid in scale development, Item Response Theory (IRT) was used to examine the psychometric properties of individual items. IRT is a set of models that describes the probability of responding to an item response category as a function of the respondent’s level on the trait being assessed (θ, theta) and characteristics of the item, for example, item difficulty or discrimination. IRT provides a useful tool in the development of a measure because it provides information about individual items. Item information functions (IIFs) can be used to examine how discriminating an item is as a function of perceived risk. In addition, option response functions (ORFs) provide information about the probability of an item response category being selected as a function of perceived risk, and can be used to decide whether a response category is redundant and could be collapsed. Unlike classical test theory, where standard error of measurement is the same across all scores in a population, IRT provides information about standard error of measurement across different levels of perceived risk [28].

The current study had the following aims: (1) Develop a pool of items assessing how people think and feel about their risk of HIV infection based on their sexual behaviors; (2) use IRT to examine item characteristics and construct a measure of perceived risk of HIV infection (the Perceived Risk of HIV Scale; PRHS); and (3), examine the criterion-related validity of the measure by exploring the relationship between perceived risk and recent sex behaviors and test results for sexually transmitted diseases. Past research has used similar measures to assess perceived risk based on past behaviors [29, 30]. It is hypothesized that when people are asked to think about their risk of HIV infection based on their recent sex behaviors, those who engage in more risky sex behaviors and those who later test positive for syphilis or HIV infection will perceive themselves to be at higher risk of HIV infection.

Methods

Item Pool Development

Items were developed to assess different dimensions of risk perception based on past literature, including cognitive assessments (e.g. likelihood judgments), intuitive assessments (e.g. feeling vulnerable to HIV) and salience of risk (e.g. is HIV risk something they have thought about). The original item pool of 30 items was reviewed by a group of subject matter experts (SMEs; N = 6) and by a focus group of individuals at risk of HIV infection (men who have sex with men and current drug users; N = 8). SMEs included researchers who had published in the field of HIV and risk perception. SMEs were provided with a definition of the construct, a description of the aim of the study and of the target sample. SMEs were asked to rate each item for relevance to the construct and item clarity, as well as to provide additional comments about item content. Individuals from the focus group were interviewed one-on-one while completing the items, they were asked to rate how clear each question was, as well as open-ended questions about what they understood the question to mean and any difficulties they had answering the question. Based on feedback from both the SMEs and focus group, items were reworded and the final item pool was reduced to 18 items focusing on the risk of HIV infection from participants’ recent sex behaviors.

Participants

Participants (N = 785) were recruited from HIV testing and prevention services in Long Beach, California. A small number of participants self-reported having previously been diagnosed with HIV/AIDS (n = 14) and data from these participants were excluded from the analysis resulting in the dataset used for the analysis (N = 771). The participants ranged in age from 18 to 79 years old (M = 41.2, SD = 12.4). In total, 37.1% of the participants were Black, 30.9% were White, 23.1% were Hispanic/Latino, 4.3% were Asian/Pacific Islander, 2.1% were Native American, and 2.6% identified as other. The majority of the participants were male (74.2%) and had graduated high school or had a GED (75.5%). Just over a third of clients identified as homeless (34.2%) and 44.6% had earned less than $500 in the last 30 days.

Procedure

Participants met individually with a trained interviewer. After reviewing the informed consent form (approved by the California State University, Long Beach, Institutional Review Board) the interviewer administered the HIV perceived-risk items followed by the risk behavior assessment (RBA). The RBA includes items assessing frequency of different types of sex, number of sex partners, sex trading behaviors, use of drugs immediately before or during sex and a single item measure of HIV risk perception. The RBA has been found to have good reliability and validity [31-33]. Following completion of the questionnaires, participants received HIV prevention counseling and/or HIV/STD testing. Participants received a small non-cash incentive for participating in the study. A total of 652 participants underwent HIV testing, the majority of these tests were performed using the OraQuick Advance® Rapid HIV-1/2 Antibody Test (OraSure Technologies, Bethlehem, Pennsylvania) (n = 601), and all preliminary positive results were confirmed with an HIV-1 Western blot. Participants who received HIV testing were also offered antibody tests for syphilis using both non-Treponemal (RPR) and Treponemal (TP-PA) tests. Participants who received HIV rapid testing were provided with the results at the end of the session. Participants who received standard testing for HIV and/or syphilis testing were asked to return in 1 week to receive the results of these tests.

Analytic Approach

IRT calibration was used to examine item characteristics and develop the final risk-perception measure. Before an IRT model is fitted to data, it is important to test the underlying assumptions of the model, including unidimensionality and local dependence. To check the assumption of unidimensionality, an exploratory factor analysis was conducted. The scree plot, eigenvalues and item loadings and alternative solutions with more than one factor were examined. The ratio of the eigenvalue of the first and second unrotated components was examined. A single-factor confirmatory factor analysis (CFA) was then performed using PROC CALIS in SAS [34]. Model fit was evaluated using the goodness of fit index (GFI), root mean square error of approximation, and the residual correlation matrix was examined for possible violations of local independence. Absolute residual correlations greater than 0.20 were considered to be possible indicators of local dependence [35].

The risk perception items were modeled using Samejima’s two-parameter polytomous graded response model [GRM; 36], using marginal maximum likelihood estimation, with Multilog version 7.03 [37]. GRM estimates a slope parameter (α) and threshold parameters (b). MODFIT [38] was used to produce plots of IIFs, empirical and predicted ORFs, test information and test standard error of measurement and to obtain χ2/df ratios to examine model-data fit.

Following IRT calibration, the concurrent criterion-related validity of the perceived risk measure was examined. Validity measures included HIV test results, syphilis test results, and retrospective self reports of number of sex partners, and episodes of unprotected sex and having sex while high on drugs. Count variables (e.g. number of partners, number of sex acts) were examined for outliers (z scores greater than 3.3), and outliers were assigned a raw score one unit larger than the next extreme value [39]. Variable distributions were examined for normality. All count variables exhibited long right-hand tails; therefore the data were transformed by taking the square root [40]. Pearson correlation coefficients were calculated to examine the relationship between retrospective self-reports of risky sex behaviors and perceived risk. T-test analysis was conducted to examine whether people who received a positive HIV test result or positive syphilis test on a prospective test had greater perceived risk than those who tested negative.

Results

Descriptive Statistics

All items were scored so that higher scores indicated greater risk perception. The item pool contained some items with similar content, for example, items assessing the likelihood judgments that used different response options. Inter-item correlations were examined for redundant items (item pairs with a correlation greater than 0.70). A total of 8 items were excluded from further analysis due to high inter-item correlations.

Unidimensionality and Local Independence

The EFA scree plot strongly suggested a single factor, as did the ratio of the first eigenvalue to the second eigenvalue (9.31:0.44). The one-factor CFA provided additional evidence that data were sufficiently unidimensional to proceed with the IRT analysis (GFI = 0.94, RMSEA = 0.087, χ2(35) = 242.0). The residual correlation matrix revealed an average absolute residual of 0.029 and no residual correlations greater than 0.12. Based on these results, it was determined that the data were suitable for IRT analysis.

IRT Calibration

Table 1 presents the parameter estimates and standard errors for the IRT calibration. Item slopes (a) ranged from 1.37 to 3.57. Slope parameters provide an indication of how well response categories differentiate between different levels of perceived risk [28], with higher numbers suggesting, in general, better discrimination. Threshold parameters (number of categories minus 1) represent the level of the perceived risk necessary for a participant to respond above a threshold category with a 0.50 probability [28]. For example, for Item 1, β2 = 0.28 indicates that a person needed a perceived risk level 0.28 of a standard deviation above the population mean to be likely to endorse categories 3 or above (somewhat likely to extremely likely) for this item. Threshold 4 of the same item, β4 = 1.94, indicates participants needed a perceived risk level 1.94 standard deviations above the mean to be likely to endorse the fifth response category (extremely likely). The threshold parameters for 10 items revealed a fairly broad range across perceived risk from −1.48 to 2.04.

Table 1

Item content and estimated item parameters

Item content and response options	Item type	α (SE)	β1 (SE)	β2 (SE)	β3 (SE)	β4 (SE)	β5 (SE)	Adjusted χ2/df
Item 1. What is your gut feeling about how likely you are to get infected with HIV? (Extremely unlikely, Very unlikely, Somewhat likely, Very likely, Extremely likely)	A	2.25 (0.14)	−0.79 (0.07)	0.28 (0.06)	1.28 (0.09)	1.94 (0.15)		0
Item 2. I worry about getting infected with HIV (None of the time, Rarely, Some of the time, A moderate amount of time, A lot of the time, All of the time)	A	2.51 (0.14)	−0.92 (0.08)	0.05 (0.05)	0.82 (0.06)	1.19 (0.08)	1.74 (0.11)	0
Item 3. Picturing self getting HIV is something I find: (Very hard to do, Hard to do, Easy to do, Very easy to do)	S	1.82 (0.13)	−0.47 (0.08)	0.72 (0.08)	2.04 (0.17)			0
Item 4. Getting HIV is something I am… (Not concerned about, A little concerned about, Moderately concerned about, Concerned about a lot, Extremely concerned about)a	A	1.84 (0.12)	−1.00 (0.10)	0.09 (0.07)	0.69 (0.08)	1.35 (0.11)		0.54
Item 5. I am sure I will NOT get infected with HIV (Strongly disagree, Disagree, Somewhat disagree, Somewhat agree, Agree, Strongly agree)	C	2.23 (0.12)	−0.93 (0.08)	−0.24 (0.07)	0.29 (0.06)	0.82 (0.07)	1.75 (0.12)	0.10
Item 6. I feel I am unlikely to get infected with HIV (Strongly disagree to Strongly agree)ab	A	1.37 (0.10)	−1.48 (0.15)	−0.30 (0.09)	0.35 (0.09)	0.87 (0.11)	1.97 (0.21)	0
Item 7. I feel vulnerable to HIV infection (Strongly disagree to Strongly agree)b	A	2.69 (0.15)	−0.099 (0.07)	−0.15 (0.05)	0.09 (0.05)	0.69 (0.06)	1.51 (0.10)	0.53
Item 8. There is a chance, no matter how small, I could get HIV (Strongly disagree to Strongly agree)b	C	1.97 (0.13)	−1.39 (0.11)	−0.77 (0.08)	−0.54 (0.07)	0.15 (0.06)	1.32 (0.10)	3.66
Item 9. I think my chances of getting infected with HIV are: (Zero, Almost zero, Small, Moderate, Large, Very Large)	C	3.57 (0.20)	−0.91 (0.06)	−0.20 (0.04)	0.64 (0.05)	1.33 (0.07)	1.80 (0.10)	0.43
Item 10. Getting HIV is something I have (Never thought about, Rarely thought about, Thought about some of the time, Thought about often)	S	1.61 (0.11)	−1.46 (0.12)	−0.27 (0.08)	1.10 (0.10)			0

Note Slope estimates (α) include a 1.702 scaling factor. The number of between category threshold parameters (β) varies dependent on the number of response categories

A affective item, C cognitive item, S salience item

aItems not included in the final version of the PRHS

bThe Strongly disagree to Strongly agree options are the same as Item 5

Fit plots from MODFIT demonstrated good fit for all items. Ratios for χ2 to degrees of freedom are in Table 1. For all items, except item 8, the ratio was lower than three, indicating good model-data fit.

IIFs, ORFs and item content were examined to identify items to be retained in the final measure. Figure 1 presents the IIF plots for all 10 items. Item information indicates how well an item differentiates among individuals across different levels of θ. Items 9, 7 and 2 provided the most information around the center of the continuum of θ. Items that had low discrimination were candidates for deletion. Item content was reviewed to ensure that items covering the three key domains (e.g., cognitive judgments, intuitive beliefs and salience of risk) were retained. A 5, 6 and 8-item version of the scale were examined and the 8-item version was selected. The 8-item version had similar test information to the 10-item version and acceptable levels of standard error of measurement (Fig. 2). For the 8-item version, for a person 2 standard deviations below the mean (at θ = −2.0), the scale information would be approximately 13 (equivalent to a reliability of r = 0.92) and SEM would be approximately 0.28. For a person 2 standard deviations above the mean (at θ = +2.0), the scale information would be approximately 21 (equivalent to a reliability of r = 0.95) and SEM would be approximately 0.22. In the theta range of −0.5 to 0.5, the loss of reliability from a 10-item scale to an 8-item scale was between 0.006 to 0.008, suggesting reducing the number of items had little effect on the reliability of the measure. The marginal reliability for θs (an average reliability across the continuum of θ) was not significantly reduced from the 10-item version (0.92) to the 8-item version (0.91). As shown in Fig. 2, the 8-item version of the measure is best at differentiating individuals who are around the mean or just below the mean on perceived risk. The reliability of the measure (r = 1 − 1/information) was above 0.85 for those with a trait level approximately 2.3 standard deviations below the mean to those with a trait level approximately 2.7 standard deviations above the mean.

Fig. 1

IIFs plots for 10 items

Fig. 2

Test information and standard error of measurement functions for 10- and 8-item scales. Solid lines indicate total information; dotted lines indicate standard error

ORFs were examined independently by two of the authors to identify whether some items response options were redundant and could be collapsed. For example, item 7 response option 3 (somewhat agree) was not likely to be endorsed at any level of perceived risk. As shown in Fig. 3, response category 3 was most likely to be selected around the mean of the trait (θ = 0). However, at θ = 0, the probability of selecting the response option 3 was approximately 16%, which is much lower than the probability of selecting response options 2 (34%) or 4 (30%), suggesting this response option is redundant. Therefore, response option 3 was recoded to 2. Similarly, for item 1, response option 4 was redundant and collapsed with option 5 (both coded as 4). For item 2, response options 4, 5 and 6 were collapsed (all coded to 4), and for item 8 response options 1, 2 and 3 were collapsed (all coded as 3).

Fig. 3

ORF plots for Items 2, 3, 7 and 9

Criterion-Related Validity

Multilog was used to estimate person location (θ, reflecting participants’ perceived risk) using maximum a posteriori (MAP) estimation. In addition, using the collapsed item categories, responses to each of the items were summed to create a total PRHS score. This total score provides an approach to scoring the measure that could easily be applied in future research. Total scores ranged from 10 to 40 (M = 22.7, SD = 7.4). Only 5.5% of participants received the lowest score on the measure (10), indicating they believed they had absolutely no risk of getting HIV because of their sex behaviors. The 8-item scale was found to have excellent internal consistency (α = 0.88). Criterion-related validity was examined for both the total score and θ, and the results were extremely similar using both methods of scoring, therefore, only results for the total score are presented below.

A total of 652 participants chose to receive an HIV test. Participants who received a positive test result (n = 13) reported greater perceived risk (M = 26.8, SD = 8.1) than those who received a negative test result (M = 22.5, SD = 7.4; t(650) = 2.09, P < 0.05). A subset of clients also tested for syphilis using RPR and TP-PA tests. Participants who tested positive for antibodies on the TP-PA test (n = 40) reported greater perceived risk (M = 26.3, SD = 8.3) than those who received a negative test result (M = 22.2, SD = 7.0; t(442) = 3.45, P < 0.001). There was a similar difference in risk perception for participants who tested positive for antibodies on the RPR test (M = 26.6, SD = 9.4) compared to those who received a negative test result (M = 22.5, SD = 7.1; t(450) = 2.58, P < 0.05).

Scores on the PRHS were positively correlated with number of sex partners in the last 30 days (r(763) = 0.31, P < 0.001) and number of days had sex (r(764) = 0.24, P < 0.001). Perceived risk was also weakly positively correlated with the number of times had unprotected anal sex (r(763) = 0.19, P < 0.001), unprotected vaginal sex (r(763) = 0.18, P < 0.001), and unprotected oral sex (r(763) = 0.25, P < 0.001). Among crack users, perceived risk was associated with having sex while high on crack (r(265) = 0.30, P < 0.001), and among amphetamine users it was associated with having sex while high on amphetamine (r(252) = 0.20, P < 0.001).

Further analyses were conducted to examine whether perceived risk differed by type of sex partner. Participants who reported having sex with an injection drug user (n = 97) reported greater perceived risk (M = 27.7, SD = 7.0) than those who did not (M = 21.9, SD = 7.1; t(750) = −7.57, P < 0.001). Participants who reported having traded money or drugs with sex (n = 65) also reported greater perceived risk (M = 28.8, SD = 6.2), than those who did not (M = 22.2, SD = 7.2; t(763) = −7.16, P < 0.001).

Scores on the PRHS were compared to a single-item likelihood measure of risk perception on the RBA. The PHRS total score was positively correlated with the single item (r(762) = 0.63, P < 0.001).

Discussion

This study developed a short measure of perceived risk of HIV infection related to sex behaviors that covers several dimensions of perceived risk. In addition to more common tools in scale development, IRT was used to examine individual item characteristics, to exclude items that provided little information and to collapse item categories. Data from the current study suggests that using 6 response options to measure perceived risk can result in redundant response categories and that 4 response options may be sufficient to differentiate levels of perceived risk. Although IRT is rather complex, this approach provided a useful tool for understanding how individual items functioned, and could be applied to the measurement of other constructs related to HIV risk behaviors.

In the past, researchers have used a variety of approaches to measure perceived risk including single item likelihood assessments [5, 41], as well as more intuitive measures of worry or feelings of vulnerability [23, 42, 43]. The 8-item PRHS incorporates items assessing cognitive assessments of risk (e.g., chance of infection), as well as intuitive assessments (e.g., feeling vulnerable, worry, gut feeling about likelihood) and salience of risk (e.g., thought about risk, can picture it happening) to provide a more comprehensive measure of perceived risk of HIV infection. The current study found that the different approaches to assessing perceived risk loaded on a single factor and were included in one scale. This finding is interesting, considering past research that has suggested measures of perceived likelihood and affective measures, such as worry, are not always closely related and may be separate constructs [44, 45]. Further research is still need to examine the relationship between different dimensions of perceived risk.

The study found evidence for the criterion-related validity of the measure. Scores on the PRHS were associated with retrospective reports of risky sex behaviors, including number of sex partners, times had unprotected sex and times had sex while high. Participants who either had a sex partner who injected drugs or with whom they had traded money or drugs with sex, perceived themselves to be at higher risk. A major strength of the measure is that scores on the PRHS differed by HIV test results. Those participants who tested positive for HIV also saw themselves to be at higher risk. Furthermore, those who tested positive for syphilis antibodies also reported high risk perception scores. As predicted, when participants were asked to think about their risk of HIV based on their past sex behaviors, those who engaged in risky behaviors saw themselves to be at higher risk.

Scores on the PRHS were compared to a single-item measure of HIV risk perception on the RBA. The two measures were strongly positively correlated, providing support for the convergent validity of the PRHS. However, the size of the correlation (r = 0.63) also suggested that a single likelihood item did not capture the breadth of content of the 8-item scale.

In the study, participants made judgments about their risk for HIV based on past sex behaviors, and these were compared to retrospective reports of behavior. This measure would be appropriate to use in similar contexts, for example, when assessing the accuracy of risk perceptions by examining associations with past behavior, or in studies examining whether an intervention has enhanced risk perception. The measure could be adapted for studies examining whether risk perception predicts future behavior by making the items conditional on a specific behavior (for example, not using condoms) rather than based on past sex behavior. By doing so it would be possible to examine whether using a broader operationalization of risk perception predicts future behavior. The cross-sectional design of the current study also limits the ability to examine the predictive criterion-related validity of the measure and longitudinal research is needed to examine the complex relationship between perceived risk and behavior.

In the present study the PRHS, was completed prior to participants reporting on their HIV risk behaviors or receiving HIV prevention counseling or testing. The order of the questionnaires was chosen to reduce the possible effect of completing a risk assessment or counseling on responses to the PRHS. However, it is possible that the completion of the PRHS may have affected participants’ responses to later questions on the RBA. For example, thinking about one’s risk of HIV may have made salient risk behaviors that the participant may not otherwise have recalled.

In the current study, both the PRHS and RBA were administered by a trained interviewer. This method allows interviewers to clarify participants’ responses, minimizes missing data [46] and is useful in settings with low adult literacy, such as Los Angeles County [47]. However, this approach may have also resulted in self-presentation biases, and in other contexts researchers may consider having participants self-administer the PRHS or using audio computer-assisted self-interviewing (Audio-CASI).

One of the benefits to using IRT analysis is that it provides an approach to examine the reliability of the measure at different levels of theta (perceived risk). The IRT analysis suggested that the PRHS was most reliable for individuals whose perceived risk fell approximately between 2.5 standard deviations below or above the mean. The PHRS may have lower reliability and criterion validity if administered in samples at low risk, or who have more unrealistic perceptions of their risk. The current study recruited participants accessing HIV prevention or testing services, and may not be representative of at-risk populations who do not access these services. Despite the recruitment setting, there was still considerable diversity on scores on the PRHS, including clients who did not perceive themselves to be at risk of HIV infection at all. This may reflect the fact that people may be motivated to seek HIV testing and prevention services for reasons other than perceiving themselves to be at risk, for example, engaging in HIV testing as part of a routine health maintenance strategy, or motivated by non-cash incentives.

Few studies address the psychometric properties of the HIV risk perception items they employ. The current study provided evidence for the reliability and validity of a measure that incorporates different dimensions of perceived risk. Although the current study has limitations, it is hoped that the topics raised will encourage others to consider examining the construct of perceived risk for HIV infection more broadly, and to use multiple items, including items examining how people feel about their risk. Given the theoretical importance of perceived risk of HIV and how commonly this construct is included in interventions aiming to reduce risk behaviors, more research is needed to understand how best to conceptualize and operationalize this construct, as well as research addressing the psychometric properties of items and scales employed.