The Individual Water Insecurity Experiences (IWISE) Scale: reliability, equivalence and validity of an individual-level measure of water security.

OBJECTIVE: The lack of a validated and cross-culturally equivalent scale for measuring individual-level water insecurity has prevented identification of those most vulnerable to it. Therefore, we developed the 12-item Individual Water InSecurity Experiences (IWISE) Scale to comparably measure individual experiences with access, use, and stability (reliability) of water. Here, we examine the reliability, cross-country equivalence, and cross-country and within-country validity of the scale in a cross-sectional sample.
METHODS: IWISE items were implemented by the Gallup World Poll among nationally representative samples of 43 970 adults (>15 y) in 31 low-income and middle-income countries (LMICs). Internal consistency was assessed with Cronbach's alpha. Equivalence was tested using multigroup confirmatory factor analysis (MGCFA), the alignment method, and item response theory. Cross-country validity was assessed by regressing mean national IWISE scores on measures of economic, social, and water infrastructure development. Within-country validity was tested with logistic regression models of dissatisfaction with local water quality by IWISE score and regressing individual IWISE scores on per capita household income and difficulty getting by on current income.
FINDINGS: Internal consistency was high; Cronbach's alpha was ≥0.89 in all countries. Goodness-of-fit statistics from MGCFA, the proportion of equivalent item thresholds and loadings in the alignment models, and Rasch output indicated equivalence across countries. Validity across countries was also established; country mean IWISE scores were negatively associated with gross domestic product and percentage of the population with access to basic water services, but positively associated with fertility rate. Validity within countries was also demonstrated; individuals' IWISE scores were positively associated with greater odds of dissatisfaction with water quality and negatively associated with lower financial standing.
CONCLUSIONS: The IWISE Scale provides an equivalent measure of individual experiences with water access and use across LMICs. It will be useful for establishing and tracking changes in the prevalence of water insecurity and identifying groups who have been 'left behind'. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Water insecurity threatens human health, nutrition and psychosocial well-being.

Experiential water insecurity scales are useful because they bring a human voice to a sector that has predominantly relied on data about water availability and infrastructure.

Individual-level data on water insecurity are needed to understand who may be left behind, but no tool to do so globally has been validated or assessed for cross-country equivalence.

We have established the reliability, cross-country equivalence, and cross-country and within-country validity of an individual-level measure of water insecurity, the Individual Water InSecurity Experiences (IWISE) Scale, using nationally representative data from 43 970 individuals in 31 low-income and middle-income countries.

The IWISE Scale can be used to obtain disaggregated, high-resolution measures of water insecurity and to compare water insecurity within and across countries.

The ability to measure the water insecurity of individuals in relationship to its potential determinants and consequences represents a significant advantage for understanding our progress towards development goals.

Introduction

We are experiencing a global water crisis.1 2 Problems with quantity (too much, too little) and quality (biological, chemical contaminants) are increasing in frequency and severity throughout the world due to myriad forces including climate change, increasing water use, crumbling infrastructure and pollution.3–5 Household water insecurity, that is, problematic availability, access, acceptability, safety or stability (or reliability) of water for household uses,6 has consequences for a range of phenomena from individual nutrition and health to food security, economic productivity, political unrest and migration.1 7–10

Most globally comparable measures and indicators of sustainable access to safe water have assessed availability,11 12 for example, per capita renewable water resources, groundwater withdrawal, or infrastructure, e.g., the WHO/UNICEF Joint Monitoring Programme (JMP) ladder for drinking water, from which the percentage of a population with access to at least basic drinking water services is estimated.13 Availability and physical accessibility are necessary but insufficient for water security.8 14 For example, millions of people cannot reliably access nearby water due to economic, political and/or other barriers. Further, measurements at the regional and community levels can obscure huge inequalities in the distribution of resources.12

Measures and indicators that are proximal to the human experience of water insecurity are often more informative because they capture the types of physical and emotional challenges on which humans act.15 As such, data on per capita water availability do not provide the details needed to fully understand the consequences of water insecurity for health and well-being, to make informed decisions about the allocation of resources to improve water security, or to evaluate the impact of interventions or shocks on human capital. Higher-resolution measurements of water access, use and stability (the major components of water security) among individuals are needed.16

To that end, the development of the Household Water Insecurity Experiences (HWISE) Scale has provided a powerful way of understanding how water insecurity shapes household well-being.17–19 This scale, which was developed using a reflective measurement model, measures manifestations of the constructs of water access and use specifically, all of which are dependent on (and are therefore also implicitly indicative of) water availability and stability. An item in the scale can reflect more than one construct. The development of the HWISE Scale was informed by a number of site-specific scales that are suitable for measuring household experiences with water access and use in a specific context,20–22 but were not suitable for global comparisons.17 The reliability, validity and cross-context equivalence (ie, measurement invariance across settings) of the scale were established drawing on data from 28 sites in 22 low-income and middle-income countries (LMICs).18 The full HWISE Scale17 and its brief, 4-item version23 have been useful in understanding how water insecurity shapes and/or is shaped by household income,24 physical injury,25 food insecurity,26 stress17 and SARS-CoV-2 pandemic.27–29 It has also been useful in assessing intervention impacts and advocating for resources.8

In addition to being a more proximal measure to health and well-being, several additional advantages are gained by shifting the level of analysis from the household to the individual level. For one, individuals are most knowledgeable about their own experiences. Furthermore, measurements of resources at the household level can obscure intrahousehold variation.30–32 Water access and use can differ by gender, age, reproductive status and other sociodemographics,32–35 that is, widespread intrahousehold variation in experiences of water insecurity is highly plausible. For example, worry and anger about water may differ depending on one’s role in water acquisition, which is often tied to gender norms and age relative to others in the household.36–40 The food security literature, which parallels the water security literature in many ways, similarly supports that food access and use differ by gender, age and other sociodemographic characteristics.41–44 For these reasons, most global data collection systems ask about individual, not household, experiences.

The importance of disaggregated data has also been recognised in discussions of clinical reporting45 46 as well as in the context of the current Sustainable Development Goals and post Sustainable Development Goal agenda.47 48 There is intention to rectify the notable absence of gender-disaggregated data from most of the current Sustainable Development Goal indicators in post-2030 agenda. Without information on sociodemographic characteristics of those who are water insecure, identifying who is ‘left behind’ is impossible.37 49

Therefore, we set out to adapt the HWISE Scale to be used for the measurement of individuals’ water insecurity experiences, and to establish the reliability, cross-context equivalence, and validity of the Individual Water Insecurity Experiences (IWISE) Scale in nationally representative surveys administered by the Gallup World Poll (GWP). Specifically, we sought to determine the reliability, equivalence and validity of the IWISE Scale for measuring the prevalence of water insecurity across and within populations in LMICs as well as its relationships to determinants and consequences of water insecurity. To that end, we evaluated (1) reliability, that is, if the scale is internally consistent within and across countries, (2) cross-country equivalence, that is, if the scale is comparable across countries, and (3) validity, that is, if the scale accurately differentiates water insecurity both across countries and across groups of individuals within countries. We expected that country mean IWISE scores would be lower in relation to greater economic and social development and better water infrastructure development. We also anticipated that individual IWISE scores would be closely aligned with reported dissatisfaction with water quality and inversely related to individual financial standing.

Methods

Development of the IWISE Scale

The HWISE Scale items ask about the frequency of 12 experiences commonly associated with the water insecurity constructs of access and use, including emotions, hygiene behaviours and consumption patterns affected by water over the last 4 weeks.17 All HWISE items are phrased ‘In the last 4 weeks, how frequently have you or anyone in your household…’.19 Response are scored 0 ‘never’, 1 ‘rarely’, 2 ‘sometimes‘, 3 ‘often/always’ and summed (range 0–36). These items were selected based on extensive qualitative and quantitative data and tested extensively in 28 sites in 23 LMICs.17 18

To create the IWISE Scale, HWISE items were modified to capture individual responses, that is, ‘you or anyone in your household’ has been changed to ‘you’. The recall period was also changed to be the prior 12 months (table 1). The change from a 4-week to a 1-year recall period made the IWISE items consistent with other items asked or measured in the year-long period in the GWP, including the Food Insecurity Experiences Scale.50 Possible IWISE responses were ‘never’, ‘in 1 or 2 months’, ‘in some but not every month’ and ‘in almost every month’, scored as 0, 1, 2 and 3, respectively. As with the HWISE Scale, the IWISE items were summed for a total score ranging from 0 to 36.

Table 1

The Individual Water Insecurity Experience (IWISE) items and guidance on their administration and scoring

Abbreviation	Introduction to be read aloud prior to asking the 12 IWISE questions: ‘I will now ask you about your experiences with water. For each experience, we want to know in how many months this happened to you during the last 12 months. Even if it happened just once during a month, we’d like you to count that month’.
Worry	(1) How often did you worry that you would not have enough water for all of your needs? Never, in 1 or 2 months, in some but not every month, or in almost every month?*
Interruption	(2) Please think about where you get most of your water, such as a tap, well, borehole, bottled water, river or stream. How often was this water source interrupted or limited in any way during the last 12 months?
Clothes	(3) How often could your clothes not be washed because of problems with water?
Plans	(4) How often did you have to change schedules or plans because of problems with water?
Food	(5) Still thinking about the last 12 months, how often did you change what you ate because of problems with water?
Hands	(6) How often were you not able to wash your hands after dirty activities because of problems with water?
Body	(7) How often were you not able to wash your body because of problems with water?
Drink	(8) How often did you not have as much water to drink as you would have liked?
Anger	(9) Still thinking about the last 12 months, how often did you feel angry because of problems you were experiencing with water?
Sleep	(10) How often did you go to sleep thirsty because there was no water to drink?
No water	(11) How often did you have no useable or drinkable water whatsoever?
Shame	(12) How often did you feel shame because of problems you were experiencing with water during the last 12 months?

*The interviewer repeats the scale responses as necessary after the first item. If respondent says, ‘In every month’ code as ‘in almost every month’. ‘Never’ is scored as 0, ‘in 1 or 2 months’ is scored as 1, ‘in some but not every month’ is scored as 2 and ‘in almost every month’ is scored as 3 for a summed score ranging from 0 to 36. Although the respondents are reminded of the time frame of 12 months (stated in the initial prompt) in items 5, 9 and 12, the interviewer should repeat the time frame more frequently if the respondent is struggling or confused.

Lengthening the recall period of the items to cover the prior 12 months instead of the prior 4 weeks increases the accuracy for making comparisons within and across countries. A year encompasses the entirety of annual variation in precipitation and temperature, and surveys were not administered at the same time of the year or season. It also allows the recall period to align with the recall period of other survey instruments (eg, the Food Insecurity Experiences Scale).51

Originally, we intended the 2020 implementation of the IWISE Scale to occur in African countries only. Accordingly, we conducted a total of 50 cognitive interviews with men and women in local languages in five sites in Africa: periurban Kenya, rural Tanzania, urban Nigeria, periurban Ethiopia, and periurban and rural Morocco. Cognitive interviews suggested that items were understood as intended and answerable (data not shown). Subsequent changes in survey logistics because of the SARS-CoV-2 pandemic, which limited possibilities for face-to-face interviews, meant that some of the African countries originally planned were replaced by countries in Asia and Latin America due to low telephone penetration (figure 1).

Figure 1

The Individual Water Insecurity Experiences (IWISE) module was administered by the Gallup World Poll in 31 low- and middle-income countries in 2020.

GWP survey procedures

GWP surveys the non-institutionalised civilian population, aged 15 and older, using probability-based, nationally representative samples. The IWISE Scale was implemented between 4 September 2020 and 24 February 2021 in 31 countries: 21 in sub-Saharan Africa, 4 in north Africa, 3 in Asia and 3 in Latin America (figure 1). Approximately 1000 individuals were sampled per country, with exceptions in India (n=12 650) where commissioned oversampling occurred, and China (n=3503).

For countries or regions in which English was not the dominant language spoken, the IWISE Scale was translated from English into the local language(s) with the help of a local translator. To ensure proper translation, an independent third-party translator reviewed and made edits to each original translation. The original translator then reviewed the edited version of the translation and accepted or rejected the suggested changes. In rare cases when the original translator rejected any major substantive changes made by the reviewer, an adjudication procedure occurred. The translated surveys were then piloted in each respective region before being formally implemented to catch any remaining issues with comprehension. All translations are available at www.hwise.org, as is implementation guidance.

All in-country partners who collected data received training on GWP standardised guidelines for selecting and recruiting respondents and conducting quality interviews. Data were collected by telephone in 29 countries; face-to-face surveys occurred in Mali, Senegal and in two of the three waves of data collection in India.

Most telephone surveys were conducted with respondents using mobile phones (table 2). Mobile phoneswere used exclusively in 21 countries; a mix of mobile and landline telephones were used in 8 countries. Thus, where face-to-face surveys were conducted, samples represent all adults (≥15 years old); in countries using a telephone survey, samples represent all adults with access to a landline or mobile phone (in countries using both types of telephones), or all adults with access to mobile phone.

Table 2

Individual and country characteristics of the 31 low-income and middle-income countries surveyed with the Individual Water Insecurity Experience (IWISE) Scale by Gallup World Poll 2020, by world region (n=43 970)

Country	Gallup World Poll Data (2020)							World Bank Data			WHO/UNICEFJoint Monitoring Programme data
	Analytical sample*	Age (years)	Female	IWISE Score	Dissatisfied with local water quality†	Annual per capita household income(Intl dollars)	Difficulty getting by on income‡	Gross domestic product per capita (2019)	Fertility rate (2018)	Infant mortality rate (2019)	Percentage of population with basic drinking water services (2020)
	N	Mean	Per cent of sample	Mean (SD)	Per cent of sample	Median (IQR)	Per cent of sample	Intl dollars	Births/ woman	Deaths/ 1000 live births	Per cent
Sub-Saharan Africa
Benin	999	31.3	53.8	7.1 (7.7)	35.7	520 (173, 1301)	68.3	3433	4.8	59.0	65.4
Burkina Faso §	981	30.1	52.0	10.7 (9.2)	41.5	293 (88, 878)	59.2	2275	5.2	53.9	47.2
Cameroon	996	31.4	53.1	15.4 (9.6)	67.3	807 (403, 1815)	69.1	3803	4.6	50.2	65.7
Congo Brazzaville	878	36.2	51.6	7.2 (8.2)	47.7	435 (193, 836)	69.8	3836	4.4	34.9	73.8
Côte d'Ivoire	935	31.0	49.6	7.2 (7.3)	43.2	565 (225, 1483)	65.0	5443	4.6	58.6	70.9
Ethiopia	1002	33.1	48.4	11.2 (8.6)	47.5	1095 (469, 2409)	61.6	2320	4.2	36.5	49.6
Gabon	987	34.0	47.0	11.0 (9.7)	69.5	738 (131, 1477)	66.9	15 612	4.0	31.1	85.3
Ghana	955	31.2	49.8	6.2 (7.4)	24.9	504 (69, 1272)	61.1	5652	3.9	33.9	85.8
Guinea	962	32.2	50.5	7.2 (7.6)	43.0	277 (37, 739)	53.2	2675	4.7	63.8	64.0
Kenya	984	30.7	50.4	12.3 (10.0)	45.8	639 (224, 1398)	63.3	4521	3.5	31.9	61.6
Mali¶	926	34.4	51.4	6.0 (7.3)	38.8	172 (26, 430)	49.9	2424	5.9	60.2	82.5
Mauritius §	949	42.1	45.3	4.8 (6.5)	15.6	3388 (1977, 5647)	39.8	23 882	1.4	14.3	>99.0
Namibia	944	32.7	53.6	11.2 (10.4)	41.0	438 (140, 1264)	77.4	10 064	3.4	30.7	84.3
Nigeria	1002	33.1	47.5	8.4 (8.6)	45.9	437 (146, 1040)	67.9	5363	5.4	74.2	77.6
Senegal ¶	978	34.2	53.4	5.7 (8.1)	43.9	524 (283, 920)	59.3	3545	4.6	32.7	84.9
South Africa	981	34.9	51.2	7.1 (8.7)	9.0	1277 (568, 3193)	47.4	13 034	2.4	27.5	93.9
Tanzania	980	31.9	51.0	9.7 (10.1)	34.0	400 (167, 999)	47.8	2771	4.9	36.0	60.7
Togo	955	32.4	50.6	8.5 (8.6)	51.9	373 (166, 646)	74.9	1667	4.3	45.8	68.6
Uganda	939	30.0	53.2	8.6 (8.2)	38.1	226 (30, 601)	76.6	2284	5.0	33.4	55.9
Zambia	976	31.0	49.9	11.6 (9.1)	57.8	769 (235, 1826)	73.9	3624	4.6	42.4	65.4
Zimbabwe	974	34.4	49.5	11.5 (9.7)	53.0	**	82.2	2961	3.6	38.4	62.7
North Africa
Algeria §	996	36.3	48.4	7.8 (7.8)	43.3	2779 (993, 4864)	25.4	12 020	3.0	20.0	94.4
Egypt §	980	35.3	47.3	7.5 (8.6)	37.2	1702 (946, 2624)	46.3	12 284	3.3	17.3	>99.0
Morocco §	955	37.4	50.8	4.2 (7.8)	30.9	1135 (340, 2723)	33.3	7826	2.4	18.3	90.4
Tunisia §	951	38.2	50.1	6.6 (7.8)	60.7	2874 (1533, 6387)	48.4	11 232	2.2	14.5	97.5
Asia
Bangladesh	1007	32.9	49.0	2.5 (7.0)	13.9	991 (413, 1718)	32.1	4964	2.0	25.6	97.7
China	3431	42.1	46.1	1.5 (3.8)	21.0	6876 (2947, 14 734)	30.1	16 804	1.7	6.8	94.3
India ††	12 349	35.9	48.2	4.2 (7.1)	16.1	816 (389, 1632)	52.1	6997	2.2	28.3	90.5
Latin America
Brazil §	990	38.7	51.9	4.7 (6.8)	22.6	3331 (1582, 6246)	31.1	15 300	1.7	12.4	>99.0
Guatemala §	1101	34.8	48.9	6.9 (8.4)	23.5	873 (277, 1872)	53.9	9020	2.9	20.7	94.0
Honduras	927	33.2	52.5	12.2 (9.8)	29.0	615 (231, 1475)	72.2	5981	2.5	14.5	95.7
Full sample‡‡	43 970	37.8	47.9	4.0 (7.0)	22.9	1813 (567, 5787)	42.9	7213	3.7	34.4	79.3

Note: All means, medians, SD, and proportions are weighted.

*Sample with complete IWISE data.

†Based on a reduced sample of 43 683.

‡Based on a reduced sample of 43 269.

§Countries that used both mobile and landlines (instead of just mobile) for telephone surveys.

¶Countries that conducted surveys exclusively face to face.

**Data on per capita household income data in Zimbabwe were considered missing due to inconsistencies in currencies used within the country in 2020.

††Surveys conducted face to face for 76.3% of respondents and by mobile telephone for the remaining respondents.

‡‡Descriptive statistics for Gallup World Poll variables among the full sample were calculated using projection weights (probability sampling weights divided by each country’s analytical sample size and multiplied by each country’s ≥15-year-old population size). Therefore, they represent the means and percentages of the overall ≥15-year-old population across these 31 low-income and middle-income countries.

In countries with considerable landline presence, GWP selected respondents using an overlapping dual-frame design using pure or list-assisted random-digit-dialling to obtain a nationally representative set of phone numbers. In countries with low landline penetration, GWP used pure random-digit-dialling from a mobile only frame to obtain a nationally representative set of phone numbers. For respondents reached by landline, random selection within the household was performed by asking for the person ≥15 years with the next birthday or using the Computer-assisted telephone interviewing programs randomly selected a respondent from a list of household members≥15 years. At least five attempts were made to contact and interview each randomly selected respondent. For respondents reached by mobile phone, in countries with high telephone coverage (>85%) no other selection was necessary except to confirm that the respondent was ≥15 years old. In the countries with low telephone coverage, the mobile phone was treated as a household device and a random adult respondent was selected using the same methods used for landline respondents.

To ensure that samples were nationally representative, GWP constructed within-country base probability weights that accounted for selection of telephone numbers from the respective frames, unequal selection probability for dual landline and mobile telephone users, and the selection of only one adult when treating the phone as a household. These base weights were used to adjust weighted samples to match population totals based on country census data or other reliable sources to account for non-response on demographic and geographic variables such as age, gender, education and region. The final respondent weights incorporated both base probability weight and poststratification adjustment. Thus, the final weights used in our analyses match sample totals as closely as possible to national totals for gender, age, education and geography. Weights were normalised so that they summed to one for each country; this means that when countries were combined, each country would contribute equally. For describing characteristics of the population for the countries combined, projection weights were created by multiplying the normalised weights by each country’s estimated 15+ years old population size in 2020 using data obtained from the World Bank Population database.52

This study was based on deidentified data made available by Gallup. Gallup World Poll followed their standard protocol for obtaining consent from participants. The authors of this paper were not involved with the consent or data collection process.

Participant involvement

Although formative work for the HWISE Scale drew on ethnographic research that included participant involvement, no participants were involved in study design, implementation, or dissemination, including the writing of this manuscript. Informed consent of all survey participants was obtained, and survey protocols were approved by Gallup’s Internal Review Board and by the governing bodies in countries where approval is required.

Criterion variables used to assess cross-country validity of the IWISE Scale

To establish validity across countries, we compared national IWISE scores to three measures of social and economic development using the most recently available data from the World Bank.52 These criterion variables were gross domestic product (GDP) per capita in international dollars in 2019, fertility rate (births per woman) in 2018, and infant mortality rate (deaths per 1000 live births) in 2019. Higher GDP represents greater economic development, whereas lower fertility and infant mortality rates represent greater social development. Selection of these variables was based on conceptual and empirical work regarding national economic, social and health development over the past 30 years,53–56 prior work demonstrating strong correlations between food insecurity and these three measures of economic and social development,57 and availability of recent data for all countries in the sample.

As a measure of each country’s water infrastructure development, we used data from the WHO/UNICEF JMP global database on the percentage of the country’s population with access to at least a basic drinking water services in 2020.58 A basic drinking water service is one that is considered ‘improved’ (ie, has the potential to deliver safe water) and from which water can be collected within ≤30 min, including round-trip travel and queuing time.13 Percentage values entered as ‘>99’ in the JMP data were truncated at 99 for analyses.

Construct and criterion variables for assessing within-country validity

Three variables from the core GWP survey were suitable to investigate validity within countries. First, we used an indicator of satisfaction with the quality of locally available water, as this was the only other question on water asked by GWP and measures a different but related construct of water insecurity (water quality). The question was phrased, ‘In your city or area where you live, are you satisfied or dissatisfied with the quality of water?’ We hypothesised that this question and the IWISE Scale were measuring a similar construct and that higher IWISE scores would be associated with higher dissatisfaction with water quality.

We then used two GWP measures of economic status as criterion variables, that is, variables that do not measure the same construct as IWISE but that we hypothesised would be strongly associated with it given previous literature suggesting a strong connection between water insecurity and economic disparities.24 59 60 Annual per capita household income quintile was our first criterion variable. This measure of financial standing was based on respondents’ report of their monthly household income in local currency (using established methods for imputing values when a range is given or if a response is not provided).61 Income data were then annualised and converted to international dollars using the World Bank’s individual consumption purchasing power parity conversion factor, and then divided by the total number of people (children and adults) living in the household.61 These values were then categorised into weighted quintiles within countries.

Our second criterion variable was respondents’ rating of the adequacy of their income. GWP asked ‘Which one of these phrases comes closest to your own feelings about your household’s income these days: living comfortably on present income, getting by on present income, finding it difficult on present income, or finding it very difficult on present income?’ As it was not clear if the differentiation between the former two responses or the latter two responses would be consistent across countries, we instead grouped responses to this question as either not having difficulty getting by (the first two response options) or having difficulty getting by (the latter two). This allowed us to simply differentiate between some difficulty getting by or not, which is more likely to be answered equivalently across countries.

Statistical analyses

Analytical sample

Responses of ‘don’t know’ or refusals to respond to an IWISE item were considered missing for the purpose of scale validation. Of the 45 555 individuals surveyed with the IWISE module in 31 countries, 7 respondents were missing data for all IWISE items, 190 were missing data on ≥4 IWISE items and 1585 were missing at least 1 response. We restricted analyses to those with complete IWISE data (n=43 970, 96.5%). There were no important differences in age or gender composition between those with complete IWISE data (mean age=34.1±0.1, 50.2% female) and without (mean age=35.4±0.7, 50.2% female). A greater percentage of those with incomplete IWISE data reported dissatisfaction with water (46.0% vs 38.4%) and difficulty getting by on current income (65.1% vs 56.6%), and those with incomplete IWISE data reported per capita household income that was on average 45.9% lower.

Of those with complete IWISE data, 287 and 674 had missing data on water quality dissatisfaction and reported difficulty getting by on current income, respectively. Those missing data on water quality dissatisfaction and difficulty getting by on current income had IWISE scores that were on average 2.0±0.9 and 3.2±0.4 points lower, respectively, than those who responded to those two questions. All Zimbabwe data (n=974) were excluded for analyses with per capita household income after GWP’s data quality team deemed the income data to be unreliable given instability in the currency during 2020. Together, these exclusions led to analytical samples of 43 683 for analyses pertaining to perceived water quality, 42 996 for analyses with income quintile, and 43 296 for analyses with reported difficulty getting by on current income.

Reliability

We assessed reliability as internal consistency by calculating Cronbach’s alpha for the 12 IWISE items within each country using Stata V.17. Values>0.80 were considered reliable.

Cross-country equivalence

The ability to compare IWISE scores across countries is dependent on the assumption that the scale measures the same latent construct (water insecurity) and performs consistently across different contexts. This is referred to as equivalence or measurement invariance.62 63 In assessing equivalence of the IWISE Scale across countries, we specifically examined whether the IWISE Scale was scalar equivalent. Scalar equivalence means that not only are the items interpreted similarly (item equivalence or configural invariance) and differences in scores are comparable (measurement equivalence or metric invariance) across contexts, but also that the definition of zero is the same across contexts (scalar equivalence or scalar invariance), allowing for comparison of average score and prevalence values.62 We used three methods for assessing equivalence across the 31 countries in our sample.

First, using classical test theory, we performed a conventional one-factor multigroup confirmatory factor analysis (MGCFA) to examine how well the 12 categorical IWISE items fit with models of incrementally increasing constraints on thresholds (ie, the degree of water insecurity needed to transition to the next response indicating greater frequency experiencing the water-induced challenge represented by a given item) and loadings (ie, the relationship between the item response and the latent construct, water insecurity). Specifically, the MGCFA compares the fit of (a) the least constrained configural model for which both thresholds and loadings can vary freely across countries, (b) the metric model that holds factor loadings equivalent across countries, and (c) the most constrained scalar model that holds both thresholds and loadings equal across countries. We tested the fit of these models with Mplus (V.8.6) using the robust weighted least squares estimator and theta parametrisation, and specifying sampling weights and strata. For goodness of fit, we examined the root mean square error of approximation (RMSEA) and its 90% CI, the comparative fit index (CFI), the Tucker-Lewis index (TLI) and the standardised root mean square residual (SRMR) value. RMSEA estimates of <0.05 with the upper limit of the 90% CI <0.06 indicate good fit and RMSEA<0.08 indicates acceptable fit.64 65 CLI and TLI estimates >0.95 and SRMR values of <0.08 are considered indicative of good fit.65 66 Changes in RMSEA and CFI estimates across the configural, metric, and scalar models that did not exceed a magnitude of 0.01 were also considered evidence of reasonable equivalence.67 68

Second, because the MGCFA method uses χ2 statistics to compare the fit of configural, metric and scalar models, these tests are often rejected when dealing with a large sample size.69 We therefore assessed approximate equivalence using the alignment method developed by Muthén and Asparahouv69 70 for examining equivalence across a large number of groups in Mplus. This method provides a less cumbersome method than the MGCFA method for identifying non-equivalent countries and items regarding both loadings (ie, the relationships between each item and the latent construct, water insecurity) and thresholds (ie, the degree of water insecurity needed to transition from one response category to the other for each item). The method uses a technique similar to the rotation criteria used in exploratory factor analysis to discover an optimal measurement invariance pattern, holding the assumption that the majority of parameters were approximately equivalent even if a few are non-equivalent.71 A set of in-depth postestimation algorithms described in detail elsewhere69 were then used to determine which threshold and loading parameters are non-equivalent. Following procedures detailed elsewhere for running the alignment test in Mplus,69 70 71 we generated models using the logit link function and the robust maximum likelihood estimator. Normalised sampling weights and region as a stratification variable were specified, and IWISE items were specified as ordinal. We compared the percentage of non-equivalent loadings and thresholds across countries and used <25% total non-equivalence (≥75% equivalence) as indicative of trustworthy alignment results.70 Because each item has three thresholds and there are 12 items for each of the 31 countries, the percentage of equivalent thresholds were calculated by dividing the number of equivalent thresholds across all countries and items by the total number of thresholds (3×12×31=1116) and multiplying it by 100. Likewise, the percentage of equivalent loadings was calculated by dividing the number of equivalent loadings by the total number of loading parameters (12×31=372) and multiplying it by 100. We performed both the free and fixed alignment models. The results were similar, so we report herein the results of the free alignment model, which may provide more accurate parameter estimates than the fixed model in situations of more than two groups and some degree of non-equivalence.69 We further confirmed the reliability of the alignment model and stability of factor means across countries by performing Monte Carlo simulations based on the parameter estimates from the free alignment model.69–71 These simulations estimated the correlations between the generated factor means and variances and those estimated in the alignment models; correlations above 98% suggest that comparisons across countries can be made despite the existence of some degree of non-equvalence.69–71

Third, using item response theory, we performed a single-parameter logistic Rasch model for dichotomised item responses (scored as 0 for never and 1 for all affirmative responses) to the 12 IWISE questions to examine an even more stringent form of equivalence that accounts for item severity scores. We used the customised R-package72 developed for validation assessments of the Food Experience Insecurity Scale across countries.51 We also examined cross-country equivalence by assessing differential item functioning using mixed-effects logistic regression models with country random intercepts and coefficients that tested the log odds of an affirmative response on dichotomised IWISE items in relation to each one-point increase in the binary IWISE score (ranging from 0 to 12).73 Further details about these methods are provided in online supplemental text 1.

Validity across countries

To test the relationship between national IWISE scores and country-level criterion variables, we correlated (Pearson) weighted country mean IWISE scores with country-level measures of economic (per capital GDP), social (fertility and infant mortality rates) and water infrastructure (percentage of the population with access to at least basic drinking water services) development. GDP was transformed using the natural logarithm due to its skewed distribution. We further performed linear regression models with robust standard errors that regressed weighted country mean IWISE scores against each of the country-level variables, and we estimated the predicted marginal national mean IWISE scores in relation to the range of values present in our dataset for GDP, fertility and infanty mortality rate, and percentage of the population with access to at least basic drinking water services. Each of these models was initially built with all 31 countries. As a sensitivity analyses, the models were repeated using only the 21 Sub-Saharan African countries. These analyses were performed using Stata V.17.

Validity within countries

We assessed the relationship between respondents’ IWISE scores and our construct variable, respondents’ dissatisfaction with water quality, by performing a pooled individual logistic regression analysis with robust standard errors and specification of sampling weights normalised to (ie, divided by) each country’s sample of respondents with complete data for IWISE and water quality dissatisfaction. This model tested the odds of reporting dissatisfaction with local water quality in relation to each one-point increase in IWISE score controlling for country fixed effects. Using the results from these individual-level pooled models, we further estimated the marginal probability of reporting water dissatisfaction for each 3-point difference in IWISE score. We performed a pooled individual linear regression analysis with robust standard errors, specification of normalised sampling weights and adjustment for country fixed effects to test the difference in mean IWISE score in relation to our two criterion measures of respondents’ financial standing: 1) each per capita household income quintile relative to the lowest quintile and 2) difficulty getting by on current income. All within-country validations tests were performed using Stata V.17.

Results

Sample characteristics

Mean age of samples across countries ranged from 30.0 to 42.2 years (table 2); 47.9% of the population represented by this sample was female.

IWISE scores varied substantially within and across countries. Mean (SD) IWISE score ranged from 1.5 (3.8) in China to 15.4 (9.6) in Cameroon, with an overall mean score of 4.0 (7.0). The most frequently affirmed items were those for experiences of interruptions in water supply, worry about not having enough water and anger because of water problems (figure 2). The three least frequently affirmed items were those related to problems with water preventing washing one’s body, hands or going to sleep thirsty.

Figure 2

Weighted mean response to each Individual Water Insecurity Experiences (IWISE) Scale item, by country (N=31) and across countries (n=43 970). Note: The score range for each item was 0 (never) to 3 (almost all months). See table 1 for full phrasing of each item. All Asian and Latin American countries are labelled, as are the three African countries with the highest mean score for the most often affirmed item, interruption. Each country was weighted equally when estimating combined country mean scores.

Reported dissatisfaction with local water quality ranged from 9.0% in South Africa to 69.5% in Ethiopia (overall mean 22.9%). Median annual per capita household income ranged between 172 international dollars in Mali and 6876 international dollars in China. The percentage of respondents reporting difficulty getting by on current income ranged from 25.4% in Algeria to 82.2% in Zimbabwe (overall mean 42.9%). In terms of water infrastructure, the percentage of the population with access to at least basic drinking water services ranged from 47.2% in Burkina Faso to >99.0% in Mauritius, Egypt and Brazil; the mean across the 31 countries was 79.3%.

Reliability

Cronbach’s alpha estimates for each country ranged from 0.89 to 0.96 (mean and median=0.91). This indicates high internal consistency in each country.

Cross-country equivalence

Goodness-of-fit statistics for the one-factor MGCFA on the categorical IWISE items supported acceptable scalar equivalence across countries (online supplemental table 1). This conclusion was supported by an RMSEA estimate of 0.053 (90% CI 0.052 to 0.054), closely in line with suggested thresholds of 0.05 (and <0.08 for acceptable fit) and an upper 90% CI <0.06. Likewise, the CFI and TLI of 0.977 and 0.983, respectively, for the scalar model were above the threshold of 0.95, and the SRMR estimate of 0.053 was below the threshold of 0.08 as indicative of good fit with the scalar model. The magnitude of change in CFI and RMSEA estimates across the configural, metric and scalar models never surpassed 0.01, further indicating reasonable equivalence.

Results from the alignment method of testing equivalence likewise suggested the IWISE Scale is approximately equivalent across countries (online supplemental table 2). More thresholds (18.5%) than loadings (3.0%) were non-equivalent across countries. This means that each of the items were similarly related to the latent construct, water insecurity, but that the extent of water insecurity that provokes a respondent to report a higher frequency of occurrence for some items may differ for some countries. For example, 66.7% of thresholds were equivalent for the item on interruption of water supply, but 92.0% were equivalent for the item on having to change plans due to water problems. Likewise, there was less equivalence in the factor loadings for the interruption item (78.5%) compared with that for the change plans item (96.2%). These results mean that the interruption item performed less consistently across countries than did the change plans item.

Overall, the mean equivalence (averaging the 81.5% of equivalent thresholds and 97.0% of equivalent loadings across countries) was 89.3%, which is well above the recommended benchmark of 75% (no more than 25% average non-equivalence across all thresholds and loadings). Furthermore, the postalignment Monte Carlo simulations based on parameter estimates of the free alignment model suggested a strong correlation between the true and estimated factor means (0.997), further indicating that the factor means were estimated well and were equivalent across countries.

The Rasch model of dichotomised IWISE responses suggested that most items had excellent fit for most countries (online supplemental table 3). The mean reliability estimates of 0.80 (range 0.79–0.84 across countries) also suggest that the dichotomised IWISE data fit the Rasch model well (data not shown). The ordering of dichotomised items by their severity scores as produced by the Rasch model aligned with the ordering of items by their mean categorical response, with the most severe items (eg, problems with water preventing washing body, hands and going to sleep thirsty) also being the least frequently affirmed items. Analysis of differential item functioning for the Rasch model found that, although some variation among countries occurred in the coefficients and intercepts from the regression of each item on the total score, the items were on the whole equivalent. This was demonstrated by coefficients for each item being of a similar magnitude (ie, similar log odds of affirming a response any given IWISE item in relation to each point increase in the binary IWISE score across countries), supporting the assumption of a Rasch model (online supplemental table 4). Furthermore, the intercepts were ordered as expected (ie, in the same order as the item severity scores), and the SD of the coefficients and intercepts across countries were small in relation to the average coefficients and intercepts, respectively.

Cross-country scale validity: weighted mean country IWISE scores in relation to country characteristics

Country mean IWISE scores were negatively correlated (−0.37) with ln(GDP) and were estimated to be on average 1.6 points lower (95% CI: −3.0 to –0.3, p=0.019) for each unit difference in ln(GDP) (figure 3A). Country mean IWISE scores were positively correlated (0.46) with country fertility rates and were estimated to be on average 1.2 points higher (95% CI: 0.3 to 2.0, p=0.010) for each additional birth per woman in the fertility rate (figure 3B). Country mean IWISE scores were marginally positively associated with country infant mortality rates (correlation 0.32; β10 deaths per 1000 live births=0.60, 95% CI: −0.1 to 1.3, p=0.077, figure 3C). Finally, for each 10 percentage point increase in the percentage of the population with access to at least basic drinking water services, country mean IWISE scores were strongly negatively correlated (−0.58) and estimated to be an average of −1.1 points lower (95% CI: −1.7 to –0.6, p<0.001) (figure 3D).

Figure 3

Predicted and observed weighted country mean Individual Water InSecurity Experiences (IWISE) scores in relation to indicators of economic and social development and percentage of population with access to at least basic drinking water (n=31). Note: Symbols represent observed weighted mean country IWISE scores. Beta coefficients and 95% CIs were obtained from simple linear regression models with robust standard errors regressing weighted country mean IWISE scores on (A) a 1-unit difference in ln-transformed per capita gross domestic product; (B) a 1-unit difference in births per woman; (C) a 10-unit difference in infant deaths per 1000 live births; or (D) a 10 percentage point difference in the percentage of the population with access to at least basic drinking water services. The black lines represent the predicted difference in country mean IWISE scores in relation to the respective country-level predictor variable, as estimated from a simple linear regression model. aData obtained from the World Bank databank53. bData obtained from the WHO/UNICEF Joint Monitoring Programme global database on household drinking water58.

When these models were restricted to only the 21 countries in Sub-Saharan Africa, the relationships between country IWISE scores and GDP, fertility rate, and infant mortality rate were much weaker (online supplemental figures 1A–2C). However, the relationship between country mean IWISE scores and the percentage of the population with access to at least basic drinking water services remained strong (correlation −0.52; β10-percentage point increase =-1.0, 95% CI: −1.6 to –0.4, p=0.002, online supplemental figure 1D).

Within-country scale validity: individual IWISE scores in relation to individual dissatisfaction with water quality and household economic standing

The pooled individual logistic regression analyses estimated that each point higher IWISE score was associated with 1.096 higher odds of reporting dissatisfaction with local water quality (95% CI: 1.091 to 1.102; p<0.001). Marginal predictions from this model estimated that those with a score of zero would have, on average, a 23.4% (95% CI: 22.5% to 24.3%) probability of being dissatisfied with their quality of water; that probability rose to 45.9% (95% CI: 44.9% to 46.9%), 69.8% (95% CI: 68.1% to 71.6%) and 86.5% (95% CI: 84.8% to 88.2%) among those with IWISE scores of 12, 24 and 36, respectively (online supplemental figure 2).

The pooled individual linear regression analyses estimated that IWISE scores were lower on average by 1.4 (95% CI: −1.9 to –0.9; p<0.001), 2.0 (95% CI: −2.5 to –1.5; p<0.001) and 2.9 (95% CI: −3.3 to –2.4; p<0.001) points among those in the third (middle), fourth and fifth (richest) income quintiles compared to those in the first (poorest) annual per capita household income quintile. The IWISE scores of the first and second income quintiles (the two poorest quintiles) were not different from each other (β=−0.1; 95% CI: −0.6 to 0.4; p=0.62).

Pooled individual linear regression models also estimated that IWISE scores were on average 3.6 points higher (95% CI: 3.3 to 3.9, p<0.001) among those reporting difficulty getting by on their current household income relative to those reporting living comfortably or getting by on their current income.

Discussion

The IWISE Scale was reliable, cross-country equivalent, and valid for differentiating water insecurity within and across the 31 LMICs in our sample. Validity was established both across countries with different values of water infrastructure development, as well as within countries, across individuals differing in dissatisfaction with water quality and household financial standing.

The Cronbach’s alpha (≥0.89 across all countries) supports high internal consistency of the IWISE items. Cross-country equivalence was demonstrated with each of the three methods for assessing cross-context equivalence. Goodness-of-fit measures for the classical test performed with the ordinal IWISE items were acceptable. The 89.3% mean cross-country equivalence for IWISE ordinal thresholds and item loadings in the alignment models exceeded the threshold of 75% for acceptability. Item functioning when fitting a Rasch model was largely non-differential among countries. Taken together, these results support that the IWISE Scale is scalar equivalent across countries, meaning that mean scores and prevalence estimates are comparable.

Validity across and within countries was also established. Country mean IWISE scores only had the hypothesised relationships with measures of economic and social development (ie, lower IWISE scores in relation to higher GDP and lower fertility and infant mortality rates) when combining data across global regions but not when restricting analyses to the 21 Sub-Saharan African countries. However, the negative relationship between country mean IWISE scores and the percentage of the population with access to at least basic drinking water services remained strong both in analyses that included all 31 countries, and in analyses restricted to the 21 Sub-Saharan African countries. The measure of water infrastructure was a better criterion variable against which to assess the cross-country validity of the IWISE Scale because of its conceptual closeness to water insecurity. In contrast, measures of economic and social development are not sensitive or specific to water insecurity.

The IWISE Scale can also validly distinguish between groups of individuals within countries. IWISE scores were positively associated with individuals’ dissatisfaction with the quality of locally available water. That IWISE scores are closely, but not perfectly related to water quality, is consistent with the understanding that water insecurity can exist even if water quality is perceived as satisfactory. The relationships with household measures of economic status were also as hypothesised. IWISE scores were higher among those reporting difficulty getting by on current income and lower among those with higher per capita household income.

Strengths of the study include the use of large, nationally representative samples from 31 countries and analysis with multiple, complementary, rigorous statistical techniques. Some limitations were that individuals without access to a telephone, who may be the most marginalised and water insecure populations within countries, may have been missed in these samples. In addition to the assumptions of the Rasch model, as described in the online supplemental text, our tests of equivalence demonstrated cross-country equivalence of the IWISE Scale for measuring water insecurity but did not examine equivalence across socio-demographic subgroups within countries. Tests of subgroup equivalence would require multiple equivalence analyses within each country, which was beyond the scope of this paper. Finally, a definitive measure (ie, a measure known to be highly accurate) of water insecurity to which we can compare the IWISE Scale scores was not available. Such a definitive measure has been developed and used in four studies for validation of food insecurity scales.62 Future research should develop and use such a definitive measure alongside the IWISE Scale.

Other future research directions include the investigation of reliability, equivalence, and validity of the IWISE Scale in high-income countries, as well as the demonstration of equivalence within countries across subgroups that differ by age, gender, water-fetching responsibilities, household location (urbanicity) and/or education level. IWISE Scale validation for other purposes, such as programme evaluation, would also be useful; a shorter recall period may be more suitable for evaluating programme impacts. Intrahousehold variation should also be investigated by surveying individuals of different genders and ages within the same household.

In conclusion, in this first investigation of an instrument to assess individual experiences of water insecurity across and within countries, we have demonstrated that the IWISE Scale provides an equivalent, valid, and reliable measure of water insecurity across countries, and a valid and reliable measure within countries. These findings suggest that the IWISE Scale is suitable for estimating population burden of water insecurity and understanding relationships between water insecurity and other national and individual characteristics. The ability to quantify the water insecurity burden of individuals across and within countries and to provide estimates that are disaggregated by gender and other social and demographic characteristics will be important next steps to track progress towards the Sustainable Development Goals and other development agendas. Furthermore, the IWISE Scale will be useful for advancing knowledge of the consequences of water insecurity at a granularity not possible with other water indicators, thereby strengthening evidence needed for advocacy. The information that the IWISE Scale generates can guide public health and economic policies and practices with the potential to improve water security at individual, household, community, regional and global levels.