Measuring deterioration of small-area housing environment: Construction of a multi-dimensional assessment index and validation in shared refugee accommodation.

Housing is an important health determinant, in particular for asylum seekers and refugees (ASR) living in state-provided accommodation and struggling for residential autonomy. However, few validated objective measurement tools exist to measure housing quality in the sense of the deterioration of the housing environment. We aimed to construct and validate an instrument to enable resource-efficient monitoring of and health research on such housing quality. After considering existing theoretical frameworks on housing effects on health, we constructed an easily applicable tool measuring the degree of "Small-area Housing Environment Deterioration" (SHED), based on the "Broken Windows" - index. In a validation study, we tested SHED index's objectivity and reliability, measuring inter-/intra-rater reliability and internal consistency and discussed its strengths and limitations by means of cognitive testing. We ran a field-test as part of a population-based, cross-sectional refugee health survey in a random sample of 58 shared accommodation centers across 44 districts of the German federal state of Baden-Wuerttemberg, enabling us to test index applicability and convergence with ASR's satisfaction with their living place. The new SHED index provides a validated and field-tested measure of deterioration of small-area housing environment with substantial reliability. Serving both researchers and policy-makers, SHED offers an easily applicable index to support epidemiological analyses on housing as a contextual and social determinant of health as well as evidence-informed decision making in questions of housing policies.

Introduction

Housing is an upstream determinant of (Bonnefoy, 2007; Marmot, Friel, Bell, Houweling, & Taylor, 2008; Shaw, 2004) as well as a fundamental requirement and resource for health (World Health Organization, 1986). Based on a multi-faceted relationship (Acevedo‐Garcia et al., 2004; J. R. Dunn, 2002b; Gibson et al., 2011), it has various effects on both physical and mental health outcomes (Ige et al., 2019; Krieger & Higgins, 2002; Kyle & Dunn, 2008; Leaver, Bargh, Dunn, & Hwang, 2007; Singh, Daniel, Baker, & Bentley, 2019). Being one of the core areas of public health research and action (Shaw 2004), concerns about the direct or material effects of housing on health have been complemented by concepts of housing in terms of “home” and “meaningfulness”, and as a manifestation of socioeconomic status (J. R. Dunn, 2002a).

In the context of refugee migration, the influence of housing on health may be of particular importance. Housing conditions of asylum seekers with unsecure residence status and refugees who have been granted international protection (asylum seekers and refugees, ASR) are shaped by limited autonomy (Au, Anandakumar, Preston, Ray, & Davis, 2019; Kotovicz, Getzin, & Vo, 2018), questions of residential instability, especially for children (Goosen, Stronks, & Kunst, 2014), and the role of privacy in shared, institutionalized accommodation centers (Hauge, Støa, & Denizou, 2017; Willems, Smet, & Heylighen, 2020). Further, ASR are usually assigned a place of residence based on administrative quota, which predominantly places them in more deprived districts in host countries (Bozorgmehr, Razum, Szecsenyi, Maier, & Stock, 2017). This may add to, or exacerbate, a pre-existing high burden of mental illness (Bozorgmehr et al., 2016) or other vulnerabilities.

Previous research among refugees and asylum seekers has studied health effects of structural and procedural aspects of housing, e.g. accommodation type (Bean, Eurelings-Bontekoe, & Spinhoven, 2007; Porter & Haslam, 2005; Walther, Fuchs, Schupp, & Von Scheve, 2020), number of relocations for children and adolescent refugees (Goosen et al., 2014), length of stay in accommodation centers (Hallas, Hansen, Stæhr, Munk-Andersen, & Jorgensen, 2007), effects of shared housing on young people's health and wellbeing (Wilkinson & Ortega-Alcázar, 2019), as well as health effects of culturally insensitive placement of unaccompanied refugee minors (O'Higgins, Ott, & Shea, 2018).

Ziersch and Due (2018) have discussed and confirmed these various physical and mental health associations of housing for asylum seekers and refugees in a recent systematic review, especially with regard to housing conditions, availability, overcrowding and tenure security. However, they note the lack of consistent housing measures across identified studies and point out that many of the studies were not specifically designed to examine the link between housing and health, especially in resettlement countries. It remains unclear how deterioration of small-area housing environment, e.g. of physical building structures or of green spaces in the immediate surrounding of the accommodation center, affects health and well-being among ASR in shared refugee accommodation.

The limited empirical knowledge on deterioration of housing environment in general may be directly linked to a lack of validated objective measurement tools. Existing housing quality indices, e.g. the Observer-Rated Housing Quality Scale (Adair et al., 2014) the Housing Quality Index (Elsinga & Hoekstra, 2005; Hoekstra, 2009) or the domain on physical and architectural resources of The Multiphasic Environmental Assessment Procedure (Moos & Lemke, Thousand Oaks) and other instruments (Shenassa, Daskalakis, Liebhaber, Braubach, & Brown, 2007; Suglia, Duarte, & Sandel, 2011; Wells & Harris, 2007), mainly focus on structural aspects. Such aspects are of permanent nature and do not include questions of deterioration which could be prevented by care and maintenance in due time. Furthermore, existing tools cannot be disaggregated or are not comprehensive because they only focus on singular issues.

We therefore aimed to

construct an easily applicable index for assessing deterioration of small-area housing environment (SHED)

assess its objectivity, reliability, internal consistency and validity in the context of shared accommodation

present results from the first field testing of SHED index including an assessment of convergence validity against ASR's satisfaction with the conditions of living place.

Methodology

Theoretical framework

Reviewing the literature on housing effects on health, we identified different frameworks on linkages and mechanisms between housing and health for the general population (Acevedo‐Garcia et al., 2004; Dunn, 2002b; Shaw, 2004). Using different terminology and sometimes a distinct perspective, the authors present and discuss similar pathways in their frameworks around “material” (or “immediate” or “physical”) compared to “meaningful” (or “social” or “soft”) and “spatial” (or “locational”) housing aspects, complemented by the role of housing as an “expression of socioeconomic status” which to some extent indirectly feeds into the former pathways.

Considering operationalization and quantification for epidemiological purposes, Cohen et al. developed and presented the “Broken Windows” - index (BWI) measuring neighborhood deterioration (Cohen et al., 2000). The authors use data on physical conditions and features of streets from an existing database as well as a separate evaluation by the authors, and on building code violations in public high schools reported by the Office of Sanitation Services, thereby focusing on ”material”, i.e. physical and built, aspects of the environment. Regression analyses demonstrated that the BWI was able to statistically explain more of the variance in Gonorrhoea rates than did a poverty index which was calculated based on income, unemployment and education.

The underlying “Broken Windows” - theory was originally introduced by Kelling and Wilson (Kelling & Wilson, 1982). They hypothesized that signs of public disorder in one's neighborhood lead to increases in crime. Untended conditions, e.g. a broken window, signal to the community that no one cares about the surroundings, creating an environment which permits disorderly behavior and vandalism.

This “Broken Windows” - theory has been gaining influence and popularity in the last decades, despite being controversial (Braga, Welsh, & Schnell, 2015; O'Brien, Farrell, & Welsh, 2019b). Transgressing the borders of social and political sciences, it has arrived in public health research and theory. Different hypothesized pathways between public disorder, subsumed under “Broken Windows”, and health have been examined and discussed in the last years, with example foci being high-risk behaviour (Cohen et al., 2000), mental health (Curry, Latkin, & Davey-Rothwell, 2008) and the organization of health services (Churruca, Ellis, & Braithwaite, 2018).

To better understand the health effects and associations reported in the literature (Gibson et al., 2011; Ziersch and Due, 2018), knowledge of potential causal pathways leading to physical and mental health effects is needed. Considering pathways from “material” disorder in the housing environment impacting health, O'Brien et al. outline three routes in their meta-analysis on neighborhood disorder and health (O'Brien, Farrell, & Welsh, 2019a). The first pathway runs across “norm violations” to risky behavior, the other two pathways both stem from fear as a consequence of neighborhood disorder. The second leads to social and physical retreat while the third increases psychosocial stress. The meta-analysis shows that neighborhood disorder impacted different aspects of mental health - i.e. stress, distress and depression - as well as substance abuse and overall (often self-reported) health. The authors conclude that the main pathway from neighborhood disorder to health is mediated through fear and psychosocial stressors, leading to decline in mental and eventually general health outcomes (Pettit, Kline, Gencoz, Gencoz, & Joiner, 2001).

Irrespective of the underlying pathways, the “Broken Windows” - index quantifies the “material” or “physical” aspects of housing environment on health, while inherently being linked to “meaningful”, “soft” or “social” aspects of housing. The level of deterioration in the “material” environment can be deemed as a proxy measure for neglect or a lack of sense of responsibility in the neighborhood or by the district administration. As Cohen et al. (2000) put it, “an environment that is filled with graffiti, deterioration, trash and abandoned cars and that is unsafe and subject to vandalism may be a signal that there are no rules and that no one cares.”

In the specific context of asylum seekers and refugees living in shared accommodation centers, the practical living reality of a neighborhood may be confined to the housing complex they are assigned to. Especially in larger reception centers for incoming ASR, important sites for daily necessities and activities may be centered within the grounds of the housing complex only, e.g. offices of social welfare and counselling, medical services or educational institutions. For anyone living in such housing complexes, the small-area housing environment can be understood as the immediate neighborhood of the lived reality where most (if not all) the time is spent.

As such, the “material” aspects of the neighborhood are of special importance based on the “Broken Windows” – theory as utilized in Cohen et al.’s BWI. Among those who are resettled or dispersed to smaller housing units within neighborhoods of the general population, objective information on the quality of the immediate housing environment may further help to disentangle potential small-area housing effects from the effects of broader neighborhood characteristics, such as proximity to social and medical infrastructure, or urban and rural characteristics.

SHED construction

In order to construct an easily applicable assessment index quantifying the level of deterioration in the small-area housing environment, we adapted and simplified the original BWI (Cohen, 2000). In a consensus meeting, we distilled relevant elements into an observer-based questionnaire called Small-area Housing Environment Deterioration (SHED) index (Table 1). While this process took place in the context of the research department's work in ASR's shared accommodation centers, we did not design the tool for the application in this setting only.

Table 1

Small-area Housing Environment Deterioration (SHED index) as used in the validation study.

Item	Question	Categories
conditions of windows/glass	Is there any damage to windows or other glass?	☐no visible damage☐minor cosmetic damage☐minor structural damage☐major structural damage
conditions of walls/roof	Is there any damage to walls or roof?	☐no visible damage☐minor cosmetic damage☐minor structural damage☐major structural damage
garbage accumulation	Is there any garbage accumulation inside or outside the house?	☐no garbage accumulation☐minor garbage accumulation☐serious garbage accumulation
graffiti situation	Is there any graffiti on the walls inside or outside the house?	☐no graffiti☐minor areas covered in graffiti☐almost all walls covered in graffiti
outside spaces	Is the space outside the house, including gardens, well-kept?	☐gardens and outside spaces well-kept☐some of garden and outside space overgrown and badly kept☐almost all of garden and outside space overgrown and badly kept
overall living environment	How would you rate the overall living environment?	☐very clean and well-kept☐some areas dirty and badly kept☐almost all areas dirty and badly kept

SHED validation

To test SHED's capacity on different aspects of test validity, we conducted a validation study to assess objectivity, inter- and intra-rater reliability, internal consistency as well as content and convergent validity of the SHED index.

As validation “on field” was not feasible, the study was conceived and designed as an experimental desk-study using digital accommodation portfolios, each consisting of a set of five pictures with different perspectives into indoor and outdoor spaces of accommodation centers. We prepared the digital accommodation portfolios with pictures which were taken in research projects in shared refugee accommodation previously as well as pictures from other accommodations when walking the city. An example of such a digital accommodation portfolio used in the validation study can be found as supplementary material to this article.

For the team of raters, we recruited participants with different levels of research and field experience to cover a broad spectrum of backgrounds. The participants performed two rounds of rating, with the second round performed after eight weeks, to enable the calculation of both inter- and intra-rater reliability measures. For each participant, the order of portfolios was randomized and stayed the same in both rounds.

SHED cognitive testing

An additional portfolio was used as a basis for a cognitive interview with the same participants to enhance our comprehension of this new SHED index regarding its strengths and limitations, as well as practical application.

We conducted the cognitive interviews with each participant individually after the second round of rating, asking them to assess the additional portfolio using the SHED index while thinking aloud. The thought and feedback process was facilitated using different probing techniques, i.e. category selection probing, emergent probing, comprehension probing and general/elaborative probing based on survey guidelines for cognitive pretesting by German Leibniz Institute for the Social Sciences (Lenzner, Neuert, & Otto, 2016). A list of all probing questions can be found as supplementary material to this article. All cognitive interviews were conducted in German, recorded and later used to synthesize issues which emerged regarding potential limitations and future improvements.

SHED field testing

As the SHED index was designed for use when visiting accommodation centers in-person, we conducted a field test in ASR's shared accommodation centers as part of the BMBF-funded RESPOND project [https://respond-study.org: last accessed on December 23, 2020] to test its applicability and convergent validity. The RESPOND study was designed as a population-based, cross-sectional survey to assess both healthcare needs as well as access to care of ASR using established instruments (Biddle et al., 2019). The questionnaire used included the EUROHIS-QOL-8 (Schmidt, Mühlan, & Power, 2006), which covers residents' satisfaction with the conditions of living place and was used to test convergent validity of the new SHED index. All data collection was carried out by trained, multi-lingual field teams.

The same team applied the SHED index during the general data collection on a total of 58 shared accommodation centers which were sampled across all 44 districts of the German federal state of Baden-Wuerttemberg using a complex random sample design on the level of a total of 1938 shared accommodation centers. These centers host ASR who have been quasi-randomly assigned based on administrative quota from state reception centers (Biddle et al., 2019). For the time their asylum claims are being processed or until 15 months have passed, ASR are required to reside in these before they are allowed to search for private accommodation. As these accommodation centers are the responsibility of the regional authorities, they vary widely in size, number and location. This includes large centers with more than 150 inhabitants in various buildings, or small houses in residential streets.

Statistical analyses

Inter- and intra-rater reliability

To assess inter- and intra-rater reliability, we calculated different measures of reliability in addition to the percent agreement.

In general, Cohen's Kappa is calculated to measure agreement between two raters beyond a sole chance-agreement (J. Cohen, 1960; McHugh, 2012). To calculate the chance-corrected agreement of more than two raters, we calculated a Kappa according to Fleiss' (Fleiss, 1971).

As Kappa is affected by presence of bias between observers and prevalence of data, i.e. distribution of data across categories, we report prevalence-adjusted and bias-adjusted Kappa (PABAK) (Byrt, Bishop, & Carlin, 1993; Chen, Faris, Hemmelgarn, Walker, & Quan, 2009). We calculated PABAK based on its equivalent coefficient of Brennan and Prediger (Brennan & Prediger, 1981) as suggested by Gwet (Gwet, 2012).

To provide sufficient statistical power to obtain significant results at a p-value of 0.05, we calculated the needed sample size in advance and recruited accordingly (Gwet, 2012). For inter- and intra-rater reliability calculations, we needed both the optimal number of subjects as well as raters, balanced for feasibility. Based on recommendations by Gwet and expected levels of medium to high reliability, the optimal study size was set to 15 subjects rated by 7 raters.

Categories for interpretation of Kappa coefficients can be drawn arbitrarily. Still, to maintain consistent nomenclature and enable cross-study comparisons, we based our interpretation on the widely used levels introduced by Landis and Koch (Landis & Koch, 1977), i.e. 0.00–0.20 equals to light, 0.21–0.40 to fair, 0.41–0.60 to moderate, 0.61–0.80 to substantial and 0.81–1 to an almost perfect agreement.

Internal consistency

To test individual index items regarding their internal consistency, we calculated Cronbach's α for each SHED item, as well as the correlation between SHED items 1–5 against SHED item 6 which is conceived as a global rating.

Convergence validity

As part of its field testing, we tested the SHED index's convergent validity with the EUROHIS-QOL-8 item on inhabitants' satisfaction with the condition of their living place (Da Rocha, Power, Bushnell, & Fleck, 2012). We cross-tabulated data on SHED assessment quintiles against the dichotomized dissatisfaction of inhabitants. By means of a Pearson's test, we tested the alternative hypothesis of dependence between SHED index summary score and EUROHIS-QOL-8 against the null hypothesis of independence at a significance level of 5%.

All statistical analyses were performed using Stata SE V15.

Results

We simplified and adapted the BWI (Cohen et al., 2000) and its application to our research objective in achieving an easily applicable assessment index for housing quality in terms of deteriorationfor, resulting in several differences between the tools.

First, to ensure applicability and low-effort implementation, we focused the tool on in-person small-area primary assessments by observers without the need to retrieve further data from other sources.

Second, compared to the BWI which focuses on block groups consisting of several city blocks and covering in average 0.04 square miles or 0.10 square kilometers, our unit of focus is the small-area housing environment, i.e. the immediate surrounding of the accommodation. An assessment of all street segments in the neighborhood of the accommodation would not be appropriate to our research objective.

Third, we excluded questions on abandoned vehicles and public high schools as they were inadequate to living conditions inside the accommodation. Instead, we added a question on outside spaces and gardens.

Fourth, we split the question on physical condition of neighborhood structures into two separate and more specific questions on condition of windows/glass and walls/roof to increase capacity to differentiate.

Fifth, we expanded the binary questions to assess physical features, e.g. garbage or graffiti, to a 3-point scale to allow more differentiated answer options.

Sixth, we simplified all questions into one questionnaire with similarly constructed multiple-choice answers, facilitating its application and the summary score calculation.

Seventh, we added a global rating on overall living environment. This offers an opportunity to separately capture all further observations and perceptions by the observers which the existing items do not cover yet.

In conclusion, the new SHED index covers five different dimensions of physical environment and their degree of deterioration, i.e. (1) conditions of windows/glass and (2) walls/roof, (3) garbage accumulation inside/outside the house, (4) graffiti inside/outside the house and (5) outside spaces, complemented by a sixth item offering a global rating of the overall living environment (Table 1). A SHED questionnaire version 1.0 for including refinements from cognitive assessments and field-testing can be found as supplementary material number 5.

A composite SHED index summary score on a scale from one to six (least to highest degree of deterioration) can be calculated after individual item's z-transformation (to standardize item scores across different scales) and normalization (to scale all item scores in the range of 0–1).

The validation desk-study took place at the Department of General Practice and Health Services Research of Heidelberg University Hospital (Germany) from November 2018 to January 2019.

In total, seven participants were recruited purposively from department staff, covering different demographic characteristics and levels of experience of working in health services and research in shared accommodation centers for ASR. The participants were predominantly female (5 of 7), 29 years of age on average (spanning 25–42 years with one missing value) and had a median research experience in the context of migration health of 24 months (spanning 0–48 months). All but one have worked occasionally (3 participants) or regularly (3 participants) in shared accommodation centers for ASR.

The validation study delivered substantial to almost perfect inter-rater reliability measures according to Landis and Koch (Landis & Koch, 1977) on almost all items including the overall SHED index (Table 2; Fig. 1 as supplementary material).

Table 2

Inter-rater reliability (reliability between raters).

Small-area housing environment deterioration	Percent agreement	Fleiss' Kappa	PABAK	Correlation of items 1–5 with item 6	Internal consistency, as measured with Cronbach's alpha
Item 1: conditions of windows/glass	0.95	0.72*** (0.44–1.00)	0.83*** (0.70–0.97)		0,88
Item 2: conditions of walls/roof	0.89	0.61*** (0.37–0.84)	0.66*** (0.53–0.78)		0,89
Item 3: garbage accumulation	0.90	0.67*** (0.46–0.89)	0.74*** (0.60–0.88)		0,85
Item 4: graffiti situation	0.97	0.87*** (0.76–0.98)	0.93*** (0.84–1.00)		0,84
Item 5: outside spaces	0.88	0.44*** (0.20–0.69)	0.69*** (0.53–0.84)		0,85
Item 6: overall living environment	0.90	0.69*** (0.48–090)	0.73*** (0.60–0.85)	0,85	0,82
SHED index	0,98	0,86*** (0.78–0.94)	0,87*** (0.82–0.91)		0,80

Confidence intervals clipped at the upper limit.

*** p-value <0.005 **p-value <0.05

The only outlier is item 5 on the deterioration of outside spaces when calculated by Fleiss’ Kappa, not considering imbalanced data patterns. When using the prevalence- and bias-adjusted Kappa, the measure crosses the border of substantial reliability at 0.6.

Intra-rater reliability measures delivered substantial to almost perfect measures for almost all items including the overall SHED index, with the exception of item 5 on outside spaces (Table 3; Fig. 2 as supplementary material).

Table 3

Intra-rater reliability (reliability between initial and repeat ratings at different points in time among each rater).

(ordinally weighted analysis)				Number of raters = 8
Small-area housing environment deterioration	Rater 1 (95% CI)	Rater 2 (95% CI)	Rater 3 (95% CI)	Rater 4 (95% CI)	Rater 5 (95% CI)	Rater 6 (95% CI)	Rater 7 (95% CI)
Item 1: conditions of windows/glass	0,87*** (0,55-1)	0,90*** (0,73-1)	0,76*** (0,27-1)	0,89*** (0,67-1)	0,83*** (0,56-1)	0,94*** (0,77-1)	0,94*** (0,80-1)
Item 2: conditions of walls/roof	0,74*** (0,57-0,91)	0,80*** (0,60-0,99)	0,77*** (0,46-1)	0,59** (0,08-1)	0,78*** (0,49-1)	0,73*** (0,38-1)	0,83*** (0,65-1)
Item 3: garbage accumulation	0,66*** (0,39-0,92)	0,85*** (0,60-1)	0,68*** (0,40-0,95)	0,79*** (0,53-1)	0,69*** (0,28-1)	0,50** (0,10-0,90)	0,86*** (0,65-1)
Item 4: graffiti situation	0,90*** (0,71-1)	1 (1-1)	0,72*** (0,42-1)	1*** (0,85-1)	1*** (0,78-1)	0,85*** (0,65-1)	1 (1-1)
Item 5: outside spaces	1 (1-1)	0,88*** (0,42-1)	0,14 (−0,27-0,56)	0,20 (−0,19-0,60)	0,44** (0,05-0,84)	0,52*** (0,21-0,83)	0,42** (0,09-0,76)
Item 6: overall living environment	0,91*** (0,75-1)	0,93*** (0,79-1)	0,86*** (0,64-1)	0,71*** (0,32-1)	0,67*** (0,24-1)	0,70*** (0,43-0,97)	0,78*** (0,50-1)
SHED index	0,93*** (0,87–0.91)	0,90*** (0,82–0.99)	0,88*** (0,77–0.99)	0,74*** (0,44-1)	0,92*** (0,54-1)	0,71*** (0,52-0,91)	0,74*** (0,88-1)
Confidence intervals clipped at the upper limit.				*** p-value ≤0.005 ** p-value ≤0.05

Both the individual items as well as the complete SHED index achieved high values of above 0.80 on Cronbach's α. Complemented by high correlation of 0.85 between the individual SHED items 1–5 against the global rating item 6, the index ensures good internal consistency on assessing the construct of deterioration across the individual index items (Table 2; Fig. 1 as supplementary material).

The individual, cognitive pretests took place at the Department of General Practice and Health Services Research of Heidelberg University Hospital (Germany) in January 2019 after the second round of rating.

The interviews helped identifying current limitations and issues to be addressed when preparing for future application of the SHED index. Participants responded positively to the simplicity of the questions and the shortness of the questionnaire but felt partially unsure about individual words and their definitions. We have summarized main issues which emerged during cognitive interviews in Table 4, including solution-oriented comments for future applications.

Table 4

Cognitive interviews with think-aloud.

Topic cluster	Issues emerged	Solution-oriented comments
Definitions	How do you define “cosmetic” damage? (Damage which can be wiped off?Damage without loss of function? …)	We define cosmetic damage as damage which is cosmetic in nature only and does not lead to a loss in structure or function.A loss in structure (e.g. a hole in the wall) is counted as a “minor structural” damage and a loss in function (e.g. statics of bear-loading walls or window insulation) as a “major structural” damage.
	How do you define garbage “accumulation”? (Do well sorted garbage piles next to trash bins count as “garbage accumulation”? …)	“Garbage accumulation” covers both “unsorted” as well as “sorted” garbage while the first is a proxy of disorder and the latter a symbol of a lack in maintenance by the facility.
	How do you define “graffiti”? (Are messages of hope counted as graffiti?Is graffiti always negative and a symbol of deterioration? …)	Depending on the context and the viewer's standpoint, graffiti can be both seen as an artform or as an act of vandalism. It is important to facilitate discussion and consolidation on how the term will be used by the assessment team.
	How do you define “spaces outside the house”? (What about directly attached spaces and gardens?What about alleys between buildings within a large housing complex?What about public streets next to the house? …)	We define “spaces outside the house” as all the space outside of the individual housing building but within the accommodation facility, thus excluding public streets.
Differentiation	In case of floor or stairway damage, do you count this as part of “walls and roof”?	Yes, please consider damages to the floor as part of the item on “walls and roof”. In SHED version 1.0, this will be added explicitly.
	In case of garbage accumulation (outside the house), do you count this both as part of the item on garbage accumulation as well as spaces outside the house?	“Garbage accumulation” focuses solely on garbage accumulation while “spaces outside the house” covers general aspects of grounds maintenance, e.g. care of green spaces, leaves, etc.
Missing categories	What about general cleanliness within the house?E.g. dirt in the hall- and stairways?	The SHED index is conceived as a simple and easily applicable tool for on the field assessment of deterioration of housing environment. Other characteristics, e.g. general uncleanliness, is to be considered when answering the global rating “overall living environment”.
	What about bad smells? Both in the house or on the grounds?	The SHED index is conceived as a simple and easily applicable tool for on the field assessment of deterioration of housing environment. Other perceptions, e.g. unpleasant smell, can be considered when answering the global rating “overall living environment”.
Miscellaneous	The rating level “almost all” (in items on graffiti and spaces outside the house) is very hard to fulfil(Is any place “almost all covered in graffiti”/“overgrown and badly kept”/“dirty and badly kept”?There is a large jump from “minor”/“some areas” to “almost all”. …)	The SHED index is conceived as a simple and easily applicable tool for on the field assessment of deterioration of housing environment. Thus, the simplicity in the answer option is on purpose but also requires courage to rate a facility with the highest rating if needed.
	How does my previous experience with housing affect my SHED rating?How can I ensure objective rating?	Firstly, it is important to acknowledge that every observer-based rating is subjective in its core.Secondly, we recommend ratings to be done in teams in order to facilitate the discussion and increase the independence of individual personal history and experience.
	Is “overall living environment” an item of its own or rather a composite of the five items before?	Item 6 on “overall living environment” is conceived as a global rating thus covering both the five previous items as well as all other remarkable aspects based on the rater's judgement.

The SHED index was piloted as part of the above outlined quantitative work package of the RESPOND study (Biddle et al., 2019), rating 58 shared accommodation centers for ASR in the German federal state of Baden Württemberg (Table 5).

Table 5

Convergent validity.

Small-area housing environment deterioration (SHED quintiles)	Number of residents dissatisfied with conditions of living places
	n	%
very low deterioration (Q1)	49	25,13
low (Q2)	18	9,23
average (Q3)	39	20
high (Q4)	57	29,23
very high deterioration (Q5)	32	16,41
N	195	100
Pearson'sχ^2(16): 12,48, p-value: 0,71

Cross-tabulating data on SHED assessment result quintiles against the dichotomized dissatisfaction of inhabitants with the condition of their living place based on EUROHIS-QOL-8, 45.64% of all ASR who were dissatisfied were living in shared accommodation centers with high to very high levels of deterioration. Yet, Pearson's test did not result in a statistically significant association, thus not supporting the rejection of the null hypothesis that dissatisfaction of inhabitants based on EUROHIS-QOL-8 and SHED index summary score are independent (Table 5).

Discussion

Principal findings

We demonstrate that the newly constructed SHED index achieves high measures on inter- and intra-rater reliability as well as internal consistency in assessing the construct of deterioration across SHED index's items. Further, we ensured content validity by (1) building a theoretical framework around health effects of deterioration of housing environment, (2) using the BWI as an existing and comprehensive instrument for quantification and (3) conducting a field test in shared accommodation centers for ASR as part of the health study RESPOND. This demonstrated the SHED index's applicability and enabled us to discuss its convergent validity with the self-rated EUROHIS-QOL-8 item on inhabitants' satisfaction with their living conditions.

We based our understanding of test validity on the common definition by AERA & APA & NCME (AERA, APA, & NCME, 2014), i.e. “the degree to which evidence and theory support the interpretation of test scores”. This entails the discussion of objectivity, reliability and score validity as core criteria for any valid measurement.

Measure of objectivity

We understand objectivity as the independence of an object's rating from who is conducting the assessment. While every assessment requires a level of subjectivity, the SHED items focus on externally visible aspects of housing environment. Through standardization of the assessment procedure and realization as a multiple-choice questionnaire, SHED achieves high inter-rater reliability as a measure of objectivity.

Measure of reliability

We conceive reliability as repeated assessments delivering same or similar results in case of no change within the assessed object. Through clear and unambiguous formulation of index items, SHED achieves high inter- and intra-rater reliability as measures of reliability, and high values of Cronbach's α ensure good internal consistency in assessing the construct of deterioration across SHED index's items.

Measure of validity

Validity, i.e. an assessment index assessing what it is conceived to be assessing, is a quality criterion of its own and the final step of ensuring a complete test validity. When constructing an assessment index to assess and quantify deterioration of small-area housing environment, validity can only be approximated but not tested in a clear-cut manner as there is no gold standard or reference yet. Thus, we based our index construction on content validity and convergent construct validity, as outlined above.

When testing the new SHED index's convergent validity with the self-rated EUROHIS-QOL-8 item on inhabitants' satisfaction with their living conditions, about half of all ASR who were dissatisfied with the condition of their living place were living in shared accommodation centers with high to very high levels of deterioration according to SHED. However, Pearson's test did not result in a statistically significant association.

There are several potential explanations for this: First, we might ask for the existence of a latent reference point for ASR when rating their current shared accommodation. SHED is an observer-based measure of objective housing environment deterioration with ASR's satisfaction with the condition of their living place based on their past history and former living circumstances.

Second, we have to consider the difference in the scope of the SHED index against the global EUROHIS-QOL-8 item on inhabitants' satisfaction with their living conditions. SHED is specifically constructed to measure the deterioration of the housing environment. It focuses on cosmetic and structural damage, as well as questions of grooming and maintenance. However, inhabitants' living conditions are naturally shaped by many other contextual factors which may influence respondents’ answer to the rather global EUROHIS-QOL-8 item, e.g. crowding, access to sanitary facilities or questions of privacy and personal space.

Future research

SHED is a measure of deterioration of small-area housing environment. The index covers housing aspects deteriorated due to negligence and lack of maintenance.

However, to fully understand assigned shared accommodation as a determinant of health for ASR, it is important to look at the multitude of contextual effects of housing environment on inhabitants’ health. Deterioration of small-area housing environment is one direct factor, but other important aspects include accommodation size, distance to medical services and availability of amenities as critical infrastructure for daily life. Further, the interaction of these factors with broader contextual factors such as district deprivation or urbanity should be considered. In the overall pursuit for a systematic understanding and quantification of housing impact on health, SHED can be combined with other measures of housing quality and serve as a building block of an all-round housing index in the future.

To this aim, future research should systematically review and discuss existing theoretical frameworks and develop a comprehensive measurement framework tailored to the particular housing conditions of ASR. These will be a valuable contribution to epidemiological studies to enhance our understanding of the circumstances and conditions shaping the health status of ASR in host countries.

Strengths and limitations

SHED offers an easily and quickly applicable assessment index to quantify the level of deterioration in the small-area housing environment during a physical visit. At the same time, being an observer-based tool, preparatory meetings are critical to ensure high validity of SHED's results. It is important for the full data collection team to review all SHED items and discuss and agree on concepts and definitions used, e.g. on what counts as graffiti.

A SHED questionnaire version 1.0, including refinements from cognitive assessments, provides sufficient validity and reliability to be used in the field and can be found as supplementary material 5. Further refinements to capture important camp characteristics may follow and lead to an updated version in the future.

Use of SHED is not restricted to ASR accommodation only and can be regarded as generic to any other housing facility. We have made use of the opportunity to collect data as part of a research project in shared accommodation centers for ASR to field-test the questionnaire, but believe it is applicable to any other housing type, especially as ASR accommodation (at least in Germany) is not “uniform” but is characterized by substantial heterogeneity, ranging from small flats with single/family rooms that are not different to accommodation for the general population, to larger centers with shared flats and more institutionalized characteristics. While being somewhat generic, the limitation is that SHED does not capture necessarily all aspects that characterizes especially those institutionalized, camp-like centers, e.g. with respect to fencing, privacy or autonomy. These aspects, however, can be captured by combining SHED with other tools or by expanding SHED in the future to accommodate for these limitations.

Separately, the validation study reported in this article covers all three important concepts of test quality, i.e. objectivity, reliability and validity, including the SHED index's internal consistency. Based on quantitative and qualitative methodology, the index has been thoroughly tested and all identified potential limitations reported.

In terms of the desk study design, three main limitations must be considered:

Firstly, digital accommodation portfolios offer a good insight into accommodation conditions but cannot replace the actual visit of shared accommodation centers. Factors such as smells in the shared accommodation centers, the “felt” distance to society, the individual atmosphere of life in the shared accommodation centers, but also the possibility to look around the facility in person may lead to different ratings in “in person” studies. While the convergent validity testing took place in the field, the reliability testing was conducted only based on the photographs. As such, our reliability measures risk being lower - not higher - than the actual values which only enforces the high reliability of the SHED.

Secondly, raters discussed confusion about which answer box to tick when they felt that they did not have enough information on the picture to answer the item. Again, this poses a risk to a lower - not higher - reliability measured in our testing based on digital accommodation portfolios compared to SHED application in reality. This could also be the reason for the tendency of index item 5 on outside spaces to be an outlier in the inter- and intra-rater reliability calculations.

Thirdly, it is important to note the lack of ASR involvement in the construction and discussion of the SHED index. This is a limitation which needs acknowledgement. However, as SHED aims to quantify the level of deterioration and not the inhabitants’ subjective evaluation of the living environment, this limitation can be tolerated.

Finally, it is important to note the relative similarities in age, sex and academic background in the participants of the validation study. This might have contributed to the high score on inter-rater reliability.

Conclusion

Discussing potential health effects of public disorder and deterioration in light of the “Broken Windows” - theory, we have constructed, validated and field-tested a new and easily applicable observer-based assessment index to measure the degree of Small-area Housing Environment Deterioration (SHED). Cognitive interviews provided valuable information on the index's applicability and issues which need to be addressed in future preparational training for data collection on housing environment.

This new validated SHED index enables policy-makers and officials to monitor deterioration in shared accommodation centers for ASR and supports evidence-informed decision making in housing policies. For researchers, this instrument provides an opportunity to conduct focused epidemiological analyses on housing as a contextual and social determinant of health for ASR living in such shared accommodation centers.

Declarations of interest

None.