Does sector matter for the quality of care services? A secondary analysis of social care services regulated by the Care Inspectorate in Scotland.

OBJECTIVES: Social services are increasingly commissioned to third and for-profit sector providers, but little is known about whether and how these changes influence quality indicators. We assessed quality-related outcomes across for-profit, public and third sector organisations delivering social care services.
DESIGN: A secondary analysis was conducted on publically available data collected by the independent regulator of social care organisations in Scotland. All outcomes are reported as predicted probabilities derived from multivariate logistic regression coefficients. Generalised ordered logit models are utilised for the quality domains and the risk assessment score and logistic regression for whether complaints or requirements were issued to organisations.
SETTING: Organisations inspected by the Care Inspectorate in Scotland. POPULATION: 13 310 social care organisations (eg, nursing homes and day care organisations). PRIMARY OUTCOMES: The quality and risk domains collected by the Care Inspectorate and complaints and requirements issued to organisations within the last 3 years.
RESULTS: Controlling for multiple factors, we find that public and third sector providers performed consistently and statistically significantly better than for-profit organisations on most outcomes. For example, for-profit services were the most likely to be rated as high and medium risk (6.9% and 13.2%, respectively), and the least likely to be classified as low risk (79.9%). Public providers had the highest probability of being categorised as low risk (91.1%), and the lowest probability of having their services classified as medium (6.9%) and high risk (2%), followed by third sector providers (86%, 8.5% and 4.5%, respectively). Public providers performed better than third sector providers in some outcomes, but differences were relatively low and inconsistent.
CONCLUSION: Public and third sector providers were rated considerably higher than their for-profit counterparts on most observed outcomes. Regulators might use this information to consider how social care providers across sector are incentivised to manage their resources. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The publically available data set provided by the Care Inspectorate allows for testing variation in quality-related outcomes across public, for-profit and third sector care providers in Scotland.

The analysis aimed to replicate past research in a new context controlling for more factors (eg, number of staff) and on a broader range of outcomes (eg, user-reported complaints).

There were few observations for organisations with very low quality levels, making comparison across sector difficult in these categories.

The findings were consistent across the quality domains collected by the Care Inspectorate and as to whether requirements and complaints had been issued to organisations.

Introduction

Health and social care services are often delivered in quasi-markets, and an increasing proportion of these services are implemented by for-profit organisations (FPOs) and third sector organisations (TSOs).1–4 Quasi-markets are characterised by free bidding from organisations across different sectors5 and can be defined as ‘…markets where the provision of a service is undertaken by competitive providers as in pure markets, but where the purchasers of the service are financed from resources provided by the state instead of from their own private resources’.6 Thus, quasi-markets primarily include services that are free or subsidised to service users (eg, healthcare or nursing homes), but where providers from all sectors (ie, public, for-profit and third) compete for service contracts.

It has been argued that quasi-markets can lead to more efficient and equitable service deliveries,5 7 but there is little conclusive evidence to support that proposition. In theory, quasi-markets may produce favourable outcomes, in that the bidding system can be thought to force organisations to respond to the needs of service users and allow the best and most innovative providers to attain the contract. However, the utility of this type of market strongly relies on the commissioning structures in place to allocate funding, which is known to have been flawed in the past.8–10 Further, it has long been understood that mechanisms of action considering behaviour and the design of social policy requires more than simply an analysis of monetary incentives.11

It has long been assumed that providers from different sectors are subject to different incentives and organisational characteristics.12–16 For example, FPOs are often perceived to be legitimised by the pursuit of profit maximisation, suggesting that they may engage in opportunistic behaviour at the potential expense of the service user without regulation. Conversely, TSOs and public sector organisations (PSOs) are often considered to be legitimised by their ability to achieve social value, suggesting that their incentive to deliver services are predominantly motivated by pro-social pursuits, thus, making them more ‘trustworthy’ providers of public services.17 These theoretical differences across sector are summarised in table 1.

Table 1

Theoretical distinction of sectors

	For-profit sector	Public sector	Third sector
Definition	Organisations that are privately owned and administered according to the pursuit of profits.	Organisations that are publically owned and administered.	Organisations that are formally structured, privately owned, non-profit distributing, self-governing and benefiting from voluntary activities.42
Primary activities	Conventionally private goods, but increasingly also public goods	Public goods	Public goods
Main type of income	Profits from sales of products and/or services	Taxation	Charitable contributions and/or commissioning
Purpose/motivation	Maximise and sustain profits	Achieve social mission (public sector motivation/social value)	Achieve social mission and accountability (social value)

Importantly, there is no general consensus on what these theoretical differences between sectors mean in practice and whether, and if so, how, they may relate to quality of service.

To explore sector differences in the context of health services, a relatively recent overview of systematic reviews investigated how ownership influences the performance of healthcare providers.18 The overview included 15 systematic reviews that were conducted in various contexts (eg, low-income and middle-income countries and high-income countries) and with different types of providers (eg, hospitals and nursing homes). The paper found mixed results with non-profit providers performing better than for-profit providers on certain indicators (eg, mortality and price), but not on others (eg, healthcare-related outcomes). Further, the results should be interpreted with the caveat that most of the included systematic reviews had significant methodological flaws and only one of the reviews was considered ‘reliable’. As a result, the overview concludes that more high quality primary research is needed, especially in exploring variation across PSOs, FPOs and TSOs, rather than just between for-profit and non-profit providers.18

However, little research has been conducted on sector difference in social care settings, even though more social services are undertaken by for-profit and third sector providers,3 19 20 and that sector variation in service quality thus have important implications for practice. To explore the merit of existing theories and to inform future policy on the contracting of social services, it is central to test whether the typology of sector accounts for variation across factors that matter to practice, such as quality and performance.

Objectives

It remains relatively inconclusive whether typology of sector is based on research conventions rather than empirical evidence.21 The focus of this paper will be to empirically test whether and if so how ‘sector’ accounts for variation in quality of social care organisations using data from the Care Inspectorate in Scotland. This focus can be summarised by the following research question:

Does the quality of social care services vary among for-profit, public and third sector organisations as assessed by the quality and risk domains of the Care Inspectorate in Scotland?

Data: the Care Inspectorate

The Care Inspectorate serves as the regulator of care services in Scotland and currently (January 2018) includes data on 13 310 organisations, which is available for download on the official website and updated monthly.22 Regulated organisations work within a range of different services, with the biggest categories being childminding, children day care services, support services, care homes for the elderly and housing support (see tables A1–A2 in online supplementary appendix A for description of included services).

To fulfil its regulating role, the Care Inspectorate regularly inspects care facilities, which entails assessing the quality and risk of services. The grading of quality is done according to the National Care Standards,23 which considers the following principles: ‘dignity and respect’; ‘compassion’; ‘be inclusive’, ‘responsive care and support’ and ‘well-being’. The ratings are graded from 1 to 6, in which 1 is unsatisfactory and 6 is excellent. Further, the Care Inspectorate distributes a risk score to care providers assessing whether a service can be said to be of ‘low’, ‘medium’ or ‘high’ risk to service users, which subsequently influences how regularly an organisation should be inspected and the quality domains to which the service should be subjected.24 The four quality domains and the risk assessment are described in table 2.

Table 2

Quality domains employed by the Care Inspectorate.

Quality and risk domains	Elaboration
Quality of care and support	How well does the service meet the needs of the service users? This includes looking at the outcomes and materials of the provided services and interviewing service users.
Quality of management and leadership	How is the service managed and does the strategy for development consider the needs of the service users? This includes interviewing staff and service users.
Quality of environment	What is the quality of the setting in which a service is delivered? This includes investigating if the site is clean and properly maintained.
Quality of staffing	What is the quality of staff in terms of, for example, qualifications and training? This includes interviewing staff and service users.
Risk of services	A joint assessment of the nature of the services (some services are inherently more ‘risky’ than others), if complaints have been issued towards a service and whether any requirements were posed after the last inspection.

The inspections are done by trained staff following a standardised procedure, which includes visiting the facility in question unannounced, conducting interviews with staff and service users, as well as physically examining the facilities.25 26 Inspected organisations must also fill out a self-assessment form, which is reviewed in combination with the organisation’s history of posed requirements and complaints as part of the grading process. Special guidelines apply to certain care services, such as organisations that work with children and young people.27

Methods

This paper is reported following best practice guidelines.28 29 All analyses were conducted in Stata V.14.

The quality and risk domains collected by the Care Inspectorate are ordinal in nature and thus lend themselves to ordered logistic regression. However, this technique rests on the proportional odds or parallel lines assumption, which requires the effect of the odds to be constant across all logits.30 Substantively, this would mean that the effect of sector on quality would be constant across all levels of quality.

We tested this assumption across sector on the quality domains and risk assessment using the ‘brant’ command,31 which demonstrated the parallel lines assumption to be violated in three quality domains (except for quality of environment) and the risk assessment, thus suggesting that the effect of sector does vary across different levels of quality and risk. To use the ordering of the data, while also allowing for the proportional odds assumption to be relaxed, we employ generalised ordered logistic regression, which is less restrictive than ordered logistic regression and more suitable for non-linear associations.30 Specifically, we will use the gologit2 command written and regularly updated by Richard Williams.32

Logistic regression is known to entail several limitations, especially when it comes to accurately presenting and interpreting ORs.33 To demonstrate the effect of sector, we report results as predicted probabilities, which are often argued to be a more robust and intuitively interpretable effect size compared with ORs.34 35 Therefore, we will not report on the actual coefficients (although these can be found in online supplementary appendix B), but instead use the gologit2 output to compute non-linear probability models.30

While marginal effects and predicted probabilities are not immune to unobserved heterogeneity,36 they are considered less sensitive to changes in the model specification than ORs.34 All predicted probabilities are derived following marginal standardisation, that is, as the average effect of sector on quality, as opposed to the effect of sector on quality on average (ie, prediction at the means).37 We use the ‘mtable’ command to derive predicted probabilities, and the ‘mchange’ command to test if the changes in probability are statistically significant.38 All differences in probabilities are reported in percentage points rather than as relative differences.

Explanatory variables

The key explanatory variables are sector, which is coded as dummies: for-profit (‘private’), public (‘local authorities’) and third sector (‘voluntary or not for profit’), with FPOs as the benchmark category. We rely on the classification of service type provided by the Care Inspectorate and the analysis thus rests on the assumption that similarly classified service types do not differ systematically across sectors. However, to account for the possibility that the effect of sector varies according to different services, all service types are included as dummy variables. We exclude service types with very limited sector variation (ie, adoption services, adult placement services, childminding, fostering services, nurse agencies and offender accommodation services). We also ran the models on the largest subcategories of services to explore generalisability across included service types (ie, care home services, children day care services, housing support services and support services). To ensure that observed effects were not driven by experience of service providers, we control for the number of years of registration with the CI. Further, we include dummy variables for whether an organisation has improved, declined or stagnated in its lowest ranked category since the last inspection. Finally, we control for the client group of care organisations (eg, children, older people, dementia patients), whether an organisation is currently active, and the number of employed staff.

Dependent variables

The dependent variables of interest are the quality and risk domains described in table 2. To test if sector predicts variation in regulation, we also investigate if organisations from different sectors vary in their probability to having been issued a requirement as a result of an inspection. Further, to ensure that the effect of sector on quality aligns to a user-reported outcome, we explore the probability of organisations having been subject to at least one complaint within the last 3 years.

Due to a limited number of observations in each group, we merged category 1 and 2 (‘poor quality’), category 3 and 4 (‘adequate quality’) and category 5 and 6 (‘high quality’) across all quality domains. However, all reported analyses were also run on the original quality scores, which demonstrate similar results. Number of requirements and complaints were treated as binary variables (having received a requirement/complaint or not within last 3 years).

Patient and public involvement

Patients and public were not involved.

Results

Descriptive statistics

The data set includes 8111 (60.94%) FPOs, 2602 (19.55%) TSOs and 2597 (19.51%) PSOs. Table 3 shows that FPOs have a higher number of employed staff on average, compared with PSOs and TSOs, but that the spread of observations for staff in FPOs is also the highest. TSOs have the highest proportion of organisations (4.66%) which improved their minimum quality rating within the last month, although the difference between sectors is modest. PSOs have, on average, been registered the longest, whereas FPOs tend to have been registered for a shorter period than PSOs and TSOs.

Table 3

Descriptive characteristics

	For-profit sector	Public sector	Third sector
Number of organisations	8111 (60.94%)	2597 (19.51%)	2602 (19.55%)
Active organisations
Active	7891 (97.29%)	2572 (99.04%)	2566 (98.62%)
Inactive	220 (2.71%)	25 (0.96%)	36 (1.38%)
Change in minimum quality rating in the last month
Same	6576 (93.44%)	2363 (93.14%)	2246 (91.08%)
Decline	215 (3.05%)	96 (3.78%)	105 (4.26%)
Improve	247 (3.51%)	78 (3.07%)	115 (4.66%)
Average number of staff	14.56 (31.87)	10.62 (27.27)	12.76 (24.68)
Average years of registration	8.58 (5.42)	13.92 (3.77)	11.53 (4.90)

Percentages in parenthesis for number of organisations, active organisations and changes in minimum quality ratings. SD in parenthesis for the average number of staff and years of registration.

Table 4 displays the cross tabulations of whether organisations have been subject to requirements after an inspection and if any complaints have been posed towards an organisation within the last 3 years. We see that PSOs received the fewest (proportionally) requirements (13.28%) and that FPOs and TSOs receive similar numbers of requirements (18.31% vs 19.43%). FPOs received a higher proportion of complaints compared with PSOs and TSOs, although the difference between the for-profit and third sector is relatively small (10.84% vs 8.65%).

Table 4

Cross tabulation of sector over quality domains, risk score and requirements/complaints posed after inspection within last 3 years

	For-profit sector	Public sector	Third sector
Quality of care (n=12 322)
Poor quality	57 (0.78)	10 (0.39)	20 (0.80)
Adequate quality	2641 (36.31)	732 (28.73)	718 (28.71)
Good quality	4575 (62.90)	1806 (70.88)	1763 (70.49)
Quality of staff (n=7933)
Poor quality	59 (2.04)	8 (0.31)	25 (1.00)
Adequate quality	1475 (51.09)	783 (30.74)	875 (35.01)
Good quality	1353 (46.87)	1756 (68.94)	1599 (63.99)
Quality of environment (n=10 293)
Poor quality	32 (0.48)	7 (0.32)	11 (0.81)
Adequate quality	2515 (37.58)	759 (35.60)	553 (40.84)
Good quality	4145 (61.94)	1408 (64.77)	790 (58.35)
Quality of management (n=12 386)
Poor quality	107 (1.47)	15 (0.59)	38 (1.52)
Adequate quality	3642 (50.12)	1041 (40.87)	1033 (41.83)
Good quality	3518 (48.41)	1491 (58.54)	1441 (56.65)
Risk of services (n=13 268)
Low risk	7263 (89.96)	2420 (93.22)	2259 (86.95)
Medium risk	504 (6.24)	140 (5.39)	231 (8.89)
High risk	307 (3.8)	36 (1.39)	108 (4.16)
Complaints received within last 3 years
No	7233 (89.16)	2453 (94.46)	2377 (91.35)
Yes	879 (10.84)	144 (5.54)	225 (8.65)
Total	8112	2597	2602
Requirements posed after inspection within last 3 years
No	4944 (81.69)	2025 (86.72)	1737 (80.57)
Yes	1108 (18.31)	310 (13.28)	419 (19.43)
Total	6052	2335	2156

Percentages in parentheses.

Table 4 further shows that there is a roughly equal proportion of TSOs and FPOs with a ‘poor’ quality rating and that PSOs tend to have the lowest proportion of organisations in this category. However, more FPOs score ‘adequately’ relative to PSOs and TSOs, and, conversely, a higher proportion of PSOs and TSOs fall within the high-quality category, compared with FPOs. One exception is the ‘quality of environment’ domain, in which the proportions across different levels of quality is roughly similar across sector. PSOs have the highest proportion of ‘low-risk’ organisations, whereas TSOs seem to be of proportionally higher risk compared with PSOs and FPOs.

Predicted probabilities

Figure 1 shows the predicted probabilities of all outcome variables (the four quality domains, risk, requirements and complaints) across sector. The probabilities are derived from the generalised ordered logistic regression models, controlling for years of registration, type of sector, whether an organisation is active, target group of service providers (eg, residents with dementia), type of service provider, number of employed staff and whether an organisation has improved, declined or stagnated in its lowest ranked quality category since the last inspection. The number of included observations in the generalised logistic models are lower than in table 5 due to the exclusion of service types with limited sector variation (ie, childminding, adoption services, adult placement services, nurse agencies and offender accommodation services). However, the observed effect of sector on quality remains similar when running more simple and inclusive models. For the binary outcome variables (complaints and requirements), the predicted probabilities are derived from multiple logistic regression models, controlling for the same variables as described above. To adjust for multiplicity, all displayed CIs are Bonferroni corrected.

Figure 1

Predicted probabilities for the four quality domains, risk, complaints and requirements across sectors with 95% CIs. All quality outcomes are based on a scale of low, adequate and high quality. The risk outcome is based on a scale of low, medium and high risk. The complaint and requirement outcomes are binary with the predicted probabilities referring to the likelihood of having been issued a complaint or requirement within the last three years. All predicted probabilities are displayed in percentages.

Table 5

This table displays the difference in predicted probabilities across sectors over the quality and risk domains with 95% CIs

	Low quality	Adequate quality	High quality
Quality of care and support (n=7011)
Public versus for-profit sector	−0.011***	−0.098***	0.108***
95% CI	−0.018 to −0.004	−0.135 to −0.061	0.071 to 0.146
Third versus for-profit sector	−0.005	−0.079***	0.084***
95% CI	−0.014 to 0.003	−0.116 to −0.042	0.047 to 0.121
Third versus public sector	0.006	0.019	−0.024
95% CI	−0.001 to 0.012	−0.015 to 0.053	0.010 to 0.261
Quality of staff (n=7007)
Public versus for-profit sector	−0.013***	−0.125***	0.138***
95% CI	−0.020 to −0.006	−0.164 to −0.087	0.100 to 0.177
Third versus for-profit sector	−0.005	−0.074***	0.079***
95% CI	−0.014 to 0.004	−0.112 to −0.036	0.041 to 0.117
Third versus public sector	0.008*	0.051*	−0.059***
95% CI	0.002 to 0.015	0.015 to 0.086	−0.095 to −0.023
Quality of environment (n=5099)
Public versus for-profit sector	−0.003	−0.106***	0.110***
95% CI	−0.010 to 0.004	−0.151 to −0.061	0.065 to 0.154
Third versus for-profit sector	0.001	−0.034	0.033
95% CI	0.004 to 0.010	−0.082 to 0.015	0.015 to 0.081
Third versus public sector	0.004	0.073***	−0.077***
95% CI	−0.004 to 0.012	0.030 to 0.115	−0.119 to −0.034
Quality of management (n=7004)
Public versus for-profit sector	−0.019***	−0.093***	0.112***
95% CI	−0.028 to −0.010	−0.133 to −0.133	0.073 to 0.152
Third versus for-profit sector	−0.007	−0.070***	0.077***
95% CI	−0.018 to 0.004	−0.109 to −0.031	0.039 to 0.116
Third versus public sector	0.012***	0.023	−0.035
95% CI	0.004 to 0.020	−0.014 to 0.060	−0.072 to 0.002

*p<0.05; ***p<0.001. Also, higher values of risk indicate higher risk of services.

Differences between FPOs, PSOs and TSOs

Figure 1 shows a similar pattern to the cross tabulations of quality across sector (see tables A3–A4 in supplementary appendix A for predicted probabilities). Again, FPOs are, in general, more likely to secure ‘adequate’ quality scores, but less likely to be in the ‘high-quality’ category, relative to PSOs and TSOs. For example, the predicted probability for FPOs to be in the ‘high-quality’ category in the care and support domain is 58.4%, whereas it is 69.2% for PSOs and 66.8% for TSOs. The predicted probability for FPOs to be of ‘adequate quality’ in the same domain is 40.1% versus 30.3% for PSOs and 32.2% for TSOs.

In table 5, we see that the differences in predicted probabilities for ‘adequate’ and ‘high’ quality are mostly statistically significant for public and third sector providers, compared with FPOs. The only quality domain which did not display this pattern was ‘quality of environment’, in which the differences in predicted probabilities between FPOs and TSOs were not statistically significant. Further, FPOs were more likely to given a ‘poor quality’ rating compared with PSOs for ‘quality of care’, ‘quality of staff’ and ‘quality of management’, although the absolute differences are relatively small (below 2 percentage points). There were no statistically significant differences in the ‘poor quality’ rating between TSOs and FPOs.

Differences between public and third sector providers

We observe several significant differences in the probabilities between PSOs and TSOs, although the differences in probability are smaller compared with the for-profit sector. For ‘quality of staff’, ‘quality of environment’ and ‘quality of management’, PSOs were more likely to be in the high-quality category compared with TSOs (66.4% vs 60.5%, 62.5% vs 54.8%, and 56.2% vs 53.7%, respectively). Similarly, TSOs were more likely to be rated of ‘adequate quality’ in ‘quality of staff’ (38.3%) and ‘quality of environment’ (44.2%) relative to PSOs (33.72% and 37%, respectively). For ‘quality of care’, ‘quality of staff’ and ‘quality of management’, TSOs are slightly, but statistically significantly, more likely to be in the low-quality group relative to PSOs (difference below 1.5 percentage points).

Risk of services

For-profit providers were the least likely to be classified as ‘low-risk’ services (79.9%), and the most likely to be in ‘medium-risk’ (13.2%) and ‘high-risk’ (6.9%) groups. PSOs had the highest probability of being categorised as ‘low-risk’ services (91.1%), and the lowest probability of having their services classified as of ‘medium’ (6.9%) and ‘high risk’ (2%), followed by TSOs (86%, 8.5%, and 4.5%, respectively). Most differences across the levels of risk between sectors were statistically significant, but the differences between PSO and TSOs were substantially smaller compared with TSOs versus FPOs and PSOs versus FPOs.

Complaints and requirements

Figure 1 shows the predicted probabilities for organisations to have been subject to a complaint or requirement in the last 3 years across sector. Again, we observe that FPOs are significantly more likely to having been issued at least one complaint, compared with PSOs and TSOs, with a difference of 17.1 and 14 percentage points, respectively (see table 6). Similarly, FPOs were more likely to having been subject to a requirement after inspection, relative to PSOs and TSOs with a difference of 10.5 percentage points for PSOs and FPOs, and 5.6 percentage points for TSOs and FPOs. PSOs were significantly more likely to having received a requirement or complaint relative to TSOs, but the differences were substantially smaller compared with FPOs.

Table 6

This table displays differences in predicted probabilities to have been issued a complaint or requirement in the last 3 years across sector

		95% CIs
If a complaint has been issued within last 3 years (n=7358)
Public versus for-profit sector	0.171***	0.197 to 0.144
Third versus for-profit sector	0.140***	0.167 to 0.112
Third versus public sector	−0.031***	−0.012 to −0.050
If a requirement has been issued after inspection within last 3 years (n=6320)
Public versus for-profit sector	0.105***	0.138 to 0.072
Third versus for-profit sector	0.056***	0.138 to 0.072
Third versus public sector	−0.049***	−0.019 to −0.079

***p<0.001.

Other explanatory factors

The number of years an organisation has been registered with the Care Inspectorate and the number of employed staff were both significant predictors of quality. The association between age and quality showed that the older the organisation, the higher probability of being rated of high quality. Further, the more paid staff an organisation had, the more likely an organisation was to be in the ‘adequate’ category, whereas the probability of being in the ‘high-quality’ category decreased with the number of employed staff. Neither the effects of staff and time in operation on quality were interacted by sector.

Subanalyses

We reran the analyses on the biggest subgroups of services (ie, care homes, support services, child care services and housing support services), which can be found in in table A5 in the online supplementary appendix A and figures B1–B5 in the online supplementary appendix C. Most of these subanalyses resulted in slightly reduced effects of sector and with less clear differences between PSOs and TSOs. However, both the raw cross tabulations and the predicted probabilities generally display the same substantive pattern and direction of effects with FPOs being more likely to be classified as of adequate quality and of higher risk relative to TSOs and PSOs, which are more likely to be rated as high quality and of lower risk.

Discussion

The present study finds that public and third sector providers are more likely to be rated of high quality and of lower risk compared with FPOs, which are more likely to be rated of adequate quality and of higher risk. This pattern was similarly observed in FPOs being more likely to receive requirements and complaints compared with their public and third sector counterparts.

The results can be thought to correspond with the notion that TSOs and PSOs are primarily driven by pro-social values and will thus be more motivated to maximise quality of services. Conversely, it may be contended that rational FPOs, being primarily motivated by profits, will coordinate their resources so that services fall within adequate standards. While PSOs were often less likely to be rated of ‘low quality’ relative to TSOs and FPOs, sector was generally not a strong predictor of the likelihood of organisations being given ‘low quality’ rating, which may be understood as all sectors being similarly incentivised not to be rated below acceptable threshold.

These findings echo a recent study on the quality of nursing homes, which utilised data from the Care Quality Commission (CQC) in England and Wales to test whether quality ratings varied according to sector finding that third and public providers were significantly more likely be rated of higher quality than FPOs, as measured by the quality index of the CQC.39 However, contrary to our findings, the paper did not identify any statistically significant differences between third and public sector nursing homes.

Strengths and limitations

This study adds to the existing research in several ways. First, the present study replicated results of past research in a new context controlling for more factors (eg, number of staff) and on a larger range of outcomes (eg, user-reported complaints) than in prior research. Second, the effect of sector on quality were robust to subsequent subanalyses and may thus be thought to generalise across the included types of care services in the Scottish context. Further, contrary to existing research, the study identified several significant differences between PSOs and TSOs, which often favoured the performance of PSOs. Yet, these differences in quality were substantially smaller compared with for-profit versus public and third sector providers.

This study is subject to several limitations, some of which are due to the nature of the data. For example, the lack of effect of sector on quality for levels of low quality (1–2) may be driven by the low number of observations in these categories. However, that origin of sector had no effect on quality is consistent to the idea that organisations from different sectors will, theoretically, have a similar disincentive to be rated as inadequate.

A more substantial limitation of the analysis is that service types may differ systematically across sectors. For example, it may be that clients served by FPOs are inherently different from those served by third and public sector providers. To address this issue, we controlled for all organisational characteristics allowed for by the data (eg, service types, years of registration and number of employees) and ran the models separately on all major service types, showing the same substantive pattern as reported in the main results. Although all models adjusted for the types of clients served by the social care providers (eg, children, older people, dementia patients), the data collected by the Care Inspectorate do not include individual-level information on clients served, and we could thus not investigate client variation in terms of, for example, socioeconomic status across sectors. To the best of our knowledge, there is no existing research investigating client variation across sectors for social care services,3 39 40 which is important to explore in future research.

The analysis is built on the assumption that the quality ratings by the Care Inspectorate can be considered reliable indicators for the latent construct of ‘quality’, which may be a flawed assumption. Yet, the observed effects were consistent across different quality domains, which suggest that the effects of sector were—at least—robust to the construct of quality employed by the Care Inspectorate. Also, the pattern observed for the quality and risk domains was consistent with the inspection requirements and complaints outcomes.

Last, the results should be interpreted in the light of the context of the data. The analysis was conducted on Scottish data on certain types of care services and the findings therefore primarily apply to that context. To test whether the observed effects do indeed transfer to other countries and service settings (eg, healthcare), further research must replicate these analyses using comparable outcomes.

Implications for policy and practice

Research on the typology of sector tend to be based on theory with limited attention on what the features distinct to each sector mean in practice and whether they influence aspects such as service delivery. Importantly, the findings of this paper do not indicate that services by FPOs are poor; in fact, the findings demonstrate that origin of sector was, for the most part, not a strong predictor of whether care organisations performed below acceptable standards. Also, similar to past research, the findings do not imply a causal effect of sector due to the observational nature of the data.41 However, the findings can be interpreted to imply that care organisations driven by pro-social motivations tend to perform better than those that are not, although this remains a theoretical interpretation, as the analysis does not include a concrete measure of motive. Going forward, it is essential to explore mechanisms to investigate whether the association found in this analysis is spurious or indeed the result of some sector-specific traits.

Conclusion

This paper investigated if origin of sector influence the quality of social care services regulated by the Care Inspectorate in Scotland to find that providers from the third and public sectors were rated of higher quality and lower risk than for-profit providers. Importantly, the analysis does not provide any empirical answer to why differences across sectors are observed. Although the results are consistent with theory and past research, more research is needed to explore what exact characteristics account for the variation in quality observed across sectors. From the findings, it follows that regulators and policymakers should incorporate more thought on how organisations across different sectors are incentivised to manage their resources, as this might influence the subsequent performance of commissioned services. However, there is little research addressing how this is best realised in practice, which warrant further analysis.