Validation of an instrument to measure inter-organisational linkages in general practice.

PURPOSE: Linkages between general medical practices and external services are important for high quality chronic disease care. The purpose of this research is to describe the development, evaluation and use of a brief tool that measures the comprehensiveness and quality of a general practice's linkages with external providers for the management of patients with chronic disease. In this study, clinical linkages are defined as the communication, support, and referral arrangements between services for the care and assistance of patients with chronic disease.
METHODS: An interview to measure surgery-level (rather than individual clinician-level) clinical linkages was developed, piloted, reviewed, and evaluated with 97 Australian general practices. Two validated survey instruments were posted to patients, and a survey of locally available services was developed and posted to participating Divisions of General Practice (support organisations). Hypotheses regarding internal validity, association with local services, and patient satisfaction were tested using factor analysis, logistic regression and multilevel regression models.
RESULTS: The resulting General Practice Clinical Linkages Interview (GP-CLI) is a nine-item tool with three underlying factors: referral and advice linkages, shared care and care planning linkages, and community access and awareness linkages. Local availability of chronic disease services has no affect on the comprehensiveness of services with which practices link, however, comprehensiveness of clinical linkages has an association with patient assessment of access, receptionist services, and of continuity of care in their general practice.
CONCLUSIONS: The GP-CLI may be useful to researchers examining comparable health care systems for measuring the comprehensiveness and quality of linkages at a general practice-level with related services, possessing both internal and external validity. The tool can be used with large samples exploring the impact, outcomes, and facilitators of high quality clinical linkages in general practice.

Introduction

Chronic disease care is complex and requires coordination between numerous providers and organisations [1], and effective teamwork [2, 3]. Good clinical linkages between organisations are related to quality of care, particularly continuity of care [4, 5]. A literature review identified the importance of organisational linkages for the development of shared care between general practice and more specialised services for conditions such as diabetes [6-8]. A recent evaluation of service integration has highlighted the importance of primary health care due to its central position in the health care system [9].

There is little research about linkages and partnerships between general practitioners (GPs) and other providers and services. In the context of this study, clinical linkages are defined as the formal or informal relationships, communication, and arrangements between the general practice and those in services or agencies with which care is being shared or other benefit is obtained for patients with chronic disease. Importantly these linkages allow for general practice integration into the primary health care system [10], and in addition to facilitating the transfer of care to other professionals, also serve to coordinate current care, and connect patients with available resources. Different linkages may be required for different chronic diseases. Although these linkages may vary, there is some evidence that links are stable over the medium term at the level of primary care organisations [11]. These linkages may be facilitated by communications [12] or the use of link-workers [13, 14], however, the latter are not the principal focus of this measure.

A comprehensive review of published inter-organisational networking research between 1980 and 1996 showed that research has primarily centred on the driving forces behind the networking, and not the measurement of the networks or their consequences [15]. The research conducted in this area has primarily been at the individual GP network level, not at the organisational, or practice, level. This is problematic because general practice is shifting from the solo GP towards multidisciplinary team care. Much of the literature about health services linkages concerns barriers to integration, or how integration is working [16], as well as interventions to enhance linkages and collaboration [17]. A change in focus is needed from the level of the individual practitioner to one that examines and measures relationships and interactions at the organisational level in order to facilitate best-practice care [18, 19].

The Assessment of Chronic Illness Care (ACIC) instrument was developed to measure chronic illness care within health systems and services [20, 21]. Three of its 28 questions measure linkages between the health delivery system (or provider practice) and community resources which play important roles in the management of chronic illness. This is a generic measure of linkages for all chronic illnesses and in relation to linkages specifically focuses on coordination with outside resources, partnerships with community organizations and regional health plans. These are not specific enough to assess the linkages between an individual practice and other health services and providers for the purpose of providing clinical care for people with diabetes, cardiovascular disease and asthma. To our knowledge, there are no validated tools to measure the presence or quality of organisational-level linkages between general practices and other services. Need to say why this is a shortcoming.

This paper reports the development, validation and application of the General Practice Clinical Linkages Interview (GP-CLI), which is designed to assess the quality of chronic disease-related clinical linkages and relationships that exist between the practice as a whole and external providers and services.

Methods

Background

The GP-CLI was developed as part of a larger Australian study assessing the impact of the organisational capacity of general practices on quality of chronic disease care. One aspect of organisational capacity proposed for study was the linkages of general practices with other health providers/organisations, but there were no suitable measurement tools available. The study was conducted between 2004 and 2005 in five states and one territory (New South Wales, Victoria, South Australia, Queensland and Tasmania, Australian Capital Territory) of Australia and in 27 Divisions of General Practice (Divisions), which are local primary care support organisations. The GP-CLI was designed for use across large samples of practices for benchmarking purposes and to determine the facilitators, impact, and outcomes of practice-based linkages.

Sample size

A priori sample size calculations on the Access factor score from the patient survey used, the General Practice Assessment Survey (GPAS), confirmed that after adjustment for clustering (our pilot patient data from the GPAS indicated a cluster effect (ICC) of 0.17 for the GPAS Access factor) it was predicted that an average of 50 patients from each of 100 practices would have sufficient power (1-β=0.8 and α=0.05) to detect an effect size of 0.24 between male and female patient groups (across all practices). Post hoc, in order to assess differences between groups of practices, adjusting for the effect of patient characteristics such as age and gender, and other characteristics such as size (25 practices were solo) and location of practices (34 practices were rural or remote), 7505 patients were required to have sufficient power (1-β=0.8 and α=0.05) to detect an effect size of 0.24 between practices with good or poor linkages.

Recruitment

The local support organisations for general practice, Divisions of General Practice, posted invitations to participate to all practice principals in their district. In order to participate, practices needed to have at least 50% of the GPs involved in the study. In each practice, 180 adult patients currently on medication for asthma, type 2 diabetes or hypertension/ischaemic heart disease, were randomly selected and invited to participate. For the purpose of this study, hypertension and ischaemic heart disease were considered as one condition as they cannot be distinguished from one another based on the patients' medication information alone, and the standards of care are similar for both conditions.

Ethics

All GPs, practice staff and patients provided full written informed consent. The study was approved by the Human Research Ethics Committees of the University of New South Wales and the University of Adelaide.

Instrument description and administration

Table 1 lists the four instruments used and provides brief details of their use.

Table 1

Details of data collection instruments used in the study.

Scale name	Format	Participant	Measures	Hypothesis being tested
General Practice Clinical Linkages Interview	20 min in-person interview	Practice principal and manager	The comprehensiveness of clinical linkages maintained by a general practice for the management of three common chronic diseases	1. The General Practice Clinical Linkages Interview has internal validity*
Regionally Available Services Survey	Postal survey	Division of General Practice CEO or project officer	The regional availability of services with which individual general practices could create linkages for chronic disease management	2. Comprehensiveness of clinical links in a practice is associated with regional availability of chronic disease services*
General Practice Assessment Scale	Postal survey	Randomly selected patients with chronic disease participating from practices	Patient satisfaction with 10 dimensions of general practice care	3. Patients have higher satisfaction with practices that have more comprehensive clinical linkages for chronic disease
SF-12	Postal survey	Randomly selected patients with chronic disease from participating practices	Patient mental and physical health status	Covariate used to control for differences in patient health status when testing hypothesis 2 and 3

Instruments developed for this study.

General practice clinical linkages interview

The Clinical Care Interview is a 20-min interview designed to measure the clinical linkages that a general practice maintains for the management of three common chronic diseases: asthma, type 2 diabetes, and hypertension/ischaemic heart disease. The information was collected by interview with the principal General Practitioner and practice manager because this method allowed the trained interviewer to use prompts as well as to make more effective use of questions to enhance data accuracy. The interview format also ensured that the data collection took place in an undisturbed environment with sufficient time for completion.

Identification of items for inclusion

Focus groups with GPs, practice nurses, practice managers and consumers identified linkages between the practice and other providers and services as important ways to ensure that patients could access other services appropriately and to receive continuity of care [22, 23].

Following consultations with several experts in the field and a review of the literature, items for the GP-CLI were chosen to reflect the clinical functions, involving external providers that are involved in best-practice chronic disease care: referral, second opinions or advice, shared care, care plans, patient education, and community access and awareness.

Once the item pool had been generated, the research team reviewed each of the items to establish its relevance and suitability for inclusion, and whether it had any value-laden, offensive or biased content. The remaining items were drafted into an interview.

For each clinical function in each of the three disease categories, respondents are asked to list up to three links that their practice currently has, and to rate the quality of the service provided (scored on a 0–5 Likert scale) and the quality of communication with the service (0–5), with 5 representing the highest quality. Communication and quality were identified as being important in other qualitative studies which we have conducted [24]. The quality of communication and quality of the service provided through the linkage is self-assessed by the practice staff based on their experience with the service or provider. In the GP-CLI, quality of communication was defined as including the quality of two-way communication between the practice and the provider involved in the care of individual patients with the condition concerned. In both the GP-CLI and the Division Survey (see paragraph ‘Regionally available services’ survey below), quality of service provided was defined as including both the accessibility, acceptability and technical proficiency of the service provided to patients referred or whose care was shared.

If the practice had no linkages for a particular chronic disease function (e.g. asthma referral), then a score of 0 was recorded. To reduce the risk of social desirability bias, respondents are asked to indicate the type of service, as well as the providers' initials or organisation's name. These two items were not scored. The communication and quality scores were averaged for however, many links the respondent listed (up to three for each question), entailing that having more than one link did not increase the overall score, as the scale was trying measure comprehensiveness and not advantage multiple links that serve the same purpose.

Interviewees were instructed that practice employees were not considered as external linkages, however, those who were associated with the practice, but not directly employed, were considered as external linkages.

Piloting, adjustment and administration of tool

The interview was reviewed by several experts and piloted with a convenience sample of four academic GPs and four pilot practices. The tool was then adjusted on the basis of the feedback from the review and pilot. The interview was administered by Research Officers who had attended a training session, and received further one-on-one supervision in order to standardise data collection.

Responses to interview questions were entered directly onto a template on a laptop computer. These data were then transferred electronically into a statistical package for analysis. This method decreased possibility of human transcription error.

The interview was in three parts, the first regarding to links for asthma, the second for type 2 diabetes and the third for hypertension/ischaemic heart disease. The format of the questions was to ask whether the practice had links at the organisational level with the four clinical functions: ‘referral or advice’, ‘shared care or care planning’ ‘patient education or self-help’ and to ‘improve community awareness of or community access to services or resources’ and then ask the interviewee to rate the communication with and quality of each of the links (Table 2). This same format was applied to all three conditions.

Table 2

Structure of general practice clinical linkages interview.

	Links at organisational level (can list up to three)	Effectiveness of communication between the organisations	Quality of service provided to your practice
Referral or advice	Present/not present	0–6 for each link	0–6 for each link
Care planning	Present/not present	0–6 for each link	0–6 for each link
Patient education or self help	Present/not present	0–6 for each link	0–6 for each link
Community awareness of or access to services or resources	Present/not present	0–6 for each link	0–6 for each link

A linkages score was calculated individually for each of the clinical functions with each of the diseases (i.e. asthma referral, asthma shared care), resulting in nine scores. These scores are the perceived quality of the linkage score (0–5). If more than one link is listed for each function, the quality scores are averaged for the reason described previously. A final score was calculated by summing the nine scores.

Regionally available services survey

To determine the concurrent validity of the GP-CLI, a survey was developed to ascertain the regional availability of services with which general practices could create linkages for the care of patients with asthma, diabetes, and hypertension/ischaemic heart disease. The survey was posted to the Chief Executive Officers of all participating Divisions for completion. For each service, the respondent was asked about service availability, as well as service quality (taking into consideration access, waiting times, etc), and public or private availability of the service. If a service is only available to some practices within a Division due to large geographical distances, the suburbs in which it was not available were recorded. This enabled a single score to be calculated for each general practice based on the suburb in which they are located.

General Practice Assessment Scale

The General Practice Assessment Survey is a commercially available 53-item patient-report questionnaire that assesses 10 dimensions of general practice care: Access, Receptionists, Continuity of Care, Communication, Interpersonal Care, GP's Knowledge of the Patient, Specialists Referral, Enablement, Practice Nursing and Overall Satisfaction [25]. The scale consists of two types of questions: report items e.g. “How long do you usually have to wait at the practice until your consultations begin?” and assessment items e.g. “How do you rate this?” Only assessment items are used in the calculation of scale scores, and they are measured on 6-point scales.

In each practice a random sample of up to 180 patients aged over 18 years who had been prescribed medication for type 2 diabetes, hypertension/ischaemic heart disease or moderate to severe asthma in the previous six months was selected using practice software. These patients were posted the General Practice Assessment Survey (GPAS). Non-responders were followed up with a second mailing.

SF-12

The SF-12 is a commercially available generic short survey designed to measure health status and has been used extensively in Australia and internationally [26]. This survey has twelve questions that relate to eight factors. These eight factors make up summary scores for mental health and physical health.

The same sample of patients received the SF-12 survey along with the GPAS. The SF-12 data were used to control for severity of illness in testing the hypothesis of the relationship between clinical linkages and service availability.

Statistical analysis

Hypothesis 1: internal validity of general practice clinical linkages interview via factor analysis

Factor analyses were carried out on the GP-CLI in order to test the construct validity of the instrument. Data from 97 practices were used and the number of quality items initially was 12, giving an item to respondent ratio of 8:1.

Tables of frequencies were inspected for any item responses showing restriction in range. As suggested by Streiner [27], items where “two responses from five categories accounted for more than 90% of the sample” were omitted. Alpha coefficients [28] were calculated as a baseline measure of the internal consistency of the items on the interview.

The aggregated mean quality scores and communication scores for the practices' clinical linkages, which together were to be a proxy for the quality of the link, were so closely correlated that the analyses could not be performed using both variables because they would have overwhelmed any other existing correlations. The mean quality of the service, as judged by the practice, was used alone as the measure for the quality of the link. Quality was chosen because it is likely that communication with a service is part of the overall assessment of quality, therefore, quality represented the broader measure of the service.

Factor analysis procedure

The method of estimation used in factor analysis was Principal Axis factoring, which assumes that the observed variables and unobservable common factors have underlying multivariate normal distributions [29]. Following the first stage of the factor analysis, the initial solution was subject to Oblimin rotation, to make the solution more interpretable. The amount of variance explained by each factor (initial Eigen values greater than one) and inspection of the Scree plot [30] were used as criteria for retention of factors.

Identifying and calculating the subscales

Simple approximation methods, suitable for research purposes, were used to calculate sub-scale scores from the original data set [31]. Items with a factor loading coefficient equal to or approaching 0.4 were included and items loading on more than one factor were screened out [31, 32]. Cronbach alpha coefficients [28] were again calculated to establish the internal consistency of the items within each sub-scale (desired range between 0.5–0.8). Subscale scores can range from 0 to 5, with a higher score indicating higher quality linkages.

Overall model fit and utility of scores

The overall model fit was assessed by a structural equation model using Analysis of Moment Structures (AMOS) [33]. The following criteria were used to assess goodness of fit; a Discrepancy ratio of less than 2; Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Tucker-Lewis Index (TLI), and Comparative Fit Index (CFI) with values in the region of 0.9 and a Root Mean Square Error of Approximation (RMSEA) in the region of 0.05 [34].

Hypothesis 2: general practice clinical linkages interview and the availability of services survey

In order to determine if service availability had an impact on a practice's clinical linkages, logistic regression was used. These analyses focused on the association between the availability and quality of services as rated in the Regionally Available Services survey and the GP-CLI factors.

The data from the GP-CLI were skewed, and therefore the scores were dichotomised into two groups: scores that were in the upper quartile were given a 1, and all others were assigned 0. The covariates included in the analyses were the number of GPs in the practice, whether the practice was rural or urban, and the overall physical and mental health of the practice's participating patients.

Hypothesis 3: general practice clinical linkages interview and patient assessment of care

Previous research on patients' assessments of care has been carried out using standard single level techniques [35, 36]. However, patients in a single general practice cannot be assumed to be independent [36, 37], and methods should take into account the correlation of individual responses within clusters [37, 38].

Multilevel regression models were used with GPAS scores as the dependent variables, GP-CLI total scores as the independent variable, and type and location of practice and age and gender of the patient, health status of patients as covariates.

The distributions were skewed, for example with more than 70% of GPAS communication scores being 80 or above out of 100, hence scores were transformed to normal scores and used as the dependent variables in multilevel analyses. The variables representing size and geographical area of practices were used as practice level independent variables.

Results

Demographics of respondents

Principal GPs and (where available) practice managers from ninety-seven general practices completed the GP-CLI. Of the 97 participating practices, 40 were large practices (4 or more GPs), 32 had two or three GPs, and 25 were sole practitioners. Forty-one of the practices were in capital cities, 22 were in other urban areas, and 34 were in rural areas with a population of less than 10,000. A sample size of 97 practices was sufficient to detect a 0.58 of a standard deviation difference in the linkage score between smaller (1–3 GPs) and larger (3 GPs) practices at a significant level of 5% and a power of 80%.

Construct validity of the general practice clinical linkages interview

Most of the items were mildly correlated and suitable for inclusion in the first factor analysis. The Cronbach alpha coefficient for the 12 items was 0.721, indicating acceptable internal consistency and suggesting item redundancy. Initial Eigen values and inspection of the Scree plot suggested that a solution with three or four factors would be optimal. Extraction of four factors terminated because the communality of one variable exceeded one. Three factors were then successfully extracted, explaining 33.1% of the variance.

Inspection of the first pattern matrix showed two items loading with coefficients less than 0.3; both items associated with “patient education” and these were removed from the analysis. A further three items loaded at values approaching 0.4; the remaining patient education item, one item each associated with shared care and care planning and referral and advice. The third factor contained all three items associated with community awareness and access with loading coefficients approaching 0.5 and above. In total, seven items loaded with strong coefficients (minimum coefficient 0.476).

The final analysis identified a three-factor solution that explained 37.3% of the variance (Table 3). The final scale comprised 9 items with all items loading with coefficients above 0.4, with the exception of the “referral and advice” quality score for asthma.

Table 3

Pattern Matrix (Final factor analysis).

	Factor
	1	2	3
ns6 Shared care or care plan quality asthma	0.786
ns19 Shared care or care plan quality/diabetes	0.552
ns32 Shared care or care plan quality/hyp/IHD	0.460
ns25 Community awareness or access quality/diabetes		0.629
ns12 Community awareness or access quality/asthma		0.529
ns38 Community awareness or access quality/hyp/IHD		0.514
ns16 Referral or advice quality/diabetes			0.657
ns29 Referral or advice quality/hyp/IHD			0.593
ns3 Referral or advice quality/asthma			0.368
			0.368

The three factors were termed:

Shared care/care planning

Community access/awareness

Referral/advice

Descriptive statistics and Cronbach alpha coefficients for the three subscales and total score are detailed in Table 4, with alpha coefficients ranging from 0.53 to 0.68. A confirmatory factor analysis using AMOS showed that the overall model was an excellent fit.

Table 4

The descriptive statistics for the final three subscales and total scales.

	Factor			Total score
	Shared care/care plan	Community awareness/access	Referral/advice
Mean	7.90	4.69	12.12	24.70
Std deviation	4.94	4.47	2.70	8.74
Minimum	0.00	0.00	0.00	0.00
Maximum	15.00	15.00	15.00	43.33
Cronbach alpha	0.660	0.587	0.533	0.675

Utility of the general practice clinical linkages interview

The Referral factor had the highest mean, followed by “shared care/care planning”, and “community access/awareness”. The means and standard deviations for the subscales and total scores are shown according to the size of practice and type of area (Table 5). The mean total clinical linkages score for practices with solo GPs was slightly lower compared with larger practices but did not reach significance. Similarly there was no association between the GP-CLI total score and the number of staff in a practice, having a practice nurse, having attached staff (allied health professionals not employed by the practice but offering services within the practice), employing a practice manager, having met accreditation quality standards from a nationally accrediting agency, or directly employed allied health staff. There were no statistically significant differences in the three subscale scores and total scores associated with urban or rural area.

Table 5

The scale and subscales by size and location of practice.

			Mean	Std deviation	Minimum	Maximum	Valid n
Area	Metropolitan	Shared care	7.39	4.97	0.00	15.00	n=63
		Comm aware	4.44	4.43	0.00	15.00	n=63
		Referral	12.24	2.94	0.00	15.00	n=63
		Total	24.07	8.61	0.00	41.33	n=63
	Rural/remote	Shared care	8.83	4.83	0.00	14.67	n=34
		Comm aware	5.16	4.57	0.00	15.00	n=34
		Referral	11.88	2.21	7.00	15.00	n=34
		Total	25.87	8.98	9.50	43.33	n=34
Size of practice	Solo	Shared care	6.97	5.87	0.00	15.00	n=25
		Comm aware	3.28	4.32	0.00	15.00	n=25
		Referral	12.06	2.85	5.00	15.00	n=25
		Total	22.31	8.99	10.00	38.33	n=25
	2–3 GPs	Shared care	8.95	4.09	0.00	14.67	n=32
		Comm aware	5.27	4.55	0.00	15.00	n=32
		Referral	12.15	2.20	7.67	15.00	n=32
		Total	26.37	7.80	10.00	43.33	n=32
	4 or more GPs	Shared care	7.63	4.90	0.00	15.00	n=40
		Comm aware	5.11	4.42	0.00	15.00	n=40
		Referral	12.12	3.02	0.00	15.00	n=40
		Total	24.86	9.16	0.00	41.33	n=40

Regionally available services survey

Of the 27 Divisions that recruited general practices for the study, 24 responded to the survey (response rate 89%). Regional service availability has no association with the quality and comprehensiveness of practices clinical linkages for asthma, diabetes or hypertension/ischaemic heart disease when controlling for practice size, rural or urban location, and physical and mental health of the practice's patients (Table 6).

Table 6

Association between regional service availability and quality and the GP-CLI factor scores by disease (logistic regression).

Factors by disease	Service availability p-value	Service quality p-value
Asthma referral	0.070	0.122
Asthma shared care	0.348	0.127
Diabetes referral	0.051	0.061
Diabetes shared care	0.334	0.500
Hyp/IHD referral	0.176	0.393
Hyp/IHD shared care	0.545	0.521

General practice assessment survey

A total of 12,544 patients attending 97 practices were mailed the GPAS and invited to participate. Responses were received from 7505 patients (a response rate of 60%), which had sufficient power (1-β=0.8 and α=0.05) to detect an effect size of 0.25 between practices with good or poor linkages.

Multilevel regression analyses showed that the total scores for the GP-CLI were likely to be associated with patients assessment of access (β=0.071, p=0.024), of receptionist services (β=0.046, p=0.018), and of continuity of care (β=0.059, p<0.001) after adjustment for type and location of practice and age and gender of patient as well as the cluster effect.

Discussion

The development and analysis of the GP-CLI has resulted in a nine-item tool with three underlying factors. The three factors are linkages for “referral/advice”, for “shared care/care planning”, and for “community access/awareness”. These factors are across the three disease categories, asthma, type 2 diabetes, and hypertension/ischaemic heart disease. That the factors are spread across the diseases reflects that there is an underlying similarity in linking for a particular function (i.e. referral, shared care), as opposed to linking for particular diseases. This indicates the usefulness of the tool as a generic measure of chronic disease linkages, rather than a measure of linkages for only the three individual conditions.

When linkages used by the practice were compared to service availability, as indicated in the Regionally Available Services Survey, the results suggest that availability is not a predictor of the quality of linkages that practices have for chronic disease. This is particularly important in planning how to improve general practice linkages in geographic areas that have varying service availability. Reinforcing this is our finding that the mean GP-CLI scores of rural and urban practice do not differ significantly.

The Regionally Available Services Survey was designed to assess the influence of service availability on the linkages measure, and while the survey was based on self report, we do not believe that there was a strong selection bias. The services asked about in the survey were not provided by the Division and a major role of Divisions is to better integrate general practice with other services and they are thus highly attuned to the services which their members can refer to or share care with. These linkages are reported annually in the Annual Survey of Divisions required of them by the Australian Department of Health and Ageing [39].

Good clinical linkages could be expected to result in improved continuity and consistency of care, better access and more appropriate referral to other services and better follow up by the practice. The relationship between the GP-CLI scores and the patients' assessments of their access to care, receptionist services and continuity of care is consistent with what is known about the importance of linkages in patient care [1, 4, 5]. From our study, however, it is not clear whether patients are more satisfied in those aspects of care because of the clinical linkages, or if the practice has better quality linkages because having better access, receptionists, and continuity of care facilitates the formation or maintenance of high quality linkages. While previous interventions to improve access to general practice have focused on appointment scheduling [40, 41], our findings indicate that there is an important relationship between clinical linkages and patient access to care, and further investigation should be undertaken to clarify this relationship.

The quality of clinical linkages in the practices was not affected by practice size, area, and the mix of staff employed in the practice. One consideration that was taken in designing the GP-CLI was that some practices have reacted to poor access to services by bringing those services directly into the practice through employing, contracting, or renting out practice space to allied health professionals and nurses. The interview was designed to measure the practices' linkages without interference from the effect of employed staff, who may perform some of the roles in the practice. Practices that have these alternative arrangements should not automatically have poorer quality linkages (i.e. a lower score on the GP-CLI) because they will still require some level of external linking for each of the functions. Our results indicate that this aim was achieved, with the mean scores of practices that employ allied health staff or nurses being no different from those that do not.

Our research had several limitations which should be recognised when considering the results. With 97 practices, the sample size was relatively small, and the study needs to be replicated to have confidence in the findings. Nevertheless, despite the limitations of sample size, the Cronbach's alpha coefficients suggest that the GP-CLI has moderate reliability [28]. However, only 37.3% of the variance in the GP-CLI is explained by the three factors and one item in factor 3 had a loading coefficient lower than the desired 0.4. This item was retained on two grounds; firstly, the coefficient exceeded the minimum acceptable value of 0.32 for inclusion [42] and retention of the item maintained the generic features of the instrument, balanced for each disease group. To test the discriminant validity of the GP-CLI, an external tool, the Regionally Available Services Survey was used. This survey was based on the GP-CLI questions, and has not been independently validated. Inter-rater reliability was not determined for this sample. Other areas in which future research is needed are in the clarification of the relationship between clinical linkages and patient satisfaction with access to care in general practice and the relationship between clinical linkages and local service availability. Additionally, there is a need to replicate these results in Australia, as well as to re-validate them in other countries.

A strength of our study is that we used both internal and external measures, and applied the tool in analyses with an existing validated instrument. The factors that resulted from the factor analysis are statistically robust with loading coefficients above 0.4 for 8 out of the 9 final items [42].

Practice level clinical linkages are important in general practice. Linkages with community referral and resources have been identified in Wagner's Chronic Care model as being related to the quality of chronic disease management [43]. Although important, there was no tool available to easily assess the quality of the links between general practices and external services. Our study has developed and validated a tool for this purpose, and the validation process has allowed for further shortening of the instrument to an approximately 15 minute interview.

Our study has shown that service availability, practice size, and staffing have no bearing on the quality of the clinical linkages maintained in the practice. This suggests that practice-based linkages are important to consider when dealing with issues such as workforce shortages and rural practice. The field would benefit from further investigation into the specifics of what services are used by practices in areas of low availability, how they are used, and whether the linkages of practices with low service availability differ from those of other practices.

The GP-CLI tool has important potential for both policy and practice. There are a number of incentives for multidisciplinary team care for patients with chronic and complex care needs [44]. Divisions of General Practice have an important role in supporting practices to develop better linkages with other service providers, especially allied and community health to deliver care for patients with chronic illness. The GP-CLI tool provides an important potential tool to use both in assessing the needs of practices for support and in monitoring change in team functioning in this setting. In evaluating the impact of policy designed to enhanced multidisciplinary teamwork within general practice, the GP-CLI provides a potential measure of the organisational linkages involved in the delivery of care for patients with specific chronic diseases.

The value of the GP-CLI to researchers is that it can be used quickly across large samples, taking approximately 15 minutes to administer. It enables practices to measure existing clinical linkages and detect changes after quality improvement interventions. Additionally, the tool enables further analysis to be done across large samples exploring the impact, outcomes, and facilitators of high quality clinical linkages in general practice. There is a need however for further research to confirm the psychometrics in another sample of practices, its stability and sensitivity to change, and its usefulness as a quality improvement tool.

Reviewers

Berrie Middel, Dr., Methodological Consultant, Department of Health Sciences, Department of Oral Health and Clinical Epidemiology, University Medical Center Groningen (UMCG), University of Groningen, The Netherlands

Lucio Naccarella, PhD, Senior Research Fellow, The Department of General Practice, The University of Melbourne, Carlton, Australia

One anonymous reviewer