Can the appropriateness of eye care be measured through cross-sectional retrospective patient record review in eye care practices in Australia? The iCareTrack feasibility study.

OBJECTIVES: The CareTrack study found that a wide range of appropriateness of care (ie, care in line with evidence-based or consensus-based guidelines) was delivered across many health conditions in Australia. This study therefore aimed to demonstrate the feasibility of using the CareTrack method (a retrospective onsite record review) to measure the appropriateness of eye care delivery.
DESIGN: Cross-sectional feasibility study. SETTING AND PARTICIPANTS: Two hundred and thirteen patient records randomly selected from eight optometry and ophthalmology practices in Australia, selected through a combination of convenience and maximum variation sampling.
METHODS: Retrospective record review designed to assess the alignment between eye care delivered and 93 clinical indicators (Delphi method involving 11 experts) extracted from evidence-based clinical practice guidelines. PRIMARY OUTCOME MEASURE: Number of eligible patient records, sampling rates and data collection time. This feasibility study also tested the ability of 93 clinical indicators to measure percentage appropriate eye care for preventative, glaucoma and diabetic eye care. A secondary outcome was the percentage of practitioner-patient encounters at which appropriate eye care was received.
RESULTS: A median of 20 records (range 9 to 63) per practice were reviewed. Data collection time ranged from 3 to 5.5 hours (median 3.5). The most effective sampling strategy involved random letter generation followed by sequential sampling. The appropriateness of care was 69% (95% CI 67% to 70%) for preventative eye care, 60% (95% CI 56% to 58%) for glaucoma and 63% (95% CI 57% to 69%) for diabetic eye care.
CONCLUSIONS: Appropriateness of eye care can be measured effectively using retrospective record review of eye care practices and consensus-based care indicators. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The feasibility of a systematic approach (based on CareTrack method) to measure appropriateness of eye care on site in eye care practices was demonstrated.

Appropriateness of eye care was in line with previously measured healthcare but this may represent a slight overestimation of true appropriateness of eye care.

Measurement of appropriateness of eye care was limited to those aspects for which clinical indicators could be derived.

The small convenience sample used in this study may introduce selection biases.

The study findings support the conduct of a larger trial to determine the percentage of eye care encounters at which appropriate care is received. This study identified the best patient record sampling methods, potential issues associated with recruiting eye care practices and with accessing, extracting, recording and analysing clinical records.

Introduction

Globally, 285 million people of all ages suffer from visual impairment with the major causes from both chronic eye conditions, including ocular diseases (eg, glaucoma, diabetic retinopathy, age-related macular degeneration, cataract) and uncorrected refractive errors such as myopia and presbyopia.1 2 The prevalence of vision problems is strongly associated with ageing,3 hence the burden of ocular health problems will increase with an ageing population. Due to the growing demand for eye care and in the context of resource scarcity worldwide, interest in measuring and enhancing the quality of eye care delivery is growing.4 5 Translation of best available evidence into clinical practice can improve the efficacy and cost-effectiveness of patient management.6 In theory, evidence-based clinical practice guidelines aim to translate research findings into easy-to-apply care recommendations that are intended to guide practitioners to improve their professional practice and optimise patient care.7 However, there is mounting evidence that such evidence-based clinical practice guidelines are not always adhered to or fully implemented in the clinical setting.8

The measurement of care quality is complex and multidimensional. In Australia, quality is considered as the guiding principle for assessing the health system’s performance and this includes nine dimensions: appropriate, effective, responsive, continuous, sustainable, accessible, capable, efficient and safe.9 In this study, we focused on appropriateness of care defined as ‘care/intervention/action that is relevant to the patient’s needs and based on established standards’ (see box 1).10 Assessment of the appropriateness of eye care delivery requires that recommendations from evidence-based clinical practice guidelines or expert-based consensus be transformed into measurable clinical indicators which are designed to assess, compare and determine the potential to improve care.11

Condition means clinical ocular conditions (eg, glaucoma and diabetes) or preventative care.

Evidence-based care involves clinical decision making based on the existing best evidence for the care (including eye care) of individual patients.37 The practice of evidence-based care means the integration of the clinical expertise of the practitioners with the best available clinical evidence and patient’s preferences.38

Appropriateness of eye care is care/intervention/action provided relevant to the patient’s needs and based on established standards.10 In this study, appropriate eye care is clinical care for a condition considered to be evidence-based or consensus-based by a panel of clinical experts in Australia in the context in which it was delivered in the years 2013 and 2014.

Clinical practice guidelines are evidence-based statements that include recommendations intended to optimise patient care and assist healthcare practitioners to make decisions about appropriate healthcare for specific clinical circumstances. Clinical practice guidelines should assist clinicians and patients in shared decision making.39

Clinical indicator is a measurable component of a standard or guideline, with explicit criteria for inclusion, exclusion, time frame and setting.13 It is a condition-specific process measurement of healthcare management, appropriate for Australian eye care practice in 2013–2014.27

Record review is a method using prerecorded, patient-focused data as the primary data source to assess quality of care.18

Eye care practice refers to practice or clinic (eg, optometry and ophthalmology practice) where a service related to the eyes or vision is provided.

Eye care provider is an individual who provides a service related to the eyes or vision.

Surveyor is a person with appropriate clinical and review experience to review patient records against clinical indicators.

There are indications that the appropriateness of care is at times suboptimal. For example, the RAND study conducted in 2000 in the USA evaluated performance on 439 clinical indicators for 30 acute and chronic health conditions and preventative care. American adults received recommended healthcare only 55% of the time.12 More recently, the CareTrack study showed similar results with 57% of Australian adults receiving appropriate care across 22 health conditions.13 Very little information can be found on the quality of delivery of eye care specifically, as ocular conditions were not included in the CareTrack study13 and only small components of eye care such as senile cataract were evaluated in the RAND study. According to the National Eye Health Survey (NEHS),14 more than 50% of Australians with visual impairment are undiagnosed. To meet the eye care needs of the ageing population and optimise the healthcare spending in this area, it is important to ensure that eye care is delivered appropriately. To do this requires a deep understanding of who is getting what eye care from whom and why in Australia.

Record review is commonly used to measure adherence to practice guidelines, the translation of clinical education into practice and the effect of interventions intending to improve care delivery.13 15–17 The greatest advantage of a record review is that the data are already collected, making it relatively inexpensive and easy to obtain a large amount of data over an extended period.18 However, patient records are not designed for research purposes, so the data can be incomplete, unavailable or difficult to interpret.19

This paper describes a cross-sectional feasibility study to test whether retrospective onsite reviewing patient records from eye care practices can be used to assess the appropriateness of eye care delivery in Australia. The feasibility study therefore aims to determine patient record sampling methods for the main trial, and explore potential issues associated with recruiting eye care practices and with accessing, extracting, recording and analysing clinical records.

Methods

An onsite cross-sectional retrospective record review of eye care practices was designed to determine the types of problems that might be encountered and to inform the future main study; this included testing the selection of eye conditions, their clinical indicators and the logistical and practical aspects of recruiting eye care practices, assessing patient records and extracting, recording, storing and analysing the data. The feasibility study was designed to assess the alignment between eye care delivered and consensus-based care indicators (see below) extracted from evidence-based clinical practice guidelines for three representative eye conditions. The representative eye conditions glaucoma and diabetic retinopathy were selected based on prevalence (sufficiently high to be measured using the proposed methodology), burden of disease and the availability of Australian and international evidence-based clinical practical guidelines against which to measure appropriateness of eye care delivery.20–23 Preventative eye care was also selected as effective prevention is a key policy initiative for all healthcare delivery.

Study settings

A sample of eight optometry and ophthalmology practices located in Sydney, Australia were selected by the investigators based on convenience and maximum variation sampling, ensuring representation from a variety of eye care practice settings using different record types (eg, paper or electronic records), electronic document and records management system (EDRMS)24 and business models (eg, franchisee, corporate and independently owned practices).

Eligibility

A random sample of adult patient records (aged over 18 years old) from each of the selected eye care practices who attended for eye examination between 1 January 2013 and 31 December 2014 were reviewed. Visits were included if they were billed as a comprehensive eye examination (ie, Medicare item numbers 10900, 10907, 10912, 10913, 10 914 or 10915),25 representing consultations longer than 15 min in length of time, or those categorised as comprehensive eye examination by the billing eye care practitioner. Postoperative visits, contact lens fitting or aftercare, unscheduled visits due to acute conditions and subsequent follow-up visits (eg, visual field test) were excluded from the sample.

To assess the appropriateness of glaucoma eye care, patients were included if they were diagnosed with glaucoma or ocular hypertension, were at risk of glaucoma or were categorised as glaucoma suspects. According to the National Health and Medical Research Council guidelines for the Screening, Progress, Diagnosis, Management and Prevention of Glaucoma,26 a glaucoma suspect is a person suspected of having glaucoma who has some but not all of the criteria required for a glaucoma diagnosis. They may have one or more of the following: suspicious optic disc, optic disc margin haemorrhage, occludable drainage angle, peripheral anterior synechiae or elevated intraocular pressure. A person at risk of glaucoma may be someone with a positive family history of glaucoma, or a history of chronic steroid use, or some other known risk factors for the disease. To assess the appropriateness of diabetic eye care, patients were included if they were diagnosed with type 2 diabetes mellitus (both with and without diabetic retinopathy). Pregnant patients were excluded from all appropriateness assessments and patients with type 1 diabetes mellitus were excluded from the assessment of appropriateness of diabetic eye care. This was because different sets of clinical indicators were expected and the prevalence might be too low to measure the appropriateness of eye care for pregnant patients and patients with type 1 diabetes mellitus.

Protocol and sampling

The study protocol was based on the CareTrack Australia protocol and the RAND methodology.12 27 Eight clinical indicators for preventative eye care (see online supplementary appendix 1), thirteen for glaucoma eye care (see online supplementary appendix 2) and seventeen for diabetic eye care (see online supplementary appendix 3) were developed—by the investigators based on recommendations extracted from relevant published national and international clinical practice guidelines (table 1)—using the Delphi method. The purpose of the clinical indicator development was not to create new sets of clinical practice guidelines or care recommendation, but to facilitate quantitative measurement of appropriateness of eye care. This rigorous Delphi review process involved a panel of three to five nationally recognised clinical experts from the relevant fields who were invited to review and rate the clinical indicators for feasibility, acceptability and impact. Experts were identified as clinical leaders in their field and typically were employed in an eye department in a large hospital or a large teaching clinic and/or held an adjunct academic appointment. Experts were invited to comment and score the indicators for their appropriateness, in the context of eye care delivered in Australia from 2013 to 2014. Experts from both the optometry and ophthalmology field were involved in the Delphi review process for all clinical indicators.

Table 1

Evidence-based clinical practice guidelines used to develop clinical indicators

Guideline	Year	Publisher	Country
Preventative eye care
NHMRC guidelines for the Screening, Progress, Diagnosis, Management and Prevention of Glaucoma*23	2010	National Health and Medical Research Council (NHMRC)	Australia
Guidelines for the Management of Diabetic Retinopathy*22	2008	NHMRC	Australia
Canadian Ophthalmological Society evidence-based clinical practice guidelines for the periodic eye examination in adults in Canada40	2007	Canadian Ophthalmological Society (COS) guidelines	Canada
Preferred Practice Pattern guidelines. Comprehensive Adult Medical Eye Evaluation41	2010	American Academy of Ophthalmology (AAO)	USA
Glaucoma eye care
NHMRC guidelines for the Screening, Progress, Diagnosis, Management and Prevention of Glaucoma23	2010	NHMRC	Australia
Canadian Ophthalmological Society Evidence-based Clinical Practice Guidelines for the Management of Glaucoma in the Patient Eye42	2009	COS guidelines	Canada
Diagnosis and Management of Chronic Open Angle Glaucoma and Ocular Hypertension43	2009	National Institute for Health and Care Excellence (NICE)	UK
Preferred Practice Pattern Guidelines. Primary open-angle glaucoma44	2010	AAO	USA
Diabetic eye care
Guidelines for the Management of Diabetic Retinopathy22	2008	NHMRC	Australia
Canadian Ophthalmological Society evidence-based clinical practice guidelines for the management of diabetic retinopathy45	2012	COS guidelines	Canada
SIGN Management of diabetes A national clinical guideline46	2010	Scottish Intercollegiate Guidelines Network	UK

*Only the recommendations related to preventative eye care were considered.

It was initially intended that 10 records per condition per practice be reviewed; however, in practice, the number of records sampled varied based on the time available and the complexity of the onsite record review. Records of preventative care patients were first reviewed, followed by records of glaucoma and diabetic eye care patients. When a list of eligible patients or eligible visits could not be automatically generated, a range of possible sampling methods were used to identify eligible records, as appropriate. These methods varied between different eye care practice settings with the final method determined based on how the records were stored. Briefly, this involved the surveyors (KCH, DR) generating either ‘a list of patients’ who were examined at the eye care practice within the study period or ‘a list of visits’ conducted within the study period (ie, multiple visits from the same patient could be included) and taking a random sample from this generated list. For practices with EDRMS that could not generate any of the types of lists mentioned above, ‘a list of random dates’ was generated with eligible visits on those dates included in the sample pool. For practices without EDRMS where paper records were used, 10 random letters from the alphabet were generated anew each time and sampling started from the patient whose surname starts with the letter selected; sequential records were then checked until one eligible patient per letter was identified. For one practice, only records from patients who visited the practice between September 2016 and March 2017 were available for sampling due to the practice having undergone extensive renovation during the mandated sampling period and their patient records having been relocated to other premises and not being accessible. Sampling for that practice therefore occurred from the pool of patients examined in 2013 or 2014 who happened to have been re-examined in the period between September 2016 and March 2017. In instances where sampling occurred by patients and not by visits, only the first eligible visit was included. In one practice, patients with diabetic mellitus could best be identified using the Medicare item 10 915 (comprehensive consultation of more than 15 min duration for a patient with diabetes mellitus including dilation25) but this was not the case for other practices sampled.

Random sampling was achieved by using a random number generator. First, a range of number started from ‘1’ was assigned to each patient or visit between 1 January 2013 and 31 December 2014. A list of random numbers was then generated and the records with the corresponding number was sampled.

In order to facilitate the progress of this feasibility study, clinical indicators from different but advanced stages of the drafting and Delphi method review process were used for certain conditions. For preventative eye care, clinical indicators drafted based on the clinical practice guidelines but that had not undertaken Delphi expert review were used in this feasibility study. For glaucoma and diabetic eye care, streamlined clinical indicators which had been reviewed in the first but not the final round of the Delphi process by the panel of experts were used.

Data extraction

Patient age, gender, ethnicity and date of visit were extracted from each patient record selected. Records were also reviewed for appropriateness by one of two trained surveyors (KCH and DR), recording whether individual indicators and subindicators were met with ‘yes’ (care provided during the encounter was consistent with the indicator), ‘no’ (care provided during the encounter was not consistent with the indicator) or ‘not applicable’ (the indicator was not relevant to the encounter) in a secure Microsoft Excel spreadsheet.

Every effort was made to minimise the occurrence of missing data. For example, for occurrences where data for an indicator were consistently missing (eg, instrument used for intraocular pressure measurement), the practice manager or the eye care practitioner were interviewed and relevant information (eg, only a single tonometer type available in the practice) used to record answers to relevant indicators. In other instances, for example, where the practices’ EDRMS only retained a record of the last recommended recall period, information about the recommended review period could not be verified retrospectively and therefore any indicators related to recommended review period were noted as ‘not applicable’.

The extracted data from two records were cross-checked at each eye care practice to ensure the inter-reliability of the two data surveyors and any differences were resolved by discussion. Kappa score was calculated to test the level of agreement between the two surveyors. Percent appropriateness of care for each indicator was averaged across all eligible records.

Patient and public involvement

In this study, patients were not involved. All data were collected from the patient records. Ethics approval (Approval no.: HC15336) was obtained and a waiver of consent to access patient records retrospectively from eye care practices was granted. Informed consent was obtained from the eye care practices.

Results

Nine eye care practices were invited and eight agreed to participate in the feasibility study. One practice refused to participate as the practice owner did not feel comfortable giving access to the patient records. This diverse convenience sample included two franchises, one corporate, one teaching clinic, one referral clinic, two independent optometry clinics and one private ophthalmology clinic (table 2). Although all eight practices used some form of EDRMS, four of the practices predominantly used paper records.

Table 2

Characteristics of the sample eye care practices

No. of optometry practices: no. of ophthalmology practices	7: 1
Electronic document and records management system (EDRMS):
Commercially available (eg, Optomate, Sunix)	3
Bespoke	5
Record types:
Electronic record only	4
Structured paper record	2
Semistructured paper record	1
Unstructured paper record only	1
Sampling method*:
By patients	5†
By visits	1
By dates	2
No. of records sampled per practice†:
Preventative eye care, median (range)	23 (21 to 50)
Glaucoma eye care, median (range)	10 (8 to 20)
Diabetic eye care, median (range)	4 (1 to 10)
No. of practitioners sampled per practice, median (range)	3 (1 to 14)
Time taken (hours per practice), median (range)	3.5 (3 to 5.5)

*One practice was able to provide a list of all eligible patients (ie, patients with glaucoma, glaucoma suspects and patients with diabetes).

†Appropriateness of preventative eye care was assessed in three of eight practices; appropriateness of glaucoma and diabetic eye care was assessed in seven of eight practices.

The purpose of this study was to explore potential issues associated with accessing, extracting, recording and analysing clinical records. Eligible records for preventative eye care were easily identified, so time was preferentially allocated to reviewing of these over the other two conditions, to identify the optimal sampling method. As a result, a median of 23 records (range: 21 to 50 records) were reviewed for preventative eye care within the allowed time in three practices, which was more than the intended 10 records.

Patient records for glaucoma and diabetic eye care were randomly sampled using different methods among the practices to determine the best feasible sampling strategy to use in the main trial as a diversity of EDRMS and record types were used in the eye care practices sampled. One practice categorised patients based on diagnosis (eg, glaucoma and diabetes) and visit types (eg, initial and follow-up visits) and a list of eligible patients was therefore provided by the practice and could be used for random sampling. Another practice categorised patients by diagnosis and a list of eligible patients could be generated from the EDMRS for random sampling. For the other five settings, the eligibility had to be checked consecutively for each randomly sampled individual patient record, until the required number of records was found; this process proved to be much more time consuming but was necessary in a majority of sampled practices.

For preventative eye care, 10 record review was completed within 1 hour for all methods, but more time was required to identify eligible records for glaucoma and diabetic eye care. If a list of patients eligible for random sampling could not be generated, sampling by patients was the most feasible (for both paper and electronic records) and least time-consuming method of identifying the eligible records for glaucoma and diabetic eye care. Ten eligible records related to glaucoma eye care could be identified and reviewed within 1 to 1.5 hours for most methods. For diabetic eye care, up to four eligible records could be identified and reviewed within 1.5 to 2 hours, whereas only one or two eligible records could be identified if sampling occurred by visits or dates. Sampling of diabetic patients by Medicare Item 10 915 was therefore suggested as a potentially more feasible option. This was attempted in the last practice where diabetic eye care was measured for the feasibility study. The overall time spent to review the records per practice ranged from 3 to 5.5 hours.

Two hundred and thirteen records were reviewed in the feasibility study, and the characteristics of the randomly selected records (patients) are shown in table 3. Appropriateness of preventative eye care was assessed in three practices; appropriateness of glaucoma and diabetic eye care was assessed in seven practices. The glaucoma and diabetic patients randomly sampled were older and a majority of the visits reviewed for these patients occurred in 2013. Three quarters of glaucoma patients were glaucoma suspects, followed by 17% with moderate glaucoma, 7% with severe glaucoma and 1% with early glaucoma. Nearly 80% of patients with diabetes did not have diabetic retinopathy, followed by 7% with moderate diabetic retinopathy and 4% mild diabetic retinopathy.

Table 3

Characteristics of the record review’s random sample

	Preventative eye care	Glaucoma	Diabetic eye care
No. of practices sampled	3	7	7
No. of records sampled	94	82	37
Patient age (mean±SD, years)	43.1±17.6	56.2±15.2	61.6±10.6
Patient gender:
Male	39 (41%)*	44 (54%)	22 (59%)
Female	53 (56%)*	38 (46%)	15 (41%)
Year of the sampled visit:
2013	39 (41%)	53 (65%)	30 (81%)
2014	55 (59%)	29 (35%)	7 (19%)
Eye care provider:
Optometrist	94 (100%)	72 (88%)	36 (97%)
Ophthalmologist	0 (0%)	10 (12%)	1 (3%)

*Gender was missing from two records.

Records review in eye care practices often required that ‘judgement calls’ be made by trained surveyors. For example, non-standard ocular acronyms were often encountered, which required specialist knowledge in the area to decipher. A review manual was drafted and continuously updated, to devise a consistent and explicit set of rules that can be employed by trained surveyors to conduct a full record review in the future.

In this feasibility study, the overall appropriateness of preventative, glaucoma and diabetic eye care was 69% (95% CI 67% to 70%), 58% (95% CI 56% to 60%) and 61% (95% CI 55% to 66%), respectively (figure 1). Overall, preventative eye care showed the highest appropriateness, most particularly for the indicator related to history taking. Only 7 of 82 glaucoma patients reviewed met the inclusion criteria for clinical indicators related to glaucoma management (see online supplementary appendix 2, indicators 4 and 5). It was notable that none of the records reviewed in this feasibility study recorded ethnicity, a common risk factor for many eye conditions including glaucoma28 29 and diabetic retinopathy.30 Substantial inter-rater agreement between the two surveyors was shown with a kappa score equal to 0.76 (95% CI 0.74 to 0.78).

Figure 1

The percentage of appropriate eye care delivery measured by domains of care. 95% CI around the mean is displayed. Preventative eye care: 94 records from three practices were reviewed; glaucoma eye care: 82 records from seven practices reviewed; diabetic eye care: 37 records from seven practices reviewed. None of the records reviewed met the inclusion criteria for the domains of care ‘management’ for preventative and diabetic eye care and ‘referral’ for all conditions, hence these were not plotted.

Discussion

Using a small, diverse sample of eye care practices (ie, different business models, practice size, EDRMS systems, record format, professions and specialties) located in Sydney, this study demonstrated the feasibility of reviewing optometry and ophthalmology records to measure appropriateness of eye care delivery. Based on these findings, a larger, more comprehensive review of appropriateness of eye care in Australia is feasible and indicated. While indicators were developed, and appropriateness of eye care assessed for three conditions only (preventative, glaucoma and diabetic eye care), there were no indications from this study that feasibility was condition specific.

Given that most of the EDRMS were not designed for record review and diagnostic coding was not commonly used, identification of eligible records was challenging. Different sampling strategies were used based on how the records were stored and the EDRMS design. The most feasible and efficient sampling method that could be used in a majority of practices was to sample randomly by patient. Other recommendations derived from this feasibility study include the use of surveyors with an eye care background, and a time allocation of 4 to 5 hours per practice for measurement of appropriateness of eye care in three conditions.

A very low proportion of patients with diabetes were found in the eye care practices that were selected for this feasibility study. This may be partially attributed to their location as the prevalence of diabetes is very low in many eastern suburbs of Sydney, http://www.diabetesmap.com.au/#/, to the patients’ profile in the sampled practices and to the specialty of the practitioners in these practices. Some practitioners also may have only recorded the presence of disease (eg, the patient having diabetes mellitus) rather than documenting both absence and presence of all relevant diseases covered during the history taking. Patients with an established diagnosis of diabetes mellitus may also be more likely to consult a secondary care specialist in diabetic eye diseases such as a medical retina ophthalmologist. Some of these patients might also have had other health conditions which required hospital care and prevented them from attending primary eye care practices.

The appropriateness of preventative, glaucoma and diabetic eye care found in this feasibility pilot study was in line with the overall appropriateness of other healthcare previously measured in the CareTrack Study.13 Regular diabetic eye checks were recommended 89% of the time in this study, a slightly higher frequency than that of 78% found by the NEHS.14

Despite evidence showing ethnicity as a risk factor for a multitude of ocular disease,31–33 none of the eight settings audited in this feasibility study systematically recorded ethnicity. Although further investigations are needed to determine whether ethnicity might factor in care decisions without being recorded, it should be noted that criticisms exist regarding the use of ethnicity as a risk factor.34–36

The process of conducting a record review and the reporting of findings to individual practices following the study completion appeared to, at times, induce some beneficial changes. These anecdotal findings originated from a single practice where the following changes were reported post study: (1) modification of record form plus staff training to reinforce the importance of addressing the chief complaint and history taking of driving status; and (2) addition of diagnostic coding for new patients with diabetes to improve patient management and promote a strong care evaluation culture.

Strengths and limitations

Important questions were answered with this feasibility study such as the sufficiency of the number of eligible patients, data collection time and preferred sampling strategy. Since the purpose of this study was to test the feasibility of reviewing patient records from Australian eye care practices to assess the appropriateness of eye care delivery, eye care practices with diverse settings were audited using convenience sampling, but this may have limited the generalisability of the results. In addition, all sampled practices were located in Sydney for convenience purposes; the appropriateness of eye care measured may thus not be representative of the rest of Australia. The majority of the patients sampled in this feasibility study were glaucoma suspect or diabetics without diabetic retinopathy and the appropriateness of eye care measured for these two conditions may therefore not reflect that of more severe presentations of disease. The unequal number of records obtained from each practice may skew the results towards the performance of the practice or practices with the most records reviewed. Finally, the indicators used were still in development and it is therefore possible that any significant changes to the indicators that occurred in the final round of review may affect the validity of these results.

Conclusion

This study demonstrated that appropriateness of eye care can be measured using record review of eye care practices. Variations from best eye care practices were identified in this feasibility study; however, this was in line with care previously measured for other health conditions. Different sampling strategies were tested to cater for the diverse nature of eye care practices, EDRMS and record types and random sampling proved to be at times challenging and time consuming. It is recommended that the personnel involved in extracting data from the clinical records should have eye care backgrounds to be able to make ‘judgement calls’ during the review process.