Impact of accreditation on documentation and staff perception in the ophthalmology department of an Indian medical college.

Purpose: National Accreditation Board for Hospitals and Healthcare Providers operates the health-care accreditation program in India. Research on impact of accreditation on eye-care centers is scarce. This article was conceptualized to scientifically evaluate the changes in documentation brought about by accreditation and its effects on staff in the Ophthalmology Department of an Indian Medical College.
Methods: This was conducted as a quasi-experimental study in four steps. First, a point-based evaluation of case sheets in both pre-accreditation and postaccreditation phase, along with statistical analysis of the results, was done. Step two involved analysis of quality indicators and patient safety indices in successive years from inception of data to post-accreditation phase. Step three comprised a survey of staff employing a validated tool, and finally, face-to-face semistructured interviews with designated authorities, including finance departmental head, completed the study.
Results: A statistically significant difference was seen in scores achieved by the pre and postaccreditation case sheets, with the postphase case sheets achieving 15% increased scores over the prephase case sheets. Quality indicator indices displayed improvements post-accreditation. There was an accompanying increase in quantity of documentation. Financial data analysis showed increased expenditure for accreditation under multiple heads. Staff believed that accreditation led to increased workload but did not express decreased satisfaction and felt that accreditation was eventually beneficial. However, staff believed rewards for improving quality can be enhanced.
Conclusion: Accreditation increases quality and quantity of documentation, and staff workload. Increased financial costs also ensue.Staff believe that accreditation improves quality, is beneficial, but desire enhanced rewards.

Accreditation has been defined by the World Health Organization as a comprehensive evaluation of the key systems that make up a health-care establishment and is an increasingly projected method for enhancing quality at the health-care delivery level.[12] Developed countries, especially in the western hemisphere, were early adopters of health-care accreditation.[345] Developing countries such as India and many Asian countries have started aggressively promoting accreditation in the past decade.[678] There have been multiple publications on the impact of accreditation on health care organizations,[891011] but research on the effect of accreditation in ophthalmology, particularly in a teaching institution, is sparse. This article was conceived with the objective of evaluating the impact of National Accreditation Board for Hospitals and Healthcare Providers (NABH) accreditation on documentation in the ophthalmology department of a teaching hospital at an Indian medical college. The study included a survey of all staff in the department with a validated questionnaire and interviews were conducted with relevant authorities to obtain a “human” perspective in this project.

Methods

This study was conducted in a medical college hospital in South India, over a period of 7 months between May and December 2019. Institute Ethics Committee approval was obtained before the commencement of the study.

The study was designed as a quasi-experimental study and conducted in four steps:

(1) Comparison of clinical document (in-patient [IP] case sheets) compliance in the pre- and post-accreditation phase

(2) Comparison of Quality indicator registers and Patient safety data indices in the pre- and post-accreditation phase

(3) Survey of staff employing a validated survey tool

(4) Face-to-face semistructured interviews with designated authorities

The first step done was an analysis of case sheets of pre-accreditation phase (pre-phase). The pre-phase was taken as July to September 2014 as the decision to get institutional accreditation with NABH was taken in October 2014, and the quality management services (QMS) department was established soon thereafter.

A systematic sampling of every fourth case sheet of the pre-phase in the medical records department was performed, and each case sheet was analyzed and scored by the authors on a points system, as detailed in Table 1.

Table 1

Results of point-based evaluation of pre-phase and post-phase case sheets

S. no.	Documentation field	Maximum points	Pre-phase case sheets mean score (±SD)	Post-phase case sheets mean score (±SD)	P (Student’s t-test; <0.05 significant)
1	Chief complaints mentioned with duration	2	1.97±0.19	1.98±0.14	0.4853
2	History of each complaint present	2	1.97±0.2	1.95±0.24	0.311
3	Past history (including systemic history)	1	0.67±0.47	0.98±0.14	0.0001
4	General physical examination	1	0.13±0.34	0.21±0.41	0.0177
5	Visual acuity examination with refraction	2	1.05±0.23	1.56±0.5	0.0001
6	Anterior segment examination	2	1.98±0.13	1.98±0.14	1
7	Posterior segment examination	2	1.96±0.27	1.96±0.19	1
8	External examination	1	0.02±0.15	0.93±0.25	0.0001
9	Adnexal examination	1	0.98±0.13	0.94±0.24	0.0245
10	Appropriate diagnosis matching examination findings	2	1.89±0.32	1.97±0.17	0.0002
11	Outline of the management plan	1	0.76±0.43	0.96±0.19	0.0001
12	Medication prescription record	1	0.99±0.12	1±0.0	0.1434
13	Discharge summary	2	1.48±0.51	1.98±0.14	0.0001
14	Investigations for surgery	2	2±0.27	2±0.23	1
15	Preoperative instructions to the patient	2	2±0.27	2±0.23	1
16	Consent form completion	2	1±0.13	1.8±0.45	0.0001
17	Operative notes	2	2±0.27	1.97±0.28	0.217
18	Postoperative evaluation notes	2	1.47±0.53	1.93±0.34	0.0001
	Average score of case sheets	30	24.15±1.87	27.98±1.54	0.0001

One point was scored for a two-point field if documentation of the field was present but incomplete. Zero points were given if documentation of the field was absent. Case sheets of patients where surgery was not done (n=4 in both groups), was not scored for serial numbers 14–18, and mean score in these fields excludes such case sheets. SD: Standard deviation

The maximum possible score was 30 for operated patients and 20 for nonoperated patients.

After completion of the pre-phase case-sheet analysis, similar point-based evaluation of post-accreditation (post-phase) case sheets was done. The post-phase was taken as July–September 2018. This was because Devkaran and O'Farrell had shown that compliance of clinical documentation to standards is higher within 3–6 months of reinspection for accreditation,[12] so we avoided the last 6 months of the final year of accreditation validity to acquire a more accurate picture of document compliance in the post-phase (NABH accreditation was valid from April 16, 2016 to April 15, 2019.)

The second limb of the study involved a year-on-year comparative study of the quality indicator indices and patient safety data indices. Documentation of the quality indicator indices was begun only from October 2015, and the patient safety data were available only from March 2016. Hence, we (the authors) collected and compared the data in the quality indicators registers and patient safety data indices from the first 3 months of availability and the same 3 months for the succeeding 3 years. The quality indicator registers inculcated data on the total number of cases posted, total cases performed, number of cases cancelled, unplanned return to the operation theater (OT), rescheduled cases, re-exploration cases, planned surgery changed intraoperatively, surgical safety checklist completion, and OT utilization hours. Patient safety data included data on percentage of hand hygiene compliance, percentage of medication errors, incidence of falls (unplanned descent to the floor with or without injury to the patient), extravasations, incidence of bedsores after admission, incidence of needlestick injuries, and surgical site infection rate.

Next, a self-administered survey tool was given to all doctors, nursing staff, and technical staff working in the department of ophthalmology for at least three consecutive years between the pre- and post-phase period [Appendix 1]. The questionnaire, available as open access from the World Wide Web, had been validated and published by El-Jardali et al.[13] and originally contained 54 items under 7 scales and elicited demographic information separately. We omitted the scales on leadership, commitment, and support (nine items) and strategic quality planning (seven items) as these were not within the scope of our study. We also avoided the elicitation of demographic information. Three new items were added to elicit information on staff satisfaction. Our final survey tool contained 32 items under 6 scales and assessed staff perceptions on quality results (3 items), human resources utilization (5 items), quality management (4 items), use of data (6 items), staff involvement in and benefits of accreditation (11 items), and staff satisfaction (3 items). A pilot study was conducted on 10 staff not involved in this study for validating the modified final questionnaire. The questionnaire was distributed by an independent case worker and did not solicit any type of identification data such as name/date of birth. Staff was requested to only circle numbers on a 5-point Likert scale, as responses. Completed forms were returned to the case worker mentioned above, who maintained confidentiality.

The fourth limb of the study involved content validated semistructured open-ended face-to-face interviews with heads of departments of QMS, human resources, facility management/services, and finance. Each head was asked to describe his/her experience during the process of preparing for accreditation and for any inputs on the changes which occurred secondary to accreditation.

Statistical analysis

All the data were entered into MS Excel version 2010 and were analyzed using SPSS version 23. For descriptive analysis, data were presented in percentages. For inferential analysis, Student's t-test was used and P value less than 0.05 was taken as significant.

Results

A total of 891 IP admissions were done in the 3-month period of July–September, 2014, and 1237 admissions were done between July and September, 2018. The relatively high number of admissions was because the institute conducted peripheral high-volume cataract out-patient camps twice a week, and all selected patients were brought back to the base hospital for admission and surgery. By systematic sampling, 222 pre-phase and 309 post-phase case sheets were analyzed, respectively.

A visible quantitative difference was present between the pre- and post-phase case sheets with regard to the average number of pages present. Case sheets of 2014 contained 20–22 sheets, while case sheets in 2018 had 30–32 sheets. The increased number of sheets in the post-phase was due to the inclusion of new pages containing checklist for assembling and deficiency of papers in the case sheet, preoperative instruction sheet, OT checklist, surgical safety checklist, nursing assessment record (Ontario-modified fall risk scoring), nutrition assessment record, activity card (for blood bank/other procedures), and patient and family information and education record.

The results of the point-based evaluation of the case sheets of pre- and post-phase are as outlined in Table 1. The mean score of pre-phase case sheets came to 24.15 while it was 27.98 for post-phase case sheets (P value 0.0001). It can be seen from the table that the chief differences between the case sheets in pre- and post-phase were an increased score for post-phase case sheets in each of past/systemic history, visual acuity with refraction, external examination, discharge summary, consent forms, and postoperative notes (P value of each being 0.0001).

The results of the analysis of the quality-indicator indices in the quality-indicator registers are as outlined in Tables 2 and 3. As the years passed, there were more number of cases performed, and a lesser number of cancelled and rescheduled cases. There was a marginal increase in utilization rate of the OT.

Table 2

Quality indicator indices—1

Period	Total cases posted	Total cases done	Number of cases cancelled	Unplanned return to operation theater	Rescheduled cases	Reexploration cases	Planned surgery changed intraoperatively	Surgical safety checklist
2015, Oct-Dec	657	638 (97.2%)	19 (2.8%)	2 (0.3%)	10 (1.5%)	0	0	657 (100%)
2016, Oct-Dec	845	836 (99%)	9 (1%)	0	1 (0.1%)	0	0	845 (100%)
2017, Oct-Dec	833	824 (99%)	9 (1%)	0	0	0	0	833 (100%)
2018, Oct-Dec	830	822 (99%)	8 (1%)	0	1 (0.1%)	0	0	830 (100%)

Table 3

Quality indicator indices—2 (operation theater utilization hours)

Period	Available hours	Utilized hours	Percentage
2015, Oct-Dec	600	238	39.7
2016, Oct-Dec	624	298	47.8
2017, Oct-Dec	600	293	48.8
2018, Oct-Dec	600	303	50.5

Oct: October; Dec: December

Table 4 contains the results of the comparative study of the patient safety data indices. These indices displayed no significant changes, as most of the parameters measured had values around or near zero.

Table 4

Patient safety data indices

Period (number of in-patients)	Hand hygiene compliance	Incidence of falls	Incidence of medication errors	Extravasations	Incidence of bedsores	Needlestick injuries	Surgical site infection rate
2016, Mar-May (864)	Not available	0	0	0	0	0	0
2017, Mar-May (983)	88%	2	0	3	0	0	0
2018, Mar-May (855)	83.6%	0	0	0	0	0	0
2019, Mar-May (1134)	84.5%	0	0	0	0	0	0

Numbers in the boxes indicate the number of incidents, except for hand hygiene, which is denoted in percentage of compliance, Mar: March

Twenty-two employees fulfilled this study's eligibility criteria, comprising four doctors, two out-patient department (OPD) nurses, six ward nurses, five OT nurses, three technicians (who also handled patient movement between OT and wards, and inside OT), one refractionist, and one optician. Twenty-one (96%) completed responses were received by the independent case worker responsible for distribution and collection of the survey form. The results of the survey are outlined in Appendix 2. Majority of the respondents were in agreement with improvement in quality results and management and also were positive about the benefits of accreditation. The one item with the highest disagreement was the one regarding rewards and recognition for improving quality. Nearly half of the respondents did not agree that nurses were rewarded and recognized for improving quality.

The first interview we conducted was with the head of QMS. She mentioned that she received full support from the hospital management during implementation and execution of the measures for accreditation and did not face any financial issues. According to her, documentation before accreditation was present but “quite haphazard and unorganised.” She added that, in her opinion, planning on fire control and safety was absent in the pre-accreditation period.

The head of the human resources department mentioned that many changes happened in the department, especially concerning documents and files. He also told us that, initially, some amount of resistance was encountered to accreditation, and training of staff “was a challenge.” He felt that there was a definite improvement in house-keeping and cleanliness due to NABH accreditation, and mandatory yearly health check-up was a useful value-added service, which was implemented as a necessity for accreditation.

The head of facility management and services told us that there were no structural changes which needed to be done to the building and rooms housing the ophthalmology OPD, ward, and OT, but his department encountered “significant increase in paper-work and filling forms” and “increased expenditure.” He did not agree that planning for fire safety was absent prior to accreditation.

The head of the finance department provided us with a sheet containing information on “Expenses incurred for NABH activities in 2014-15,” including the anticipated expenditure for the next five financial years, toward accreditation activities in the institute. A total of nearly 3.4 crore rupees (half a million US dollars, grossly) was spent in the initial year as expenditure, specifically toward accreditation. This expenditure was for the whole 1000-bedded hospital and not specifically for the ophthalmology department, but 58.3% of these costs was on recurring expenses, including periodic certification and licensing, printing and legal/processing charges (6.2%), extra salaries and wages (20.0%, mainly because a new department, the QMS, had to be established), and house-keeping man-power services (15.9%). The nonrecurring expense of 41.7% was on furniture and fixtures.

Finally, we discussed the findings of our study with the medical director of the institution to get the perspective of the management. The director was of the opinion that accreditation has widened knowledge of quality standards and has contributed significantly in upgrading staff skills to provide quality care. According to him, the largest part of expenditure was related to fire compliance, which he mentioned as mandatory regardless of accreditation. He also stated that employees must certainly be recognized for exceptional performance and in the institution this is done by rewards in the form of a monthly quality champion award.

Discussion

Numerous authors have investigated the impact of accreditation and have used various methods to do so.[891011141516] Similar studies from India have been scant. The government of India has established the NABH for accreditation of health-care organizations, but accreditation is voluntary, not compulsory. Despite the increasing adoption of accreditation by hospitals across India, Issue 1 of AIOS Times, the bulletin of All India Ophthalmological Society, contained articles titled “NABH Accreditation – Is it worth the trouble?” and “NABH in Eye care – Boon or bane”.[1718] These articles were subjective opinions of experienced ophthalmologists. We formulated an objective clinical and reproducible method to evaluate the impact of accreditation. This can be replicated even in the currently prevalent era of day-care ophthalmic surgery and digital (electronic) storage of records.[1920] While this study has focused on the Department of Ophthalmology, we believe the results and inferences are applicable to all departments in a health-care institute.

There was a statistically significant difference between the mean scores attained by case sheets in the pre-phase and the post-phase period. On average, a post-phase case sheet scored around 15% more than a pre-phase case sheet. One prime reason why this was so was because the post-phase case sheets had a specially prepared “cataract case-sheet” with space provided under headlines such as chief complaints, associations, past history, personal history, systemic examination, local examination, head posture, facial symmetry, extraocular movements, visual acuity, acuity with pinhole, acuity with glasses, anterior segment, and fundus, among others [Supplementary Figs. 1 and 2]. Anyone writing the history and completing the examination findings had to complete each head, thus creating a stimulus for increasing compliance in the case sheet. This pre-printed case sheet had been introduced by the QMS, with the express intention of improving document compliance. This stimulant was absent in the pre-phase case sheets, each of which had a general history sheet, which was applicable for all departments across the hospital. Hence, it would be more probable that a resident or worker completing a case sheet would miss writing some ophthalmic information. When we analyze the items in Table 1, we can see that the mean scores for past history and external examination are significantly lesser in the pre-phase. Most of the pre-phase case sheets had no mention of external examination findings. The vast majority of pre-phase case sheets, especially when admitted for cataract, had visual acuity mentioned but no mention of refraction, leading to a lower mean score in pre-phase case sheets for vision and refraction.

In actuality, these changes in the document were not necessarily mandated by accreditation. We can infer that once the institute made the decision to get accredited, improved compliance with documentation was taken as a priority, and the idea of pre-printed case sheets with headings was implemented.

Completed consent forms for surgery were found in all sampled case sheets of both phases, wherever the patient had undergone surgery. However, there existed one matter of concern in the pre-phase consent forms. The pre-phase consent form had no mention of surgeon sign at all. Consequently, all pre-phase case sheets had completed consent forms, but none had signs of the operating surgeon. In the post-phase, all consent forms were completed, with signatures present of both patient and surgeon. Here again, it is not that accreditation guidelines alone mention the need for the surgeon's signature on the consent form. We infer that due thought was given to the legitimacy of each document once the decision to go in for accreditation was made.

In both phases, all sampled case sheets in which the patient had undergone surgery contained completed operative notes, postoperative evaluation notes, and discharge summaries. But in the pre-phase period, each of these had to be manually entered in blank progress note sheets, whereas in the post-phase case sheets, there was a pre-printed operative note sheet [Supplementary Figs. 3 and 4]. A resident or a surgeon simply had to mark the correct choice on the sheet. For example, under the heading of “Conjunctival Flap,” one of “Fornix based/Limbal based/No flap” would need to be marked, or under the heading of “Incision,” one of “Clear corneal/Limbal/Scleral” would need to be marked. An accreditation board does not provide guidelines on what the format of a document should be—it just checks if the document is present and completed. By implementing these changes, the institution ensured improved compliance with documentation.

Unlike the case sheets, collection of data on quality indicators and patient safety began only after a decision was made to get accredited. NABH guidelines mandate the collection of data for ascertaining these indices, but it can also be construed that the institute put in increased thought to improve quality and safety and began data collection on these parameters. As can be seen from Table 2, the number of cancelled or rescheduled cases was more in the first 3 months of data collection. In the later years, these parameters were a fraction of the first year. By deciding to go in for accreditation, the institution put in more thought to collection and compliance of data, which resulted in increased quality of documentation.

While these changes improved the quality of documentation, they were accompanied by an increase in the quantity of documentation. Increase in amount of “paper-work” was not just in the therapeutic domain alone but also for nurses and medical record departmental staff, as can be gleaned from the names of some of the new documents introduced, such as nursing assessment record, nutrition assessment record, activity card (for blood bank/other procedures), patient and family information, and education record (which contained signatures of patients, nurses, and doctors) and checklist for assembling and deficiency of papers in the case sheet (which was completed by medical records staff after discharge of the patient).

It would be obvious that the increased documentation would lead to increased work for the staff. What was the impact of this on employees? Our survey provided us with insightful information regarding the attitude of staff to the increased work. Every single respondent (100%) agreed that the department encourages nurses to keep records of quality problems through documentation, and 76% agreed that accreditation increases the workload of the employee. Despite this, more than half the respondents (57%) agreed or strongly agreed that accreditation increased staff satisfaction at the workplace and only one response (5%) displayed a contrary opinion. Most of the respondents agreed with improvement in quality results and quality management. The majority were also positive about the benefits of accreditation. Our interpretation of these results is that while accreditation increases documentation and workload of staff, and may encounter initial resistance, in the long term it does not cause significant staff dissatisfaction as it also enhances quality and is perceived as beneficial as a whole. A notable point was the item with the least number of agreements—only ten (47%) agreed that nurses are rewarded and recognized, financially and/or otherwise, for improving quality, and seven (33%) disagreed with this. We construe that staff are ready to take on the increased workload in the interests of increased quality and patient benefit but would appreciate increased rewards and/or recognition for the augmented workload.

When a health-care institution decides to get accredited, increased attention to documentation and compliance of documentation ensues. Collection of various data across different fields of quality and safety is mandated. This enhanced attention to improving document compliance, quality, and safety achieves its results but comes at an increased financial cost and an increased workload for staff. If enhanced quality and safety are taken as nonnegotiable goals in health care, then this trade of increased quality and safety for increased work and cost is worth it.

Here, we will introduce the subjective opinion of the head of facility management. In his opinion, and we quote him verbatim, “If NABH work can be done without NABH accreditation, it would be better! But it is not possible.” Our final words would be a modified resonation of the quote—can the results achieved by accreditation be achieved without going in for accreditation?

Conclusion

Accreditation increases the quality of clinical documentation. It also increases the quantity of clinical documentation. There is an increased expenditure when an organization decides to get accredited. Employees surveyed in this study believed that accreditation improves the quality of care and services and increases workload of the employee. The increased workload has not significantly reduced staff satisfaction in this study, but nearly half the surveyed staff feel that rewards for improving quality can be enhanced.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Cataract case sheet page 1

Cataract case sheet page 2

Cataract operative note sheet page 1

Cataract operative note sheet page 2