| Literature DB >> 31818835 |
Feng Zhao1, Olena Doroshenko2, Valery N Lekhan3, Lilia V Kriachkova3, Alona Goroshko4.
Abstract
OBJECTIVES: This article reviews the applicability of a customised version of the Appropriateness Evaluation Protocol (AEP) to evaluate the magnitude of inappropriate hospitalisations in two regions of Ukraine. DATA AND METHODS: The original AEP was modified to develop a customised tool, which included criteria for the appropriateness of hospitalisation and duration of inpatient stay. The customisation of the tool followed the Delphi procedure. We randomly selected 381 medical records to test the feasibility and reliability of the method and 800 medical records to evaluate the scope of inappropriate hospitalisations. We used descriptive and analytical statistics, receiver operating characteristic curve analysis and Cohen's kappa to check the consistency between the findings of primary reviewers and experts. RESULT: We observed high levels of agreement in conclusions of primary reviewers (reference standard) and experts during testing of the reliability and validity of the method. The external validity check showed that the use of the tool by different experts provided high accuracy: 95.1 sensitivity, 76.6 specificity and area under ROC-curve (AUC)=0.948 (р<0.001) for analysis of the appropriateness of admissions; 95.3 sensitivity, 84.7 specificity and AUC=0.900 (р=0.001) for the duration of hospitalisations. Cohen's kappa coefficient (κ) indicated agreement in expert evaluations of 0.915 (95% СІ 0.799 to 1.000) and 0.812 (95% СІ 0.749 to 0.875), respectively.We found that over one-third of admissions (38.1%; 95% СІ 33.9 to 43.5) and over half of total bed-days were unnecessary (57.4%; 95% СІ 56.4 to 58.5). The highest levels of stay were observed in hospitals' general medicine departments (64.6%; 95% СІ 63.0 to 66.3)compared with other departments included in the analysis.Entities:
Keywords: Ukraine version appropriateness evaluation protocol (AEP); avoidable hospitalizations; hospital admissions; reliability and validity of the method; unjustified inpatient stay
Year: 2019 PMID: 31818835 PMCID: PMC6924815 DOI: 10.1136/bmjopen-2019-030081
Source DB: PubMed Journal: BMJ Open ISSN: 2044-6055 Impact factor: 2.692
Figure 1Schematic presentation of the steps followed during the development and evaluation of the UA-AEP method.
Inter-rater agreement of experts’ evaluations and operational characteristics of the method for assessment of the appropriateness of hospitalisations
| Statistics | General medicine n=100 | Cardiology n=70 | Neurology n=119 | Surgery n=92 | Total pilot sample n=381 |
|
| |||||
| Sensitivity (%) | 84.8 | 96.3 | 96.4 | 97.3 | 95.1 |
| Specificity (%) | 73.1 | 87.5 | 68.5 | 94.7 | 76.6 |
| Area under ROC curve (рvalue) | 0.865(0.002) | 0.977 (<0.001) | 0.945 (<0.001) | 0.991 (<0.001) | 0.948 (<0.001) |
| Overall agreement (%) | 84.0 | 94.2 | 88.2 | 90.5 | 92.8 |
| Cohen’s κ(95% CI) | 0.702 | 0.916 | 0.888 | 0.934 | 0.915 |
|
| |||||
| Sensitivity (%) | 88.4 | 96.3 | 96.6 | 97.2 | 95.3 |
| Specificity (%) | 85.9 | 87.5 | 75.0 | 94.4 | 84.7 |
| Area under ROC curve (рvalue) | 0.870(0.009) | 0.920 (<0.001) | 0.860(0.007) | 0.960 (<0.001) | 0.900(0.001) |
| Overall agreement (%) | 86.7 | 91.9 | 90.1 | 94.3 | 91.2 |
| Cohen’s κ(95% CI) | 0.737 | 0.838 | 0.772 | 0.902 | 0.812 |
ROC, receiver operating characteristic.
Figure 2Shares of inappropriate admissions by department type, total and in the two regions of Ukraine (%, 95% СІ).*р<0.001 compared with the general medicine department.
Volumes of inappropriate hospitalisations
| Characteristic of the sample | Number of inappropriate admissions | Total bed-days for inappropriate hospitalisations | Average length of stay in bed-days (95% СІ) | Unnecessary days of inpatient stay, |
| Total for two regions | ||||
| By facility level | ||||
| Secondary care | 217 | 2255 | 7.1 (6.9to 7.2) | 58.8 (57.5to 60.1) |
| Tertiary care | 88 | 1274 | 8.2 (7.8to 8.5) | 54.8 (52.9to 56.6)* |
| Both levels | 305 | 3530 | 7.4 (7.3to 7.5) | 57.4 (56.4to 58.5) |
| By speciality of departments | ||||
| Neurology | 79 | 1123 | 7.9 (7.7to 8.0) | 54.6 (52.6 to 56.5)† |
| General medicine | 151 | 1083 | 8.8 (8.6to 9.0) | 64.6 (63.0to 66.3) |
| Surgery | 54 | 838 | 5.9 (5.7to 6.0) | 52.4 (50 to 54.7)† |
| Cardiology | 21 | 486 | 7.3 (7.0to 7.5) | 51.2 (48.1 to 54.3)† |
| Central region | ||||
| By facility level | ||||
| Secondary care | 105 | 1252 | 8.2 (8to 8.4) | 50.6 (48.7 to 52.6)‡ |
| Tertiary care | 49 | 740 | 8.5 (8.3to 8.7) | 53.4 (51.0to 55.9) |
| Both levels | 154 | 1993 | 8.3 (8.2to 8.4) | 51.7 (50.1 to 53.2)‡ |
| By speciality of departments | ||||
| Neurology | 79 | 1123 | 7.9 (7.7to 8.0) | 55.0 (51.2to 58.8) |
| General medicine | 151 | 1083 | 8.8 (8.6to 9.0) | 58.4 (56.1 to 60.7)‡ |
| Surgery | 54 | 838 | 5.9 (5.7to6.0) | 41.9 (38.9 to 44.9)†‡ |
| Cardiology | 21 | 486 | 7.3 (7.0to 7.5) | 46.6 (42.9 to 50.2)†‡ |
| Western region | ||||
| By facility level | ||||
| Secondary care | 112 | 1002 | 6.0 (5.7to 6.3) | 65.9 (64.2to 67.6) |
| Tertiary care | 39 | 533 | 7.7 (6.6to 8.8) | 56.6 (53.8to 59.4)* |
| Both levels | 151 | 1535 | 6.5 (6.3to 6.7) | 63.1 (61.7to 64.6) |
| By speciality of departments | ||||
| Neurology | 58 | 830 | 7.8 (7.5to 8.2) | 54.4 (52.1 to 56.7)† |
| General medicine | 59 | 366 | 7.8 (7.3to 8.3) | 72.7 (70.3to 75.1) |
| Surgery | 27 | 237 | 3.8 (3.4to 4.3) | 67.4 (63.9to 70.8) |
| Cardiology | 7 | 102 | 4.9 (4.1to 5.6) | 63.0 (57.3 to 68.7)† |
*р≤0.001 compared with the secondary level facilities.
†р<0.001 compared with the general medicine departments.
‡р<0.001 compared with the other region.