Infusion phlebitis assessment measures: a systematic review.

RATIONALE, AIMS AND
OBJECTIVES: Phlebitis is a common and painful complication of peripheral intravenous cannulation. The aim of this review was to identify the measures used in infusion phlebitis assessment and evaluate evidence regarding their reliability, validity, responsiveness and feasibility.
METHOD: We conducted a systematic literature review of the Cochrane library, Ovid MEDLINE and EBSCO CINAHL until September 2013. All English-language studies (randomized controlled trials, prospective cohort and cross-sectional) that used an infusion phlebitis scale were retrieved and analysed to determine which symptoms were included in each scale and how these were measured. We evaluated studies that reported testing the psychometric properties of phlebitis assessment scales using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines.
RESULTS: Infusion phlebitis was the primary outcome measure in 233 studies. Fifty-three (23%) of these provided no actual definition of phlebitis. Of the 180 studies that reported measuring phlebitis incidence and/or severity, 101 (56%) used a scale and 79 (44%) used a definition alone. We identified 71 different phlebitis assessment scales. Three scales had undergone some psychometric analyses, but no scale had been rigorously tested.
CONCLUSION: Many phlebitis scales exist, but none has been thoroughly validated for use in clinical practice. A lack of consensus on phlebitis measures has likely contributed to disparities in reported phlebitis incidence, precluding meaningful comparison of phlebitis rates.

Introduction

The insertion of a peripheral intravenous cannula (PIVC) for intravenous (IV) fluids and medications is the most common procedure in hospitalized patients worldwide 1. A frequent PIVC complication is phlebitis, that is, inflammation of the vein, which may be mechanical, chemical or bacterial in origin 2,3. Phlebitis causes a cascade of unwelcome repercussions – significant pain, failure of the PIVC, interruption to prescribed therapy and requirement for insertion of a new PIVC with associated increased equipment costs and staff time. Phlebitis compromises future venous access 4, and untreated bacterial phlebitis may lead to bloodstream infection 5; therefore, early detection of complications and removal of the PIVC is crucial.

Phlebitis may be localized to the insertion site or travel along the vein. If extravasation (also called infiltration) of fluids in the interstitial space occurs 6, oedema may prevent recognition of phlebitis symptoms, such as induration (hardened tissue), because of difficulty in palpating the vein. Phlebitis may occur during catheterization or up to 48 hours after removal 7.

This systematic review sought to address the following questions:

Which diagnostic criteria are used to determine infusion phlebitis in the clinical setting?

Do any existing infusion phlebitis assessment scales have strong measurement properties, including reliability, validity, responsiveness and feasibility?

This review is intended to inform clinicians about existing methods of phlebitis assessment, based on evidence of the measurement quality of existing assessment scales.

Methods

We searched the Cochrane library, Ovid MEDLINE and EBSCO CINAHL for research articles in English, using the following search terms: infusion phlebitis; thrombophlebitis; peripheral IV catheter; phlebitis score; phlebitis grade; and phlebitis assessment. Research studies (randomized controlled trials, prospective cohort and cross-sectional) that reported phlebitis incidence in adult patients with PIVCs or that evaluated a phlebitis scale were included. No date limitations were applied, with citations published until September 2013 included. Titles and abstracts were initially screened for relevance. Full texts of potentially relevant articles were obtained and evaluated for inclusion. The reference lists of these articles were checked for other studies of potential relevance, and these were also retrieved.

All articles that examined infusion phlebitis assessment in adults as a primary outcome measure were retrieved, but only those that used a phlebitis assessment scale were included in the final review. Each scale was examined to identify which signs and symptoms were included in the measurement of phlebitis. Figure 1 illustrates the study selection process. The role of the phlebitis assessor, how often assessment was performed and if training in phlebitis assessment had been provided were noted. Information regarding each scale's psychometric properties, if provided, was also recorded.

Figure 1

Process of selecting studies for the review.

This review used definitions of measurement properties and parameters consistent with those provided by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) 8,9. Relevant measurement properties for phlebitis assessment include reliability (inter-rater, intra-rater, test–retest), validity (content, face, criterion, construct) and responsiveness. Because phlebitis scales are formative indexes rather than reflective scales 10,11, neither internal consistency nor structural validity is relevant.

In addition, our review also considered attributes associated with excellence in clinimetrics. Feinstein's 12 approach to developing clinical assessment tools, especially relevant for formative indexes like phlebitis scales, was taken, including evaluation of the ‘sensibility’ of clinical instruments. Sensibility includes several properties covered in COSMIN (e.g. content validity, responsiveness), but also includes acceptability and feasibility, that is, ease of practical application of clinical instruments. Feasibility takes into consideration such issues as length of time to complete the scale, ease of administration and clarity of the items and instructions 12.

Results

Although phlebitis incidence related to PIVCs was reportedly measured in 233 studies, 53 (23%) articles did not provide any definition of phlebitis. Of the 180 studies that described the method of phlebitis assessment, 101 (56%) reported using a scale and 79 (44%) used a definition alone. Seventy-one phlebitis assessment scales including 15 symptoms were identified.

The 15 symptoms included in phlebitis assessment scales were pain, tenderness, erythema or redness, oedema or swelling, palpable venous cord, induration or hardness, frank thrombosis, streak formation or red line, purulence or exudate, local warmth, local coolness, infusion slowed or stopped, fever or pyrexia, tissue damage and impaired function. The prevalence of these symptoms captured in phlebitis assessment scales is shown in Fig. 2.

Figure 2

Frequency of symptoms reported in 71 phlebitis scales.

Phlebitis assessment scales

Large disparities were found among the 71 phlebitis assessment scales. Some authors used a previously published scale; others modified an existing tool or created their own. When a published tool such as the Visual Infusion Phlebitis (VIP) 13,14, Infusion Nurses Society (INS) 15–18, Maddox 19,20, Baxter 21, Lipman 22 or Dinley 23 scale was used, many authors did not state which version they had used, despite wide variations between different versions. Other authors did not report the source of their scale at all.

Assigning a phlebitis assessment score or grade was commonly performed in one of two ways. Phlebitis scores were either cumulative (assigning points for each symptom and adding them up) or progressive (based on more points for a specified progression of symptoms). Cumulative scales scored 0–2 points for each phlebitis symptom, depending on the presence, measured length (in centimetres), or severity, and their total potential scores ranged from 0–6 to 0–7, to 0–9 and to 4–16. Total phlebitis grading also varied considerably for progressive scales, ranging from 0–2 to 0–6.

The symptoms required for phlebitis varied considerably. Only erythema was reported as a phlebitis symptom in every scale. Several authors scored patients as positive for phlebitis with the finding of pain alone 24–28, erythema alone 29,30 or either 31–34. Some authors considered a palpable venous cord alone to be sufficient for phlebitis 35–37, although the length of palpable cord required varied from 2.5 7,38,39 to greater than 15 cm 40. Exact measurement of symptoms, such as distance of erythema and oedema from insertion site, was undertaken in several studies, but the length or diameter required for concern varied considerably, from greater than 2 41 to greater than 3 cm 36,37. Some authors measured local warmth objectively, using a differential thermometer 41–43, but in most cases, temperature appeared to have been subjectively evaluated. Finally, some authors using progressive scores considered a patient had phlebitis when symptom severity met the criteria for a score of 1; others reported phlebitis only when severity scored as 2 or 3.

Phlebitis incidence

Not all authors reported phlebitis in the same way. Some reported phlebitis incidence per patient (potentially including multiple PIVCs); others reported phlebitis incidence per PIVC. Reported phlebitis incidence varied dramatically for studies using a scale – from 0% 44 to 91% 45.

The phlebitis assessment process

Frequency of reported assessment ranged from every PIVC access for medication or infusion, to twice daily, daily or second daily assessment. A handful of studies reported continued phlebitis assessment after cannula removal up to 24 hours 24, 48 hours 7,46 and 3 days 47. One study reported follow up of patients until the phlebitis resolved; in one case of phlebitis, pain lasted for 5 months 48. Assessors ranged from ward nurses, research nurses, experienced IV teams, medical students, doctors, to independent IV assessors. Some researchers reported providing phlebitis assessment training to staff, but the majority did not.

Psychometric evaluation of infusion phlebitis assessment scales

Although there are dozens of phlebitis assessment instruments, formal evaluations of their measurement properties are rare. Several scales were used in multiple studies, such as the Baxter scale 21,31,33, the Dinley scale 23,49,50 and the Lipman scale 22,51,52, but appear never to have been formally assessed. Thirteen articles reported evaluating some psychometric properties of their assessment scale (see Table 1), but only three provided detailed information. This section describes the psychometric adequacy of those three scales: VIP scale, INS phlebitis scale and PVC ASSESS.

Table 1

Studies that reported measuring some psychometric properties of a phlebitis assessment scale

Study, year, design, country (alphabetical)	Population	Primary outcome	Phlebitis scale, definition	Source of scale	Symptoms measured	Reported phlebitis rates and grade	Assessor(s)	Assessment frequency	Measurement property assessment
Ahlqvist et al., 2006 44 Cross-sectional survey (pre/post) Sweden	2001: 107 PIVC in 67 medical and surgical patients; 2002: 99 PIVC in 63 medical and surgical patients	Effect of introducing guidelines for PIVC care on incidence of thrombophlebitis, nurses' care, handling and documentation	Scale 0–3 Phlebitis defined as ≥1	Lundgren et al., 1993 48	Redness Tenderness Pain Swelling Increased temperature Palpable cord Pus	% PIVC 2001 survey: 39% grade 1 7% ≥ grade 2 2002 survey: 27% grade 1 0% ≥ grade 2	7 nurses not employed on study wards	Second daily	Reliability, validity, feasibility assessed in pilot study. No data provided.
Ahlqvist et al., 2010 86 Cross-sectional Sweden	67 PIVC§	Inter-rater reliability of phlebitis assessment using PIVC assessment tool	PVC ASSESS Points per symptom 1 or more symptom(s)	Hershey et al., 1984 7	Pain Tenderness Erythema Oedema Induration Purulent exudate Streak formation Palpable cord	N/A	Nurses trained in phlebitis assessment	Group A (3 RNs) assessed at bedside Group B (3 RNs) assessed photos of same PIVCs 4 weeks later	Inter-rater and intra-rater reliability assessed; content validity informally assessed; acceptability and feasibility assessed‡
Bostrom-Ezrati et al., 1990 53 Prospective cohort United States	514 medical and surgical patients at 4 hospitals¶	Incidence of IV site symptoms, and associated patient and practice factors	Maddox scale 0–5	Maddox, 1983 20	Erythema Swelling Induration Pain Palpable venous cord	% patients 22.6% grade 1 17.3% ≥ grade 2	Nurse data collectors	Twice daily	Inter-rater reliability assessed
Boyce & Yee, 2012 24 Prospective cohort United States	24 PIVC in 12 patients	Incidence and severity of phlebitis in patients given peripherally infused amiodarone	Modified INS scale 0–4+ Phlebitis defined as ≥0+ (pain)	INS, 2006 16	Pain Erythema Oedema Streak formation Palpable venous cord Purulent drainage	% PIVC 50% ≥ grade 0+	Staff nurses	Every 4 hours until 24 hours after infusion ceased	Content validity and feasibility assessed. No data provided.
Campbell, 1998 31 Prospective cohort Northern Ireland	90 medical patients from 13 wards¶	Incidence and severity of phlebitis, contributing factors, extended length of stay, IV complications	Baxter scale 0–5	Baxter, 1988 18	Pain Erythema Swelling Induration Palpable venous cord	% patients 26% grade 1–3	Staff nurses	None stated	Test–retest reliability reported as being assessed. No data provided.
Catney et al., 2001 40 Prospective cohort United States	411 medical and surgical patients¶	Relationship of dwell time to phlebitis and infiltration	Authors' scale 1–3 Phlebitis defined as ≥2	None stated	Pain or tenderness Erythema Swelling Palpable venous cord	% patients 7.3% ≥ grade 2	6 IV team staff	Twice daily	Inter-rater reliability, construct validity and feasibility reportedly assessed in pilot study. No data provided.
Dibble et al., 1991 54* Prospective cohort United States	514 patients in 4 hospitals¶	Frequency of IV site symptoms	Modified DeLuca and Maddox scale 1–5 used	DeLuca et al., 1975 101; Maddox et al., 1977 19	Pain Redness Swelling Induration Palpable cord	% patients 39.9%	66 research assistants (nurse educators, RNs, nursing students)	Twice daily	Inter-rater reliability assessed
Dryburgh & Imlah, 2002 77 RCT Canada	38 outpatients receiving IV antibiotics for cellulitis¶	Incidence and severity of phlebitis with two antibiotics	Modified Phlebitis Rating Scale 1–4	American IV Nursing Standards, 1982†	Erythema Tenderness Pain Swelling Induration Purulence	% patients 27% cefazolin 59% cloxacillin	Community nurses	Daily	Test–retest, face validity and construct validity assessed. No data provided.
Gallant & Schultz, 2006 13 Prospective cohort United States	851 PIVC in 513 adult cardiac surgical and cardiothoracic patients	Reliability of a phlebitis scale	VIP scale 0–5 Phlebitis defined as VIP ≥ 2	Jackson, 1998 14	Pain Redness Warmth Oedema Purulence Palpable venous cord > 7.6 cm	% PIVC 6.2% VIP ≥ 2	Research IV team nurses trained in VIP scale	Daily	Authors reported testing inter-rater reliability, but actually assessed criterion validity. Content validity informally assessed.‡
Groll et al., 2010 29 Cross-sectional study and chart audit Canada	416 PIVC observations in 182 patients	Psychometric properties of phlebitis and infiltration scales in an acute and community care setting	INS phlebitis scale 0–4 Phlebitis defined as ≥1	INS, 2006 16	Pain Erythema Edema Streak formation Palpable venous cord Purulent drainage	% patients 18.3% ≥ grade 1 observed 7.7% episodes of phlebitis documented in chart	Two research nurses	None stated	Inter-rater reliability, acceptability and feasibility assessed. Authors reported concurrent (criterion) validity, but actually tested convergent validity.‡
Larson et al., 1984 58 Prospective cohort United States	876 PIVC in 707 medical-surgical patients	Relationship between selected risk factors and incidence of phlebitis	Maddox scale 0–5 Phlebitis not defined	Maddox et al., 1977 19	None stated	% PIVC** 25.6%	Quality assurance research nurse	Daily	Inter-rater reliability assessed in pilot trial
Powell et al., 2008 30 Retrospective review United States	679 PIVC in inpatients§	Relationship between dwell time and phlebitis	INS phlebitis scale 0–4 Phlebitis defined as ≥ 1	INS, 2006 16	Erythema Pain Streak formation Palpable vascular cord Oedema Purulent drainage	% PIVC 3.7% ≥ grade 1	3 IV team nurses	Daily	Inter-rater reliability assessed. No data provided.
Washington & Barrett, 2012 74 Point prevalence United Sates	188 PIVC in 169 medical-surgical patients	Point prevalence of phlebitis rates	INS phlebitis scale Phlebitis defined as ≥2	INS, year not stated	Pain Erythema Oedema Streak formation Palpable cord	9.5% ≥ grade 2	10 data collectors	One assessment only	Inter-rater reliability and feasibility assessed. No data provided.

Bostrom-Ezrati et al., 1990 and Dibble et al., 1991 reported on the same study.

No reference given by authors and unable to locate reference.

Measurement property values are shown in the text of the paper.

Number of patients not stated.

Number of PIVC not stated.

Phlebitis grade not reported.

INS, Infusion Nurses Society; IV, intravenous; PIVC, peripheral intravenous cannula; RCT, randomized controlled trial.

VIP scale/Jackson scale

As part of a randomized trial published in 1977, US pharmacists, Maddox and colleagues 19 created a phlebitis assessment instrument to grade phlebitis presence and severity using six symptoms: pain, erythema, swelling, induration, palpable venous cord and frank vein thrombosis. The scale ranged from 0 to 5+; a score of 1 was considered indicative of phlebitis. Their report included no evaluation of the scale's reliability or validity. During the 1980s and early 1990s, several researchers used the Maddox scale or a slightly modified version of it 27,47,53–61, but psychometric assessments were still not reported.

In the UK in 1998, Jackson 14 published guidelines for scoring phlebitis based on an adaptation of the Maddox method and a scale developed by Lundgren and colleagues in 1993 48, which was relabelled the VIP score. This scale grades phlebitis progressively from 1 (no observable phlebitis symptoms) to 6 (advanced thrombophlebitis), and each grade is associated with a recommended action (e.g. cannula removal). The VIP scale assesses the presence/absence of six symptoms: pain, erythema, swelling, induration, palpable venous cord and pyrexia. Neither Jackson nor other researchers who subsequently used the scale 13,32,62–67 reported information about the scale's measurement properties.

A formal assessment of a modified version of the VIP scale was undertaken in the United States in 2006 by Gallant and Schultz 13. They monitored 851 PIVCs in 513 cardiac surgical patients in one hospital. Jackson's original grading from 1–6 was recalibrated to 0–5; a score of 5 indicated purulent drainage, redness and a palpable cord greater than 7.6 cm. Other modifications were not described in detail, although pyrexia as a symptom was removed. Phlebitis was considered present if the VIP score was ≥2, with associated recommendation for PIVC removal. Despite modifying the scale, the authors continued to use the label of VIP scale. Therefore, several versions of the VIP scale, including Jackson's original scale, are available and in use.

Staff nurses (number unreported) from two wards received training in the use of the Gallant and Schultz VIP scale, and then completed daily PIVC assessments. Inter-rater reliability was assessed by correlating each research nurse's VIP score with that of the principal investigator, a senior clinical nurse. The type of correlation (Pearson's r, Spearman's rho, intraclass) was unreported. The number of PIVC assessments included in the inter-rater reliability checks was also unreported. Each nurse was said to achieve an acceptable inter-rater reliability correlation of ≥0.85. However, inter-rater reliability was not computed between the nurses themselves, which is a more standard approach. A key unanswered question that remains is whether rating consistency across similarly trained observers can be achieved with this scale.

In terms of validity, the report stated that expert nurses in the cardiac surgery unit ‘established content validity for the modifications of the Jackson VIP scale’ (p. 341). Data on content validity, using a quantitative assessment of agreement such as the content validity index 68,69, were not provided. The scale's criterion or construct validity was not discussed. However, assessment of inter-rater reliability could be construed as testing criterion validity. If the principal investigator was an ‘expert’ in phlebitis, then her scoring can be accepted as a ‘gold standard’ against which the nurses' ratings were tested. This study reported no analysis of specificity or sensitivity, which are standard parameters for criterion-related validity in scales such as the VIP that have a ‘cut-point’ for the presence/absence of an outcome. The researchers also did not specifically assess construct validity.

Gallant and Schultz concluded that their version of the VIP scale is a reliable and valid measure for assessing and determining the removal of a PIVC. However, the evidence for the scale's adequacy is extremely limited. The reliability assessments did not establish that nurses could be consistent in their evaluations of phlebitis symptoms with each other (inter-rater), nor with themselves (intra-rater). Test–retest reliability was not examined. The study yielded some information about criterion validity, but the VIP scale's specificity and sensitivity were not tested. Construct validity was not considered. Responsiveness – the ability to detect true changes in symptoms – was also not examined. Post-study, the hospital made a decision to adopt the VIP as a standardized assessment tool, which suggests they found it easy to use in clinical practice; however, no data regarding feasibility were provided.

INS phlebitis scale

The INS in the United States developed the first INS phlebitis scale in 1998 15–18. The INS scale has changed over time, with the current version being a progressive score from 0 (no symptoms) to 4 (all symptoms present: pain, erythema, oedema, streak formation, palpable venous cord >2.54 cm in length and purulent drainage) 16. Any score of 1 or greater is considered phlebitis. Several studies included in the current review used an assessment tool based either on the INS scale or an adaptation 24,25,29,30,34,70–76. Despite widespread use, the INS scale has had limited scrutiny for psychometric properties. Boyce and Yee 24 adapted the INS scale and consulted a panel of 18 experienced nurses to assess the revised scale's face validity, resulting in several further changes to the tool. Following pilot testing, the tool and instructions were modified to be ‘more user-friendly’ (p. 30). No other psychometric evaluation appears to have been undertaken by these authors. Dryburgh and Imlah 77 appeared to have adapted an early version; they assessed it for face validity and what they called ‘test-test’ reliability in 10 patients, without providing data. Washington and Barrett 74 reported assessing inter-rater reliability, but did not provide values. Powell et al. 30 reported agreement in rating phlebitis between two members of the IV team, but the ratings appear not to have been independent or blinded.

A more in-depth study by Groll and co-researchers 29 was undertaken in Canada to evaluate the psychometric properties of the most recent (2006) version of the INS phlebitis scale 16. In the study, adults with a PIVC were recruited from a community hospital and a visiting home nursing agency. Pairs of independent research nurses who were not providing direct patient care undertook 392 observations of 176 patients. No information regarding the training of the research nurses was provided, nor did the report state how many pairs of nurses performed ratings. The study aimed to yield evidence regarding the INS scale's reliability (inter-rater), validity, acceptability and feasibility.

For inter-rater reliability, two nurses simultaneously scored the INS scale for each patient. The kappa statistic was used for the reliability index; proportion in agreement was not reported. It was not reported whether the kappa statistic was calculated based on agreement for the full scale's 0–4 range (i.e. a weighted kappa), or on a simpler dichotomous rating of phlebitis presence (≥1) or absence (0). Furthermore, although different pairs of raters assessed different sets of patients (i.e. the design was not fully crossed), it is unclear whether the appropriate statistic – Fleiss's kappa 78,79 rather than Cohen's kappa 80 – was used. In any event, the reported kappa was 0.45, which is considered ‘moderate’ using Landis and Koch's 81 standards (kappas of 0.21–0.40 are ‘fair,’ 0.41–0.60 are ‘moderate,’ 0.61–0.80 are ‘substantial,’ and 0.81 and greater are ‘almost perfect’). Standards for kappa are controversial 82,83, but few would argue that a kappa of 0.45 offers strong evidence of assessor agreement.

In terms of validity, Groll and colleagues assessed what they called ‘concurrent validity’. Concurrent validity, a form of criterion validity, requires a ‘gold standard,’ which in this case was documentation of phlebitis in the patients' charts. The Spearman correlation between the number of times observers said phlebitis occurred based on the INS scale and the number of times phlebitis was documented in the chart was a modest 0.39. Research nurses using the scales identified more than twice as many cases of phlebitis as were recorded in patient charts. An entry in a patient's chart is a questionable choice for a ‘gold standard.’ Indeed, the authors noted that the discrepancy between the charts and the scale ‘underscores the need for the use of validated tools’ (p. 389). Within COSMIN's classification, the procedure would best be described as convergent validation (i.e. evidence that two separate measures of a construct are correlated) rather than criterion validation.

The INS scale was also assessed for acceptability and feasibility. The nurses completed the instruments relatively quickly, with a mean completion time of 1.3 minutes (range 1–15 minutes, SD 0.9 minutes) to complete both the phlebitis scale and the INS infiltration scale (the INS infiltration scale is not covered in this review). Feedback from six research nurses indicated that the phlebitis scale was acceptable for the purpose of identification and measurement of phlebitis, the instructions were clear, and the scale was deemed easy to use and clinically appropriate. Acceptability was further supported by the fact that there were only limited amounts of missing data.

The researchers concluded that the scale was ‘valid and reliable in both the acute care and community settings’ (p. 390). However, the values of both kappa for inter-rater reliability (0.45) and the correlation coefficient for the validity evaluation (0.39) are modest. The researchers perhaps interpreted statistically significant differences as evidence of the scale's good properties. However, statistical significance is of limited interest in assessments of measurement properties, and indeed they are seldom reported in inter-rater reliability studies 84,85 because the focus is on how close the reliability coefficient is to 1.00, not whether it is different from 0.00.

Although the researchers provided initial data on the psychometric properties of the widely used INS scale, the research did not comprehensively examine reliability and validity. There were no assessments of reliability over time (test–retest and inter-rater), and the validation efforts did not generate sufficient evidence of validity. Furthermore, the scale's responsiveness was not evaluated. The assessments of feasibility and acceptability were the most comprehensive published to date, but the findings would have been of greater value with a larger sample than six nurses. It would be advantageous to replicate this research in additional centres and with a better ‘gold standard’ phlebitis criterion, such as evaluation of patients by an infusion expert.

PVC ASSESS

Ahlqvist and a team of Swedish co-researchers 86 developed a 45-item tool called PVC ASSESS to assess the management, documentation, signs and symptoms associated with PIVC use and complications. Only 11 of the items measure phlebitis symptoms – 5 based on patient reports (pain, tenderness, communicating) and 6 based on nurse observation. All observation items are dichotomous, indicating presence or absence of erythema, oedema, purulent exudate, induration at insertion site, streak formation and palpable cord. In the methodological paper describing the instrument, there were no guidelines for combining scores from the 11 symptom items into an overall score, nor any discussion about a discrete ‘cut-off’ score for phlebitis.

Reliability was assessed at the item level and only for the six items that required nurses' observations. Inter-rater reliability was estimated with 3 nurses and 66 patients. The researchers calculated proportion of agreement and kappa, using multi-rater kappa. Proportion of agreement ranged from 0.77 (erythema) to 0.95 (exudate and palpable cord). Because of low prevalence of most symptoms, kappa was computed only for one item (erythema), a modest 0.40 [95% confidence interval (CI) = 0.18–0.62]. Inter-rater reliability was assessed for three items that could be evaluated via colour photographs for 67 patients, using a different set of three nurse assessors. Proportion of agreement ranged from 0.76 to 0.89, and the only kappa value – again for erythema – was 0.58 (95% CI = 0.44–0.72).

The researchers also assessed intra-rater reliability (which they incorrectly called test–retest reliability) using photographs. Three nurses examined colour photographs of 67 patients. They rated the presence or absence of three signs (erythema, exudate and streak formation) on two occasions, 4 weeks apart. Commendably, the order of presentation of the photos was altered at the second viewing. Across the three raters, intra-rater kappas ranged from 0.49 (nurse 1, streak formation) to 0.76 (nurse 2, purulent exudate). The median intra-rater kappa was 0.59. This study was the only one in which intra-rater reliability (constancy of assessment by the same rater over time) was evaluated.

In terms of validity, only content validity was considered. The report indicates that the research group ‘confirmed content validity … through comparisons with guidelines and published scientific literature in the field’ (p. 1109). It does not appear that a formal content validity assessment was performed. The team did undertake an assessment of acceptability and feasibility. A sample of 27 nurses and 93 nursing students informally used the instrument with nearly 600 patients, and then provided feedback about the clarity and content of items, and the usefulness and layout of the tool. A few changes were made after this feedback, but results of the feasibility assessments were not provided.

Although the researchers considered their tool as ‘reliable,’ kappa values for the inter-rater and intra-reliability of nurse-observed phlebitis items were modest. No information about the reliability for the five patient-reported items was provided, and one of these items (‘Communicating’) was not defined. Test–retest reliability (short-term stability of scores across different assessments) was not evaluated. In terms of validity, no evidence was offered regarding the criterion validity (e.g. comparison to a ‘gold standard’) or construct validity, nor was responsiveness of the index evaluated. Feasibility information was limited. Göransson and Johansson 87 also used the PVC ASSESS tool, but did not report any psychometric evaluation.

Discussion

In this systematic review of research studies using phlebitis as the primary endpoint, we found numerous definitions of phlebitis, 71 different phlebitis assessment scales, a wide variation in assessment techniques and reported phlebitis rates, and very little psychometric evaluation of the existing scales. While it was surprising to find such an array of confounding factors in phlebitis assessment, of even greater concern was the fact that many studies reported phlebitis as a primary endpoint without providing any definition of phlebitis at all.

Among the 180 studies that explained how they determined phlebitis, either by scale or definition alone, we found a broad range of definitions. The Centers for Disease Control and Prevention 88 defines phlebitis as warmth, tenderness, erythema or palpable venous cord, citing Maki and Ringer 89, although this is not the definition used by those authors. Other commonly used descriptors include pain, swelling, induration and purulent drainage.

With cumulative scales, no uniformity exists as to how many signs must be present to qualify as phlebitis and/or warrant the removal of the PIVC. Many tools consider the presence of two or more symptoms as phlebitis, with others requiring only one sign, and others several signs. Furthermore, differentiating phlebitis from extravasation may be difficult when tenderness and oedema are the predominant signs 90.

Numerous progressive scales with grading according to symptom severity have been developed over the past 40 years, but persistent limitations include the following: (1) not all ‘required’ symptoms may be present, yet the PIVC is not working properly 91; and (2) a patient may not develop the signs in the particular sequence outlined by the scale, and thus does not meet the threshold for phlebitis despite patient/staff concerns that trigger PIVC removal 60,91.

Phlebitis rates ranged widely in this review. This can be attributed in part to the absence of a universally accepted scale with strong demonstrated reliability. The INS 16,17 recommends a phlebitis rate of 5% or less as acceptable, but differences in definition and assessment procedures, study design (prevalence versus incidence), casemix of research trials and rate calculation methods make comparison difficult. The INS also recommends that phlebitis should be calculated as the number of phlebitis incidents per total number of PIVC multiplied by 100 15,17,18, but this review found that reporting methods varied considerably: per patient, per PIVC and per 1000 catheter days.

The regular clinical use of a phlebitis tool is believed to provide a trigger, alerting nurses to take action if problems occur 92. The review found that the most commonly used tools were the INS, VIP, Jackson, Baxter and Maddox scales; however, all of these have been modified by various authors and several versions of each scale exist, with some researchers continuing to use older versions. Typical modifications include the addition or removal of phlebitis symptoms and variations in the scoring process, including the number of symptoms required for diagnosis and changes to the numerical scale. The INS phlebitis scale is a popular tool, but several variations exist 15–18, and we found that many authors further modified the tool for their own purposes. The UK Royal College of Nursing recommends the VIP scale 93 because specific actions, such as PIVC removal, are given as severity of phlebitis increases. However, the VIP scale exists and continues to be used in multiple modified versions. In the United States, the INS currently recommends using either the INS tool 15,16, as evaluated by Groll and colleagues 29, or the VIP scale, as per Gallant and Schultz 13.

Frequency of phlebitis assessment ranged from every cannula access, to twice daily, daily or even second daily assessment. Accessibility or visibility of the PIVC site was not mentioned in the majority of studies, although presumably some used gauze and tape dressings, which are acceptable 94 but preclude visual inspection of some symptoms.

Assessors ranged from student nurses and ward nurses to experienced IV teams, and medical and nursing researchers. Although some authors reported providing education on phlebitis assessment, the majority did not. Inter-rater reliability of phlebitis assessment has proved to be problematic. A 2002 epidemiological literature review 60 reported that no diagnostic criteria for phlebitis had been proven valid or reproducible. Since then, several authors have reported measuring inter-rater reliability, but none has addressed the full psychometric properties of the scale used, as discussed earlier. The studies reviewed suggest that it is extremely difficult to use existing scales with confidence, given the modest inter-rater reliability values.

With the current state of knowledge about scale quality, we cannot recommend a particular phlebitis assessment scale. None of the existing scales has been subjected to rigorous and thorough psychometric testing. For example, sensitivity and specificity have not been calculated for any scale. With the current evidence, no scale stands out as being of particularly high quality. In particular, inter-rater reliability estimates tend to be quite modest.

This review highlights priorities for future psychometric evaluations of phlebitis scales. The most critical measurement properties to assess are inter-rater and intra-rater reliability, as well as criterion validity (although other properties, such as responsiveness, would be of interest). With respect to inter- and intra-rater reliability, it is statistically unlikely that any scale will show high kappa values due to the generally low prevalence of phlebitis among a group of hospital patients at one moment in time. Future evaluations of reliability should provide the actual proportions of phlebitis assessments with positive agreement and negative agreement 95, to assist in interpretation of kappa estimates. Byrt et al.'s formula, which corrects for unbalanced prevalence, to present kappa values may also be useful 96. Although Hoehler 97 has argued against Byrt et al.'s formula replacing Cohen's kappa formula, it would be very useful to present both kappa estimates, so that users could see the potential degree of agreement 85,98.

In terms of criterion validity, evaluators need to select a suitable criterion, that is, ‘gold standard,’ such as rating by a phlebitis expert. Most existing scales grade severity, which implies the need for analysis using a receiver operating curve that establishes the appropriate ‘cut-off’ value for phlebitis diagnosis, and to ascertain that area-under-the-curve values are acceptable (commonly 0.70 or higher is desirable 83). It is also essential to calculate the scale's sensitivity and specificity (how often will it correctly test negative in those who do and do not have phlebitis?).

Lastly, it would be extremely useful to compare two or more scales for their psychometric adequacy in the same study. A direct comparison of reliability and criterion validity using the same sample of patients and raters would make it much easier for clinicians to select a phlebitis assessment scale with optimal properties.

Limitations

Our study has several limitations. Firstly, we only retrieved studies published in English that assessed infusion phlebitis in adults, so we cannot extrapolate the findings to paediatrics or non-English-speaking countries. We did not contact study authors to request potentially unpublished psychometric data. We were unable to locate several older articles (pre-1985) that reported phlebitis, so it is possible that we missed some older phlebitis tools. It is also possible that there are newer phlebitis scales in use and as yet unpublished.

The extreme number and variation of measurement options for phlebitis, combined with the paucity of evidence for reliability and validity, is of great concern. Up to 80% of all hospital patients require IV therapy with about 330 million PIVCs sold each year in the United States alone 5,99. Although phlebitis scales are quick to complete 29, the number of PIVCs used multiplies to significant nursing time and paperwork. In the United States, if 100 million PIVCs are used for an average of 3.5 days, and nurses assess PIVCs once each 8-hour shift, this accounts for about 23 million hours of skilled nursing time being used with questionable value each year in that country alone.

Conclusion

The selection of appropriate measurement tools is essential to clinical practice 100. Yet, it is unclear how best to assess phlebitis because no existing scale has undergone rigorous psychometric testing. This likely contributes to the wide variation in reported phlebitis incidence, which precludes meaningful comparison of studies. The current state of the evidence underlying phlebitis scales holds serious implications for PIVC assessment internationally.