Response to "Variability of Tear Osmolarity Measurements With a Point-of-Care System in Healthy Subjects-Systematic Review".

The article[1] entitled “Variability of Tear Osmolarity Measurements With a Point-of-Care System in Healthy Subjects—Systematic Review” contains several serious errors in fact and method that so thoroughly skew the results, the unsupported conclusions and speculative discussion of this article should be rejected.

Most notably, contrary to the stated selection criteria, the authors relied upon an article[2] that used a Fiske 210 Osmometer to set the low end of their estimate, not a TearLab Osmolarity System. The article in question, Garcia 2014, reported values for normal subjects (270 ± 4.4 mOsm/L) more than 3 standard deviations away from the mean of the remaining studies. In fact, it is physically impossible to obtain such values from a point-of-care TearLab Osmolarity System because the TearLab device reports only numbers greater or equal to 275 mOsm/L. A subsequent article by Garcia et al,[3] which is also cited in the meta-analysis entitled “Lack of Agreement among Electrical Impedance and Freezing-Point Osmometers,” makes clear that the “TearLab tear osmolarity measurements were higher than those of the Fiske 210 measurements” and “the 2 osmolarity values cannot be used interchangeably.” Accordingly, this article by Garcia should be excluded from a meta-analysis of the TearLab Osmolarity System.

At the high end of the analysis, Baenninger et al relied on a study[4] that reported a result more than 3 SDs above the mean of the remaining studies, contained no validated qualification of the subjects as to whether they were dry eye or normal, and did not report any liquid quality control to ensure proper calibration of the TearLab. The article in question, Eperjesi et al,[4] was a very early study of the TearLab Osmolarity System. Although it was known that clinical signs and symptoms of dry eye did not correlate at that time,[5] it had not been reported with tear osmolarity until well after the study of Eperjesi et al had been conducted.[6] Eperjesi et al[4] noted that “Prior to tear osmolarity testing, the participants were asked whether their eyes felt particularly sensitive that day, and all responses were negative.” This was the extent of subject qualification in the study. Given that the characteristic heteroscedasticity of dry eye disease (increase in test-to-test variation >8 mOsm/L) was also published well after Eperjesi et al had conducted their study,[7,8] it is very likely that dataset included a series of dry eye subjects in the previously undiagnosed cohort based on the reported variances. Most importantly, from a basic physiology standpoint, the 328 mOsm/L average reported by Eperjesi et al cannot be representative of healthy subjects. A 2016 study[9] demonstrated that compared with a 290 mOsm/L control, human corneal epithelial cells exposed to 308 mOsm/L medium begin to display changes in vital dye staining and cells exposed to 338 mOsm/L exhibit signs of impending cell death including a multitude of membrane blebs. A 328 mOsm/L average is simply too high an osmolarity to represent a healthy population. Based on these facts, the study by Eperjesi et al should be excluded from the meta-analysis.

Analysis of the distribution of means cited by Baenninger et al in Figure 1 reveals the extent to which the articles of Garcia 2014 and Eperjesi are statistical outliers. The remaining studies show a tightly clustered group with a mean of 298.8 ± 7.4 mOsm/L and 95% confidence interval (CI) of 284.0–313.6 mOsm/L. This is strikingly different than the results Baenninger et al reported, which suggested CIs for “Mean osmolarity for healthy eyes” could plausibly range from 261 to 365 mOsm/L. In addition, as written, the results of Baenninger et al conflate individual measurement CIs with those of a CI on the mean. This is unacceptable when reporting an expectation, especially because the postulated range is derived entirely from the 2 outlier manuscripts.

FIGURE 1.

Distribution of osmolarity studies included in the Baenninger et al[1] meta-analysis.

In addition to the outlier articles, there are still several other manuscripts within the remaining set that one could object to inclusion; articles that contain subjects known to have dry eye disease (DED) subjects in the datasets, skewing averages upward, and broadening even the nonoutlier distribution. For instance, Szalai et al[10] have published a letter to the editor in Cornea,[11] detailing the failures in qualification in that study, stating that “refractive surgery patients were recruited based solely on their asymptomatic status…over 50% of the ‘healthy’ group had abnormal lid-parallel conjunctival folds (LIPCOF) scores, almost 40% had abnormal breakup time (BUT) values, and one quarter had abnormal meibomian gland scores, statistically identical to the number of abnormal values observed in the Sjögren group. The ‘healthy’ group was clearly heterogeneous.”[11] Or Gagliano C et al,[12] where 22 post-menopausal women with aqueous deficiency, an average ocular surface disease index (OSDI) of 28.5 ± 18.34 and tear film breakup time (TBUT) values between 6 and 8 seconds comprised the “healthy” control. Or Messmer et al[13] who stated “Only 16 of 200 patients showed no signs and/or symptoms of dry eye syndrome (DES). In 71 patients, up to 2 signs/symptoms of DES were obvious. These individuals constitute the control group.” Another inconsistency includes the citation of Oncel et al[14] that reported an average of 298.7 ± 7.8 for the healthy control group, yet Baenninger et al included the higher average of 306.3 ± 6.6 mOsm/L in their analysis, which was derived from eyes of patients with deposits of pseudoexfoliative material, rather than the correct value for healthy controls.

In summary, to reach the conclusions stated in Baenninger et al, the authors had to rely on a study that was physically impossible for the TearLab to reproduce, a study that was physiologically impossible to be true, and studies with known DED subjects in the normal cohorts. The authors then applied an incorrect statistical analysis that dramatically overestimated the range of expected normal osmolarity. Accordingly, the speculative claims in the discussion should be rejected.

If Baenninger et al had not made such strong claims and speculative statements in the manuscript, an article of this type could otherwise stimulate valuable discussion. The best way to harmonize inclusion criteria for normal subjects remains a challenge in DED studies. Diagnosing patients with DED with specificity is nontrivial and is one of the main reasons devices such as the TearLab exist. Unfortunately, the manuscript, as published, sets the field back at least a decade and is similar to other early articles that did not have the benefit of literally hundreds of articles on tear osmolarity that have since added knowledge to this field and dispute the claims made herein.