Reply: the limitations of psychometric outcome studies in plastic surgery.

Sir:

In their response to my article,[1] Panchapakesan et al[2] argue that methodological standards apply to utilization (authors’ italics) of an outcome study rather than development of an outcome study. However, none of the authors’ references[2] support the contention that their test development may be excused from basic considerations that lead to scientific soundness, a standard that Pusic et al[3] have previously recognized as essential to validity and reliability. Panchapakesan et al[2] comment that using unpublished instruments is “notoriously fraught with bias,” citing an article on schizophrenia. Ironically, the test questions and scales for the BREAST-Q and FACE-Q remain unpublished. Panchapakesan et al[2] wrote that “presentation of the technical details of psychometric tests is essential to allow for critical appraisal and acceptance of a measurement instrument.” Surely such details include the test questions.

The ruler analogy[2] is interesting. No one would dispute that the spaces between the numbers on a ruler must be the same. For this reason, a question on pain level, for example, would be on a different ruler than a question on nipple numbness. The BREAST-Q combines test responses into a 0–100 scale.[4] What clinical meaning can be derived from this overall index? Alongside validity and reliability, Pusic et al[3] previously acknowledged that a survey must be “clinical meaningful” and address “those issues considered important to patients and their surgeons.”

Panchapakesan et al[2] do not recognize the fundamental difference between a survey and a test, calling the issue “just semantics,” despite the fact that this distinction is made in the psychometrics literature.[5] A survey evaluates how well plastic surgeons are doing their job in providing patient satisfaction.[1] A test evaluates the subject with respect to an abstract quality, such as intelligence or scholastic aptitude.[5] Such tests are the proper domain of psychometrics.[5]

The terms of the licensing agreement do not appear in publications,[4,6] but are available on the BREAST-Q Web site.[7] This 11-page contract[7] insists that the users not only pay a licensing fee (despite study funding from the Plastic Surgery Foundation) but also cite two specific references, insert the owners’ copyright notice on all pages presenting the BREAST-Q, withhold test questions, and provide the owners with a copy of the article for approval before submission.

Plastic surgeons are instructed to upload their patient data to the owners’ Web site for analysis and a “QScore.”[8] Plastic surgeons are capable of using standard statistical tests to analyze their data; there is no need to outsource this task.[1] I am not the first plastic surgeon to conclude that psychometric applications have little relevance for plastic surgeons.[9] Of course, I do not take issue with psychometrics as a discipline,[2] just its misapplication to plastic surgery.[1] Alderman et al[10] dismiss ad hoc studies but do not specifically criticize their design or methodology. The fact is, only ad hoc studies[11,12] have been used successfully to compare operations.

DISCLOSURE

The author has no financial interest to declare in relation to the content of this article. The Article Processing Charge was paid for by the author.