Concurrent diseases and conditions in cats with renal infarcts.

Dear Editors of JVIM,

I recently read the paper by Hickey et al., “Concurrent Diseases and Conditions in Cats with Renal Infarcts.” Unfortunately I have serious concerns about the quality of scholarship and the thoroughness of the peer review process.

This study does not appear to have been designed to investigate more than a handful of concurrent conditions. The only conditions examined are HCM, other cardiomyopathy, neoplasia, hyperthyroidism, cardiac thrombus, pelvic limb thrombus, thoracic limb thrombus, and distal aortic thrombus. Most of these conditions are known risk factors for arterial thromboembolism (with 4 of them actually defined by the presence of ATE).

The conclusion stated in the abstract, and discussed again at the end of the paper, is that cats with renal infarcts should be screened for occult HCM. Unfortunately, the authors never actually classified cats as having occult vs. clinical HCM.

The authors evidently did very little to control for confounding factors. Confounding occurs when exposure is related to some third factor that is also associated with the outcome. The only variable they mention as a confounder is age, and their attempt to control for this possibly introduced more bias into their analysis than if they had not controlled for it at all. For reasons that are unexplained, they broke age down into 3 categories: (1‐6, 7‐14, and 15‐21 years). The Journal's guidelines dictate that cutoffs chosen to categorize continuous variables must be justified. Of particular concern is the category of 7‐14 years. This category covers an age range where many cats develop conditions such as renal insufficiency, hyperthyroidism, and heart disease. This could introduce residual confounding into the analysis.

Another limitation is the lack of appropriate presentation of results. Contrary to the guidelines of the JVIM, there are no tables of descriptive data, and many of the findings are either not mentioned at all or inadequately described. For instance, the authors leave the reader with no information about the distribution of study variables between cases and controls except to say that, “Breed, reproductive status, and age of the control group and case group were comparable.”

They probably mean that the p‐values for the distributions of breed, reproduction status, and age between cases and controls are greater than .05. However, you don't need to look at p‐values to determine whether 2 numbers are comparable, you just need to look at the numbers themselves. If there is a difference of any scientific/medical/biological importance, then you have to consider the possibility of confounding. The compatibility of that difference with some null hypothesis, or the mechanism that produced that difference, is irrelevant. For example, most articles reporting the results of randomized clinical trials present p‐values as evidence for the presence or absence of differences in baseline characteristics between the treatment and placebo group. Why? One does not have to look at a p‐value to know that there is a 100% probability that the differences are due to chance, since a random process created the differences. The differences still warrant investigation in the analysis. Additionally, confounding involves relationships between the confounder and the exposure, and the confounder and the outcome. The p‐values in the first table of articles that have tables only apply to one of those relationships. When it comes to controlling confounding, p‐values are at best uninformative, and at worst completely misleading. It is the responsibility of the authors to present the data so that the readers can decide if a difference seems important or not.

There are additional examples of inappropriate interpretation of p‐values. For example, they state that “there was no association between the presence of renal infarcts and hyperthyroidism.” The odds ratio for this relationship is 1.6, with a 95% CI of 0.93‐2.7. Lack of statistical significance at some arbitrarily defined alpha‐level is not the same as “no association”. The measure of an association is the effect measure, in this case an odds ratio. An OR of 1.6 shows a moderately strong relationship. The confidence interval is narrow, indicating a relatively low amount of random error. The confidence limits are much more supportive of a positive association than no association, with the majority of the CI well above 1.0. Contrast this with the estimate for the relationship between ATE and renal infarcts. The 95% CI goes from 2.55 to 25.4. This is still supportive of a strong association, but the high level of random error makes us unsure how strong the effect is. This is a good example of a case where the estimate with a lower p‐value is actually much more affected by random error than the estimate with a larger p‐value, and highlights a major flaw in the concept of null hypothesis significance testing.

Additional flaws in the analysis are revealed in later discussion of hyperthyroidism and neoplasia. In the last sentence of the results the authors state that after stratification by age, “renal infarcts did not have a significant association with hyperthyroidism or neoplasia.” Let's assume that this means age is a confounder, and adjustment caused the effect to disappear (adjusted OR close to 1.0). Remember that earlier they stated that the age distribution was “comparable” between cases and controls, with no data to support this. If the distribution of age was comparable in any sort of clinical/scientific sense, then age could not act as a confounder of these relationships. Unfortunately, they fail to report the actual age distribution, and they fail to present the adjusted estimates. The reader is not given any information to judge the veracity of these statements. Also, despite finding that the relationship between neoplasia and renal infarcts disappeared after adjustment for age, an entire paragraph is devoted to exploring mechanisms to explain why the unadjusted estimate did reveal an association. If an association disappears after adjustment for a confounder, then confounding is the explanation for the association.

In the second paragraph of the discussion, they claim that their finding of no association between renal infarcts and hyperthyroidism are in line with four other studies, all of which are case‐series of cats with distal aortic or arterial thromboembolism.1, 2, 3, 4 Since every cat in those studies has the outcome of interest, it is impossible to measure associations of any kind. Nevertheless, in one of the studies there were enough hyperthyroid cats that those authors concluded that “thyroid disease may pose a risk factor for ATE.”1 Furthermore, those authors state that this finding is in line with the results of a study by Laste et al.2 The authors of the current article also reference Laste et al, as one of the four showing no association between hyperthyroidism and ATE. Additionally, one of the cited studies never actually mentions hyperthyroidism at all.3