Biased data lead to biased algorithms.

Although machine learning and use of machine-learned models are a technology that will revolutionize medicine and provide more opportunities to improve health outcomes, I was disappointed that none of the articles in the recent CMAJ series on the subject discussed what is known to be a major, and topical, issue with algorithms and machine learning — that they replicate social biases that exist in the systems they are supporting.1 If biased data are used, then biased algorithms follow. Populations that are over- or underrepresented in data will experience the continued marginalization and failure of machine learning.

Examples of bias in health abound. A recent article showed that the use of a biased algorithm resulted in much sicker Black patients being given the same severity score as white patients. Boys are given a higher pain score than girls based on a cultural belief of “stoic” males.3 Although Cohen and colleagues suggested that models could learn “bad habits”4 and Antoniou and Mamdani discussed threats of using an incorrect data set on an “increasingly ethnically diverse population,”5 both articles shied away from clearly stating that these models rely on data sets that are situated in a system and within structures that have, unfortunately, embedded racism.

Machine learning could revolutionize medicine, and its implications are clearly exciting, but a move toward its use should acknowledge that substantial bias exists and that automatization of this bias will result in more harm. The first step is for scholars to directly acknowledge the existence of these biases.