First Do No Harm: Legal Principles Regulating the Future of Artificial Intelligence in Health Care in South Africa.

What sets AI systems and AI-powered medical robots apart from all other forms of advanced medical technology is their ability to operate at least to some degree autonomously from the human health care practitioner and to use machine-learning to generate new, often unforeseen, analysis and predictions. This poses challenges under the current framework of laws, regulations, and ethical guidelines applicable to health care in South Africa. The article outlines these challenges and sets out guiding principles for a normative framework to regulate the use of AI in health care. The article examines three key areas for legal reform in relation to AI in health care. First, it proposes that the regulatory framework for the oversight of software as a medical device needs to be updated to develop frameworks for adequately regulating the use of such new technologies. Secondly, it argues that the present HPCSA guidelines for health care practitioners in South Africa adopt an unduly restrictive approach centred in the outmoded semantics of telemedicine. This may discourage technological innovation that could improve access to health care for all, and as such the guidelines are inconsistent with the national digital health strategy. Thirdly, it examines the common law principles of fault-based liability for medical negligence, which could prove inadequate to provide patients and users of new technologies with redress for harm where fault cannot clearly be attributed to the healthcare practitioner. It argues that consideration should be given to developing a statutory scheme for strict liability, together with mandatory insurance, and appropriate reform of product liability pertaining to technology developers and manufacturers. These legal reforms should not be undertaken without also developing a coherent, human-rights centred policy framework for the ethical use of AI, robotics, and related technologies in health care in South Africa.

Introduction

From time immemorial doctors have sworn to treat their patients to their greatest ability and to do them no harm. This spirit is retained in the revised Geneva declaration in which doctors also pledge to respect patient autonomy and dignity, eschew discrimination, and maintain patient confidentiality while sharing their medical knowledge in the interests of the patient and the advancement of medicine.[1] But how do regulators ensure that autonomous artificial intelligence (AI) systems, medical robots and related technologies are designed to obey the same laws and ethical codes? This is an urgent question as AI is set to play a growing role in all aspects of public and private health care and health research, including the making of great advancements in clinical diagnostics and decision-making and health care management. For example, during the COVID-19 pandemic AI facilitated disease surveillance and outbreak monitoring across the globe.

The capacity of AI systems to operate at least to some degree autonomously from the human health care practitioner and to use machine-learning to generate new, often unforeseen analyses and predictions is what sets AI systems and AI-powered medical robots apart from all other forms of advanced medical technology. A key priority is to develop laws and policy to support the “ethical and transparent use” of these new technologies,[2] and the transparent and secure management of health data sets on which algorithmic models can be built.[3]

While a core set of general principles for the ethical development of AI has emerged,[4] those principles must still be operationalised through legal regulations,[5] and this is particularly important in a high-risk area such as health care. The enactment of comprehensive data protection laws, while important, is not sufficient to address the unique regulatory challenges posed by AI.[6] South Africa has no laws specifically regulating AI.[7] Thus existing legal principles must be adapted, or new principles developed to mitigate the risks to human well-being (comprising of both health-related and human rights-related risks) while not stifling innovation and leading (unintentionally) to non-compliance.[8]

This article examines the extent to which current South African laws and policy in health care align with the normative framework of international principles for ethical AI and the values underpinning South Africa’s constitution. It examines three legal issues central to the effective regulation of AI: the regulatory oversight mechanisms for the registration of new AI health technologies, the health professions ethics framework governing the use by health care practitioners of these new technologies, and the common law principles of liability for harm caused to a patient or user of the technology. It concludes with recommendations for the development of a clear AI strategy with clear ethical guidelines centred in a human-rights narrative for the implementation of AI in health care in South Africa.

Artificial intelligence: the future for health care in South Africa

Artificial intelligence is expected to boom in Africa in the coming years.[9] AI could help to address a lack of access to health care facilities and a shortage of skilled health care practitioners, and lead to advances in health care policy and delivery through the better prediction, prevention, diagnosis and treatment of disease.[10] But despite these possibilities AI is “rarely deployed in medical practice, due to technical, regulatory and ethics concerns”,[11] and in Africa it is also being held back by a lack of access to the robust open data sets on which the development of AI depends.[12]

The primary application of AI in health care considered in this article concerns patient interactions that are directly mediated by a human health care practitioner who is assisted by AI. For example a KwaZulu-Natal Department of Health initiative to meet UNAIDS’s[13] “90-90-90 target” in the treatment of HIV/AIDs empowers rural health care workers and Department of Health Services administrators with AI-powered insights through Vantage, a South African information and communications technology (ICT) start-up.[14] The project is just one example of the potential of AI to increase the ability of health care practitioners to mediate successful patient outcomes, and the synergy between the policy goals of improving the conditions of each South African, and empowering small, medium and micro-sized enterprises (SMMEs) to work competitively in the ICT sector, not simply as consumers of technology but as developers of innovative new applications of technology.[15] AI has innumerable promising applications in health care, ranging from the interpretation of medical images, laboratory results and time series data, to biomedical text mining, electronic health record analysis and medical decision support systems.[16]

Defining key terms for a new regulatory framework

Artificial intelligence has not yet been authoritatively defined. The European Union (EU), which is currently at the most advanced stage worldwide in the development of AI laws and regulation,[17] has proposed that it be defined as:

a system that is either software-based or embedded in hardware devices, and that displays intelligent behaviour by, inter alia, collecting, processing, analysing, and interpreting its environment, and by taking action, with some degree of autonomy, to achieve specific goals.[18]

The Organisation for Economic Co-operation and Development (OECD) has adopted a similar definition:

An AI system is a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varying levels of autonomy.[19]

AI now uses big data analytics[20] powered by complex algorithms[21] to collect and interpret data. The term “algorithm” refers to the computational process or set of coded “instructions” that will be implemented by the computer programme to perform a function or solve a problem.[22] However, new machine-learning (ML) techniques[23] enable AI to “complete tasks in a way that would be considered intelligent were they to be completed by a human”[24] as the machine can move beyond a coded set of instructions to adapt and improve as it “learns” from the data.[25] In a health care setting one can distinguish broadly between ML techniques applied to the analysis of structured data, such as imaging, genetic and electrophysiological data, and natural language processing techniques used to analyse unstructured data, such as clinical notes in digitised health records, and generate machine-readable structured data for further analysis.[26] In both instances the “deep learning” enabled by adaptive algorithms means that the manner in which the machine responds to data is no longer pre-determined and entirely predictable.[27]

Similarly, advances in ML mean that one must now distinguish between “deterministic” robots, which can act autonomously but will do so in a predictable manner determined by pre-programmed instructions, and “cognitive” robots, which are powered by stochastic or adaptive algorithms that enable the robot to take decisions based on the input it receives from its environment but means that the robot’s actions are not always predictable.[28]

Normative framework for ethical AI development

The development of specific laws to regulate AI remains in its infancy. Although the Council of Europe’s ad hoc committee on AI (CAHAI) has put forward a proposal for an AI treaty, the work planned for 2021 remains at the stage of a study of its feasibility and scope.[29] However, guiding normative principles have been developed by several international organisations and are largely convergent, emphasising respect for human rights and freedoms[30] alongside transparency, fairness, security and, more broadly, beneficence and accountability as core components of ethical AI development.[31] These values are encapsulated in the OECD’s five Principles on AI:[32]

AI should benefit people and the planet by driving inclusive growth, sustainable development and well-being.

AI systems should be designed in a way that respects the rule of law, human rights, democratic values and diversity, and they should include appropriate safeguards – for example, enabling human intervention where necessary – to ensure a fair and just society.

There should be transparency and responsible disclosure around AI systems to ensure that people understand AI-based outcomes and can challenge them.

AI systems must function in a robust, secure and safe way throughout their life cycles and potential risks should be continually assessed and managed.

Organisations and individuals developing, deploying or operating AI systems should be held accountable for their proper functioning in line with the above principles.

As a member of the United Nations Educational, Scientific and Cultural Organisation (UNESCO), it is to be expected that South Africa will be guided in its national legislative and policy development agenda by the Recommendation on the Ethics of Artificial Intelligence adopted by UNESCO’s General Conference at its 41st session on 24 November 2021.[33] In addition, as a member of the G20 South Africa should take guidance from the G20 AI principles[34] adopted in 2019, which are in turn modelled on the OECD Principles on AI. These principles strongly overlap with the EU framework for “trustworthy AI”,[35] the United Nations Educational, Scientific and Cultural Organization (UNESCO) recommendation,[36] and industry-led commitments to ethics such as those of the IEEE,[37] Microsoft,[38] Google[39] and DeepMind.[40]

However, differences in how these “soft” principles are interpreted and the extent to which they are applied by corporate actors[41] require the development of enforceable obligations in laws, regulatory policy and professional codes of conduct.

South African legislative and regulatory policy framework for AI in health care

The artificial intelligence applications developed for or used in a health care setting must operate in full compliance with the National Health Act 61 of 2003, the Health Professions Act 56 of 1974, the Medicines and Related Substances Act 101 of 1965 and the Hazardous Substances Act 15 of 1973. In addition, legislation governing consumer products or services,[42] the protection of personal information,[43] access to personal information[44] and electronic transactions[45] must be applied where relevant. The development of policies, standards, and certification mechanisms for AI applications in health care will thus require constructive dialogue and co-ordinated action by the Information Regulator, the Department of Health (DOH), the South African Health Products Regulatory Authority (SAHPRA) and other stakeholders in South Africa’s digital health strategy.[46]

Artificial intelligence in digital health policy

South Africa adopted a telemedicine strategy in 1998 but failed to achieve the targeted improvements in access to health care in under-resourced rural communities that telemedicine promised.[47] Policymakers have since set their sights even higher on a global digital health strategy led by the World Health Organisation (WHO),[48] which still includes telemedicine in the broader rubric of e-health,[49] but now also includes 4IR technologies such as AI, big data analytics and robotics.[50] At a regional level digital health is also a key pillar in the African Union (AU)’s Digital Transformation Strategy,[51] and the Policy and Regulation Initiative for Digital Africa (PRIDA) is developing Africa’s digital health strategy.[52]

South Africa’s latest digital health policy strategy adopts the WHO definition of digital health[53] and therefore sets a clear green light for the development and deployment of AI applications in health care in pursuit of the strategic vision and detailed objectives of the policy. But the policy itself and the existing legislative and regulatory policy environment in South Africa are lacking in substantive principles to guide such development or deployment.

The term “health technology” refers to “machinery or equipment that is used in the provision of health services”,[54] excluding medicines.[55] At national and provincial level, the Health Council is to advise the Minister of Health on

policy concerning any matter that will protect, promote, improve and maintain the health of the population, including- … (v) development, procurement and use of health technology.[56]

The acquisition of any “prescribed health technology” by a health establishment is subject to the issue of a certificate of need by the Director-General.[57] The Minister of Health, after consultation with the National Health Council, may promulgate regulations[58] and prescribe quality requirements and standards relating to health technology,[59] and the Office of Standards Compliance and the Inspectorate for Health Establishments must monitor and enforce compliance by health establishments with such standards.[60] The framework thus exists in which the use of AI in health care could be evaluated, but it continues to face challenges in implementation.[61]

Artificial intelligence software as a medical device

The Medicines and Related Substances Act 101 of 1965, as amended,[62] defines the term “medical device” widely to include inter alia any “machine” and “software” intended by the manufacturer for use in the “diagnosis, treatment, monitoring or alleviating” of any disease or injury, and the “prevention” of any disease. Many but not all possible applications of AI in the field of health care will fall within this definition,[63] including software that can assist with diagnosis in a clinical setting, and the hardware embedded with AI software that makes robotic surgery assistants, nursing aides and nano-robots possible. In both examples the AI software is clearly intended by the manufacturer to be used for the medical purposes defined. General software that is not specifically intended for such a purpose is not a medical device, “even if it is used in a health care setting.”[64]

The lines become blurred in the area of smart wearable devices and “fitness” and “health” mobile apps for smartphones, which may be considered “lifestyle” or “general wellness” products that mostly fall outside the ambit of health care regulations.[65] So, too, a chatbot developed in Kenya to offer sexual and reproductive health care information (but not medical “advice”) and the chatbots developed during the COVID-19 pandemic to provide symptom checking, reporting and exposure services would not prima facie be classified as medical devices as they are not being used in the diagnosis of disease (or a prescribed course of treatment). Nevertheless, there can be clear health implications if these chatbots incorrectly direct a patient, raising ethical concerns and the question of how they should be regulated to prevent the risk of harm.[66]

However, the involvement of a human health care practitioner is not a requirement imposed by the definition of software as a medical device under the Medicines and Related Substances Act 101 of 1965. Thus, currently medical devices intended for self-monitoring by a patient, for example blood pressure monitors or blood glucose tests, fall within the definition. It is conceivable that in future AI-powered devices that provide an interpretative analysis of data for a diagnosis of the underlying disease or injury would fall within the definition, provided the device is objectively intended by the manufacturer to be used in this way.

Interpretative clarity on the ambit of the definition is essential to ensure that the developers of such software are directed to appropriately consider the risks posed by the software and to implement a quality management system for the software lifecycle, which is especially important when software is used outside of a clinical setting.

The need for reform of regulatory oversight mechanisms

Medical devices that meet defined “standards of quality, safety, efficacy and performance”[67] are registered by SAHPRA after evaluation and assessment. SAHPRA may declare that a medical device (or any class, or part of any class, thereof) must be registered.[68] The sale of any medical device that has not been registered as required by such a declaration is prohibited.[69] The process by which applications for registration are reviewed by SAHPRA is governed by section 15 of the Medicines and Related Substances Act 101 of 1965, and requires SAHPRA to receive particulars and “where practicable” samples of the medical device.

This single stage model for regulatory review according to pre-defined, static specifications and standards cannot adequately address safety, quality and efficacy concerns as AI systems are “adaptive”, with the software algorithms being trained from large data sets so that the machine may change its behaviour over time in response to new insights learned from real-world applications.

The United States Food and Drug Administration (FDA) have proposed a “total product lifecycle”[70] regulatory oversight mechanism for software such as medical devices in health care. Pre-market certification of software would require manufacturers to provide the FDA with a “pre-determined change control plan” outlining the modifications that can be anticipated, coupled with transparent monitoring throughout the product lifecycle.[71]

In the EU, Regulation 2017/745 on medical devices[72] expands the definition of medical device to include the “prediction and prognosis” of disease, which may bring certain mobile applications such as heart rate monitors on smartphones and smartwatches into the regulatory regime.[73] Further, a specific classification standard for software has been introduced.[74] To complement sectoral product safety legislation the EU has also adopted a proposal for an AI Act to regulate the conditions applicable to the development and marketing of all AI-products and services and has established post-market controls.[75]

At an international level the Focus Group on AI for health (FG-AI4H), established in 2018 by the International Telecommunications Union (ITU) in partnership with the World Health Organization (WHO), provides

a standardized assessment framework for the evaluation of AI-based methods for health, diagnosis, triage or treatment decisions.[76]

In 2021 the WHO published a framework to guide the evaluation of clinical evidence supporting AI software development, software validation and reporting, deployment, and post-market surveillance.[77] The framework is a ground-breaking development that will assist in ensuring that safety and performance claims are supported by robust, transparent evidence. Importantly it emphasises that evidence must be free of the existing biases in healthcare on racial, ethnicity, age, socio-economic and gender lines that are perpetuated when they are encoded into the data used to train AI algorithms.[78]

It is essential that consideration be given to these developments to reform the regulatory regime in South Africa.[79] Public authorities must have oversight and the ability to intervene at all stages of the AI product lifecycle. The development of technical standards, robust ethical guidelines and a certification process could be considered as means to ensure oversight before market launch, so that health care practitioners and patients have access to trustworthy AI products and services only.

In the case of high-risk use, where indicated by a risk assessment, there would be a general obligation upon developers to deposit the documentation on the use, design and safety instructions with public authorities, and where “strictly necessary” this might include information on the “source code, development tools, and data used by the system”.[80] Allowing authorities access to the data, software and computer systems of developers and deployers of AI technologies is necessary to verifying not only the intended purpose but also the actual uses to which AI is put.[81] Such access must of course take place with safeguards to protect data, privacy, intellectual property rights and trade secrets.[82] In this regard, without duplicating duties, there needs to be co-operation between the Information Regulator and the health sector regulatory bodies to ensure that new technologies identified as “high risk” are developed and deployed in accordance with legal and ethical obligations[83] and an approved certification process.[84] Consideration also needs to be given to support for end-of-life products, and “independent trusted authorities” must have the means to provide services such as maintenance, repair and software updates and patches to the users of “vital and advanced medical appliances” where the developer or deployer of the technology ceases to do so.[85]

Need for regulatory reform of ethical guidelines

The Health Practitioners Council of South Africa (HPCSA)’s ethical guidelines for practitioners remain rooted in the outdated era of telemedicine.[86] Telemedicine is defined in the guidelines as:

The practice of medicine using electronic communications, information technology or other electronic means between a health care practitioner in one location and a health care practitioner in another location for the purpose of facilitating, improving and enhancing clinical, educational and scientific health care and research, particularly to the under serviced areas in the Republic of South Africa.[87]

Thus, telemedicine seeks to replicate traditional face-to-face practitioner-patient consultations using ICTs such as video conferencing. It could also include the exchange of information electronically (between practitioner and patient or, for example, between the primary and secondary health care practitioner for a specialist diagnosis or a second opinion) but an

actual face-to-face consultation and physical examination of the patient in a clinical setting by at least one of the health care practitioners remains mandatory.[88]

The guidelines are further restricted by the requirement that both the consulting practitioner and the servicing practitioner must be registered health care practitioners, either in South Africa or in the country where they are located.[89] A medical examination must be performed and documented, with a clinical history of the patient, before any course of treatment is prescribed or prescription issued.[90] No course of treatment or prescription may be issued on the basis of a questionnaire alone,[91] and informed consent must still be obtained when a prescription is issued electronically.[92] The guidelines have been relaxed recently, but only for the duration of the COVID-19 pandemic, and only to the extent of permitting “telehealth”[93] even where there is not “an already established practitioner-patient relationship”.[94]

The HPCSA ethical guidelines are thus inadequate to regulate the lawful and ethical development and deployment of AI applications. Worse, they may in fact inhibit the adoption of new technologies in health care in South Africa by virtue of the threat of sanctions against health care practitioners if they are found guilty of unprofessional conduct[95] or a breach of the professional duties imposed by common law.[96] The HPCSA’s statutory mandate under section 3 of the Health Professions Act 56 of 1974 is subordinate to national health laws and policy. Presently the outdated guidelines are inconsistent with the national policy on digital health, which includes innovation through the adoption of new technologies such as AI as one of five key principles underpinning the strategy.[97] While the report of the Presidential Commission on the Fourth Industrial Revolution (4IR) recognises that there remains a role for telemedicine in bridging disparities in physical access to health care services,[98] it underscores the need to leverage new technologies such as AI for efficiency and cost saving in health care planning, as well as advancements in the medical treatment of patients.[99]

Although machine-learning has transformed the role of the medical device from a mere tool to a powerful collaborator with the health care practitioner,[100] there is no room in the guidelines to regard an AI system as a servicing practitioner working in partnership with the consulting practitioner.[101] While South African law recognises juristic persons, it does not presently afford any legal status to “things”.[102] A radical re-imagining may be necessary to address the new risks and roles of AI and there is, at least in principle, no reason why a statute cannot create a statutory right of action against an AI system (the thing) which would impeach it (without necessarily citing or requiring jurisdictional competence over the person who owns or operates the thing).[103] However, without comprehensive, insurance-backed provisions for recourse in the event of harm, such provisions may be meaningless.

Guiding principles for the development of civil liability for medical harm in an AI context

As a corollary to the development of a regulatory oversight and professional ethics framework for the development and use of AI, consideration must be given to the basis upon which civil liability may be attributed when technology fails and causes harm. In this section two guiding principles are put forward to guide future regulation in this area.

Informed consent from the patient must always be obtained

Informed consent is the bedrock to the provision of any health care service. Sections 6 and 7 of the National Health Act 61 of 2003 respectively provide the way a patient is to be informed, and stipulate that a health service may not be provided to a user without that user’s informed consent, save in limited exceptional circumstances.[104] In terms of section 7(2),

[a] health care provider must take all reasonable steps to obtain the user’s informed consent.

The only guidance available on the use of technology in a health care setting is that in addition to obtaining the patient’s informed consent to a prescription or any course of treatment, the patient must also give informed consent to the use of the technology.[105] While the technologies underlying telemedicine such as video conferencing and email are now so commonplace that one can see little difficulty in providing an understandable explanation to the patient, the same cannot be said about AI. While this may change somewhat as new technologies infiltrate all areas of daily life, it is unlikely to ever be the case that an average patient will understand the complex algorithms that power AI systems. The scholarly debates taking place around the legal requirement for “transparency”[106] (or “explainability”)[107] must be tempered by pragmatism. Just as case law has held that a detailed explanation of a complex medical procedure is more likely to bamboozle than inform,[108] an unduly technical explanation of the computing processes underlying AI systems, robotics or related technologies would be counterproductive. A purposive interpretation of the consent requirement must focus on the need for the patient to understand enough about the risks of the process to make an informed decision about whether to proceed.[109]

The National Health Act 61 of 2003 sets out the principle that the “user”[110] of health care services is to have “full knowledge”[111] in that the health care provider must inter alia inform the “user” of “the range of diagnostic procedures and treatment options generally available”[112] and the “benefits, risks, costs and consequences generally associated with each option”,[113] as well as any implications, risks or obligations arising from the “user’s” exercise of the right to refuse treatment.[114] Moreover the explanation must “where possible” be given in a language and in a manner that the user can understand.[115] This qualification is a paradox. Informed consent simply cannot take place where the patient has not understood the explanation. South African law requires that the patient have “full knowledge” and there is a statutory,[116] common law[117] and ethical duty[118] to obtain informed consent. How this requirement is to be met in practice requires careful consideration. Besides the obvious difficulties of explaining complex technologies in understandable terms, we must also explain what is presently unknown. Providing the patient with full knowledge may paradoxically require explaining that even the developers of the software and the treating doctors do not always fully understand the inner algorithmic workings of the AI.[119] Further, we must put in place mechanisms to provide patients with additional information when it becomes available, and to obtain informed consent for sharing clinical data for research and development.[120] Electronic patient consent and record management systems make this feasible.[121]

The primary health care practitioner bears legal responsibility

As illustrated above, the assumption underlying the existing legislation and ethical guidelines in health care in South Africa is that all instances of patient diagnosis and treatment are mediated through a human health care practitioner registered with the HPCSA in terms of the Health Professions Act 56 of 1974. In many instances this will continue to be the case and therefore, no matter how complex the AI system may be, “the last call”[122] rests with the human health care practitioner.

At common law a health care practitioner’s liability when a treatment or diagnosis causes harm to a patient is based on the Aquilian action and involves applying a test for negligence based on an interrogation of what a reasonable medical professional ought to have done in the same situation.[123]

There is no reason to relax the ordinary standard of professional conduct because of the limitations of the technology or medium of communication used. A doctor could be found liable for harm on common law fault-based principles for failing to apply his or her own mind to the diagnosis or recommendations generated by the AI-software. The HPCSA guidelines state that professional discretion in relation to the course and scope of treatment “should not be limited by nonclinical considerations”[124] such as the constraints of any technology. The consulting health care practitioner is also responsible for ensuring that the patient’s well-being comes first, and the patient’s rights to privacy, dignity, information about their condition and confidentiality are respected by servicing health care practitioners.[125] They must ensure that adequate measures are in place to ensure the quality of service, as well as the confidentiality and security of the patient’s information, both in respect of their own employees as well as of non-health care personnel providing auxiliary or technical services,[126] the optimal functioning of the technology,[127] unauthorised access to patient information,[128] and damage to or the loss or alteration of patient information.[129]

Thus, when a servicing health care practitioner is consulted the primary health care practitioner remains responsible. The primary health care practitioner must interpret and apply his or her own mind to results in advising a patient on treatment options, risk, and likely outcomes. By analogy, when AI systems are used the health care practitioner remains liable for errors and omissions in a diagnosis or treatment that were reasonably foreseeable[130] or would not have been made by a reasonable practitioner in the same branch of the profession.[131] Likewise the practitioner remains liable for a failure to obtain informed consent from the patient.[132] To the extent that a greater degree of skill and care is required in the use of new and complex AI technologies, the practitioner would be expected to meet this higher standard,[133] and could face civil or even criminal liability for the consequences of acting without the required knowledge and skill in the use of new technologies.[134]

There is, however, no guidance in case law on how to apply the principles of fault-based liability in a scenario where the outcome is primarily attributable to an unknown flaw or failing in the AI system that could not reasonably have been anticipated. One could theorise that if there is no causative fault on the part of the doctor,[135] he or she would escape liability altogether, with the unfavourable outcome that the injured patient is left without recourse.[136] Even if one turned to the legal doctrine of vicarious liability, there would be great difficulty in establishing, firstly, that the AI system “acted negligently” and, secondly, that the medical practitioner exerted a sufficient degree of control over the AI system to be held responsible.[137] Moreover, one may well see an increase in the use of contractual exemption clauses to exclude all liability, save where the harm was intentionally caused,[138] which all points to the need for clear legislative and policy guidelines to be developed in this area.

Opening the black box: an argument for strict liability

The principle of “explainability” requires that AI developers give clear, understandable explanations of how the algorithms function and present results to data protection and consumer protection authorities and the end user.[139] This is the bedrock of consumer trust in new technologies, “even if the degree of [explicability] is relative to the complexity of the technologies”.[140] Nevertheless, it is impossible in some cases even for the developer of the technology to explain how an algorithm arrived at a particular result,[141] and this has given rise to the term the “black box algorithm”.[142]

Strict liability for operators of AI technology

When the machine makes a mistake that cannot be anticipated or explained, this raises difficulties about how to apply the common law of fault-based liability to the human health care practitioner. In simple terms, the doctor cannot be held liable on any standard of reasonableness. Moreover, the existing statutory and ethical framework does not impose any duty of care on the developers of AI applications in health care to prevent harm or obtain informed consent from the users of those technologies. At common law there is no general duty to prevent harm to others; and liability can be imputed for conduct only that is found to be wrongful when tested against the legal convictions of the community and the values embodied in the Constitution.[143] In addition, causative fault in the form of negligence or intentional wrongdoing must be proved.

While there is a basis for imposing strict liability for high-risk activities under South African common law,[144] legislation developed for the health care sector would be preferable in that it would provide a clear and certain framework to facilitate widespread adoption of and trust in such new technologies by health care practitioners and patients.

The latest EU legislative proposal on civil liability generally proposes joint and several fault-based liability on the operator(s) of AI systems.[145] Health is classed as a “high risk” use case based on the sensitivity of health data and the potential for harm and the infringement of human rights, alongside consideration of the specific purpose or proposed use of the technology in any particular case, as well as the severity of possible harm.[146] For this reason, strict liability (and mandatory insurance schemes) for health care practitioners are under consideration.[147]

Liability of developers and manufacturers of AI technologies

Product liability

At common law, when a product fails liability is attributed either under the terms of the supply contract, using contractual warranties and service level agreements, or through the imposition of fault-based product liability for manufacturers and so-called expert retailers. This presented an “often insurmountable challenge”.[148] For the non-lawyer, the term fault-based liability refers to the requirement that in addition to providing that the product was defective and caused harm, the claimant must prove that the supplier was negligent by failing to act in a reasonable manner and that the harm was caused by this negligence. Fault-based liability must therefore be distinguished from strict-liability, in terms of which a supplier is liable even if there was no fault.

One solution being considered in Europe is the application of the existing provisions of statutory product liability regimes, subject to appropriate amendments to incorporate digital goods and services within the ambit of the legislation.[149]

Product liability is governed in South Africa by the Consumer Protection Act 68 of 2008.[150] Section 61 of the Act attempts to impose strict liability for product defects upon all parties in the supply chain, which would in theory include manufacturers, doctors, and hospitals. However, the Act provides for several defences that considerably vitiate its effectiveness.[151]

The Act also provides for a statutory warranty of quality and safety enforceable jointly and severally against “the producer or importer, the distributor and the retailer” but only for six months after purchase.[152] Leaving aside the limited scope and duration of the warranty, the first problem is that the provision of goods and services to the State falls outside the ambit of the Act.[153] There are also problems with the statute’s scope of application to private sector health care. Patients are unlikely to be parties to any transaction supplying AI software as a medical device (save in relation to mobile apps and wearable health monitors), although they may be able to claim protection under the Act as the term “consumer” is defined widely to include the end-user of the product or the recipient or beneficiary of the service,[154] and would in those instances most likely seek to claim against the health care practitioner.[155] When the health care practitioner uses AI technology in the course of performing a health care service or at any health care facility, the provisions of section 58(1) require that “any risk of an unusual character or nature” be disclosed, potentially widening the ambit of the informed consent obligations.[156] The health care practitioner or facility that has purchased or used the AI technology will ordinarily be unable to rely on the Act for recourse against the developer. The Act’s protections apply to a consumer, and its provisions do not apply to a juristic person (which includes partnerships) with an annual turnover above R2 million.[157] The application of the Consumer Protection Act 68 of 2008 to AI is thus an area requiring further research and possible reform.

Cross border enforcement difficulties

The first obvious problem with any proposal to impose liability on developers is that most AI applications will be developed outside South Africa. The solution in the Telemedicine guidelines is that

the practice of medicine takes place where the patient is located at the time the telemedicine technologies are used.[158]

This simple solution remains fit for purpose in relation to the liability of the health care practitioners treating the patient if it is extended to include all AI, robotics, and related technologies. However, for the purposes of establishing jurisdiction over the developer or deployer of such technology, or service-providers processing or storing the data on their behalf, it is inadequate. The elegant solution in article 3 of the EU Framework proposal could be considered as a model for a similar South African regulation:

This regulation applies to artificial intelligence, robotics and related technologies, where any part thereof is developed, deployed or used in the Union, regardless of whether the software, data or algorithms used or produced by such technologies are located outside of the Union or do not have a specific geographical location.

The provision overcomes the difficulties associated with the fact that technology components may be developed, manufactured, deployed, and operated by multiple parties in multiple jurisdictions. Pinning down the place where the cause of action arose and establishing personal jurisdiction over the responsible parties by the application of ordinary common law principles of jurisdiction may be cumbersome, if not impossible in some cases. While jurisdiction is commonly settled by agreement and recorded in the terms of the contact between the parties, this may also be an inadequate solution if it limits South Africans who have suffered harm to rights to action in a foreign court, where the cost and difficulty of enforcing their rights may render the rights nugatory.

Policy considerations

Competing policy considerations must be carefully weighed up, which in the field of health care include not only the protection of the individual but the broader policy goals of innovation and the widespread, cost-effective availability of new technologies.[159] On the one hand, onerous strict liability regimes that leave health care practitioners with no recourse to claim an indemnity from the developers or manufacturers of AI products are unduly burdensome.[160] Doctors and health facilities must rely on contractual service level agreements, software and hardware warranties and indemnity clauses to seek recourse against the supplier of AI products, or compulsory insurance schemes must be in operation which may in themselves be prohibitively costly. On the other hand, to impose direct liability on manufacturers and developers or to overregulate the field may stifle innovation, investment and SMME participation.[161]

The importance of a human rights-centred narrative in national policy

South Africa presently has no overarching national AI strategy, which contrasts poorly with the approach in countries such as Canada[162] and China,[163] that are moving forward swiftly with a 4IR policy agenda. The reports for the 4IR commission and the work of C4IR and ASSAf are moving in this direction. However, it is imperative that technical frameworks be developed in tandem with the guiding ethical principles and the review of the legal frameworks.

At their core, ethical AI principles seek to defend human autonomy, which is the very essence of the rights to dignity and privacy,[164] against machine profiling and the practices it enables, which range from the somewhat innocuous (even helpful) functions of behaviourally targeted advertising and content suggestions to the subtle and insidious re-enforcement of hidden bias and discrimination. The cornerstone of a human rights-centred regulatory framework is the recognition that AI is made by people for people. It should therefore be designed “to serve people and not to replace or decide for them.”[165]

The regulation of AI in health care must therefore take due cognisance of the constitutional rights of dignity[166] and privacy,[167] alongside equality,[168] life,[169] bodily and psychological integrity,[170] access to health care services, including reproductive health care,[171] and access to information,[172] as well as the rights in the Patient’s Rights Charter,[173] including the right to the confidentiality of one’s information required by the National Health Act 61 of 2003. There is a strong alignment between the international normative framework of principles for ethical AI development and the rights in the Bill of Rights under the Constitution of South Africa.

There is a robust body of constitutional case law recognising that there is a “strong privacy interest” in maintaining the confidentiality of health information,[174] and that

[t]he more intimate that information, the more important it is in fostering privacy, dignity and autonomy that an individual makes the primary decision whether to release the information. That decision should not be made by others.[175]

However, the conceptualisation of privacy purely in terms of the right to decide whether to disclose data at all, for example, must make way to permit the free flow of data for research and innovation but still respect the individual’s human rights. In doing so the central challenge to the ethical development of AI is to ensure that we do not reduce the human being to an object “to be sifted, sorted, scored, herded, conditioned or manipulated.”[176] A human rights-centred narrative in any AI strategy is thus essential.

South Africa’s digital health strategy places a “person-centred focus” as the first of five key principles underpinning the strategy[177] and highlights the need for digital health solutions to respect “patient privacy”.[178] The report by the Presidential Fourth Industrial Revolution Commission[179] recognises that AI could herald great advances in health care but that “the data ecosystem also brings about the critical need for policy and legislation relating to the use of data, including ethics and security.”[180] Referring to the “central productive force of data”[181] in the 4IR, the report recognises

perhaps more importantly, that fundamental human rights are now intertwined with the protection of data.[182]

The danger I point out is that trite references in passing to “patient privacy” are insufficient, and a clear commitment to and detailed treatment of human rights issues such as that contained in the EU “trustworthy AI” approach[183] is required.

Conclusion

South Africa has neither an overarching AI strategy nor any specific laws governing AI. Although there may be some temptation to adopt a “wait and see” approach,[184] early and proactive engagement in the regulatory endeavour is important to ensure that laws are not Western “imports” but are fashioned to be appropriate to the South African context.[185]

The development of a national policy framework of guiding ethical principles would in no way undermine the existing legislation and ethical guidelines governing health care practitioners, which must be read alongside AI guidelines, and implemented to their full effect.[186]

This article has examined three key areas for legal reform in relation to AI in health care. The first is that the regulatory framework for the oversight of software as a medical device needs to be updated to develop frameworks for adequately regulating the use of such new technologies. In this regard the WHO framework[187] provides a solid starting point for the planning of clinical and research studies and the reform of South Africa’s regulatory system to accommodate AI software as a medical device.

Secondly, the present HPCSA guidelines for health care practitioners in South Africa adopt an unduly restrictive approach centred in the outmoded semantics of telemedicine. This may discourage technological innovation that could improve access to health care for all, and as such the guidelines are inconsistent with the national digital health strategy. As a first step, such guidelines should be amended to expressly permit the use of AI and to provide additional guidance on informed consent in such contexts.

Thirdly, the common law principles of fault-based liability for medical negligence could prove inadequate to providing patients and users of new technologies with redress for harm. Consideration should be given to developing a statutory scheme for strict liability, together with mandatory insurance, and the appropriate reform of product liability pertaining to technology developers and manufacturers. It is suggested that the EU model should be considered as a starting point for developing an AI Act for South Africa.

These legal reforms should not be undertaken without also developing a coherent, human rights-centred policy framework for the ethical use of AI, robotics, and related technologies in health care in South Africa.