Ethics of Artificial Intelligence in Medicine and Ophthalmology.

BACKGROUND: This review explores the bioethical implementation of artificial intelligence (AI) in medicine and in ophthalmology. AI, which was first introduced in the 1950s, is defined as "the machine simulation of human mental reasoning, decision making, and behavior". The increased power of computing, expansion of storage capacity, and compilation of medical big data helped the AI implementation surge in medical practice and research. Ophthalmology is a leading medical specialty in applying AI in screening, diagnosis, and treatment. The first Food and Drug Administration approved autonomous diagnostic system served to diagnose and classify diabetic retinopathy. Other ophthalmic conditions such as age-related macular degeneration, glaucoma, retinopathy of prematurity, and congenital cataract, among others, implemented AI too.
PURPOSE: To review the contemporary literature of the bioethical issues of AI in medicine and ophthalmology, classify ethical issues in medical AI, and suggest possible standardizations of ethical frameworks for AI implementation.
METHODS: Keywords were searched on Google Scholar and PubMed between October 2019 and April 2020. The results were reviewed, cross-referenced, and summarized. A total of 284 references including articles, books, book chapters, and regulatory reports and statements were reviewed, and those that were relevant were cited in the paper.
RESULTS: Most sources that studied the use of AI in medicine explored the ethical aspects. Bioethical challenges of AI implementation in medicine were categorized into 6 main categories. These include machine training ethics, machine accuracy ethics, patient-related ethics, physician-related ethics, shared ethics, and roles of regulators.
CONCLUSIONS: There are multiple stakeholders in the ethical issues surrounding AI in medicine and ophthalmology. Attention to the various aspects of ethics related to AI is important especially with the expanding use of AI. Solutions of ethical problems are envisioned to be multifactorial.

Artificial Intelligence (AI) is poised to play a major role in medicine and health care in the foreseeable future as a tool to aid in screening, diagnosis, and management of various diseases. Ophthalmology is one of the leading specialties in the field of AI and ophthalmology generates an enormous volume of imaging and data that could be used to train and validate AI. However, implementing AI in medicine raises many ethical dilemmas that span the fields of medical education, research, and practice. These dilemmas impact patient care, doctors’ roles, and the roles of those involved in the field of medicine-governments, regulators, insurers, payers, and other providers. The role of AI is still evolving and has the unique potential of functioning independently from human beings. AI’s role in medicine should be transparent in order for its benefits to outweigh its potential harms.[1] Opportunities of medical AI exist in health care delivery, health care management and administration, research, and prevention. Its challenges are classified as technical, ethical, methodological, regulatory, social, and infrastructural.[2]

METHODS

Keywords were searched on Google Scholar and PubMed between October 2019 and April 2020. The results were reviewed, cross-referenced, and summarized. A total of 284 references including articles, books, book chapters, and regulatory reports and statements were reviewed, and those that were relevant were cited in the paper. These references were relevant to ophthalmic AI, AI use in various medical specialties, and AI bioethical, technical, and regulatory subtopic keywords. eTable 1, http://links.lww.com/APJO/A88, explains the workflow of literature search, sorting, summarizing, and reporting.

RESULTS

Bioethical challenges of AI implementation in medicine were categorized into 6 main categories. These include machine training ethics, machine accuracy ethics, patient-related ethics, physician-related ethics, shared ethics, and roles of regulators. eTable 1, http://links.lww.com/APJO/A88, summarizes the 6 categories of ethics in AI.

Machine Training Ethics

Health care data is the fuel for deploying AI in medicine. The data is generated through various resources, such as electronic health records, clinical trials, and health actuaries. It is utilized to supplement regulation of health care in order to support research and modify health policies.[3] Data resources need to be connected, standardized, and uniformly formatted to become optimally useful in propelling the application of AI forward in the health field.[4]

Data Ownership

Data ownership indicates authority to control, process, or access data. It also conveys profitability from the right to sell data or to receive compensation.[6,8,9] Medical data similar to physical property data ownership should be covered byproperty lawsand/or intellectual right laws should decide ownership.[7] These data could be public or private (eFigure 1, http://links.lww.com/APJO/A98).

In addition to patients, other parties may claim the right to data ownership such as:

Health care providers, doctors, practices, and hospitals (private or public). Public (governmental) data facilitates data access for research and warrants their security.[10] Private partnership incentivizes flexibility of data distribution;[4]

Health actuaries and insurers for administering and funding the process of data generation;

Developers, designers, manufacturers, and cloud storage providers (corporations or individuals) responsible for data generation, processing, and storage.[11] eFigure 2, http://links.lww.com/APJO/A99, illustrates the data ownership issues. Table 1 demonstrates the consequences of data ownership.

TABLE 1.

Consequences of Data Ownership

Meaning of Data Ownership
Authority	Profitability
- To access data	- To sell data
- To control data	- To be compensated for data revenues
- To process data

Data Protection

Confidentiality and Privacy.

Privacy concerns regarding the implementation of AI revolve around data collection and utilization. Medical data privacy should not be strictly venerated. Rather, it should be treated contextually—contextual rules about information flow depend on the parties involved, the process of accessing information, and the frequency and purpose of access.[12] These contexts for data handling should be set in informed agreements between data sources and data users.[13]

Patients’ confidentiality is twice subject to exposure. First, exposure when fed into the electronic medical records, and second when electronic medical records get connected to AI systems.[14]

Data privacy violations lead to many issues such as:

Discrimination or deprivation of insurance or employment;

Emotional stress from sensitive health data exposure;

Mental health consequences such as embarrassment, paranoia, and mental pain;

Deontological concerns about personal data vulnerability;

Erosion of trust;

Failure to seek health care service or withholding information to protect privacy;

Group-based harm.[11,13]

Privacy overprotection puts economic and technological constraints on innovation and big data development. Companies have and could continue to exploit privacy rules to guarantee secrecyand proprietary authority, eroding public trust in AI systems.[13] To increase the societal confidence in AI research, clinical trials are advised to mandate availability and transparency of data in the trials results. Informed choice of data disclosure should be offered to trial participants. However, intellectual rights for trial conductors should also be considered.[15] eFigure 3, http://links.lww.com/APJO/A100, illustrates the concerns about data privacy violations.

Confidentiality Protection Solutions.

AI data confidentiality solutions include governmental regulations, technology advances, or funding incentives.[16]

Regulations to Protect Confidentiality.

The European Union (EU) has legislatures of General Data Protection Regulation (GDPR), Cybersecurity Directive, and Medical Devices Regulation. In the US, the Health Insurance Portability and Accountability Act (HIPAA) is suggested as a counterpart for European legislation to cover wider confidentiality issues in medical data.[14]

Technological Solutions.

Some technological approaches to protect data privacy, each with its promises and shortfalls:

Ad hoc deidentification;

Pseudonymous data;

Provable (differential) privacy adds a small calculable amount of noise into the data input to deidentify the output. It is a controllable and guaranteed method for data deidentification;[17]

Anonymous resolution technique enables advanced data correlation while using only irreversible cryptographic hashes. It helps discover records of common interest (eg, identities) across systems without transfer of any personally identifiable information. It significantly reduces the risk of unintended info disclosure;[18]

Privacy audits ensure appropriate use and security standards to guard against unauthorized use.[13]

Cybersecurity.

Cybersecurity protects data pools from hacking. In the US, the Food and Drug Administration (FDA) is responsible for ensuring that companies abide by cybersecurity standards in their approved products. In the EU, the Directive EU Cybersecurity Act 2016 is the regulatory framework for protection against cyberattacks.[14,20]

Sharing Patients’ Data.

Data sharing implies data accessibility.[21] Sharing patients’ data form a Machine training amongst institutions and countries should not compromise patients’ confidentiality. Data sharing issues reflect a conflict between individuals’ desire for privacy and desire for providing evidence to drive health care.[22] Routes of medical data sharing include collaboration within health care communities, sharing with research institutes (outsourcing), sharing with pharmaceutical and insurance companies, dissemination for legal purposes, open-access and proprietary dataset portfolios, and governmental data cloud initiatives.[19,27,28]

Advantages of medical AI data sharing include: 1) interoperability between medical AI systems, 2) coordination and exchange of data between health care providers’ data pools to ensure they are cohesive rather than fragmented, 3) scientific discovery to improve individual and population health, and 4)setting equity in health care service quality nationally and globally.[19,25]

Meanwhile, data sharing concerns include: breaches of privacy, failure to protect the confidentiality, individual data control, false-positive signals, cybersecurity challenges, and pressure imposed on practitioners to collect more data to cope with and contribute to emerging health care technology trends.[22,23,24,26,27]

Machine Accuracy Ethics

Machine Accuracy

Machine accuracy means performing certain tasks with greater consistency and speed than humans when benchmarked against gold standards. AI accuracy has been proved in diagnostic, therapeutic, and prognostic studies.[29,30] Machines consider their outcome absolutely accurate, whereas human decision in medicine is always dealt with as one alternative of the truth.[31,37] Diagnostic accuracy studies in medical AI research have often been suboptimally reported.[32,33] Most AI ophthalmology studies are validated in silico on retrospective datasets. Ideally, real, prospectively collected datasets should be used to validate AI.[34,36]

With continuous data processing and learning, the device accuracy is not guaranteed. Status quo (frozen) AI programs are more reliable after approval. The FDA conditioned freezing of the algorithms to ensure stability of performance after approval.[41]

Exaggerated accuracy due to intrinsic problems in algorithmic design is called overfitting. Artificial neural networks overfit either from too small sample sizes or too high numbers of nodes. Noise of random fluctuations in the training data are picked by the model as learned concepts. These concepts do not generalize to the new validation data because they are highly customized to the training data. Overfitting exaggerates accuracy and overestimates the model’s clinical performance.[42-44]

Intelligibility and Transparency

Professional societies and ethical committees issue policies to mandate transparency in developing AI projects in health care and research. They also mandate guidelines for posting results.[48,49] AI transparency enhances scientific advancement and improves public quality of life.[25,45] However, AI tools lack transparency either because of their innate complexity, or for intentional proprietary purpose.[49,50]

AI transparency covers:

Transparency of data at all stages of the process;

Transparency in sample selection for AI training data to guarantee inclusiveness, equity, and generalizability of health care research results in all population sectors;

Intelligibility of data processing maneuvers.[46,47] Table 2 demonstrates intelligibility and transparency issues.

TABLE 2.

Intelligibility and Transparency Issues in AI

Where transparency in medical AI should be sought?
Transparency of data in all stages.
Transparency in sample selection.
Transparency in data processing between input and output stages.
Transparency in Decision-making.
Any AI system is opaque (unintelligible) for 2 reasons
Innate complexity of the system itself.
Intentional proprietary design for the sake of secrecy and proprietary interests.

AI indicates artificial intelligence.

Highly accurate AI models are usually not intelligible. They might follow subtly biased rules to give management decisions ignoring important health care determinants. Low accuracy, rule-based systems are more transparent and more ethically appealing.

There is always a trade-off between accuracy and intelligibility.[39]

High transparency issues consist of:

Jeopardized data security;

Increased vulnerability to attacks;

Increased exposure of private patients’ data;

Compromised intellectual copyrights for the designers and developers;

Frustrated proprietary benefits for companies.[53,54]

Black-Box Phenomenon

Black-box tools are feared because they are opaque, complex, and lack transparency.[51] This phenomenon is a hurdle to the process of certification. Black-box AI tools are not easily amenable for governance. Continuous appraisal of research about the safety and efficacy of black-box devices is mandatory for their validation but difficult.[52] Black-box intelligent tools transform regulatory rules—most notably rules of liability.[51] If black-box AI tools attain autonomy, they become even more opaque, less controllable, and harder to regulate in our current medical governance systems.[51] eFigure 4, http://links.lww.com/APJO/A101, illustrates the potential consequences of lack of transparency in AI systems.

Biases

Bias in medical machine learning and AI is part of a broader ethical problem with many consequences. In most instances, an AI tool that gives a wrong decision usually reflects biases inherent in the training data. Biases might beracial, ethnic, genetic, regional, or gender-based. Mental shortcuts in human logic, such as hypothetical-deductive reasoning, individual judgment, and heuristics are based in biased judgment. Bias in machine judgment is thought to stem from human bias.[56,58] When biases are more subtle, they will be harder to anticipate and more damaging to resolve.[55,57] Table 3 demonstrates solutions for potential biases in AI.

TABLE 3.

Solutions for Potential Biases in AI

Issues		Solutions
Providing robust data sources	Equal distribution of data features in all population sectors	Defining the source of bias in the training datasets
Fair representation of the population in training datasets	Creating gold standards to benchmark medical AI	Using AI itself to detect real-time bias
Existing standards to assess the risk of bias in prediction models	Continuous postapproval tracking	Intentional oversampling of under-represented populations

AI indicates artificial intelligence.

Here are some approaches to the issues of bias:

Robust data sources fed into algorithms;[59]

Creating gold standards in all fields of medicine;[59]

Equal representation of populations’ sectors in training data-sets;[60]

Intentional or preferential (explicit) unequal distribution of datasets to mimic the real distribution of a condition in the population;[61]

Sometimes, accepting real data with their implicit bias;[39,44]

Thoroughly examining training data for bias;[27]

Exclusion of some sectors of the population to mitigate bias based on disease representation. For example, excluding dia-betic retinopathy to find the actual distribution of less common vasculopathies;[62]

Cooperation between doctors and AI to eliminate sources of bias;[63]

Using machine learning as a tool to assess and categorize risk of bias in randomized trials;[55,64]

Standards like PROBAST tool used to assess risk of bias in prediction models and aid algorithm developers in selecting representative training sets and appropriate predictor variables;

Continuous tracking for algorithm predictions;

Creating simulated data sets with high numbers of omitted variables and conducting counterfactual simulations to determine predictions’ sensitivity to detect omitted variable bias;

Oversampling under-represented populations.[65]

Patient-related Ethics

Patients’ ethical rights in research work were initially summarized in the Belmont report, which addressed ethical principles of research on human subjects. These principles are beneficence, nonmaleficence (safety), autonomy, and justice.[66] Patient-related ethics in AI are based on these rights. Patients exercise their rights either explicitly through informed consent or implicitly through norms of confidentiality or regulatory protections.[67]

Informed Consent

Informed consent is based on the principle of autonomy.[68] The informed consent could authorize partial or complete role of algorithms in health care services and detail the process of reaching diagnostic or therapeutic decisions by the machines.[69] Clinicians should explain details of these processes to their patients. Patients should have the choice to opt in or out of allowing their data handling, processing, and sharing. Patients who opt out should have their electronic health record data excluded from AI algorithms connected by any means to the electronic health record.[6,67] Informed consent should reach satisfactory levels of fulfilling patients’ awareness about the scope for data use.[71] Table 4 demonstrates the informed consent issues.

TABLE 4.

Patients’ Rights and Informed Consent

Patients’ Rights Implicated by Informed Consent
Right to possess their data
Right to sell their data
Right to destruct their data
Right to access their data
Right to authorize doctors to access their data
Right to block access to their data
Shared responsibility for patients and doctors to all the above rights
Joint authority on patients’ data with other multiple parties[72]

Confidentiality

Patients’ confidentiality is legal obligation and a code of conduct. Confidentiality involves the responsibility of those entrusted to handle and protect patient’s data. It encompasses the use, storage, access, and dissemination of these data.[6] With AI, it means trusting and comfortably confiding information to machines through all the stages of AI training, validation, and output acceptance.[59] The confidentiality concept needs to be reimagined in the era of AI.[14,55] Confidentiality risks include inappropriate datasets use, inappropriate disclosure, and limitations in data deidentification techniques.[73]

Confidentiality issues and their solutions are considered according to economic, cultural, and social, and educational disparities in different countries.[74] Global and local regulations should consider the implications of private and public authority over patients’ data, and the conflict between private and public interests.[75]

Physician-related Ethics

Dependence on Machines

Almost any new technology leads to a degree of dependence.[76] Physicians’ dependence on AI can mislead them by making them vulnerable to inflicting harm and subject to litigation.[70,77] Consequently, using AI might result in decreased efficiency in decision making.[38,40] Overdependence on AI, also termed “automation bias,” is a short-term symptom. The long-term symptom is loss of skill. Doctors’ skills are either completely or partially replaced by technology, leading to possible loss of self-confidence or competence.[37,44,78]

Machine Rivalry and Substitution of Doctors Role

Machine rivalry with doctors threatens to negatively impact doctor-patient bonds or overtake jobs.[79] Even if AI could perform some clinical tasks more safely and efficiently than doctors,[27] it is not projected to cover all the clinicians’ judgment. It should always be kept in mind that the aim for adopting AI is to augment and assist doctors, not to replace them.[80] The trend of fearing automation could be just another repeated fleeting fear, and AI could one day be an established part of medical practice.[81] Doctors’ jobs should not be taken by AI, rather, they should be reidentified by AI. Doctors could abdicate some roles to AI—they may then be displaced by it rather than replaced.[82]

Trust

Trust in doctors is based on the human capacity to react emotionally toward patients. It has piled over thousands of generations of performance in medicine. Trust in intelligent machines might not need that long. Yet, it certainly needs time and effort to form in the public mind.[83] Patient-machine interaction should be based on trust bonds, just like patient-doctor human bonds.[87] Ethical governance and long-term auditing of AI clinical safety and performance are needed to secure trust in the machine era.[86] The concept of trust consists of 3 components: competency, motive, and transparency.

Competency is to master the role in caring for patients. AI can enhance doctors’ competency and empower patients by facilitating their access to their data and access to health services and by making them shareholders in the economic value of their data. Biased, inaccurate, ineffective, and unexplainable AI still compromises trust and impacts patients’ autonomy. Motive means solely acting for the patients’ interest. If AI gives physicians more time to attend to bonding with patients, then AI will promote doctors’ motives. On the other hand, AI will harm trust if it is merely used to increase workflow and to add more patient numbers, and put doctors under the pressure of dealing with more cognitively demanding tasks. Transparency is the openness of patients to divulge more of their personal data. For physicians, it means openness about their uncertainties. Machines’ transparency means intelligibility and avoidance of black-box nature.[84,88]

Empathy

Empathetic skills and knowledge need to be further incorporated into medical education and training programs. AI performing some tasks offers space for doctors to utilize empathy in medical education and training.[89] Patients expect doctors to be more empathic than machines and prefer a bigger role of doctors in their care.[90] Although doctors are not excused for lacking empathy, machines can allow them to exercise empathy by providing them more time.[92] Patients contribute their data for algorithm training, development, and validation, and algorithms give decision output about patients’ health. This novel interaction needs to be ethically and empathetically calibrated.[95]

Supposedly, by feeding thousands of patients’ scenarios into an algorithm, AI will learn to generate empathetic reactions. AI develops “artificial affection” to feel and express feelings of pain, allowing machine personhood to be enhanced. This agency serves 2 purposes:

Machines’ ability to empathize with patients;

Machines’ liability for harm inflicted by their actions.[91,93]

In business as well as in health care, AI is being used to improve customers experience by showing empathy, responding to client dissatisfaction, and offering solutions.[94]

Shared Ethics

Liability and Culpability

Liability of harmful consequences due to AI diagnostic and therapeutic uses is one of the obstacles of deploying AI. Rectifying the issue of liability will enhance the role of AI in health care services. Medical AI has many applications, all of which come with their concerns of malpractice and harm.[96] Liability in medical laws is based on comparing the clinician job with the highest clinical standard. It is suggested that medical AI set such a standard for comparison. Two aspects to be considered for AI liability standards:

The standard of care is changing overtime;

Medical AI tools are evolving overtime.

Two possibilities follow these considerations:

Doctors become liable when they incur harm while following AI recommendations;

DoctorsmaybeheldliablefornotfollowingAIrecommendations if the AI recommendations become the standard of care.[97,100]

The more autonomous and unsupervised the intelligent system, the more refinement of liability rules will be needed.[101] Current medical legislations deal with liability issues by:

Imposing responsibility to recover the injury if it is recoverable;

Compensating for it if it is not recoverable.

For medical AI, both legislative and regulatory laws need to be reconstructed to fit issues of liability.[98,99]

Multiple parties share liability in the eye of law. These entities include physicians (negligence laws), health organizations (vicarious liability), and manufacturing companies (products liability).[98]

Legal suggestions to cover laws of liability:

Designers should be held liable for any harm inflicted by their machines;

Systems behaving autonomously without human control should be held liable;[101]

Machines entitled for personhood being treated with laws of tort liability to be covered with malpractice insurance just like physicians;

“Common enterprise liability,” which combines elements of medical malpractice, products liability, and vicarious liability. Many entities share the burden of liability to compensate the injured party;[51,98]

Liability could be inferred on third parties, such as manufacturers, investors, and insurers for medical AI systems;

No-fault liability compensation for harm to circumvent negligence laws.[97]

These suggestions need to be subsequently audited and appraised for AI medical implementation.[70]

Accountability and Responsibility

Responsibility for clinical decision—making regardless of any harm inflicted on the patient applies when using AI. Doctors are responsible for full awareness of the capacities of the machines they use. AI companies are responsible for full disclosure of the functions and limitations of their products as well as training on use of these machines, but standards are few.[27]

Cost

Medical intelligent machines have attendant costs for data storage, data curation, and model maintenance and updating. These costs and related needs may simply replace current costs of health care service with different and potentially higher costs.[102]

Role of Regulators

Regulatory concerns about medical AI are divided into 3 groups:

Data sourcing;

AI device development and design;

AI clinical deployment.[67]

Regulatory standards are awaited to settle issues of liability, privacy, and ownership of data in AI. In the diversity of global economic, social, and regulatory standards, AI exaggerates inequality.[68] In regulating technology and protecting consumer data, Europe has adopted a more consumer-oriented, conservative, precautionary approach whereas the US adopts a more market-driven, liberal, and permissionless approach.[14,103,105] eFigure 5, http://links.lww.com/APJO/A102, illustrates the various regulatory issues.

Standardization and Quality Assurance

Global standardization customizes machine bioethics to fit the diversity of human societies and determine the roles of health care providers. These standards set the rules for implementing AI in medicine to guarantee its contribution to health care practice and research.[93] Standardization also means formulating data nomenclature, collection, formatting, storage, and retrieval. This facilitates data exchange and integration into clinical practice.[106] Standardization is a difficult task because health care data consists of imaging, laboratory values, unstructured language information of history and examination in electronic medical records, and so on. These categories of data should be uniformed into standard formulas to be fed into algorithms.[27] Any inconsistencies in the quality of health care data reflect inconsistencies in the quality of AI performance.[83,107]

Marketing of AI Systems as Medical Devices

Because of their progressive development, self-modifiability, and trainability, learning machines could develop beyond the existing approval and licensing criteria. A graduated approach to regulation is suggested for AI systems. Regulators must focus on postmarketing surveillance to ensure continuously improving AI performance.[60]

Licensing of AI Systems as Intelligent Players in Health Service

If some AI systems are licensed to fulfill the demand of a doctor’s role in medicine, then they should be dealt with in a system identical to doctors’ licensing systems. That is, they should go through licensing, training, continuous medical education, and relicensing.[79]

EU GDPR

The new EU GDPR mandates explicit informed consent for data use. Through informed consent, the patients are given the right to opt in or out from any potential processing of their data through the “right of explanation.” GDPR also empowers patients to track or remove their data through the “right to be forgotten.”[27] GDPR imposes transparency in medical practice and research by:

Giving the patient the right to access and check accuracy of their data;[108]

Deconvoluting black-box algorithms before using them in patient care.[43,46]

HIPAA and HIPAA Privacy Rule

HIPAA was passed in 1996 and is very strict against data sharing. It sometimes even prohibits sharing data with the patient.[109] Although HIPAA mandates data deidentification, it does not protect them from reidentification.[111] Compared with GDPR:

HIPAA has narrower types of data coverage—it only covers health insurance data;

HIPAA is stricter than GDPR in requiring permission of data use and disclosure;[110]

HIPAA allows data disclosure without consent when different parties share the personal data for the purpose of treatment;

HIPAA does not acknowledge the right to be forgotten;

HIPAA has similar prompts for organizations to notify any breach of sensitive patients’ data.[4]

FDA

The FDA embraces several regulatory pathways for ophthalmic and medical AI.[104] The FDA recognizes that AI-based technologies are distinct from traditional medical devices. It has adopted a new category called Software as Medical Device (SaMD). Based on the level of risk, an SaMD is either streamlined for or exempt from premarket review.[27] Three risk classes balance patients’ safety and benefit. Class I (mild risk) devices with no treatment decisions are exempted from premarket approval. Class II (moderate risk) and Class III (severe risk) need premarket approval. Based on the existence of a previous, similar technology, an “evolutionary device” has a predecessor, whereas a “revolutionary device” has none.

According to the degree of human clinician input, imaging algorithms and clinical decision—support software have 2 categories—computer-aided detection for detecting diseases with no characterization and computer aided diagnosis for assessment, diagnosis, differential diagnosis, extent, and prognosis.

The following are FDA pathways to evaluate medical technology:

Premarket approval, suited for Class III revolutionary devices, is the most stringent and requires rigorous clinical and nonclinical studies. It evaluates safety and efficacy on a substantial population;

De novo approval for revolutionary Classes I and II devices;

510k suited for Class II, evolutionary (with predicate) devices;

Humanitarian exemption.[7]

To guarantee the accuracy of approved AI devices, the FDA and other regulatory systems emphasize some or all the prerequisites listed in the following table.[31] Table 5 demonstrates prerequisites for AI accuracy.

TABLE 5.

AI Device Accuracy

Prerequisites to Guarantee AI Device Accuracy
Accurate input and output
Defined disease prevalence
Defined study settings
Defined sample size and statistical metrics
Reproducibility of sample in certain population
Defined disease classification system
High quality images
Robust research work
Reproducible research process Generalizable research results

AI indicates artificial intelligence.

Additional Ethical Dilemmas With Machine Learning Integration

Can Machines Function Ethically?

Machines must function ethically to play a part in the clinical process. They have repeatedly been delegated to perform human tasks. However, society must ensure these machines function in an ethical manner with patients’ best interest as the primary value.[86]

Can Machines Learn Ethics, or Can Ethics be Taught?

Machines should be taught a universal code for ethics, or multiple local codes. Ethical bases for AI differ from human ethical codes. Human ethical codes are not fully established—they are a subject of a long-lasting debate and cultural diversity. Therefore, to standardize ethics of medical practice, what ethics to train machines on must first be decided. Approaches of top-down, bottom-up virtue ethics all need to be explored to decide what fits medical AI ingeneral and specific domains.[96] Top-down ethics impose a theoretical educational model on machine decisions. In a bottom-up approach, machines learn ethics themselves by development or experience. Both approaches have their limitations, so hybrid approaches are being suggested to compensate for these limitations.[112]

Moral calculation concept is the machine counterpart for human moral psychology. Binary ethical controversies and questions are put into an algorithm following yes/no cascades or if/then rules.

Can Ethics be Put Into Algorithms?

Ethical questions are formulated in algorithmic mathematical formulae and fed into algorithms. Implicit moral machines are fed with ethical rules. Explicit moral machines can calculate their ethical rules.[87] Ethical machines are judged based on their autonomy and sensitivity to moral values. However, machine ethics are too complex to consolidate in algorithms, successfully putting these ethics into algorithms would allow for efficiency beyond human judgement and decision making.[85]

Does AI Need Special Ethical Guidelines and Considerations Other Than the Current Conventional Ethical Rules Governing the Marketing of Medical Devices?

AI tools ethical guidelines cover marketing and licensing:

Marketing of AI systems as medical devices is the current role of the FDA and Conformitè Europeënne (CE, European Union CE Marking);

Licensing and relicensing for machines as intelligent counterparts for “human doctors.”

How Can AI Systems, While Undergoing Machine Learning and Developing Algorithms, Assess Risk, Determine Benefit/Risk Ratio, and Prioritize “No Harm” While Making Health Care Decisions?

This is the net result of all the above discussion. All ethical guidelines on AI implementation and standards of responsibility/liability are hinged upon the principle of “doing no harm.” As early as the 1950s, the theoretical bases for robotic ethics, based on the fictional work of Isaac Asimov, formulated the 3 robotic laws of Asimov. The first rule stated that a robot should not harm a human, neither by action, nor inaction. Medical AI bioethical research has always considered the Asimov laws, no matter how primitive they were, in the bioethical design of medical AI.[93]

CONCLUSIONS

Ethical aspects of using AI in medicine and ophthalmology have been discussed in literature resources that studied AI use in medical education, practice, and research. Significant components to consider for the ethics of AI implementation in medicine include machine training ethics, machine accuracy ethics, patient-related ethics, physician ethics, and shared ethics. Many texts suggested solutions to ethical challenges and gave answers to ethical questions. Nearly all literature resources praised AI and projected its potentially beneficial role in transforming medicine. However, all these text resources showed that there should be universal standardization for ethicaland regulatory considerations for the optimal implementation of AI in medicine and ophthalmology.