On January 18, 2024, the World Health Organization (WHO) published, “Ethics and governance of artificial intelligence for health Guidance on large multi-modal models,” which follows an earlier 2021 guidance on AI and addresses regulatory issues surrounding generative AI, specifically large multi-modal models (LMMs).
ABOUT LMMs
According to the WHO guidance, LMMs are models that can “accept one or more type[s] of data input and generate diverse outputs that are not limited to the type of data fed into the algorithm.” Although similar to other forms of AI, the guidance states that “how LMMs are accessed and used is new, with both novel benefits and risks that societies, health systems and end-users may not yet be prepared to address fully.”
Often referred to as general-purpose foundation models, LMMs are also said by the guidance to differ from the “500 AI models for clinical medicine [that] have been approved by the Food and Drug Administration,” in that while those models were approved for “only one or two narrow tasks,” LMMs are versatile and can be, “used in numerous tasks, including some for which they were not explicitly trained.”
The guidance builds on the WHO’s 2021 prior guidance on the ethics and governance of artificial intelligence for health, which identified the following principles: (1) protect autonomy; (2) promote human well-being and safety and the public interest; (3) ensure transparency, “explainability” and intelligibility; (4) foster responsibility and accountability; (5) ensure inclusiveness and equity; and (6) promote AI that is responsive and sustainable.
REASONS FOR PROVIDING GUIDANCE
In sharing its reasons for issuing the guidance, the WHO noted that it has been “predicted that LMMs will have wide use and application in health care, scientific research, public health and drug development.” Additionally, the organization stated that even though relatively new, “the speed of their uptake and diffusion led [it] to provide this guidance to ensure that [LMMs] could potentially be used successfully and sustainably worldwide.”
More specifically, the WHO stated that such speed “requires that governments rapidly develop regulations and specific criteria for using these AI algorithms in health-care systems and for other scientific and medical purposes.”
POTENTIAL BENEFITS & RISKS OF LMMs IN HEALTHCARE
Before exploring the regulation of LMMs, the guidance identifies potential benefits and risks across the areas of diagnosis and clinical care, patient-guided use, clerical and administrative tasks, medical and nursing education, and scientific research and drug development.
For example, as to diagnosis and clinical care, potential benefits offered by LMMs cited by the guidance include the following:
- They can assist in managing complex cases and with the review of routine diagnoses;
- They can reduce the communication workload of health-care providers; and
- They can provide novel insights and reports from various unstructured forms of health data.
Potential risks in this area include the following:
- Inaccurate, incomplete or false responses;
- Poor quality training data;
- Bias (of training and responses);
- Automation bias;
- Degradation of skills of health-care professionals; and
- Issues surrounding the informed consent of patients.
Potential noteworthy risks when it comes to patient-guided use also include “inaccurate, incomplete or false statements,” and such things as the risk of delivery of care outside of the health system.
When it comes to scientific research and drug development, potential benefits include innovations through de novo drug design as well as the generating of insights from data. A key potential risk is that it is difficult to hold algorithms accountable for content.
It should also be noted that the guidance focuses on risks to health systems, such as those potentially caused by the dependence of health systems on ill-suited LMMs, and cybersecurity risks.
THE AI VALUE CHAIN: DEVELOPMENT, PROVISION, AND DEPLOYMENT
The guidance identifies three stages in the AI value chain: development, provision, and deployment. Deployers at the deployment stage and providers at the provision stage are distinguished by providers not being referred to as health care providers, but as third parties who use a general-purpose foundation model through an active programming interface (API) for a specific use. (Such third parties can include healthcare providers).
Providers may get involved in fine-tuning an LMM, which may include additional training of the foundation model; integrating the LMM into applications or a larger software system that provides service to users; or integrating components known as “plug-ins” to channel, filter and organize the LMM into formal or regulated formats.
Deployers, who according to the guidance can be thought of as customers of the providers, use the acquired or licensed LMM-related products or applications directly for patients.
When it comes to developers, the guidance refers to those who develop the LMM models, whether they be general-purpose foundation models, health or medical domain specific foundation models, or health system specific foundational models. This can include large and small technology corporations, universities, and health systems, among others.
GOVERNANCE FRAMEWORK
The WHO guidance notes that the three stages of the AI value chain above can be thought of as one of the methods used to frame governance. This is because certain risks can be addressed at each stage. Additionally, the guidance notes that allocation of responsibility between developers, providers and deployers is affected by stage, noting that, “there are clear areas in which each actor is best placed or is the only entity with the capacity to address a potential or actual risk.”
The guidance applies the following three questions to each stage:
- What risks should be addressed at each stage of the value chain, and which actors are best placed to address those risks?
- What can a relevant actor do to address the risks, and which ethical principles must be upheld?
- What is the role of government, including relevant laws, policies, and regulations?
At the development stage, the guidance states that responsibility lies with developers and that the role of governments with regards to them is to “set laws and standards to require or forbid certain practices,” relevant to this stage.
At the provision stage, according to the guidance, responsibility lies with both providers and developers, and it takes the position that governments are “responsible for defining the requirements and obligations of both entities.”
At the deployment stage the guidance states that certain responsibilities and obligations continue for developers and providers, such as when they are also the deploying entities, or when there are risks that can only be addressed after deployment. At this stage, the guidance also takes the position that individuals harmed by an LMM should be protected by liability rules/regulations, further stating that governments should consider applying strict liability standards.
GOVERNANCE CATEGORIES
The WHO touches upon various aspects of governance when it comes to LLMs.
While it doesn’t explicitly breakdown these aspects along the following categories, such categories nevertheless surface throughout the entire guidance and its various sections and consist of:
- Assessment methods such as audits and impact assessments;
- Best practices such as the inclusion of transparency principles;
- Credentialing-related areas such as certification, licensing, and registration; and
- Notice requirements such as operational disclosures.
Assessment Methods for AI in Healthcare
The guidance touches upon numerous methods that can be used to assess various aspects of LMMs. For example, in the case of audits, the guidance recommends their use in the initial stages of the development of foundation models, in the performance of the models, at the post-release stage, and even for downstream products and services built on LMMs. It also recommends performing governance audits of providers of LMMs. Datasets are also included, and it is recommended that there be audits to evaluate disability bias.
Other assessment methods addressed in the guidance include red teaming, which is the “evaluation of a model or system that [identifies] vulnerability in real-world simulations,” and impact assessments, including those that last for the duration of the lifecycle of an LMM.
Best Practices in Healthcare AI Governance
The guidance addresses a number of principles that could be considered as falling under the category of best practices. One such area, as seen under the governance framework, focuses on the proper burden allocation between developers, providers and deployers, with the WHO taking the position, for example, that the “regulatory burden on a provider should increase if the product or application substantially diverges from or changes the foundation model in ways that are out of the control of the developer of the model.”
Another instance can be seen where an LMM is regulated as a medical device, with the guidance stating that governments should ensure that “the developer and/or the provider is responsible for the burden of proof that the device performs as marketed.”
Another principle emphasized in the guidance is transparency. Here it is recommended that certain aspects of an LMM and its applications “should be transparent to allow oversight and assessment of its safety and efficacy by regulators. These aspects may include the source code, data inputs, model weights, and the analytics approach used.
The WHO further recommends that the disclosure of the performance of an LMM or application in internal testing should be considered, and as to post-release impact assessments, mentioned above, the WHO recommends that the results be published and should “address outcomes and impacts disaggregated by type of user, for example, by age, race or disability.”
Credentialing for AI in Healthcare
The guidance contains various registration, licensing, and certification recommendations, which together can loosely be considered as credentialing requirements.
Concerning registration, the WHO explores the idea of requiring developers to register early-stage AI algorithms that would be used in health care and medicine, saying that it could “encourage publication of negative results and prevent publication bias or over-optimistic interpretation of results,” among other things.
Along similar lines, the guidance notes the WHO’s previous recommendation that developers consider licensing requirements for developers of ‘high-risk’ AI, specifically including for health.
The guidance also says that pre-certification programs mandated for developers can help identify and avoid ethical risks. It also recommends that developers of LMMs that could or are intended to be used in health care, scientific research or medicine, “should consider certification or training to align themselves with requirements in the medical profession and also to increase trust in their products and services.”
It can also be noted that various training and educational recommendations made by the WHO might also fall under this category.
Notice Requirements for AI
Although related to transparency, the providing of warnings or notices for risks associated with LMMs is also important and according to the guidance, such risks should be “advertised clearly,” and have been compared to nutrition labels in food. Such notices include disclosures that content produced by an LMM is generated by an AI system, and that LMMs have a propensity to hallucinate.
According to the guidance, some examples of laws that can be considered related to LMMs are in the areas of data protection, human rights, medical devices, and consumer protection.
Categories: AI, Uncategorized
Leave a Reply