Emerging artificial intelligence (AI) is predicted as revolutionary in many application fields including medical and healthcare. However, AI systems in medical and healthcare (AI medical systems) would exhibit “new factors specific to AI” not included in conventional technologies. This paper sheds spotlights to clarify the characteristics and potential issues of AI medical systems from viewpoint of the regulatory science – characteristics and clinical positioning of AI medical systems, and safety factors rooted in the nature of AI, as well as reliability of data sets in machine learning.
AI medical systems have unique characteristics including 1) plasticity causing changes in system performance through learning, and need of creating new concepts about the timing of learning and assignment of responsibilities for risk management; 2) unpredictability of system behavior in response to unknown inputs due to the black box characteristics precluding deductive output prediction; and 3) need of assuring the characteristics of datasets to be used for learning and evaluation.
This paper is an English summary of a technical report published by the Science Board of the Pharmaceuticals and Medical Devices Agency (PMDA), Japan. Original report can be accessed at: https://www.pmda.go.jp/rs-std-jp/outline/0003.html (in Japanese).
Figure. Classification of AI medical systems classified by output (un)predictability. Characteristics of AI medical systems are; plasticity – continuingly learning and changing performance; unpredictability – AI algorithms tend to be ‘black box’ with a degree of autonomy; data management – quality assurance of the data is the key. While most those have been known as factors of AI, (un)predictability of output is another factor to consider. This figure illustrates different levels of output predictability; (a) selected from finite number of pre-set solutions, (b) chosen within given finite range, and (c) without given constraint. The degree of autonomy increases by this order.
