AI and Society

Medical artificial intelligence (AI) has enormous potential to transform medicine. Over 521 medical devices enabled by AI or machine learning have been approved for use in the USA. Yet, according to labour trends, healthcare lags noticeably behind other industries in AI adoption. One reason is that we cannot currently measure the trustworthiness of AI systems in healthcare, including more-recent generative AI tools. To measure trust in medical AI tools, we should score the explanations that they generate against the reasoning provided by medical experts. This process must include transparency of training data and open community vetting.

AI algorithms used for diagnosis and prognosis must be explainable and must not rely on a black box.

Artificial intelligence is changing medicine and it will relieve physicians from the burden of rote knowledge. Here, I discuss how this might affect medical training, drawing from the example of how automation in aviation redefined the role of the pilot.

Machine Intelligence (MI) is rapidly becoming an important approach across biomedical discovery, clinical research, medical diagnostics/devices, and precision medicine. Such tools can uncover new possibilities for researchers, physicians, and patients, allowing them to make more informed decisions and achieve better outcomes. When deployed in healthcare settings, these approaches have the potential to enhance efficiency and effectiveness of the health research and care ecosystem, and ultimately improve quality of patient care. In response to the increased use of MI in healthcare, and issues associated when applying such approaches to clinical care settings, the National Institutes of Health (NIH) and National Center for Advancing Translational Sciences (NCATS) co-hosted a Machine Intelligence in Healthcare workshop with the National Cancer Institute (NCI) and the National Institute of Biomedical Imaging and Bioengineering (NIBIB) on 12 July 2019. Speakers and attendees included researchers, clinicians and patients/ patient advocates, with representation from industry, academia, and federal agencies. A number of issues were addressed, including data quality and quantity; access and use of electronic health records (EHRs); transparency and explainability of the system in contrast to the entire clinical workflow; and the impact of bias on system outputs, among other topics. This whitepaper reports on key issues associated with MI specific to applications in the healthcare field, identifies areas of improvement for MI systems in the context of healthcare, and proposes avenues and solutions for these issues, with the aim of surfacing key areas that, if appropriately addressed, could accelerate progress in the field effectively, transparently, and ethically.