Interrogating the research and development pipeline of artificial intelligence (AI) in health : diagnosis and prediction-based diagnosis

Despite the potential significant benefits of AI in health and particular in healthcare, its uptake is slow. Main barriers include trust issues, understandability of outputs and patient safety concerns. While "trustworthy AI" has been embraced as a policy programme by various global regions, issuing high-level guidance documents, it is currently unclear to which extent the clinical research community is considering these during early steps of the life cycle. To tackle this evidence gap, we systematically interrogated the biomedical literature over 6 months, focusing on the R&D phase of the life cycle in five health application areas of AI. Our analysis builds on three pillars: 1) concerned key attributes, e.g. AI solution availability, type of AI technology, clarity on interpretability considerations, medical field(s), geographic origin. 2) examined awareness and reporting of sensitive concepts related to trustworthy AI. 3) explored case studies, covering various medical fields and development stages. The current report focuses on "diagnosis and prediction-based diagnosis", summarising results of pillars 1 and 3. We found that the majority of the research articles focus on developing or fine-tuning existing algorithms, with deep learning techniques being the most commonly used. Typically, the publications did not report why specific methods had been used or whether and how interpretability was considered. Our data show considerable innovation, in particular in China, the EU and USA but highlight that clinical researchers seem insufficiently aware of critical issues including the consequences of AI techniques on interpretability. This might delay or complicate later uptake of such AI solutions.