AI Medical Device Software

Requirements for clinical evaluation and post-market clinical follow-up

Recently, the European Parliament adopted the Artificial Intelligence Act: MEPs adopt landmark law. The regulation aims to safeguard fundamental rights, democracy, and environmental sustainability. The new set of rules implies a need for more careful examination of the evidence regarding the continuous safety and performance of AI-driven medical device software.

Pre-market phase: technical documentation medical device software

Frequently absent from the technical documentation supplied by manufacturers of AI medical device software, is comprehensive information regarding the development and upkeep of the model. Generally, during the pre-market phase, the model is constructed utilizing retrospective datasets for training, validation, and testing purposes. Additionally, when the model interfaces directly with the medical setting it operates within, prospective clinical investigation is often necessary.

"What we often see is that in the technical documentation, and especially in the clinical evaluation report, the developed AI model is merely described as an input or component without further elaboration", said Natascha Cuper Product Reviewer at Kiwa Dare. "However, this is not sufficient. Unlike traditional medical device software based on known algorithms, an AI model cannot be fully verified. For that reason additional information about the datasets is necessary."

Clinical performance

The quantity and quality of the datasets utilized directly influence the clinical performance of an AI model. Therefore, it is crucial that information on AI model development and details of the used datasets is summarized and referenced in the clinical evaluation report. This checklist provides a clear overview of the necessary information to document for AI model development. Guideline for AI for medical products can be found here.

Post-market phase: maintaining your AI model

Beyond initial deployment, maintaining AI models in the post-market phase is equally vital. Natascha Cuper said: "Initially, it's crucial to gather real-world evidence on the utilization of the AI medical device software in clinical settings. This is particularly vital because the model's clinical performance relies directly on how relevant the datasets used are to the real clinical scenarios. Any deviation should be closely monitored."

Any deviation should be detected by means of collecting real world evidence of the use of the AI medical device and the suitability of the outcomes provided by it.

Natascha adds: "Secondly, AI can be prone to concept drift. The reason for this is that over time, clinical practice changes, and the datasets used in the past to develop the model, might become less relevant for the actual situation."

For instance, consider an AI model trained in 2019 to diagnose various medical conditions based on symptoms like coughing, rhinitis, and breathlessness. However, with the sudden emergence of COVID-19 in 2020, the model's effectiveness would become inadequate. This can also happen in less visible ways: slight changes in insights, common medical practice and the target population will downgrade the performance and accuracy of the developed AI model.

Natascha concludes: "Therefore, it is important to consider as part of the post-market clinical follow-up plan, how concept drift and regular maintenance of the model will be considered. For example by planning regular performance checks with up-to-data datasets and making plans for retraining when the threshold values for the continuous reassessment of the benefit-risk analysis are no longer met."

In need of assistance?

Incorporating AI into the medical device sector presents various opportunities and challenges. It demands a systematic approach, encompassing rigorous clinical assessments and ongoing monitoring of post-market performance. Should you have any inquiries regarding this matter, please feel free to reach out to us at medical@kiwa.com.

Do you want more information on our services for the medical device sector? Visit our page at www.kiwa.com/medical.