26 January 2021
In December, the British Association of Dermatologist (BAD) President’s Bulletin included an article from Dr Rubeta Matin with an update from the recently formed BAD Artificial Intelligence Working Party Group. We wanted to welcome the formation of this group, and also welcome the message they set out of a willingness to support ‘appropriately regulated and governed uses of AI interventions to enhance clinical practice’. We echo their sentiment that pre-market regulatory evidence will only ever go so far, and understand that safe, authorised deployments must be accompanied by strong real world evidence.
In this, my first blog as Medical Director of Skin Analytics, I wanted to share with the wider dermatology community our position regarding our regulatory status and why you can safely deploy our medical device DERM:
Regulation
Regulatory documents are long and I’m sure I’m not alone when I say they can be hard to penetrate. This complex area has been made even more confusing over the last few years with plans for a transition from the Medical Device Directive (MDD) to the Medical Device Regulations (MDR), a subsequent corrigendum, the impact of Brexit and the wait to see what new guidance will come from the MHRA.
This complexity is why we at Skin Analytics have prioritised having in house expertise in the form of a Quality and Regulatory Affairs (QARA) manager/director for the last 2 years. By having this deep knowledge of the regulations and changes, we are not only given confidence that we can traverse the theoretical regulatory landscape, but more importantly, we have been able to design, build, test and deploy our medical device DERM in partnerships with the University Hospitals Birmingham NHS Trust and Mid and South Essex Health & Care Partnership.
We would be happy to talk to anyone in more detail, but to try and simplify the regulatory position, I thought I would see if I could set out Skin Analytics’ position in 5 bullet points… but I couldn’t so we have ended up with 6.
1. Skin Analytics registered DERM as a Class I medical device with the Medicines and Healthcare Products Regulatory Agency (MHRA) on the 15th of February 2019 under the Medical Device Directive 93/42/EEC (as amended) (“MDD”).
2. The MDD was due to be replaced by the Medical Device Regulation (EU) 2017/745 (“MDR”) in March 2020 and under the MDR, DERM is considered a Class II medical device.
3. Only an approved notified body can confirm conformity with the regulatory standards but in July 2019 BSI notified clients that they were not accepting further applications due to their workload.
4. In recognition of this, and a similar picture across the other notified bodies, on the 25th of November 2019, the EU issued a Corrigendum enabling Skin Analytics’ Class I MDD declaration to remain valid until 26th of May 2024.
5. The MHRA has confirmed that the MDD, which is given effect in UK law through the UK Medical Devices Regulations 2002 (SI 2002/618, as amended) continues to have effect following the Brexit transition period. This means that the procedures for placing a medical device on the Great Britain market are still based on the requirements derived from the MDD. Therefore Skin Analytics registration as a Class I medical device remains valid and the company is compliant with regulatory standards in line with its intended use.
6. In recognition of the fact that DERM will be up-classified under the MDR, and that the product is considered a Class III device under United States federal regulation, Skin Analytics is building DERM to a rigor which meets not just current Class I regulatory requirements under the MDD, but to the level of compliance required by Class II EU MDR and Class III US FDA regulation.
Safe deployment
At Skin Analytics we believe we have helped to set the bar for good quality clinical evidence in AI within the healthcare setting, and are passionate about implementing AI the right way in clinical pathways (see our earlier response to the NHS AI Code of Conduct).
In 2019, we published the first powered prospective clinical study in AI for skin cancer, demonstrating DERM’s ability to detect melanoma at an equivalent level to a skin cancer specialist reviewing patients in person as normal in a two week wait cancer clinic in the UK1. After the publication of the CONSORT-AI and SPIRIT-AI guidelines in September last year, our paper was recognised by Eric Topol as being one of “only about a dozen” prospective trials in a clinical environment assessing performance of AI deep neural networks2. That is something we are incredibly proud of, but we are not resting on that paper nor our December 2019 publication in Dermatology Practical & Conceptual3. In addition to a significant amount of internal validation that we conduct, we continue to recognise the importance of good quality clinical research. To this end we currently have one paper under review with a journal and two more prospective UK clinical trials underway, though as with a lot of non-COVID related research this has been impacted by the pandemic. We are also finalising protocols for further research in the USA (working with the US FDA as part of our breakthrough device designation) and in Australia.
In the President’s Bulletin, Dr Matin cited concerns that ‘pre-market regulatory evidence requirements may not be robust’ and we would agree that too often, pre-market evidence is based on retrospective evidence alone, and real world performance may not match. We also recognise that it is our responsibility to now demonstrate that results we saw in our prospective clinical trials and internal validation are matched in a real world deployment. That is why we have worked with our partners to develop pathways which utilise DERM in line with its regulatory clearance and stated intended use, but which also have the additional checks and balances in place to make sure they are first and foremost safe whilst still allowing us to continue to evaluate the performance through clinical audit, a fundamental pillar of good clinical governance..
We do this in a number of ways, but perhaps the most significant has been our decision to implement a ‘Safety Net’. This ‘Safety Net’ ensures that any case where all lesions are recommended for discharge (or treatment as Actinic Keratosis) by DERM, a Consultant Dermatologist from the GMC Specialist Register also assesses the case. They have the ability to agree with or overturn the decision from our AI and only when this step has taken place will any patient’s clinician be informed of the final outcome.
Another pragmatic step we have had in place since we launched has been to use the work of our wonderful biostatistician Chronis who has developed a way to meaningfully monitor performance as we get histopathology results back. This process gave us confidence that we would have been able to detect early if our rates of detecting melanoma, squamous cell carcinoma or basal cell carcinoma were outside of the range we have agreed and stated.
Anyone who has spoken to us knows that it has always been our intention to write up and hopefully publish our results and learnings from our UHB deployment, working with the dermatology team there to do so. We hope that this will be ready in the coming months.
Finally, and perhaps most importantly I wanted to state that I firmly recognise that the burden is on us as a company to demonstrate our compliance to regulatory standards and continue to evidence our performance. What I can promise is that we are, and will continue to be a transparent organisation who are actively seeking the scrutiny of groups like the BAD AI Working Party Group, understanding that it is only by hearing the concerns of the community and addressing them that patients and clinicians will start to really get the benefits of new technologies like ours.
If you have any questions or would simply like a chat to find out more about what we are doing then please reach out to me via dan@skinanalytics.co.uk
Dr Dan Mullarkey,
Medical Director at Skin Analytics, NHS GP
1 – https://jamanetwork.com/journals/jamanetworkopen/article-abstract/2752995
2 – https://www.nature.com/articles/s41591-020-1042-x
3 – https://dpcj.org/index.php/dpc/article/view/dermatol-pract-concept-articleid-dp1001a11