Table of Contents
Cancer Wait Times for dermatology remain challenging
Three months ago in our original response to the BAD Letter “The use of Artificial intelligence (AI) to tackle 2WW referrals”, we noted the challenge in dermatology with Cancer Wait Times. In that time we’ve not seen any material change in performance with the latest provisional NHS data suggesting that now 11% of urgent skin cancer referrals are waiting more than 4 weeks for a first assessment, amounting to 17,454 patients in Q3 FY 22/23.
Nationally urgent skin cancer 2WW referrals are increasing more than 11% each year. Without innovation, we cannot expect our Dermatology teams to be able to achieve these targets in the future. With that background, and lack of viable alternatives, it is increasingly frustrating that the BAD continues to make statements against the use of AI that are not supported by evidence.
In all these conversations, critically we don’t see any acknowledgment about the risk of harm associated with the status quo. In the short term, this may well cause more harm to patients across the country as they continue to wait for dermatology assessments and treatment.
Skin Analytics helps our partners deliver better skin cancer services
Skin Analytics technology is a medical device that is governed by law and regularly audited by an independent, MHRA approved body. We work closely with our partners to ensure services that we deliver together ultimately benefit patients. Our services are safe and effective and we will continue to support NHS organisations looking to put their patients first.
Defining an indirect response
It has come to our attention that Skin Analytics (SA) has chosen to respond indirectly to our letter by providing marketing information which is not specifically related to the purpose of our letter.
It is not immediately clear to us whether the BAD intends this statement to mean that our response was not directly addressed to the BAD or that our response didn’t deal directly with the purpose of the BAD letter.
To address the former, although we only received their December letter after it was forwarded to us by several supportive partners, we sent our response to no less than six BAD representatives including the BAD President, CEO and Chair of the BAD AI Working Party Group. We had hoped that would have opened a direct line of dialogue but once again we are only finding out about this clarification after being informed by a third party. Despite this, we again intend to distribute our response to the BAD.
If the statement refers to whether our response was directly relating to the original letter, we would urge NHS teams to review the original letter (available here) and our response and form their own opinion. We would argue that it would be difficult to be any more direct than to break down each argument as we have done, prefacing the section with the BADs own words rather than paraphrasing them.
With this letter, we will again try and provide a balanced, evidence based response to the latest set of BAD statements. Finally, we noted the BAD’s referral of our evidence based response as marketing material, again we urge NHS teams to review the original letter (available here) and our response and form their own opinion. We think most will agree that it serves no one to allow the conversation to degenerate further and will not respond directly to the claim.
BAD Argument 1
Any doctor knowingly diagnosing and treating these skin cancer patients must be employed by their NHS hospital and be a core member of the skin MDT (1). Consultants who diagnose skin cancer and do not carry out any surgical work should be named as an extended member of the MDT. Their attendance is required four times a year which includes level 4 skin cancer mandatory case discussions of their patients. The consultant’s role on a Skin MDT is required for each hospital where they are seeing (triaging) 2WW referrals.
This means any Trust using the SA AI app and the company’s employed Consultants used to review benign classification of lesions resulting from the 2WW referrals will need to comply with the requirements above.
We request the BAD provide more information regarding this requirement. Upon reviewing the reference, a NICE guidance released in 2006, it is important to note that in May 2019 NICE decided to update this document with the reasoning below replicated in full (bolding is Skin Analytics).
Topic expert feedback received during surveillance indicated that this NICE guideline is considered outdated, does not reflect current service structures, and is no longer fit for purpose. Key issues include changes in cancer infrastructure and strategy, and developments in assessment, staging and management of skin cancer since the publication of the guideline.
In December 2020, NICE decided not to update the guideline but retain it for clinical usefulness rather than to describe the necessary operational structures. Further it should be noted that NICE guidance is not mandatory and certainly some consideration is worthwhile about which elements are still valid when the guidance is 17 years old and predates much of the current NHS strategy.
In summary, the conclusion that the use of Skin Analytics services, which leverages NHS Consultant Dermatologists on the GMC specialist register, requires those specialists to be members of each Trusts MDT is not clear to us from the reference provided. We call on the BAD to amend their statement to be accurate or provide more evidence to support their claim.
BAD Argument 2
Trusts should also consider the cost implications and benefits in duplicating photography processes when the SA AI App is used directly by department Consultants, as well as the risk of sharing large-scale patient data with a commercial company.
Cost
Cost is a key consideration within any procurement process and will typically also consider the relative savings compared to alternatives such as waitlist clearing initiatives or additional spending on locum clinical workers. We welcome the inclusion of that additional spend in the business cases our partners develop to decide on whether our service supports their goals.
Photography duplication
Regarding the duplication of photography processes, we would direct the BAD back to a section of our previous response, included again below for ease:
There is no requirement for duplicate images because of the use of AI. Our technology runs on images that are also suitable for review by human clinicians and is captured on hardware approved by the BAD [15 16]. In our case, we have made the decision to restrict the types of image capture devices (cameras and dermatoscopes) that we use with our AI to ensure that we have appropriately tested performance using these devices. The same images that are used for our AI assessment can be used by clinicians.
In some cases, our partner dermatologists request their teams to capture additional DSLR images and of course we support them to do this.
Data sharing risk
Regarding the risk of sharing patient data, we would again point out to the BAD that there are suitable guidelines issued by the NHS and other regulatory bodies in this regard. At Skin Analytics, we take patient data seriously and Skin Analytics meets GDPR requirements. Any use of patient data beyond delivering our services is done only with explicit patient consent. We would hope that in this regard, we are exactly the same as any commercial outsourced / waitlist clearing company who have access to patient data.
Beyond this, we have implemented an Information Security Management System that meets ISO/IEC 27001 standards. We were last audited by BSI in January 2023 who noted that:
- Leadership commitment consistently demonstrated
- Good management review processes complemented by regular meetings
- Objectives well documented with meaningful metrics
- Extensive Risk processes in place – and with good involvement from all areas of the business
- Internal Audit process well documented
- Continuation of good IS Awareness and level of staff communication in place
- Dedication to the continual improvement of the management system
- Verification of new staff office for collaborative work – conducted and the relevant physical security controls seen to be in place.
- No findings have been raised.
BAD Argument 3
Trusts need to be aware that the responsibility for AI used in clinical decision making will fall to the clinician who uses the technology, the deploying organisation, the industry innovator that developed the technology or those who validated and approved the technology for clinical use. Various legal frameworks may be applicable including negligence, product liability and vicarious liability. This is an evolving area but to date, despite AI technologies increasingly becoming available, there has been no legal change regarding the duty of care by the responsible doctor to the patient.
As an important first premise and as outlined clearly in our previous response, our AIaMD has been validated and approved for clinical use according to law under The UK Medical Device Regulations 2002 (“UK MDR”) which is overseen by the MHRA, the UK’s medicines and healthcare products regulatory agency. The legal basis for the use of DERM is clear.
Despite this, we acknowledge that the legal landscape for AI is nascent and that more clarity would be welcomed. Regarding the duty of care, we can’t state this point strongly enough, risk assessments require that risks are weighted against their likelihood of occurrence. Ultimately, to date we have found that many NHS Trusts and clinical teams believe that it is not in the patient’s best interest to tolerate increasingly long waits for cancer and routine care whilst legal clarity is refined and all edge cases are considered.
We both insure and closely monitor our services with a leading post market surveillance approach to ensure patient safety is paramount. Our latest performance shows the current version of DERM has successfully referred 103/103 (100%) melanoma, 113/113 (100%) SCC and 240/247 (97.2%) BCC.
BAD Argument 4
There is currently minimal externally validated evidence supporting the safety of AI when deployed outside of pilot studies or at scale, and little consensus as to how this safety is assured. Safety concerns include AI bias and differential performance across patient groups when deployed in a new setting or population that differs from its original training context; the impending arrival of algorithms that will continuously update in response to new data, provides an opportunity for continuous improvement but risks continuously evolving further away from the algorithm for which the original regulatory approval was given. Finally, there are issues of interpretability and the extent to which the user can understand how the algorithm reached its output and challenge any decision arising from that output. These safety concerns were also highlighted in The Regulation of Artificial Intelligence as a Medical Device November 2022 written by Lord Darzi (2). Many of the stakeholders consulted in the report, expressed concerns about the efficacy and safety of AI as a Medical Device (AIaMD) when deployed in ‘real world’ settings with potentially very diverse local demographics and settings.
We would reiterate our point from our previous response that our AIaMD is classified as a Class IIa medical device overseen by the MHRA and has been rigorously assessed across a number of areas including clinical evidence. Further, the substantial real world post market surveillance data shared in the previous response is fully auditable by our NHS partners. We can not ignore the state of Cancer Wait Times in the UK or the impact this is having on routine dermatology patients.
We note and welcome the reference to Lord Darzi’s report on regulation and AIaMD. It would be just as accurate to say that many stakeholders consulted in the report were excited by the opportunity to improve healthcare to the benefit of patients. Not all AI companies have chosen to invest in regulation as we have done and Skin Analytics both supports and works with regulatory bodies on the development of regulations more specifically tailored to AIaMD.
However, we are only able to comply with the legal frameworks set by appropriate regulatory bodies. We look forward to the publication of new UK regulations that are developed for AIaMDs and we are confident we already meet those requirements.
We have already responded in detail regarding the risk of bias and differential performance when used in different population groups and welcome any new information the BAD would like to discuss regarding this.
One point we didn’t cover was the “impending arrival of algorithms that will continuously update…”. This is an interesting point made by the BAD but we fear it shows a lack of practical understanding of current medical device requirements and our AIaMD. To be definitive, our AIaMD does not continuously update itself. Any improvements are released according to the appropriate regulatory processes.
BAD Argument 5
In areas where the AI app has been piloted in primary care, we would expect to see a reduction in 2WW referrals (which go directly to the patient’s local hospital). Our review of publicly available NHSE 2WW skin cancer referral data for Trusts in the areas where SA is in operation (in primary or secondary care) has actually shown an incremental increase in this activity. This is despite the repeated claims by SA to be able to reduce 2WW referrals and the evolving information available on their website.
We appreciate the BAD’s clarification around the expectation of a reduction of 2ww only when used in primary care. However, we note the sentence immediately following makes the same original mistake of including secondary care which we find somewhat concerning.
In our previous response we presented data that shows that our partners total case volume is around (but generally less than) what would be expected based on the steady growth of skin cancer referrals over the last decade and again point out that we have no impact on the 2ww referrals when deployed in secondary care. Rather we reduce the number of dermatologist appointments required based on the 2ww referral volume.
We have two primary care deployments. One is at University Hospitals Birmingham as a community diagnostic hub which was set up as part of the AI in Health and Care Award and it is simply too early to make any statements on referral volumes. While anecdotally referrals are lower, based on the early stage, it would be irresponsible to suggest any conclusions can be drawn at this point – it is certainly inappropriate to suggest referrals have increased.
The second primary care deployment is with the Mid and South Essex Health and Care Partnership. In Q3 of 2021/22, they had 3,553 2ww referrals. For the same period in 2022/23 they received 3,817 2ww referrals. This represents a 7.4% increase compared to the average 11.45% per annum increase across England over the last 10 years. Further, in this period, Skin Analytics assessed less than 20% of 2ww referral volumes, resulting in 28% of them avoiding a 2ww referral.
It is just patently incorrect to suggest that Skin Analytics has resulted in increased 2ww referrals.
We believe it is fundamental to support claims with evidence and despite repeated claims of increased 2ww volume because of Skin Analytics, the BAD has failed to provide any evidence. We call on the BAD to share the reviewed data they refer to so that it may be externally validated. If they fail to do so yet again, we compel the BAD to stop making unsubstantiated claims.
BAD Argument 6
As the professional body for dermatologists and advocating for safe and effective care for patients with skin disease, we have many specialised working party groups which include academics and technical experts with the requisite knowledge in fields including AI. Crucially, we are yet to receive any published peer reviewed evidence assessing the relevant outputs for the different clinical settings where the Skin Analytics AI app is deployed (and any other commercial AI application for that matter) that can be independently verified. We also look forward to the planned interim analyses from the sites who are involved in the Phase 4 NIHR NHSX AI Lab Award.
We have worked in this field since 2012, launched teledermatology services in 2015 and AI services in 2020. We have assessed more than 50,000 patients and believe we have insights that would be invaluable to the BAD’s understanding in this field. Further, given the analysis BAD presented, we believe our statistical approaches may be valuable training to help them better evaluate the impact of AI services.
With respect to independently verified work, we have taken part in a number of novel evaluations conducted by third parties, including agreeing to be the first use case of the Medical Algorithmic audit first described in The Lancet Digital Health.
We have also collected data on a large number of our patients as outlined in our earlier response and submitted a number of abstracts for this year’s BAD conference in Liverpool. Several of these have been accepted and we eagerly await the opportunity to present and discuss the results in this forum.
Further, recently one of our partners shared a case study with their experience of working with us. You can find the case study the University Hospitals Leicester wrote and NHS England published here.
BAD Argument 7
There are several issues regarding the regulation of AI / software as a medical device which the BAD AI WPG has continued to highlight in its position statement. These challenges were also reiterated in The Regulation of Artificial Intelligence as a Medical Device November 2022 written by Lord Darzi (2). We continue to work with Notified Bodies and the MHRA to better understand and support robust and transparent regulatory processes.
Regulation is incredibly important to both protect patients and foster innovation and we work with regulators in a number of countries to appropriately regulate our service. We are the only Class II AIaMD in dermatology, a process that required significant investment, a dedicated team and many years to achieve.
The BAD has repeatedly over the years challenged our regulatory position, indirectly, by approaching our partners and more directly by approaching the UK regulator, the MHRA. We have worked with both groups to ensure comfort with our approach. Meanwhile, the BAD has never responded to our offer to help them understand our approach to regulation and the quality systems that underpin it.
We support the paper written by Lord Darzi and work with regulatory bodies around the world as they define the future regulatory requirements. We would suggest the open engagement that group took with industry is critical to realising the benefits to patients and the UK that underpin Lord Darzi’s report.
BAD Argument 8
We trust that this clarifies the purpose of our initial letter to the Trust CEOs regarding the use of AI in the 2WW skin cancer pathway. We recommend Trusts request the following information prior to any AI implementation in clinical pathways:
- the clinical evaluation report,
- the protocol approved by the MHRA which led to the clinical evaluation,
- the specific intended use statement that the regulatory approval applies to,
- and a clear outline of the proposed project and outcomes which will be collected.
The BAD welcome members to contact us if they have specific queries and can assist with the critical appraisal of any evidence presented.
It is difficult to be precise about the true purpose of these letters however the conclusions are undermined by the data supplied and an apparent lack of good faith when discussing how best to leverage AIaMDs to the benefit of the UK’s population. What is much clearer is the impact on our partners and prospective partners each time the BAD releases incorrect information.
It is worth noting that the letter wasn’t just sent to Trust CEOs but also to ICB leads and Cancer Alliances. It was not sent to Skin Analytics and neither was this most recent Clarification letter.
Regarding the recommendation for information requested by Trusts prior to implementing AI, this highlights one of the challenges facing this industry currently. There are a lot of different bodies advocating for different information to be supplied and ultimately there is a very significant overlap which increases administrative burden for the procuring NHS site. Based on the critical appraisal conducted by the BAD to date, we would strongly recommend Trusts to conduct their own evaluation, following NHS guidelines already in place.
Our Conclusion
We weren’t surprised to be sent this Clarification letter by some of our NHS colleagues, given the BADs continued refusal to engage in dialogue. However, we were surprised by the content of the Clarification. Firstly, the content appears to be much the same as their original Letter which we showed was false and no further evidence was given.
Where the content differed was startling, the BAD moved away from stating there is no legal remit to use AI in the pathways we’ve launched with our partners and instead attacked our use of their own Consultant Dermatologist members to complement the role of our AI.