In July 2019 the Department of Health and Social Care and NHS England published a new code of conduct for the use of digital technology in health and care. As a company at the forefront of using AI algorithms to identify Skin Cancer, we know that AI provides significant benefits for patients, clinicians and healthcare systems. However, like all new technologies, AI brings challenges that need to be addressed in order to build products that are safe and secure.
That is why we are pleased to have engaged in consultations with NHS England in 2018, to provide feedback on perspectives that have helped to shape the principles set out in the report. We believe that all responsible AI companies providing services to the healthcare sector should comply with the code of conduct and we have written our response to the code of conduct below.
This blog sets out how Skin Analytics complies with the AI code of conduct. The full publication can be found here: https://www.gov.uk/government/publications/code-of-conduct-for-data-driven-health-and-care-technology/initial-code-of-conduct-for-data-driven-health-and-care-technology
1. Understand users, their needs and the context
“Understand who specifically the innovation or technology will be for, what problems it will solve for them and what benefits they can expect. Research the nature of their needs, how they are currently meeting those needs and what assets they already have to solve their own problems. Consider the clinical, practical and emotional factors that might affect uptake, adoption and ongoing use.”
We have been focused on skin cancer and skin cancer pathways since we were founded in 2012. We have spent this time working with a great many research organisations, NHS bodies, and live clinical services to get a deep understanding of skin cancer pathways.
A few years ago, when the evidence was not yet in place to support the safe use of an AI solution in skin cancer, we set up an analogue model using dermatologists to assess images of skin lesions (a practice called teledermatology). This model has helped us learn how an AI solution can be implemented effectively and integrated with different types of care pathways already present in the NHS.
We continue to focus deeply on user research, and understanding the needs of clinicians in practice. To this end we conducted a prospective clinical validation study of our product across 7 NHS Trusts (published here), and we are conducting two additional clinical studies across a number of UK Trusts. We are also currently working on a human factors study with Imperial College, Health Economics evaluations with Imperial College and the York Health Economics Collective, as well as a planned clinical study in Australia looking at the impact of our AI tool on clinicians’ decision making.
2. Define the outcome and how the technology will contribute to it
“Understand how the innovation or technology will result in better provision and/or outcomes for people and the health and care system. Define a clear value proposition with a business case highlighting outputs, outcome, benefits and performance indicators.”
Our mission is to help more people survive skin cancer by using AI to enable innovative skin cancer pathways that deliver better patient outcomes that are sustainable for health systems globally.
We are clear in our value proposition and business case. Our AI system will provide dermatologist quality decision making outside of Trusts, increasing diagnostic accuracy (sensitivity) at primary care from 78% to 95%.
Further, using our AI service can reduce referrals from primary to secondary care by 63% leading to considerable cost savings and better patient experience. We charge a per case fee that is much lower than the referral reduction.
3. Use data that is in line with appropriate guidelines for the purpose for which it is being used
“State which good practice guideline or regulation has been adhered to in the appropriate use of data, such as the Data Protection Act 2018. Use the minimum personal data necessary to achieve the desired outcomes of the user’s needs and the context.”
Skin Analytics takes the use of patient data seriously and balances the need to store a legal health record against the storage of personal data. We have developed our service to inform patients of how we will use their data and explicitly separate any consents required to provide the service from those we ask for to process the data outside the provision of the service, in accordance with the Data Protection Act 2018.
We are certified to ISO 13485:2016 and EN ISO 13485 as a medical device manufacturer. As part the Quality Management System (QMS) we operate, we have data privacy standards covered in our following policies and standard procedures:
- Data Protection – SA-SOP-G006
- IT security policy – (SA-PLY-G001)
- Pseudonymised data policy – (SA-PLY-012)
- Access Control – (SA-SOP-G001)
- Asset Management – (SA-SOP-G003)
In addition to the above policies, our Secure Software Development Lifecycle (SSDLC) policy puts measures in place (e.g. encryption) to ensure the security of the data that we hold.
We also comply with the DSP toolkit and are in the process of securing ISO27001 which we expect by Q4 2020.
4. Be fair, transparent and accountable about what data is being used
“Utilise data protection-by-design principles with data-sharing agreements, data flow maps and data protection impact assessments. Ensure all aspects of the Data Protection Act 2018 have been considered.”
Our products and services have been designed with patient data protection in mind. We collect only the necessary patient information required and utilise pseudonymisation and anonymisation practices to ensure we use only what we need and do so legally and securely, as set out in our Privacy Notice. Systems that store or process personally identifiable data are subject to Data Protection Impact Assessments. We put agreements in place with NHS Trusts and CCGs and other healthcare partners to ensure patient medical information is limited and appropriate.
We have a Senior Information Risk Owner (SIRO) and a Data Protection Officer (DPO) appointed, and will shortly have a Caldicott Guardian in place.
5. Make use of open standards
“Utilise and build into the product or innovation current data and interoperability standards to ensure it can communicate easily with existing national systems. Programmatically build data quality evaluation into AI development so that harm does not occur if poor data quality creeps in.”
From our live teledermatology services we know the value of interoperability to the healthcare system. We are committed to integrating as seamlessly as possible with major EHR systems in the UK. For launch we will pass patient results back to EHR systems automatically but are working as fast as EHR systems allow at integrating case creation from within the main systems.
Our AI algorithm does not self learn based on data from our service. Any additional data added to our training set is reviewed, and the results of updates to the algorithm are carefully reviewed as part of our release process to ensure that the quality of the tool has improved with each release.
6. Be transparent about the limitations of the data used and algorithms deployed
“Understand the quality of the data and consider its limitations when assessing if it is appropriate for the users’ needs and the context. When building an algorithm, be clear about its strengths and limitations, and give clear evidence of whether the algorithm you have published is the algorithm that was used in training or in deployment.”
Diagnosis of skin cancer today is defined by histopathological examination and interpretation, and is taken as the gold standard diagnostic methodology. Throughout the development of our AI algorithm, we have focused on using images of lesions with histologically confirmed diagnosis to ensure the algorithm is trained on the highest possible data quality available in healthcare today. As benign lesions are rarely biopsied we also train on benign lesions confirmed as benign by trained dermatologists.
Our intended statement of use as a medical device makes clear the uses of our service that are appropriate or not.
In the technical file that goes along with any product release we make clear the evidence that supports the intended use of the product. This includes a clear statement of which version of the algorithm is the deployed version and how it performs compared to previous versions of the algorithm..
The technical file also includes a critical analysis of the evidence available and risk analysis of the product, including identification of situations where the available evidence does not support the use of the product – for example in different patient populations or for conditions not specifically trained and tested for..
Finally, we have conducted a powered prospective clinical validation study which assessed how well our algorithm identified (histopathologically confirmed) melanoma in images of pigmented lesions. External dermatologists and skin specialists also provided their clinical assessment of the lesions, allowing us to demonstrate that our algorithm performs with a similar accuracy to specialists. A similar study aiming to demonstrate the performance of our algorithm in identifying NMSC is about to start. We hold ourselves to a high standard for clinical evidence both in reducing bias in study design and by publishing all our research.
7. Show what type of algorithm is being developed or deployed, the ethical examination of how the data is used, how its performance will be validated and how it will be integrated into health and care provision
“Demonstrate the learning methodology of the algorithm being built. Aim to show in a clear and transparent way how outcomes are validated.”
We have developed our algorithm to perform a diagnostic test on skin cancer. Because of the importance of this test, our algorithm does not “self learn’ but changes are manually released as part of a release package that improves performance or functionality.
Decisions to release deployment versions of our algorithm are made in line with our ISO:13485 quality management system. The process uses a robust evaluation of changes to the algorithm, the impact of which is fully risk assessed prior to approval for release. These decisions are made by a release committee made up of Skin Analytics employees and independent advisors including NHS Consultant dermatologists and health economic experts who review clinical safety assessments as well as algorithm performance before any release.
Our service has been designed to sit easily within existing skin cancer pathways. The performance of our algorithm has been assessed in line with NICE guidelines (see Principle 8).
8. Generate evidence of effectiveness for the intended use and value for money
“Generate clear evidence of the effectiveness and economic impact of a product or innovation. The type of evidence should be proportionate to the risk of the technology and its budget impact. An evidence-generation plan should be developed using the evidence standards framework published by NICE.”
Our AI service is a tier 3b service for effectiveness under the NICE digital standards evidence framework as it helps clinicians diagnose skin cancer. We currently are at the low level of financial risk to the payer status, requiring basic economic analysis. This may change if adoption becomes more widespread.
We have adopted the best practice standard of evidence for the effectiveness of our service for melanoma. We have undertaken a prospective clinical study that compares the performance of our algorithm with the relevant comparator (skin cancer specialists operating in a secondary care setting) against the gold standard of histopathology. The results of this study have been published in a peer reviewed journal (JAMA) showing our AI solution performed at a comparable level to the skin cancer specialists.
For nonmelanoma skin cancers (NMSC) due to a lower risk profile, we have completed an observational study of more than 7,000 lesions confirmed by histopathology and compared the AI to a meta analysis of GP and dermatologist performance with and without dermoscopy which was published in Dermatology Practical & Conceptual.
We initially prioritised melanoma due to the higher risk caused by this condition and are currently running a prospective clinical study to obtain the best practice standard of evidence for Non Melanoma Skin Cancer with results expected Q3 2020.
At our current scale of deployment we require only a budget impact analysis for 1-2 years. We have prepared internal health economics papers to support this but are working with Imperial College and the York Health Economics Consortium on independent health economics papers to support this.
9. Make security integral to the design
“Keep systems safe by safeguarding data and integrating appropriate levels of security into the design of devices, applications and systems, keeping in mind relevant standards and guidance.”
We have built our service in line with the DSP toolkit and our Secure Software Development Lifecycle (SSDLC) and Product Development Lifecycle (PLC) policies build in security by design for our products.
Our SSDLC follows a rigorous risk based approach to software design and development where risks are identified up front and used as part of the design inputs for the product. These design inputs ultimately become formal product and software requirements for the product. This risk based approach enables us to continually assess software risks as the project progresses and make changes as necessary to mitigate these risks.
Our Product Development Lifecycle (PLC) process gives specific consideration to Information & Security Risk management and Security Testing Activities, which include carrying out independent security audits (ie penetration testing) when appropriate.
Security forms an integral part of our risk management process and we have a number of internal policies that provide a high level of security when designing our software. We update our policies regularly to ensure that we are aware of the latest security risks that may affect our software. This includes OWASP recommendations as well as our own best practise procedures. In addition, our maintenance policies ensure that we regularly check our existing software and apply patches and updates as required.
10. Define the commercial strategy
“Purchasing strategies should show consideration of commercial and technology aspects and contractual limitations. Consider only entering into commercial terms in which the benefits of the partnerships between technology companies and health and care providers are shared fairly.”
We believe that our service offers a clear health economic benefit to commissioning NHS users and stands alone on this basis as a commercial product for us and as a commissioned service to NHS users. Contracts will make clear the roles and responsibilities of each party in this commercial product.
Our service does not depend on the receipt of health outcomes data from the NHS and can operate as a purely commercial medical device purchased by the NHS. The effectiveness of the tool has been confirmed via clinical studies based on the data we have obtained to date.
However we understand the value that in-use case studies as well as outcome data have to continue to support the effectiveness of our service to health systems around the world, and to continue to improve the quality of the algorithm by providing health outcome data for cases assessed. Where opportunities exist to work with NHS organisations along these lines we will work to ensure that appropriate value is given to the NHS for the role that it plays in these separate relationships.