Are we on the verge of releasing skynet on the world? Not even close, but a set of standards for the application of AI to healthcare is overdue.
Would then stifle innovation or provide us with the landscape to deliver real value to the healthcare industry? That was the question I asked myself time and again after we were recently invited to an event run by Nesta and BSI and attended by other small firms such as the NHS, Google Deep Mind and Babylon Health.
We were all sitting in a rather dreary room along Victoria Embankment here in London debating the role of regulation and standards for AI used in healthcare. At Skin Analytics we’re generally advocates for proportional regulation based on the risks and associated impacts. The new Medical Device regulations are a great step I believe as many Class I products, such as ours, are outside the risk profile that was originally envisaged software was capable being.
Ultimately, the last thing we want to see is innovation being suppressed under volumes of red tape. At the same time, we’re trying to change healthcare, meaning that we’re dealing with people’s health and that’s not an area to “move fast and break things” for fairly obvious reasons.
Back to our dreary room however, and we proposed a number of areas that need to be part of any standards developed in this area but there are two I feel quite strongly about.
The first is the requirement for a well designed clinical evaluation that is suitable for the approach and intended use of any AI service. Desk based research just doesn’t cut it in the world of AI. Too often we see the real world performance of an AI system to be significantly below expectations. Only a prospective clinical study can meet this requirement.
The second is defining an algorithm update process that ensures that no AI system in healthcare is evolving without a clear set of acceptance criteria for any modifications to the algorithm. In our world this means building a test set that is held apart from the training data and enables new versions to be benchmarked against the old.
These shouldn’t stifle innovation in any way. Good clinical evidence is the lifeblood of evidence based medicine and there is simply no way around it. Services that go around the medical community should be closely scrutinised by the regulatory authorities to ensure adequate evidence exists.
Dispelling the myth that all AI is updating itself in unknown ways is important and defining the release of improved versions of any solution is a core part of any well designed software process.
Delivering value in healthcare requires any innovator to ensure that they are delivering a quality solution that enhances or improves the health outcomes in a cost effective way.
That may take longer to deliver and consequently more blood, sweat and tears from founders but it if those principles are at the core of the idea, you will be more likely to achieve your goal. Unless skynet takes over that is.