Run before you can walk: The way to roll out AI

Trying to implement Artificial Intelligence (AI) in the same way as other tech doesn’t work. Dr. Rishi Das-Gupta, CEO of the Health Innovation Network (HIN) South London; Dr. Dominique Allwood, CEO of Imperial College Health Partners; and Dr. Chris Laing, CEO of University College London Partners explore why a new model is needed for a new technology.

The traditional quality improvement model of trialing technology can be described as “crawl, walk, run”. This approach, which starts small but then accelerates, tests an idea and steadily refines it with support. At each phase of work, problem solving allows for progressive scaling up - up to five times the size of the previous phase. It has the advantage of being a proven and safe way to improve healthcare.

However, this model might not be the most appropriate for rolling out AI technology. As AI products evolve quickly, technical expertise is spread across multiple people in a healthcare organisation. It becomes inefficient for each organisation deploying technology to repeatedly answer all technical, safety, and security questions. Moving this slowly leaves patients and clinicians holding risk.

A new model, offering the necessary flexibility, might be more accurately described as “run, walk, drag”.

"Run, walk, drag" model

The first “pilot” phase is often fast.

Developing a product and deploying prototypes tends to happen quite quickly in a closely supervised environment. This is also helped by the fact that pilots are often run with local support from AI enthusiasts who help to demonstrate a proof of concept. Hence, projects often start off at a “run”.

A recent example is the use of ambient voice technology in General Practice. Although products are not yet well integrated into the workflow or GP software packages, as they need local adaptation to get them to work seamlessly, tech-savvy GPs have been early to adopt them because they see benefits to staff and patients. They have developed workarounds for themselves and trained their colleagues directly where needed. They have “run” with the solution during pilots.

The early scale-up phase typically sees a decrease in pace.

Here, solutions are being deployed in a wider range of services with more staff involved. Close supervision by the enthusiasts is no longer enough to carry an idea through. Typically, integration into workflows and training staff is needed. All these problems can be overcome but they require time and effort.

We typically see unexpected challenges surface at this stage - particularly when a product changes staff roles in a patient’s care. For example, using AI to interpret CT scans at an NHS Trust in London has meant an early indication of abnormal findings is available almost immediately when the scan is completed. This put radiographers who performed the scan in the position of having to reassure patients, or break bad news. This isn’t a role that they have done traditionally. Some of them have rightly asked for training and support on how to sensitively deliver the news, to provide appropriate care for the patients they treat.

We should expect early roll-out to be slower and, sometimes, more expensive than pilots. The use of mixed-methods evaluation of the roll-out, together with knowledge sharing, can support scaling AI and technology solutions in this phase. This support from health innovation networks accelerates this phase and provides valuable inputs for the final phase.

As technology diffusion continues, it then becomes important to engage those who are reluctant to use it. Although we sometimes see this reluctance due to unfamiliarity of technology or a general reluctance to change, more often it is that the product offers less benefit in their specific practice. General services and primary care might find that some product innovations would only benefit a very small proportion of their patients. Therefore, it’s often easier to roll-out new products in specialist centres where most patients would benefit from the innovation.

For the groups showing reluctance, it isn’t necessarily enough that a solution is widely used. They will often demand that all potential risks and issues are worked through, and every potential downside is mitigated, before a change is made. Doing this requires more time and a sustained effort by transformation teams and local champions - hence this being the ‘drag’ phase. In the programmes we have run in our health innovation networks, simple activities such as sharing stories and impact from earlier adopters, are critical to maintaining momentum in roll-out.

Eventually, when many staff are on a platform roll-out, it starts to accelerate again. When using the solution becomes the norm, staff expect to have access to it and the benefits it offers. At this stage, services which haven’t adopted it often have technical or financial reasons for not proceeding with changes, and these can be addressed directly.

Understanding innovation

With the deployment many technologies and AI solutions over the coming years, we believe we need to look at our processes and culture for innovation, as well as the products and solutions. Our thinking around this comes from discussions at a London AI roundtable that explored ideas on how to accelerate the adoption of AI. We recognised that the majority of healthcare providers are in the early stages of AI adoption. In creating an understanding around the processes and culture of innovation, we can help accelerate the adoption of innovation - benefiting patients and staff, while reducing costs at a critical time for the health and care system.

There's an important question around what is done at the national level (once) and what is done at local/regional levels. This is especially important given the reshaping of functions currently housed in NHS England. For example, we believe we would benefit from a national team focusing on general aspects of safety and compliance, such as the Digital Technology Assessment Criteria (DTAC) compliance and data security. However, deployment needs to be tailored locally which is where Health Innovation Networks (HINs) can play a critical role.

Technology adoption often has many components which are best handled locally because they depend on interactions with other operational processes and local technology solutions. The Health Innovation Network South London can support the rapid adoption of technology and new ways of working by helping to solve problems, evaluating roll-out, and sharing good practices across the network.

How could this work?

In London, there is an ongoing pilot for ambient voice technology which is due to report back soon. As part of this, London has agreed to adopt a framework called “T.E.S.T.” (Technology, Evaluation, Safety, Test) to rapidly and objectively evaluate readiness for spread. It focuses on:

We believe using the above framework mitigates the risk of the current system in that it transparently looks at the impact on inequalities, acknowledges that a clinical mistakes will happen in both current and future practice (but should be managed), and appreciates the need around data security and the likelihood of poor co-ordination leading to inefficiency.

Summary

Our experience suggests five key learnings: