Development of a Digital Outcomes Framework

May 15, 2023

Do you want help developing an outcomes framework?

This article outlines an example of our work in developing an outcomes framework to facilitate the evaluation of digital-first projects across London. The Health Innovation Network provides support to help organisations understand the benefits of outcomes frameworks and deliver outcome monitoring solutions.

Background Information

NHS England is supporting primary care to move towards a digital first approach, where patients can easily access the advice, support and treatment they need using digital and online tools. These tools need to be integrated to provide a streamlined experience for patients, and quickly and easily direct them to the right digital or in-person service.

This means patients should be able to use online tools to access all primary care services, such as receiving advice, booking and cancelling appointments, having a consultation with a healthcare professional, receiving a referral and obtaining a prescription.

The London Digital First team are currently driving the digitisation of primary care to achieve these aims, through several innovative workstreams across five ICS geographies. Multi-year funding has been awarded and ICS teams are currently in their final year of delivery of this programme from 2023-24. To demonstrate the impact of all the initiatives underway, the Digital First London team commissioned the Health Innovation Network (HIN) to collaboratively develop an Outcomes Framework which will be implemented by each ICS team.

This is a working document to help local project teams to capture, review and track their outcomes relating to digital transformation. This will build into local and then regional outcomes registers to show the quantitative outcomes related to the Digital First Programme from year one. Further work is in progress at the HIN to capture qualitative patient perspectives of the Digital First Programme to provide a complete narrative of the programme by the end of 2023/24.

Outcomes Framework Design

The Outcomes Framework encompasses a series of structural, process and outcome measures which can be used to consistently monitor project/programme performance towards achieving two core goals – Improving Patient Outcomes and Improving Service Efficiency.

Why are outcomes needed?

To build the case for continued investment into digital transformation of primary care.
To demonstrate and measure the impact and achievements of the Digital First Programmes within each ICB and share regionally/nationally to shape the future of digital transformation priorities.
To understand if Digital First teams have successfully met their programme objectives and where there are opportunities for improvement.
To provide information on priorities for programme areas and provide an overall purpose for programmes of work.

HIN support included:

Needs articulation with the Digital First team to understand expectations and requirements for a digital outcomes framework, and defining the two core goals this framework would evidence progress towards – Improving Patient Care and Improving Service Efficiency.
Stakeholder mapping and interviews with ICS teams to gain an overview of the breadth of digital projects being implemented and which metrics would be most important for teams to help demonstrate the success and value of their work for clinicians and patients.
Iterative design and testing of the outcomes framework with input from each ICS region, resulting in the final framework being available in multiple formats as per ICS requests to meet the individual requirements and ensure ease of use for the different teams involved.
Utilisation of the breadth of HIN expertise’ for project support, including obtaining specialist evaluation expertise to advise on framework design and identifying suitable metrics for monitoring processes and outcomes of digital projects.
Extensive desktop research across the ICS regions to identify and share a collection of useful, readily available NHS data sources to support ICS project teams in identifying suitable sources to collect evidence for their project outcomes.
Design and facilitation of in-person and hybrid workshops for each ICS team to socialise their outcomes framework and help implement the framework into their unique projects – attended by over 50 project, programme and clinical leads from the Digital First teams.
Adaptation of outputs for additional workstreams – the Outcomes Framework is currently being used to support monitoring of the project outcomes for the 11 Digital First Automation Grant pilots

Stakeholder feedback

“At a regional level the workshop has helped our team to understand the application of the framework to our programme and the challenges that our ICB colleagues are facing.”
Sharon Sheldon (Senior Programme Manager, Digital First for Primary Care London)

“Being provided with space and time to work through the template as a team rather than individually was a valuable process. It enabled us to hear other people’s thoughts and gain insights we wouldn’t usually gain. A beneficial and engaging workshop.”
Ian Leigh (Programme Manager, Digital First Programme)

Outcomes Framework resources:

The framework is now available to access and download on the NHS Futures page.

To support the adoption of the Outcomes Framework a series of materials were developed. These include:

Outcomes framework guide detailing the approach to implementing outcomes and providing a blank template for completion.
An Excel version of the above pro-forma (as requested by ICS Digital First Teams)
A useful data sources repository.
An outcomes tracker for monitoring outcomes across multiple projects and life cycles.

Further information

The HIN’s Digital Transformation and Technology team have extensive experience in supporting outcomes monitoring and development for the healthcare sector, including public health, and primary and secondary care services.

For further information on our outcomes framework and how we might support your organisation in adopting outcomes monitoring, please contact hin.technology@nhs.net