Translate this website

Remote Consultations in Mental Health: Learning from Evaluation Report

Introduction

COVID-19 turbo-charged the use of remote consultations in mental health, with many services having to switch to video or telephone appointments almost overnight. But what has this meant for service users and staff, and what lessons can we take for the future?

This report, published jointly with south London mental health providers, local system partners, academics and service users examines the evidence on the impact of the shift to remote consultations, shares learning and provides recommendations for future practice. This work is ongoing so please check back to the Remote consultations in Mental health resource page for updates and the latest published papers.

Executive Summary

Remote Consultations in Mental health – Learning from Evaluation Executive Summary


COVID-19 has meant significant changes in how mental health services have been delivered. Appointments that would normally have taken place face-to-face have had to be moved to video or telephone consultations.

These changes are likely to have had an impact on all the people involved in mental health care – from service users to clinicians and other mental health professionals. There may have been positive and negative effects, or unexpected consequences. Currently, however, there has been no comprehensive evaluation of these effects.

This project is led by the three south London NHS Mental Health Trusts, working in conjunction with service users and academia to develop the evidence base in this area and form a learning healthcare system.

Through conducting a robust evaluation of the current evidence and identifying any potential gaps, the project aims to guide ongoing research, disseminate best practice, and inform the delivery of services now and in the future.

This report details phase one of this project with the thematic analysis of the findings from the three workstreams- systematic evidence reviews; a synthesis of patient, carer and staff surveys and a survey of ongoing evaluations.

Executive-Summary-Read-only-1Download

Foreword

The new coronavirus SARS-CoV2 was first identified in late 2019 with the first cases of COVID-19 infection reported in the United Kingdom at the end of January 2020.  From mid-March 2020 onwards, social distancing measures were introduced to reduce the spread of the virus and health services rapidly increased their adoption of remote consultations. This included inter-professional, service user and carer facing interactions using the internet, telephone, video conferencing and text messaging. Video conferencing software called ‘Attend Anywhere’ was made freely available to secondary care from 31 March 2020.

In response to this rapid change, a number of partners across south London mental health service providers, service users, service and innovation connectors and academic evaluators came together with the ambition of forming a learning health care system[1] to evaluate experiences, implementation, and effectiveness of remote consultations in the context of existing and emerging evidence to inform and improve service delivery, during the pandemic and beyond.

Our interested organisations joined in June 2020 as an informal partnership across south London. Together we identified and agreed on a programme as part of COVID-19 response work. Phase one saw the delivery of a survey of ongoing evaluations being conducted in south London; a synthesis of patient, carer and staff surveys; and systematic evidence reviews.

Phases two and three were centred around the dissemination of our findings and establishing a ‘Learning Healthcare System’ on remote consultations in mental health settings.

This local-level learning has the potential to be captured, synthesised and shared across organisations; to identify evidence gaps, create positive change within services and improve experiences and outcomes for patients, carers and staff. This report shares our methods, findings and tools developed and highlights the gaps in evidence and future research opportunities. Please stay up to date with our work develops via our webpage and use the contact form to get in touch with the team.

Professor Fiona Gaughran – Lead Consultant Psychiatrist, National Psychosis Service, Director of Research and Development SLAM, Reader in Psychopharmacology and Physical Health Kings College London, Applied informatics research lead NIHR ARC South London

[1] The concept of a healthcare system collectively gathering information and synthesising knowledge about how well or otherwise service delivery is working then using this understanding to drive ongoing improvement can be described as a ‘learning healthcare system’.  The Institute of Medicine defines a learning healthcare system as a system in which “science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience.”  

Background

Service changes were introduced extremely rapidly across the NHS in response to the COVID-19 pandemic, including in mental health. These included a shift from face to face service delivery to a model where most outpatient contacts were conducted remotely, either on the telephone or by video consultation. These service changes may have significant advantages for many staff and patients, such as convenience, safety, time savings; they may also have disadvantages, drawbacks, or unintended consequences, and may exclude some key populations. 

Our partnership sought to understand the impact of such service changes, what has worked and what has been less successful, and for whom, in order to either embed or adapt, new and emerging models going forward, to ensure the greatest benefits for patients, carers and staff. This report also provides case studies that describe the implementation processes that supported the introduction of these novel models of care in the first phase of the COVID-19 pandemic.  

The project is set mostly in south London and includes the three mental health Trusts – South London and Maudsley NHS Foundation Trust (SLaM), South West London and St George’s Mental Health NHS Trust (SWLSTG) and Oxleas NHS Foundation Trust (Oxleas), working closely with the Health Innovation Network (HIN), Kings Improvement Science (KIS), NIHR ARC South London and the SLaM Quality Centre (a collaboration between the Institute of Psychiatry, Psychology and Neuroscience (IoPPN) at Kings College London (KCL) and Kings Health Partners (KHP) along with the Mental Health Policy Research Unit. University College Health Partners (UCLP) were also involved in the partnership contributing to the core group and sharing the e-survey across their mental health providers in north east and north central London.

Figure 1:

Increase in Microsoft teams call at South London and Maudsley NHS Foundation Trust March 2020 – May 2020.

Figure 2:

Project over-arching objectives: a South London approach to establishing a ‘learning healthcare system’ across health and care

Aims

To bring together the findings from the three workstreams to inform clinical practice and to determine ongoing gaps in knowledge. The workstreams are listed below.

  1. Systematic literature reviews of the evidence on remote consultations pre and during the COVID-19 pandemic
  2. Thematic synthesis of Trust-wide surveys on remote consultations
  3. E-survey of projects on remote consultations across south London mental health NHS Trusts

The specific objective of this work was to identify commonalities and areas of difference; highlight gaps in the evidence base around remote consultations in mental health services that may need to be addressed in future research, and inform future information gathering approaches. The evidence gathered supports mental health services and service users to learn what is already known about remote consultations.

Figure 3:

Methodology

Partnership development

Our interested organisations joined in June 2020 as an informal partnership across south London. Together we identified and agreed on a programme of work as part of COVID-19 response work. The group have formed as the MOMENT (reMOte MENTal health group).

Programme methodology

A project group was established with resources, timelines, and a governance structure across the multiple partners.  A project plan and data capture process were established and a forum (the core group) that allowed sharing of tools and approaches. A dissemination plan and engagement strategy were written to connect with colleagues across London and help create a larger learning network. A group of four experts by experience with a range of backgrounds, including experience of using health and social care services and caring responsibilities, were recruited by the KIS Patient and Public Involvement (PPI) Coordinator through the KIS involvement bulletin. The project group used the learning health system as an organising framework and co-designed the work programme with providers, users, and evaluators.

Programme structure – The programme was split into three workstreams.

  • Workstream one – Evidence reviews

Two reviews were conducted. One ‘Umbrella Review’ on the evidence pre-COVID-19 and a systematic review of evidence during the pandemic

Umbrella review

We conducted an ‘umbrella review’, also known as a ‘review of reviews’, of research literature and evidence-based guidance on remote consultations in mental health, including both qualitative and quantitative literature, conducted and published prior to the COVID-19 pandemic. The aim was to identify the pre-COVID-19 literature on guidance, effectiveness, implementation and economic effectiveness of remote consultations in mental health. Nineteen reviews met our criteria, reporting on 239 studies and 20 guidance documents. The review included studies on telephone counselling, videoconferencing for diagnosis, therapy and education across a range of diagnoses.

Systematic review

We also conducted a systematic review of the evidence on remote consultations in mental health services during the COVID-19 pandemic. Our aims were to identify the adoption and impact of remote “telemental” mental health approaches and facilitators or barriers to optimal implementation. Seventy-seven relevant papers were synthesized.

  • Workstream two – Trustwide patient and staff surveys on remote consultations

A thematic analysis was conducted to synthesise the questions and findings across four organisation wide surveys that were carried out within the three south London mental health Trusts in the summer of 2020. This analysis included results from patient and carer surveys, one survey of Consultants and Specialty and Associate Specialty (SAS) doctors, and one survey that collected responses from patients, carers and staff (see table one). Please see here for copies of each individual survey report.

Table 1:

Table 1: summary of Trust surveys on remote consultations
  • Workstream three –  e-survey

An electronic-survey (e-survey) (Appendix one) was conducted to collect information about evaluation, research or quality improvement projects studying any aspect of remote consultations (both patient-facing and interprofessional) within mental health services. The survey was initially conducted within the three mental health Trusts in south London, though some of the projects described were in local voluntary sector organisations or had national / international reach. The survey was then carried out in north east and north central London following minor modifications.

The focus of the survey was to collate information about project aims and methods; we did not capture findings.

The south London e-survey was initially open to responses between 13th July 2020 and 1st September 2020 and was briefly electronically reopened on 11th November 2020 to allow entry of five additional projects that became known to members of the project group. In north east and north central London the e-survey was open between 5th October 2020 – 15th November 2020.

Summary of findings

Triangulation of evidence and data: A thematic analysis of the findings from the three workstreams was undertaken by a multidisciplinary team of researchers and project managers, further scrutinised by a team of experts by experience, and by clinical and managerial members of the project group. An infographic summarising the findings from this analysis is being developed by the experts by experience (underway).

Workstream One: Evidence reviews

Umbrella review of pre-COVID-19 literature- Please see [https://www.jmir.org/2021/7/e26492] to access the paper.

This systematic review of systematic reviews revealed the following:

  • Remotely delivered mental health services can be as efficacious and acceptable to staff and patients as face-face formats, at least in the short-term.
  • There was little evidence on large scale implementation of remote consultations and effectiveness in ‘real-world’ (i.e. outside of a research study) settings.  
  • Further, the findings of this umbrella review did not provide us with evidence relating to digital exclusion and how it can be overcome and was not able to provide conclusions on particular contexts, for example children and young people’s services or inpatient settings.

A systematic review of COVID-19 literature from during the pandemic– (link will be added to paper when available)

A synthesis of 77 relevant papers from 19 countries demonstrated that globally, many countries had been able to rapidly shift to remotely delivered mental health services during the pandemic. In general, these studies suggest that:

  • Telemental health has been reasonably well accepted, particularly where the alternative was no contact.
  • A mixture of telephone and video-based calls have been offered, with people expressing different preferences for these.
  • Concerns about remotely delivered services are raised in relation to new service users, physical healthcare, and privacy and confidentiality.
  • A small number of studies have formally investigated how telemental health may best be implemented, though suggestions made within this body of literature to support implementation include:
    • Staff training, champions for telemental health, providing service users with access to technology and guidance on how to use it
    • Providing staff with guidance on identifying whether a remote offer is appropriate in different situations / with different individuals.

Overall, the literature suggests that the delivery of telemental health has been largely successful within the context of a pandemic. Nonetheless, longer-term evaluation and better evidence is needed as restrictions on physical distancing between people evolve.

Figure 4:

Infographic the PPI Group produced.

Workstream two: Summary of findings from the Trust patient and staff surveys

Patient experience of remote appointments can be summarised as follows:

Oxleas:

  • 90% of patients responded “Yes” or “Somewhat” when asked if they were happy with the care and treatment received in their remote appointment.
  • 79% of patients responded “Yes” or “Maybe” when asked if they would like to be able to have remote appointments in future.

SWLSTGs:

  • 97% of survey participants reported that they would either ‘definitely’ or ‘probably’ use the system again, were they to be offered the option, despite issues with video and audio quality reported in the survey.

Joint patient and staff survey at SLaM:

From responses to a question on experience and one on future intent, three profiles of virtual contact users was constructed.

• Resistant (n=84): those who reported that their virtual contact experience was “worse/ much worse” than that in face-to-face contact, and they are “somewhat/ very unlikely” to want it in the future

• Ambivalent (n=338): those who did not find virtual contact experience better than that in face-to-face contact, yet they showed no intention to reject it in future

• Receptive (n=123): those who found virtual contact “better/ much better” than face-to-face contact and are “somewhat/ very likely” to want it in future More detailed information on each of the surveys is included in table two in the appendix.

The themes produced were considered according to whether the survey responses had been collected from patients or staff.

The following themes were generated, a range of opinions were expressed in relation to each theme.

  • Convenience.
  • Environment and privacy.
  • Choice.
  • Openness during consultations.
  • Limitations compared to face-to-face.
  • Longer-term use.
  • Resources required for better implementation. 

The analysis of themes across all the surveys is listed in table three in appendix two.

Important gaps in the information available following the thematic synthesis were identified.

  • There was a lack of demographic information about participants. Considering the information about respondents that was available and combining this with knowledge about the survey sampling and distribution methods, we are able to conclude that respondents are not representative of the population and as such the findings of the thematic analysis may not be generalisable
  • We also recognise that the surveys were designed to capture a snap-shot of perspectives at a particular point in time within the context of a pandemic and that view-points shift over time.

 Convenience

“Would prefer to use this system rather than face to face. It is more convenient for me as I work full time and means I do not have to leave work early”

South West London & St George’s Mental Health NHS Trust

“Logistically more convenient, no travel expense and in an era of COVID-19, feels safer.”

“Benefits: less travelling time, more productive”

OXLEAS

 Benefits: less travelling time, more productive

“Easier to manage work life balance, less tired as reduced travel”

South West London & St George’s Mental Health NHS Trust

Making best use of data – improvement analytics

In addition to the Trust surveys, both SWLSTG and SLaM rapidly developed dashboards to track access and productivity to remote consultations over video, in person and telephone. You can view a case study from SLaM here.

Improvement analytics supports delivery of a Trust Data Strategy where the Trusts aim to become a data-driven organisation, where all staff have the capacity and ability to use data to inform decision-making and improvements in order to improve outcomes for the patients and communities. 

This includes:

  • Visualisation of data for improvement
  • Diagnostic data
  • Measurement plans
  • Power Business Intelligence dashboards
  • Data for improvement resources and coaching

These dashboards provide visualisation of data which is intended to be the foundations of a conversation starter.

Please see Figure five and six for snapshots of the dashboards. The SWLSTG dashboard shows the increase in telephone and video consultations from the point of the first lockdown. It also illustrates the time period of the patient survey described in workstream two.

Figure 5:

Figure 5- Example of the SLaM dashboard

Figure 6:

Figure 6: SWLSTG dashboard in relation to their Trust patient survey on remote consultations

Workstream three: Summary of findings from the E-survey

Responses

Responses from south London based mental health organisations described 22 projects. A further 10 projects being conducted in north east and north central London were captured by the survey. The findings describe all 32 projects (please see appendix two for more detail on each project).

Focus of projects

The majority of projects (29 out of 32) sought to assess patient and/or staff perspectives on experience and/or access via surveys or interviews.

Design and methods

There were 16 service evaluation projects; five quality improvement projects; five service evaluation and quality improvement projects; five research projects; and one strategy discussion.

The broad methodological approaches being used are: qualitative, e.g. interviews/focus groups (three projects); quantitative, e.g. analysis of routinely available data (one project); survey (20 projects); mixed methods (7 projects), unclear (1 project).

Patient and public involvement within the projects:

Just over a third of projects (11 out of 32) stated intention to involve their patients/public members within the project team, for example in aspects of project planning and delivery, as opposed to involving patients/service users/carers as participants in the project.

Service areas and patient groups:

A range of service areas and patient groups are included across the 32 projects.

  • Child, adolescent and young people [8 projects]
  • Community Mental Health Services [4 projects]
  • Education and training [1 project]
  • Forensic [1 project]
  • Gender services [1 project]
  • Integrated Psychological Therapy Team [1 project]
  • Aging/older people’s services [2 projects]
  • Learning disability [3 projects]
  • Memory services [1 project]
  • Parents and carers of young people using mental health services [1 project]
  • Personality disorder [1 project]
  • Psychosis [1 project]
  • Adult mental health services [1 project]

Some projects span multiple services, others are Trust (or organisation) wide.

Demographics:

Under a third (nine out of 32) demonstrated intention to collect demographic information from participants. This is potentially an important gap in terms of better understanding for whom remote consultations does and does not work well for.

Remote consultations technology: type, function and support

Many projects included the use of multiple types of phone or video-based technology within their questions / data collection. Technology solutions specified included:           

  • Attend Anywhere
  • Cisco WebEx
  • FaceTime
  • Iaptus Virtual Consultation
  • Microsoft Teams
  • Skype
  • Telephone
  • Zoom

Microsoft Teams and Zoom were the most mentioned platforms.

Across the 32 projects, remote consultations was listed as being used to facilitate:

•            Individual assessments

•            Routine clinical appointments

•            Individual psychological therapies

•            Group psychological therapies

•            Online arts psychotherapy

•            A listening service

•            Emergency appointments

•            Patient reviews

•            Interprofessional communication and administration including meetings

•            Service evaluations & surveys

Respondents were asked whether support is offered for remote consultations to staff and/or patients (e.g. training, introductory video, or technical assistance). Some respondents left this question blank or indicated that no support is offered. Where it was indicated that support is available, this took the form of webinars; patient leaflet / instructions; informal training from staff (unclear whether this relates to staff-to-staff or staff-to-patient), support for staff from Attend Anywhere Team; and staff supporting patients to complete e-surveys.

Questions about preferences or comparisons between different remote consultations platforms are included within ten projects. In addition, nine projects will collect information about technical issues (e.g. hardware, connectivity, inter-operability) or anticipate that this may emerge within qualitative findings

Outcomes being studied:

A minority of projects (four out of 32) were set to assess the effectiveness of remote consultations on clinical outcomes or examine cost. None of the projects were assessing cost effectiveness.

Very few respondents gave clear details of service level outcomes (for example relating to numbers of patients supported or impact on team working) assessed within their projects, though one project will collect service level data on Did Not Attend (DNA) rates and travel.

Participants were asked what (if any) implementation outcomes (e.g. acceptability, adoption, appropriateness, feasibility, fidelity, penetration, sustainability) are being studied and how these are being measured or assessed. Nine projects will assess one or more implementation outcomes via qualitative interviews or surveys with acceptability being mentioned most frequently (four projects). It is unclear whether any of these projects intend to use validated scales to assess implementation outcomes.

A minority of projects assess unintended outcomes (positive or negative) or inequities (for example characteristics of or number of people not accessing remote care).

Theories and frameworks:

The majority of respondents did not report applying a specific framework, theory or model to guide their reported project. 

Potential gaps in research and evaluation:

Several topic areas are not well covered by the 32 projects and are potentially under-researched or evaluated:

  • Contextual details including numbers of patients in contact with services; the proportion of contacts delivered remotely versus face to face; the characteristics of patients who are (or aren’t) accessing services in different ways; and how each of these variables are changing over time
  • Digital exclusion / inequities and gaining an in-depth understanding of which groups of people are not well served by remote consultations
  • Patient and public involvement in designing and executing projects
  • Understanding impact on staff interprofessional working
  • Evaluating the support offered to facilitate remote consultations (e.g. training, webinars, patient leaflets)
  • Effectiveness studies looking at clinical outcomes
  • Cost effectiveness of remote consultations
  • Evaluating implementation of new pathways including hybrid/blended approaches to service delivery (a mix of face-to-face and remote delivery) and de-implementation of old ways of working
  • The use of implementation frameworks / theories / models to understand, contextualise and generalise project findings

However, it should be noted that many of the above topics are indeed touched upon or may emerge as areas of exploration or as findings within some of the projects, particularly within those projects that are collecting qualitative data from participants.

Limitations of the e-survey:

The e-survey provided a snapshot of projects planned or underway between July and November 2020. Many projects will have been instigated subsequently.  

While every effort was made to disseminate the survey widely across the eligible mental health services, we are aware that some relevant projects already underway were missed.

While a small number of responses were received from charitable / voluntary sector organisations, there is likely to be a body of work being undertaken by such organisations that is not captured here.

The survey distribution method used by each organisation did not follow a uniform approach. It is possible that some distribution methods were more effective than others in reaching the relevant people and encouraging project leads to complete the survey. Direct email contact with individuals known to be engaged in relevant projects appeared to be the most successful way to encourage survey completion.

It would have been helpful to obtain more detail about the methods being used within each project, and any repeat surveys in the future should include questions that would elicit this information. For example, where it is indicated that the project involves a survey, we do not have detail about whether the survey was distributed in an electronic (e.g. email or text) or paper format and we often do not have information about the types of questions asked.

The questions on service outcomes, implementation outcomes and unintended consequences were frequently left blank by respondents, potentially indicating that participants did not understand what the question was asking or that they felt it was not relevant to their project. It would be useful to obtain feedback on the survey questions from staff members should the survey be repeated. Furthermore, in future iterations it may prove helpful to follow up each response via telephone interview, allowing exploration of questions where limited detail was initially offered.

It is possible that the methodological quality of future projects could be strengthened by providing team members with training or relevant resources on implementation.

The original aspiration was to embed patient and public involvement within this piece of work. The short timeline meant that it was not possible to involve public members within the development of the e-survey, however, the findings have been discussed in group meetings which include experts by experience.

Commonalities across workstreams and recommendations

The following themes emerged from the three workstreams and are inter-related.

Perspectives are not universal

Our collective findings across the workstreams clearly demonstrate that there are a variety of perspectives regarding remote consultations between staff and patients, and regarding remote consultations more broadly (including staff inter-professional working).

The remote delivery of mental health services works well for some people but not others and is appropriate in some situations or on some occasions but not others for many individuals. There is no ‘one size fits all’ and an individualised approach will always remain the gold standard. This breadth of experience holds true for patients, carers, and staff.

It follows that the effectiveness of remote consultations on a range of outcomes – for example clinical symptoms for patients, or wellbeing or productivity for staff – may not be universal either; this too remains to be established.

Recommendation: Further research is required to better understand under which circumstances remote consultations is beneficial, for whom, and why, in order to make evidence-informed offers regarding the mode of service delivery and to provide increased choice.  This research should purposively seek to tap the views of under represented populations e.g. racial and ethnic minority groups, carers and do deep dives within clinical populations.

Acceptability

Findings from our thematic analysis of the organisation wide surveys mirror findings from our two literature reviews in terms of the acceptability of remote consultations to patients, carers and staff. Both workstreams suggest that while there are different opinions, and while face-to-face contact may be preferred, remote service delivery can be acceptable to patients, carers and staff, at least in the short-term, with many participants indicating that they are satisfied with this way of working. Levels of satisfaction may be higher when video calls are used as opposed to telephone calls. It is noteworthy that the pre-COVID-19 review indicates that these findings may apply outside of the context of a pandemic. Again, findings from the staff and patient surveys, in particular, illustrate the point that individuals may find remote working and/or remote consultations acceptable on some occasions / in some circumstances but not others. Furthermore, the likelihood of non-response bias (where people who take part in a study are systematically different to those who do not) is a key caveat here as participants across workstreams were firstly able and secondly motivated to engage in research and provide their feedback or data may not be representative of wider populations.

Recommendation: A set of questions to be routinely asked as part of future projects should be developed. As an example, questions should elicit participants’ demographic information in order to better understand whose perspectives and data are being captured or excluded. This would help us to understand whether there may be differences according to ethnicity or living in an area of relative deprivation/advantage, for example. This work has commenced by the project team please check for updates.

Recommendation: Future research and evaluation strategies should specifically target the groups who have been under-represented in the data sets analysed to date, including but not limited to: older adults, children and young people, people with learning disabilities, people with an autism spectrum disorder, inpatients, drug and alcohol clients, prison leavers, homeless people and carers.

Accessibility

For many people, for example, those with diverse communication needs, the widespread adoption of remote technologies at the start of the pandemic removed choice and reduced the ability to access mental health services. Findings from the staff and patient surveys demonstrated that some patients had received text messages inviting them to a video-based consultation and including a link to join the virtual meeting without any prior conversation about whether this format was appropriate for their needs. Similarly, some consultations were offered via telephone without assessment of whether this was an appropriate means of communication for individual patients and carers. There are also some good examples of services using innovative ways to engage service users in remote consultations – (Click here to see how the Recovery College at SWLSTGs have supported their students in accessing resources remotely and here to see how SWLSTGs have assisted their patients and carers in accessing Attend Anywhere via a step by step video guide) – and where guidance was produced to help clinicians with decision making tools for choosing between remote, in person or blended consultation.

Recommendation: Organisations and services should ensure that the NHS Mandatory Accessible Information Standards are adhered to when offering remote consultations or indeed when staff are engaged in remote work more broadly. We need to be asking service users and carers about their capabilities and confidence and addressing this.

Convenience

Three of the systematic reviews included within the pre-COVID-19 umbrella review assessed convenience, with most patients indicating that engaging with therapy sessions from home via remote interventions was convenient. Convenience was also a main theme arising from the thematic analysis of survey findings. Many respondents highlighted the convenience and time and/or money saving nature of remote consultations or remote work by virtue of not needing to travel. Further, some people felt that remote appointments could facilitate the attendance of more people from a multi-disciplinary team, though others suggested the opposite was true. Importantly, however, there was a consistent message that some people find remote consultations (or for staff remote work more broadly) inconvenient some or all of the time. Patients cited difficulties with computer literacy, having an appropriate private space, involving family members or carers in appointments where this was wanted, and poor virtual meeting etiquette (e.g. being left in ‘waiting rooms’ for lengthy periods). Staff also noted problems with meeting etiquette (e.g. meetings over-running), unsuitability of the environment for privacy and ergonomic reasons, and the tiring nature of virtual meetings.

Therapeutic alliance

Findings from both the staff and patient surveys and the evidence reviews suggested that for some people it is possible to develop a good therapeutic alliance remotely, although it is perceived that therapeutic alliance may be better when services are delivered face-to-face. In our pre-COVID-19 umbrella review, female older adults and veterans generally expressed a preference for talking to therapists in person. One of the studies included in the during-COVID-19 systematic review reported that 88% of clinicians found it more difficult to establish a therapeutic relationship with new clients when consultations were held remotely. Similarly, two systematic reviews within the umbrella review included findings demonstrating poorer clinician ratings of the therapeutic alliance during remote work. There was some suggestion that therapeutic alliance may develop more easily in consultations held using video-conferencing software as opposed to the telephone.

Technological challenges

Within our patient and staff survey findings, specific issues relating to the use of technology included: user confidence and knowledge around using technology, issues with Wi-Fi and connectivity, ability to access (and cost of) appropriate equipment and software subscriptions, and security/information governance challenges. Having access to technology and appropriate support to use this technology were identified as key barriers to uptake. These findings applied across patients, carers and staff. Within our umbrella review, three of the systematic reviews included mentioned technical difficulties as a challenge, however, none of these reviews implied that technical difficulties had been a severe barrier to implementation. However, issues were reported around mistrust in technology, low image resolution, and connectivity problems.

Exclusion

Findings across workstreams raise the possibility that many people may have been excluded from accessing mental health services with an impact on their wellbeing and their families, or have had their access reduced, as a result of the rapid shift to remotely delivered services. This is mirrored by the presumed exclusion of people who are not routinely using remote technologies from much of the research and evaluation data that have been analysed to date. However, we do not have systematically collected data to demonstrate the extent of digital exclusion or to draw conclusions about which groups of people are most adversely affected. There are a small number of projects (that we obtained details of via the e-survey) that are seeking to understand the perspectives of some of those groups who are more likely to be digitally excluded (for example people with learning disabilities and older adults). Within our staff and patient survey synthesis, it was recognised that the perspectives of older adults are mostly unknown. While the umbrella review included data relating to some groups who are thought more likely to be digitally excluded (e.g. older adults) there was a lack of evidence for other groups including children and young people and inpatients, and overall, as outlined above, a lack of demographic information about people who had participated in the research studies (which was also a key limitation within the e-survey and evidence reviews).

Recommendation: The co-creation of research/evaluation and service delivery strategies to help understand and address digital exclusion and inequities will be vital and careful consideration will need to be given to assess how best to involve those who are under-represented and/or digitally excluded within the development of these strategies. Engagement from a variety of services, for example, assessment centres, food banks, probation services, supported accommodation and community charities may be needed to reach those who are under-represented. It is acknowledged that this work would be challenging, but it will be essential for services where digital is the primary route to care.

Guidelines

Responses to staff surveys synthesised indicated that staff would appreciate clear guidelines on how and when to offer remote consultations as opposed to face-to-face. This was echoed in the during-COVID-19 literature review. Our umbrella review included one systematic review of guidelines for video-conferencing based mental health treatments[1]. This review encapsulates guidance on decisions about the appropriateness of remotely delivered mental health services; ensuring competence of mental health professionals; legal and regulatory issues; confidentiality; professional boundaries; and crisis intervention.

Recommendation: It may be beneficial for those who are developing new guidance on video-based consultations within mental health services to draw upon the recommendations made within the systematic review by Sansom-Daly et al. (2016).

[2] Sansom-Daly UM, Wakefield CE, McGill BC, Wilson HL, Patterson P. Consensus among international ethical guidelines for the provision of videoconferencing-based mental health treatments. JMIR mental health. 2016;3(2):e1

Case studies

See all the case studies

Remaining research and evaluation gaps

Reaching those who are least able to engage in remote consultations

All three workstreams likely under-represent the voices of those who are least likely and able to engage remotely which represents a significant and worrying gap in the available evidence. Data collection mechanisms to date have been overly reliant on electronic means – for example surveys administered by email. Innovative methodologies will be required to proactively reach digitally excluded people and enable their participation in both developments of a research strategy and in the research itself, especially under the restrictions in movement and socialisation placed upon the population during the acute phase of the COVID-19 pandemic.

Uptake

Some of the data considered within this project suggest that the offer and uptake of remote consultations varies according to service. Further work could be done to understand the reasons underpinning choices made by services and differences in uptake amongst different patient groups. The routine collection of data relating to the mode of service delivery over time in each clinical pathway would likely form an important next step in the development of a ‘learning healthcare system’ that collectively gathers information and syntheses knowledge about how well or otherwise service delivery is working then uses this understanding to drive ongoing improvement.

Change over time

We are currently unable to draw conclusions about whether perceptions in relation to remote consultations are changing over time nor whether viewpoints will evolve as we move beyond the acute phase of the COVID-19 pandemic and towards a situation where face-to-face contact poses less risk of spreading infection. Longitudinal data are needed to inform future choices and investments. The most rigorous way to assess change in perceptions and experiences over time is to ask the same set of people the same questions repeatedly. This requires each participant to have a unique anonymous identifier assigned to their survey responses in order to track change over time. This methodology is often more time and resource intensive to set up, and the attrition rate (where people drop out / don’t complete surveys) may be high. Further, people who are motivated to participate in completing a series of surveys may have different characteristics or perceptions compared to people who do not participate.

Blended models of service delivery

We currently know very little about models of delivery, experiences and effectiveness of mental health services that are delivered through a combination of remote and face-to-face consultations.

Recommendation: Research is needed to evaluate the implementation of new pathways including hybrid/blended approaches to service delivery (a mix of face-to-face and remote delivery) and de-implementation of old ways of working.

Effectiveness

While the pre-COVID-19 umbrella review demonstrated that remotely delivered services can be as good as face-to-face appointments in improving clinical outcomes in some circumstances, we cannot say with certainty whether this finding holds true in the case of fast and widespread implementation due to the pandemic, as there was a lack of high quality quantitative evidence within our during-COVID-19 literature review . 

Recommendation: It is important that future work addresses questions of clinical effectiveness and better ascertains for which groups of people and which clinical pathways remote consultations are and are not effective before being routinely offered as the norm post-pandemic.

Cost and cost effectiveness

Given the relative dearth of evidence on the effectiveness within the pandemic context, it follows that little is known about the cost effectiveness of remotely delivered mental health services. Within our umbrella review, two systematic reviews examining either costs or cost effectiveness met our inclusion criteria. One systematic review concluded that tele-psychiatry can be cost effective compared to face-to-face interventions, particularly in rural areas where the number of consultations required before telepsychiatry becomes more cost effective (combatting initial equipment costs) is lower. In the second systematic review which looked at costs, 60% of included studies reported that telepsychiatry programmes were less expensive than in person care, due to savings such as travel time and reduced need for patients and their families to take time off work. However, eight studies in this review concluded that telepsychiatry programmes were more expensive, particularly due to videoconferencing equipment costs. A final study included in the review found no difference in costs.

Recommendation: Further research regarding costs and cost effectiveness is needed, particularly as video-conferencing software is now more widely and cheaply available.

Implementation effectiveness and support

Our COVID-19 specific literature review had a focus on exploring barriers and facilitators to optimal implementation of remote consultations and the emerging evidence on this is summarised. the pandemic led to remote consultations and remote work being implemented urgently as a matter of need and not choice. This presented little chance to study implementation effectiveness in real-time, thus work remains to be done to establish best practices in terms of implementing remote consultations. Such studies are now feasible as remote options are likely to be offered long-term in some settings. The existing implementation science literature may be helpful in designing better implementation support going forward. Furthermore, we may be able to apply frameworks retrospectively in order to generate additional learning from implementation efforts undertaken within the context of a crisis.  

Next steps

The programme has achieved a great deal since June 2020 and the work continues using the Learning Healthcare system model as our guiding principle. 

Evaluating remote consultations is a priority for the London Healthcare system.  Research resource is being established for this for the NIHR Applied Research Collaborations (NIHR ARCs) that we hope will continue to inform our work.

We’re here to help

Find out more about our work in mental health.

Explore

Joint Pain Advice in the workplace

Work and Health Challenge Fund Project

About

The Joint Pain Advice (JPA) model was developed by the Health Innovation Network (HIN) as a result of an identified need for accessible, personalised, understandable information and practical advice and support about how to self-manage the impact of chronic pain. Pilot studies have demonstrated that JPA supports participants with chronic knee, hip and back pain to reduce their pain and symptoms, increase their physical activity levels and improve their physical function and mental wellbeing in both clinical and community settings.

HIN and the Bone and Joint Research Group (part of Royal Cornwall Hospitals NHS Trust) were awarded funding from the Work and Health Challenge Fund to test the JPA model delivered in the workplace to employees.

The evaluation of this service improvement project aims to understand the effectiveness of JPA as a workplace intervention.  It explores the clinical effectiveness of JPA and the acceptability and feasibility of the programme for participants (employees), JPA Advisors and workplaces, delivered in a ‘real world’ workplace setting.

Executive Summary

Overview

In the UK over 8.75 million people have osteoarthritis (OA) and around 10 million people have chronic low back pain. These problems impact an individual’s personal, social and working lives, affecting physical and mental health and wellbeing1. They also have a wide socioeconomic impact through substantial health and social care expenditure and lost productivity2. Although National Institute for Health and Care Excellence (NICE) guidelines core recommendations for the management of OA3 and low back pain4 are to provide information and advice to improve understanding of the condition and management strategies and to encourage increased physical activity and support weight loss (where appropriate) as effective ways of managing the condition and its impact, in reality, few people receive this advice.

Joint Pain Advice (JPA) supports people to self-manage their chronic hip, knee and/or back pain, based on NICE guidelines. Within the JPA model, participants are invited to up to four face-to-face appointments over six months. This intervention has been successfully piloted in primary care and community settings with c600 participants who reported improvements in pain, physical function, physical activity and mental wellbeing5,6. It was also shown to reduce follow-up appointments with a GP5. A social return on investment evaluation found that the JPA intervention offered a social return on investment of between £2 and £4 for every £1 invested7.

JPA in the workplace was one of 19 initiatives across the UK selected to receive funding from the Work and Health Challenge Fund. As part of this project, JPA was offered to employees across four large organisations in London and 16 organisations in Cornwall, including several small and medium enterprises (SMEs). Two different models of delivering JPA were tested:
In London, existing members of staff of the larger organisations were trained to deliver JPA to their organisation’s employees. They are referred to in this report as ‘in-house’ Advisors.
In Cornwall, JPA Advisors were trained and travelled around from workplace to workplace to deliver JPA to employees of several organisations, including small and medium enterprises (SMEs). They are referred to in this report as ‘peripatetic’ Advisors.

Challenge-Fund-JPA-Workplace-Report-V4-1Download

We’re here to help

We are always happy to hear from people who want to know more about the JPA in the workplace project or our other work in musculoskeletal.

Get in touch

Psychologically Informed Collaborative Conversations
(PIC-C)

Pain is common in the general population, affecting between one-third and one-half of UK adults. The connection between pain and psychological wellbeing is reciprocal – with increased awareness that when pain persists it often leads to increased psychological distress and high levels of psychological distress can increase the likelihood that painful musculoskeletal conditions will either persist or develop.

Helping clinicians implement psychologically informed treatment strategies and techniques offers the promise of improving patient care for people living with pain.

In 2020, The Health Innovation Network AHSN, in partnership with St George’s Hospital NHS Foundation Trust, Kingston Hospital NHS Foundation Trust, the Physiotherapy Pain Association and Duke University, USA secured funding from Q Improvement Lab, a Health Foundation Initiative, to test innovative ideas to improve care and services for people living with pain. An evaluation report of this project – Psychologically Informed Collaborative Conversations (PIC-C) – now available to download.

Download the full report

“PIC-C training has changed my practice quite a lot. I’m much more happy to sit and listen to the patient. Give them more time, get the whole story, take more time on the subjective and include more of the biopsychosocial [components], rather than just looking at the physio and the exercises.”

Course participant

Report summary

PIC-C is an evidence based online training and supervision programme to help physiotherapists increase their confidence in delivering psychologically informed care and support to patients presenting with pain. The design is based on four key themes:

  • building a therapeutic alliance;
  • reducing perceived threat of pain;
  • reconceptualising pain beliefs and somatic experience; and
  • fostering self-efficacy for pain control.

PIC-C was co-created with patients who were equal members of the project team, involved in reviewing, critiquing and contributing to content.

PIC-C was tested with 38 physiotherapists. Evaluation two weeks after completing the course showed a number of significant benefits of the training.

Graph showing reported improvements in compassion, confidence, resilience, work related fatigue and work satisfaction
Graph indicating the impact of PIC-C training on participant attitudes and feelings (n=38)

  • 100 %

    of participants reporting increased resilience following the training

  • 96 %

    of participants reporting that the training had increased their compassion for others

  • 50 %

    reporting that the training had reduced their work-related fatigue

“I definitely feel more confident now, in seeing this particular group of patients. Talking through other people’s experiences and my own has definitely made me feel more confident with the idea of [psychologically informed practice].”

Course participant

PIC-C was tested with 38 physiotherapists.  Evaluation showed: A reduction in physiotherapist anxiety and increased confidence with 100 per cent of respondents stating PIC-C had a positive impact on their confidence in delivering psychologically informed consultations.

A change in practice in using psychologically informed strategies and techniques with patients as participants progressed through the training. The biggest change being that Physiotherapists attended more to the therapeutic alliance. Pre-PIC-C, 19 per cent of participants stated that they focused on ‘developing a therapeutic alliance’ as a standard part of their practice. Post-PIC-C, this increased to 81 per cent of participants.

Participants valued the supervision element of the programme, with participants recognising this was a unique benefit of the course offer. Having protected time to discuss the practical application of techniques learned including complex cases helped embed learning and strengthened relationships with colleagues.

PIC-C influenced a change in practice, evidencing a shift in confidence in adopting psychologically informed approaches. PIC-C provides benefits to practitioners themselves and the organisations in which they work.

The PIC-C training course is relevant to all physiotherapists and health professionals and therefore will be of interest locally, nationally and internationally. Supervision post teaching sessions was identified as a ‘missing link’ by physiotherapists in the literature and our exploratory focus groups, and its inclusion sets this training apart from other packages. 

Presently, there is no similar learning programme available to physiotherapists in the UK.  We therefore recommend PIC-C is made available to health professionals as a stand-alone programme or incorporated into wider educational learning packages e.g. postgraduate courses.  We also recommend securing accreditation to satisfy quality control and continuing professional development requirements and recognition for Advanced Clinical Practice.

Additional resources

We’re here to help

If you have any questions about the PIC-C project or would like more information please contact Amy Semple, PIC-C Project Lead.

Get in touch