Mindset XR Module 7: How to document clinical evidence for medical devices

In regulatory terms, conceptualisation is done by writing an Intended Use Statement. This is where you clearly define:

• ---

  Intended medical indication e.g. mild or moderate depression.

• Intended patient population e.g. english-speaking adults.

• Intended user groups e.g. patients, clinicians, technical staff.

• Intended parts of the body e.g. headset in contact with the face.

• Intended use environment e.g. home or clinic setting, with a stable internet connection.

• Operating principle:

  • Clinical: how does your product fit in the care pathway? At a more granular level, how does a user navigate through your product and use it? 
  • Technical: how do you implement your product technically? What are the high-level architectures of any AI models you use? What is the process for any data transfer? 

  ---

At the end of your intended use statement, it’s also helpful to define any ways in which your device could be foreseeably misused, and what mitigations you have in place to prevent that from happening. 

Finally at the end of the Intended Use, you should define the risk classification of your device, using the specific regulatory rules as described by each jurisdiction you want to deploy your device in. 

This is a high-level document (10-15 pages) which describes all the steps you are going to take to demonstrate scientific, analytical, clinical, and post-market validity. Each activity should be itemised, and milestones should be set to meet before moving onto the next step.  

The clinical evaluation plan is also where you define your clinical benefit. 

Conducting a thorough and systematic literature review can be intensive but is an essential part of the process. A systematic literature review involves following a pre-specified and reproducible search protocol allowing you to find and critically evaluate published scientific evidence relevant to your device. Being systematic in your approach is essential so that your review can be reproduced and updated regularly as new evidence becomes available.

One of the central questions to ask yourself when approaching the literature review is:  

• Is there a valid association between the outputs of your technology and the clinical/biological target? 

Unlike academic literature reviews, you also need to try and answer the following questions 

• Is your technology based on sound scientific principles? 
• How well does your approach work? 
• Who are your commercial competitors? 
• What is the current clinical gold standard in each target jurisdiction? 
• What are the gaps in the literature? i.e. what studies do you need to do? 

During your review you should also look out for best practices in investigation design: 

• Sample sizing.
• Control/comparator arms if needed.
• Validated outcome measures and endpoints (both clinical and statistical).
• Gold-standard/current standard of care clinical practice.

We recommend you follow the PRISMA guidelines to ensure your literature review is comprehensive and unbiased. 

When documenting the findings of your literature review, it’s helpful to: 

• Tabulate all results with reasons for inclusion / exclusion.
• Tabulate performance benchmarks for clinical practice, SotA and any competitors.
• Include any risks identified.

In the discussion section of your literature review, you should cover the following: 

• Answer the question – is there a valid scientific association between our claim and the literature? 
• What is that association, and how strong is the evidence? 
• What performance metrics are we aiming to prove? 
• What are the common methodologies for investigations in this use case? 
• What are the evidence gaps we need to demonstrate? 

When documenting your literature review process, you should be thorough. In particular, you should: 

• Ensure consistency across reviews.
• Document in enough detail to enable others/new hires to conduct the review if required.
• Review search terms regularly to include State of the Art.
• Maintain a competitor analysis matrix.
• Must be documented to feedback into adverse event reporting, risk register and evidence requirements.

You must repeat your literature review at least annually: 

• This is a minimum requirement criteria under the MDR.
• You should include any and all publications relating to your device, whether positive or negative.
• Good practice is to repeat the searches 3/6 monthly, and hold an internal clinical review to include/exclude any new studies.

Analytical validity is more related to technical/engineering teams rather than clinical teams, so we won’t focus as much on these documents. Briefly, you need to document:

• ---

  Product requirements and risk register (PRR): itemise the product requirements and risks associated with failure of each.

• Software verification plan (SVP): outline the unit tests you will run to show your software works. Report the results in the software verification report (SVR).

• Risk management plan (RMP): show how you will mitigate against identified risks of the device. Report the results in the risk management report (RMR).

• How you ensure that your product is cyber secure and compliant with General Data Protection Regulation (GDPR).

A clinical investigation is essentially a study protocol which outlines:

• The risks and benefits of conducting the investigation of an experimental medical device.

• A justification of the investigation design.

• The investigation design itself, including a justification of sample size and statistical analysis plan.

• Additional documentation e.g. investigation brochures, case report forms, informed consent forms and monitoring plans for adverse events.

At this point, it’s also helpful to remember logistical considerations of clinical investigations: 

• Have agreements in place with partner sites or clinical trials units that outline responsibilities regarding clinical, data and risk management.

• Consider the need for ethical approval (likely), Health Research Authority (HRA) approval (if recruiting patients via the NHS), and regulatory exemptions.

ISO 14155 is an international standard which outlines best practice in conducting clinical investigations of medical devices. Within the standard, there are outlines of what needs to be considered in clinical investigation plans, and what needs to be documented in clinical evaluation reports. We suggest you buy the standard, because auditors will check that you have it! 

Researchers conducting investigations on human participants need to have Good Clinical Practice (GCP) certification. This is a free, 4 hour course available on the NIHR website and outlines the fundamental requirements of being safe and ethical during investigations. 

If you are using an experimental device for a medical purpose in a study, you need to notify the MHRA, so they can evaluate the risks of the study before you conduct it. Further detail can be found here.

MHRA notification can be done at the same time you request ethics approval via the IRAS portal. Following approval from the MHRA, you will need to pay a fee and submit quarterly reports as your investigation progresses. There is a similar process in the US (with the FDA) for higher risk devices. 

After conducting your investigation, you need to document and discuss the results in a clinical investigation report. This is a detailed, accountable and traceable audit of all devices used in the study. As well as discussing the clinical findings, you should also discuss any adverse events and risks that arose. Remember, regulators care about safety and risk as much as performance. You should also indicate any findings you may need to further investigate, either pre- or post-market. 

Once again, ISO 14155 has helpful guidance and headings for content you will need to cover in your clinical investigation report. 

Post market surveillance plans need to clearly define:

• The activities you are going to undertake to actively and passively monitor for user feedback, complaints, safety and adverse events. 

• Document:  

  • 1. Who is responsible for conducting each activity 
  • 2. How frequently the activity will be undertaken 
  • 3. Who will be responsible for addressing any shortcomings that arise 

Here, you need to document: 

• What type of PMCF study you plan to conduct (e.g. a continuous study recruiting a few participants every few months vs a discrete study) and justifying why that design is suitable.

• How many participants you aim to recruit, with justification.

• How many sites you will recruit from.

• What the endpoints are going to be.

PMS and PMCF can be combined into a single document – but all aspects must be covered! 

The clinical evaluation report is the pivotal document that regulators review when evaluating the clinical evidence for a medical device. This document can end up being quite long (sometimes over 100 pages), depending on how complicated the device is.  

The CER starts with a summary of the intended use, benefits and risks of the device. Then, it goes on to summarise and compile the clinical evidence from all steps of the clinical evaluation process: scientific, analytical, clinical validity (and in future updates, also post-market data).  

At the end of the document, manufacturers are expected to show how they meet the general safety and performance requirements laid out by the regulations, and how frequently they are going to update their CER and literature review.