Mindset XR Module 3: Research design

Welcome to the Mindset Extended Reality (XR) Innovation Support Programme learning resources, which include three series delivered in conjunction with our expert Mindset-XR programme partners:

• Medical regulation

• Clinical evidence

• Lived experience involvement

Mindset-XR is helping to catalyse the growth of immersive digital mental health solutions in the UK, through funding, tailored support and training. It is delivered by Innovate UK and the Health Innovation Network South London (HIN).

This series focuses on research and clinical evidence, with key insights from King’s College London’s Institute of Psychiatry, Psychology and Neuroscience. Across a number of modules, these resources will guide you through your research journey, from establishing what you plan to investigate, to conducting research and disseminating your findings.

Outline

Welcome to Module 14: Research design. In this section, we’re focusing on:

What makes good research design?
Things to consider before delving into research design.
Design options in research
Understanding the different research design methods available.
Common pitfalls
A list of common pitfalls to be aware of when designing your research project.

What makes good research design?

All good research design follows from the research question.

Before considering your design, you must consider if your research question is:

Well defined
Useful
Possible to answer within the context of the research

In Module 13, we covered how to settle on the right question (or questions) to shape your research.

Design options in research
Prospective or retrospective
These types of design methods refer to time.
- Prospective refers to recruiting people and studying them forward in time from a specified starting point (also called longitudinal research).
- Retrospective involves looking at interventions and outcomes that occurred in the past.
Analytic or descriptive
- Analytic design involves stating in advance and then testing a formal hypothesis of what theory suggests the research will show.
- Descriptive seeks to identify and describe patterns in data without a formal hypothesis about the patterns.
Observational studies
In this type of design method, you record what happens in an environment or situation without influencing what happens in that environment or situation.
Quantitative or qualitative methods (or mixed methods)
- Quantitative – collection and analysis of data in which key constructs can be operationalised or defined in numerical terms and subject to formal statistical analysis.
- Qualitative – focus on individual experiences of participants using methods such as interviews or focus groups.
- For developing and evaluating digital therapeutics, a combination of quantitative and qualitative methods (or mixed methods) is often helpful.
Experimental (or quasi experimental)
This method involves manipulation of the environment – typically by including an intervention, which is sometimes referred to as ‘the exposure’.
- A key difference: experimental designs involve random allocation of people to groups, whereas quasi experimental research does not.
Experimental research designs are often positioned as the ‘gold’ standard with respect to clinical evidence. In particular, the randomised controlled trial (RCT).
Randomised controlled trial (RCT)
In a RCT, people are recruited and complete the battery of outcomes at baseline. They are then randomly allocated to either receive the intervention or be in a control or comparison group. The same outcome measures are then collected at pre-defined future time points after the completion of the intervention.
- Randomisation is a powerful tool because it considers possible group differences that can impact on outcomes, these are called unmeasured confounders.
By using an RCT design, you are then generating evidence that allows you to speak about cause and effect. This means you can speak of the efficacy of the intervention you have developed. Answering questions like: does this intervention work? If so, how well does it work?
While RCT is considered gold standard across some contexts, it is important to remember that different research designs are equipped to address different research questions.
- A blend of experimental and qualitative evidence can provide converging evidence for the evaluation of XR interventions.

Common pitfalls

Moving too fast

Not taking sufficient time to plan can be detrimental to your research. Allocating sufficient time at the start to carefully design the research will ensure the efforts you put in will lead to meaningful outcomes.

Being too ambitious

Attempting too many things, or answering too many questions within a single piece of research is another common pitfall. The risk is you fail to address the main aim or research question you had at the start.

It is helpful to consider how the current project you are designing is going to fit into a bigger programme of research.

Losing sight of the human angle

Focusing so much on the research design element sometimes makes it easy to forget human beings are the ones involved in the XR research.

This can often be seen when researchers are developing their battery of outcome measures. There can be a tendency to keep adding ‘just one more’ measure – and end up with a battery of measures that is too long. You are then likely to face issues in terms of higher drop out and poorer data quality.

Careful consideration of the research question and the involvement of experts by experience throughout the study are key ways to mitigate this risk.

Summary

Making sure your research question is well defined and useful lays the groundwork for successful research design.

Different research designs are equipped to address different research questions. A blend of research design methods can be beneficial for the evaluation of XR interventions.

Taking the time to acknowledge and work through common pitfalls in the design process will help you avoid them in your own research journey.