A randomised controlled trial (RCT) seeks to measure impact by identifying a causal link between an intervention and subsequent outcomes.
An RCT is a trial carried out on two (or more) groups where participants are randomly assigned to either an ‘intervention’ (i.e. the intervention group) or ‘controlled condition’ (i.e. the control group, which doesn't receive the intervention). Each group is tested at the start and end of a trial and the results from the groups are compared to see if the intervention has made a difference. In other words, has the intervention achieved its desired outcome? If the randomised groups are large enough, we can be confident that differences observed are due to the intervention and not some other factor.
A RCT may be run alongside a process evaluation of a programme or intervention to see how the intervention has been implemented. This often takes the form of qualitative interviews and seeks to find out if the intervention has been carried out as intended. This checks the fidelity of the intervention and is crucial before assumptions are made about what the results mean. For example, if one teacher is following the intervention in every lesson but another teacher only does the intervention in one lesson a month, this may affect the impact of the intervention. It's important to know whether the observed outcomes are related to the intervention itself, or to the way it has been implemented.
Due to the unique blend of expertise at NFER, we offer a service that covers all aspects of RCT delivery in an education setting, including both impact and process evaluations. The impact evaluation assesses whether the initiative has made an impact on the intervention group in comparison to the control group. The process evaluation is run alongside the impact evaluation and explores how interventions and new approaches in teaching are implemented, to inform the sustainability, scalability and roll out of a trialled scheme.
Our expertise in running trials
NFER is also experienced in trial management, overseeing a trial’s protocol and testing arrangements, understanding the strict methodological requirements of randomised controlled trials and liaising efficiently with delivery partners. We have a dedicated trial logistics team skilled in participant recruitment, maintaining participation and data security, as well as other crucial elements of randomised controlled trials. In particular, our Telephone Unit and field force of 402 Test Administrators in England alone, are instrumental in running successful RCTs at NFER. Our Centre for Statistics provides experienced statisticians to support the design and analysis of the trials we run.
A further strength of how we execute and report trials is through adhering to the principles of open science. At the beginning of a trial we publish a protocol and statistical analysis plan. These ensure trial design and analysis are pre-specified and not vulnerable to data dredging or multiple significance testing. Furthermore, we publish the results of every trial to avoid publication bias and, where a repository exists, we upload data and analysis code at the end of the trial to allow results to be reproduced. All of these strategies help to counter the replication crisis.