Commissioned vs. researcher-led research: a retrospective cohort evaluation of NIHR funded randomised controlled trials

Problem

The NIHR was established in 2006 to promote and support high quality health related research focused on the needs of patients and the public. There are several “pathways” to research funding, with the two over-arching schemes referred to as “researcher-led” and “commissioned” streams. The latter is used to stimulate research groups to address topics considered NHS priorities. Within the NIHR Health Technology Assessment (HTA) programme projects are commonly selected for commissioned research by inviting experts and the public to submit topics, which are then prioritised and summarised into defined “commissioning briefs”. Research groups are expected to adhere closely to the brief. There is uncertainty regarding which stream provides the highest quality, and best-value for money research. We aimed to assess the impact of commissioned, compared with research-led projects, using research metrics.

Approach

The NIHR Funded Portfolio database was searched for randomised controlled trials (RCTs) funded by any programme offering both commissioned and researcher-led funding opportunities, ending in 2018, giving at least 5 years to publish results, with the following data extracted: total and per participant funding, study progressed beyond the initial pilot, final monograph published, and journal impact factor where main results published. Summary statistics were used to compare commissioned and researcher-led RCTs.

Findings

108 eligible RCTs were identified, of which 44 were commissioned and 64 researcher-led. Of these, 90.9% and 93.8% of commissioned and researcher-led projects respectively progressed beyond the pilot phase (X2 (N=108) =0.31, p=0.58). Commissioned study funding was lower than researcher led (mean £671,427 vs. £883,295; p=0.08). The final monograph was published in 90.9% of commissioned projects, versus 98.4% of researcher-led projects (X2 (N=108) =3.33, p=0.05). No difference was found between commissioned and researcher-led projects with main study results published in a non-NIHR journal (X2 (N=108) =0.51, p=0.48), but we did find the main study results from commissioned research were published in lower impact factor journals than researcher-led projects with mean impact factors of 38.56 (SD=16.92) and 111.20 (SD 15.57), p-value < 0.001.

Consequences

Final monographs are more likely to be published, and results published in higher impact factor journals, if the original research was researcher-led vs. commissioned. Further research is needed to establish if this is because researcher-led projects are of higher quality, or commissioned research addresses harder-to-do questions.