Perceptions and experiences of general practice trainers and trainees of workplace-based assessment in the United Kingdom: cross sectional survey

Problem

Workplace based assessment (WPBA) forms a major component, with the Applied Knowledge Test and Recorded Consultation Assessment, of UK general practitioner (GP) licensing. WPBA comprises formative elements including Case based Discussion, Consultation Observation Tool, Mini-Consultation Exercise, Multisource Feedback, Patient satisfaction Questionnaire, Clinical Examination and Procedural Skills, Clinical Supervisors Report, and Educational Supervisor Review together with other requirements which combine to form a summative Annual Review of Competence Progression during each year of training. While the WPBA has been long-established, there has been little previous study on its validity in GP specialty training.

Approach

We used a national cross-sectional survey, including Likert scaled and free text responses, to a convenience sample of GP trainees and trainers to investigate their experiences and perceptions of WPBA. We aimed to investigate GP trainees and educators’ perceptions and experiences of WPBA in relation to validity and fairness of individual assessments and WPBA as a whole. In doing so, we aimed to understand fairness of WPBA for different groups of trainees, particularly those in protected characteristic groups. Quantitative analysis included descriptive statistics, scale development measuring positivity towards WPBA and regression analyses of responder characteristics associated with positivity towards WPBA. We also undertook qualitative analysis of free text responses supported by NVivo and integrated quantitative and qualitative responses.

Findings

There were 2,088 responses from 1,176 trainees and 912 trainers. The two groups were similar in sex, but responders who were trainers were significantly more likely to be White British and Irish and less likely to be Asian and Black, compared with GP trainees. They were also significantly more likely to be to be UK trained rather than trained outside the EEA, to be from an urban practice, a devolved UK nation and less than full time. Both trainers and trainees were generally positive towards WPBA, with trainers more positive or similar to trainees towards the individual assessments. In a multivariable regression model accounting for sex, ethnicity and country of primary medical qualification, trainees were significantly less positive than trainers (B -0.48, 95% confidence interval [CI] -0.54, -0.42, p<0.001) while international medical graduates (IMGs) were significantly more positive than UK graduates (B 0.29, 95%CI 0.20, 0.37, p><0.001) towards WPBA. Qualitative analysis revealed varying concerns about validity and relevance, assessment burden, potential for bias from assessors and patients, fairness to those with protected characteristics, gaps in assessment, and perceptions of individual assessments.

Consequences

Perspectives of trainers and trainees towards WPBA were generally positive overall, although trainees were less positive and expressed greater concerns than trainers. Trainers’ greater positivity compared with trainees towards most elements of WPBA accords with their greater experience as assessors. Despite concerns among stakeholders about bias, IMGs were significantly more positive towards WPBA.