Investigating the impact of case-mix on general practice cancer diagnostic outcome indicators

Problem

The Cancer Services profiles report indicators of cancer diagnostic activity for all English general practices. A recent study reported that several indicators were dominated by chance, with some practice-level variation explained by the practice’s age-sex profile. It is often argued that where variation in indicators are driven by differences in the population served by different providers, that adjusted performance on such indicators facilitates fairer comparisons. Here we assess two potential methods for adjustment by quantifying the variation explained by patient-level case-mix and establishing whether the practice-level data on the age-sex profile of registered patients adequately adjusts for this.

Approach

We considered five indicators from Cancer Waiting Times (2016/17, 6050 practices) or Routes to Diagnosis (2015, 6355 practices) data: Two Week Wait (TWW) conversion rate (the percentage of TWW referrals resulting in a cancer diagnosis) and TWW detection rates (the percentage of incident cancer cases diagnosed via a TWW referral) and the percentage of cancer cases diagnosed as either an emergency, following GP referral, or by another route. Mixed-effect logistic regression was used adjusting for patient-level case-mix, using cancer registration data on age, sex, deprivation, referral/cancer-type and, where possible, ethnicity and stage at diagnosis. Further models also adjusted for the practice-level age-sex profile.

Findings

Chance explained 60-85% of the observed between practice variation, whilst the combination of chance and patient-level case-mix explained between 75% (TWW conversion rate) and 89% (emergency diagnosis proportion) of the observed variation. For TWW conversion rate, there was considerable overlap in the variance explained by practice- and patient-level factors. For the other indicators, practice- or patient-level factors were largely independent.

Consequences

Chance is not synonymous with case-mix and is the dominant source of variation in practice indicators. Therefore, we recommend the continued aggregation of data over multiple years. For most studied indicators, adjustment for the age-sex profile of the whole practice population is not a substitute for case-mix of individual cancer patients and so should not be used. Rather, we suggest it is likely that the ages of patients served by a practice may affect a GPs propensity to refer or investigate, even when faced with the same type of patient. Given patient-level case-mix adjustment leads to only a modest reordering of practices routinely adjusting these indicators may not be a priority.