When I wrote recently about 2gether NHS Foundation Trust’s results in the 2013 NHS Staff Survey, I referred briefly to the results for individual parts of the trust, the directorates. This time I’m looking at the individual directorates in a bit more detail.
There are seven directorates in the trust. Five are in Gloucestershire:
Children and Young People’s Service (CYPS)
and there are two others:
The 28 Key Findings (KFs) in the survey results cannot be compared directly because they are so inconsistent. Some are percentages, while others are on a scale of 1 to 5. Even among the percentages, or among the scaled results, some fall in a narrow range and others in a broad range.
To create a consistent scoring for each KF, I simply counted the number of other mental health trusts that had scored worse, separately for each directorate . These points scores benchmark the directorates against the performance of other trusts in a consistent way, to allow meaningful comparisons between KFs to be made.
The directorates are entirely different in each trust, so in general it is not possible to compare directorate with directorate across trusts. (But there’s more to say about this below.)
For example, as I noted in my previous post, CYPS scored worse than the worst trust in England for KF24. Because no other mental health trust scored worse on KF24, I gave CYPS no points for that.
But for KF16, CYPS scored better than the best mental health trust in England. Because all 56 of the other mental health trusts scored worse on KF16, I gave CYPS 56 points for that.
Each of my points scores is thus between zero and 56. The surprising thing is the number of zeros and 56s. Only North had no zeros (but it had a 1). Every directorate in the trust had at least one 56. Overall, there were 28 zeros and 19 scores of 56 points.
Statistically, the number of zeros and 56s is less surprising, or perhaps not surprising at all. A directorate has fewer staff than an entire trust, so its results come from a smaller sample. The small sample size makes extreme results like zeros and 56s more likely. How much more likely, I’m not sure. I don’t know an appropriate statistical test that would tell me whether the number of zeros and 56s is statistically significant.
The best directorate in the trust overall, judging by the median points scores, was Countywide with a median 39 points. This makes it roughly equivalent to a trust in the top third of mental health trusts, much better than the trust’s overall ranking (in the bottom third).
Countywide did better than the best mental health trust in England on four findings: KF3 (work pressure), KF4 (team working), KF11 (stress) and KF27 (equal opportunities).
However, it did worse than the worst mental health trust in England on two findings: KF13 (errors, near misses and incidents) and KF16 (violence from patients, relatives or the public).
The worst directorate in the trust overall, judging by the median points scores, was CYPS with a median of just 1 point. This makes it roughly equivalent to the second-worst mental health trust in England.
CYPS did worse than the worst mental health trust in England on thirteen findings — nearly half the survey results. On seven of these findings other directorates in the trust were as bad or worse. The six findings where CYPS was uniquely the worst were: KF6 (job-relevant training), KF8 (well structured appraisals), KF23 (job satisfaction), KF24 (recommendation as a place to work or receive treatment), KF25 (motivation) and KF27 (equal opportunities).
However, CYPS did better than the best mental health trust in England on four findings: KF16 and KF17 (physical violence), KF20 (pressure to work when unwell) and KF18 (abuse from patients, relatives or the public).
These findings, based on my home-made points scoring system and affected by small sample sizes, are no more than general indicators of what might be going on in the directorates. A major worry is that comparing directorates with whole trusts is not comparing like with like.
Perhaps it’s unfair to imply that CYPS is doing badly. Perhaps working with children and adolescents is just particularly difficult. In that case, similar directorates in other trusts would show the same kind of results.
As a quick sanity check, I looked at a handful of other mental health trusts. This was not a proper random sample, just the first few reports that I grabbed. Some of them don’t provide separate results at all for children and adolescents (often called CAMHS — child and adolescent mental health services). To save time, I only looked at the overall staff engagement scores in CAMHS-like directorates.
The results do not show any consistent tendency for CAMHS-like directorates to be worse than other directorates. In the table, Score is the overall staff engagement score for the CAMHS-like directorate, and Worse is the number of directorates in that trust with worse scores:
|Southern Health (Hampshire)||3.78||6|
In the handful of reports that I grabbed, I didn’t find any other trust where a CAMHS-like directorate was the worst directorate. In Derbyshire, CAMHS was the best of their seven directorates for overall staff engagement.