A new online resource, Living a long life?, presents information about why people who are mentally ill don’t generally live as long as the rest of us. Early death associated with serious mental illness deserves to be more widely understood, so I was pleased to see this initiative. But I soon discovered some very considerable problems.
The website is a project of the RSA (the Royal Society for the encouragement of Arts, Manufactures and Commerce), which since 1754 has been pursuing the goal of “21st century enlightenment“. Well, maybe the slogan is actually more recent.
The project is part of the RSA’s Open Public Services Network (OPSN) programme, which aims to provide “independent assessment of government and public services performance data.” It presents some data in pretty charts and maps, together with background information, so that you can learn about various aspects of mental health and life expectancy.
Living a long life?
How mental health impacts life expectancy
People with a mental health condition are more likely to die before the age of 75 from a range of treatable illnesses.
The title page is quite scary. Suppose you’ve been, say, depressed at some time in your life. Are you really more likely to die before the age of 75? Suppose you’re 74 when you become depressed. It doesn’t give you long. No wonder you’re depressed. The reality is more complicated. Whoever wrote the title page didn’t understand what’s really happening.
Throughout the analysis, the definition of “mental health condition” wibbles and wobbles all over the place. In some parts of the analysis, even mild depression or “bad nerves” counts. In some parts of the analysis you have to be psychotic. Other parts are in between. Sometimes, I suspect, the term “serious mental illness” includes mild psychosis but excludes serious depression. This all makes the results thoroughly untrustworthy.
The age of 75 is the arbitrary cutoff for recording premature death in the NHS Outcomes Framework. There’s nothing special about 75. It’s just that if you die at the age of 75 or over no one takes any notice, as far as any association with mental health goes. So if you are otherwise destined to live to the age of 98, but because of serious mental illness you only make it to 78, the national statistics ignore you.
The title page is only a taste of worse to come.
Physical health issues
The next page looks more promising (even though it isn’t a proper page — this is one of those annoying websites where you have to scroll down to see the next page). The title is:
“Find out how people with mental health problems also suffer physical health issues”
Well, “Find out…” looks promising. The phrases “mental health issues” and “physical health problems” are a bit woolly, which is a bad sign. The previous page used the phrase “a mental health condition”, so the wibbly-wobbly approach to mental illness is already beginning to emerge.
Below the title there are four numbered boxes. It said “Find out…” and it looks as if you can click on each box to find something out. Except…when you click on them nothing happens.
The first one would have been really interesting:
Which illnesses do account for the higher death rates revealed by the NHS Outcomes Framework? It would be good to know.
Death rates compared
The next pseudo-page down reveals the answer…or maybe not. It has a bar chart comparing years of life lost before the age of 75, for men and women separately, and for various categories:
“Most of this is due to physical illness and unintentional harm.”
Unfortunately, the largest category for both men and women is “other illness and unintentional harm”. How can anyone think it’s meaningful to categorize illness in such a way that the largest category is “other” and it includes something that’s not an illness at all? There’s no explanation. The caption credits an unpublished paper.
Anyway, the four categories of illness are: cancer, respiratory, circulatory and suicide. Let’s ignore the niggle that that suicide isn’t really an illness and take it that a higher rate of suicide is intrinsic to mental illness. Whoever analysed the figures should have taken the suicide rate for people with serious mental illness, and subtracted the suicide rate of people who aren’t mentally ill when they choose to end their lives, to get the intrinsic suicide rate associated with mental illness. I have no reason to be confident that this is how the figures were analysed, though.
But anyway, it’s good to have four actual categories of illness to work with. Onwards and…er…downwards to the next pseudo-page.
Deaths from treatable illnesses
On this pseudo-page we again have four boxes, one for each category of illness. And this time they are clickable! Hooray!
That was the good news.
The bad news is that the four categories aren’t the ones in the bar charts just above. Diabetes has suddenly appeared. Circulatory has split into heart disease and stroke. Respiratory has survived unscathed. Cancer and suicide have vanished. If a conversation with your GP went like this you’d think she was showing signs of early dementia.
The boxes tell you about the relationship between the illness and mental illness. For example:
“People with serious mental health conditions are two to three times more likely to die of heart disease.”
More likely than…who? what? It doesn’t say.
I know someone who is quite likely to commit suicide because of her serious mental illness. Is she really “two to three times more likely to die of heart disease” than of suicide? The claim either means nothing or it means something that it’s not saying. It’s written just to look scary, not to mean anything.
Pop-ups in the four boxes provide references, but there isn’t any way to click on a reference in order to read it. Two of the references are to an article about the United States, only looking at schizophrenia (just one of the forms of psychotic illness), using a cutoff age of 64. The article isn’t public, as far as I can see, despite the RSA’s claim elsewhere that it wanted to present only public data. Although it’s given as a reference for the diabetes box, the article’s abstract doesn’t mention diabetes at all. The article’s recommendations include reducing “tobacco and substance abuse”, which the RSA pretty much ignores.
The other two references are to a lobbying exercise by the Royal College of Psychiatrists in 2010, No health without public mental health: the case for action. The college’s position statement mentions heart disease in three places and stroke in two places, but nowhere does it support the information in the RSA’s boxes, as far as I can see.
Clicking on any of the boxes takes you to a pop-up overlay where the pseudo-pages scroll sideways, (just in case you were getting used to scrolling up and down). On these pseudo-pages you can read standard blurb about diabetes, heart disease, stroke and respiratory disease.
Mental illness isn’t mentioned anywhere in the blurb, as far as I could see. For example, on the heart disease pseudo-page it mentions smoking, which is good to see. But it doesn’t mention the strong association between smoking and mental illness, which is blatantly incompetent for a website like this. Also, we got here from a page that said all these illnesses are treatable, and it doesn’t mention anything about treatment.
There’s a link to another webpage where you can download some of the data. I’ll come back to that.
What is happening in your area?
The page ends with a box where you can type your postcode to see some data about your area. This reinforces the popular meme that healthcare is a “postcode lottery”. There’s no evidence that it is a lottery, though. Populations, circumstances and local priorities really are different in different areas, but the website doesn’t explain any of this.
I typed a Glasgow postcode and it just came up blank. The data behind the site is only for England, but whoever designed the site doesn’t seem to have been told that. English postcodes seem to be mapped to clinical commissioning groups (CCGs), but the page doesn’t tell you that either.
The page title mentions “death rates” but there are no death rates shown. You see a bar chart with the caption, “Gap in life expectancy in years” but the chart doesn’t show life expectancies and it’s not calibrated in years. It shows your chances of dying before the age of 75, as a ratio relative to the average for England, supposedly.
For example, according to the chart, people who have serious mental illness are 2.4 times more likely to die before the age of 75, on average for the whole of England, so the middle bar on the chart always shows 2.4× regardless of your CCG.
For a particular CCG, the third bar shows the ratio for “Your area for people with serious mental health conditions”. If this is lower than the middle bar, you live in a good area, in some sense.
The best area is the Isle of Wight (postcode PO30) with a ratio of just 0.3. What does that mean? I have no idea. It seems to mean that as someone with a mental illness you are far less likely to die than anyone else, if you live on the Isle of Wight. Or does it really mean 1.3, a 30% extra chance of dying? Does the middle bar’s 2.4× really mean 3.4×?
The worst place is possibly Newcastle (postcode NE1) with a gap ratio of maybe around 4, but the bar chart just says “no data”. This is perhaps because the data is old and the CCG has since been abolished. Whoever designed the website didn’t map postcodes to the CCGs that existed at the time the data was collected.
The worst place you can actually see a bar chart for is maybe Wigan (postcode WN1) with a gap ratio of 3.7. Or maybe it’s Bath (postcode BA1) with a gap ratio of 3.1. Strangely, for Bath the ratio is shown as 3.1 in figures but the height of the bar is more like 3.8 against the scale of the chart.
It’s impossible to discover these best and worst places from the website directly. I worked them out approximately by downloading the data, but the data doesn’t contain the actual figures or any obvious way to calculate them, so I might be wrong.
The first bar shows the ratio for “Your area all deaths”, but weirdly it is sometimes negative. This is bonkers. You can’t have a negative chance of dying (except possibly in a zombie apocalypse, but I don’t plan to cover that scenario in any detail). I suspect that where the bar incorrectly shows, for example, -0.1 it should really show 0.9, but I’m not sure. These charts look plausible, but on closer inspection they seem to be intended as some kind of performance art showcasing the pleasures of innumeracy.
What’s your chance of dying, really?
How do your chances of dying before the age of 75 really change if you’re mentally ill? The text beside each CCG bar chart says:
“In England, people living with a serious mental health condition are more than twice as likely to die before the age of 75.”
The actual figure on the bar chart, the middle bar, says 2.4 times as likely to die, and it seems possible it really means 3.4 times.
There’s no data for England as a whole in the data download, so I got some relevant data from the Health and Social Care Information Centre (HSCIC) indicator portal, and I looked at the figures for 2013/14. The figures I give here are approximations to keep things simple. I’m only after very rough estimates.
In passing, I noticed that the HSCIC data also has four categories of illness, but they’re different again: cancer, circulatory, liver disease, respiratory. Liver disease is the one with the greatest effect on death rates in people with mental illness according to this data. This reflects, of course, the importance of alcoholism, which the RSA plays down.
According to HSCIC, that year there were 38 million people in England aged 18 to 74 and 146,000 of them died. So if you were one of those people your chances of dying in that year were: 146,000 / 38 million = 0.0038
Now, suppose you were 18 in that year, and, to keep the calculation simple, suppose your chances of dying remain the same every year (which isn’t actually true, but it’s probably good enough as a very rough estimate). You are going to face that same chance of dying every year until you’re 75, which is 57 years. Once you reach 75 we don’t care what happens to you!
Your chance of dying before the age of 75 is therefore: 57 × 0.0038 = 0.22
But suppose that at the age of 18 you have a serious mental illness, so serious that you are in secondary mental health care. HSCIC data shows that in that same year, 2013/14, there were 2 million people in secondary mental health care and 24,000 of them died that year. So your chances of dying that year were: 24,000 / 2 million = 0.012
And suppose you remain seriously ill until your late 20’s, say for ten years. After that you’re a bit more mature and your medication has stabilised your condition, so you’re transferred back to primary care, the care of your GP. Over the ten years your total risk of dying will have been: 10 × 0.012 = 0.12
And then you’ll have 47 more years until we stop caring about you. During that time you’re not in secondary mental heath care, so your total risk of dying will be: 47 × 0.0038 = 0.18
That means over the whole period from age 18 to 74 your total chance of dying will have been: 0.12 + 0.18 = 0.30
Well, if you hadn’t been mentally ill your total chance of dying over that period would have been 0.22, so mental illness pushed it up to 0.30, which is a factor of: 0.30 / 0.22 = 1.4
To put it another way, a 40% increase. That’s quite a bit different from the bar chart, which shows a whopping 240% increase, or maybe 340%. The bar chart’s estimate is six times my estimate, or maybe eight-and-a-half times. I don’t believe the bar chart.
What can I do about improving my health?
Below the chart, a section on improving your health gives some general-purpose advice unrelated to postcode. It’s unrelated to mental illness, too.
“If you are already living with a long-term condition such as heart disease or diabetes, you will need regular visits to your GP and other health professionals for check-ups and advice.”
Wasn’t all this supposed to be about people with a mental illness? Now it’s about people with incurable physical illnesses.
That reminds me that when I was discovered to have heart disease I was screened for mental illness. This kind of screening means that mental illness is more likely to be detected in people who already have heart disease. It may account for some of the statistical correlation between mental illness and heart disease, and there may be similar effects with other long-term physical conditions.
Three information boxes at the bottom advise you to seek help, get health check-ups and send feedback to Healthwatch. There are links to Mind, NHS Choices and Healthwatch England.
If you download the data you get some data in a spreadsheet, some definitions and other information in a separate spreadsheet, and a short paper on methodology. The data spreadsheet contains links to other files that aren’t in the download.
As a source of data for the main website, the spreadsheet is essentially useless. The numbers that appear in the postcode-related bar charts aren’t in the spreadsheet, and there doesn’t seem to be any obvious way to calculate them.
The national average data (the source of the 2.4 in the middle of all the bar charts) is missing and there’s not enough data on the CCGs to calculate it even if you knew the formula. (You would have to know the size of the population each CCG serves.) In the data from the HSCIC I didn’t see anything like the same numbers, as I explained above.
The paper on methodology is baffling. It describes four indicators that were developed by the project:
- How well is my GP looking after my physical health needs?
- What is the likelihood of getting access to the right psychological therapies, and what is the outcome if I do?
- Am I more or less likely than average to be prescribed anti-depressants?
- How well am I supported to live well with my condition?
The paper claims the methodology relies to some extent on advice from the Organisation for Economic Co-operation and Development (OECD), but as far as I can see the warnings that the OECD includes about misuse of its methodology were disregarded. The paper seems to conclude, essentially, that the results obtained were arbitrary, depending more on the methods of analysis than on the data.
I didn’t find these indicators in the least bit credible. The download doesn’t contain the files needed to reconstruct them from raw data.
The website displays the indicators on colour-coded maps, but pretty colours don’t make the results meaningful. It’s not that the indicators don’t set out to measure interesting things. The problem is that as measures they appear to be bogus. I didn’t find any evidence to validate the indicators as useful measures of the things they claim to measure.
Also, the indicators don’t relate to serious mental illness or length of life in any obvious way. They look like a separate, unrelated and failed project.
Even if the indicators had some validity, complex indicators like these are useless both for promoting improvement in services and for promoting public health agendas, precisely because of their complexity. The numbers that emerge are opaque. They’re about as convincing as if they had been conjured up behind the locked doors of secret chapels by sorcerers chanting spells in some arcane language. (In fact the arcane language used in this project is known, cryptically, as R.)
A separate blog post about the project links it unconvincingly with stigma and discrimination, and blames GPs. None of this is actually in the main site, which doesn’t, for example, tell you how to complain about or change your GP.
Some flimsy shock horror statistics are quoted out of context. For example, you can find “3.25%”, and it’s meant to make you think it’s a disgrace. But what’s the national target? What would the percentage be in an ideal world? You’re not meant to ask those kinds of questions, presumably.
One of the indicator maps is shown, with the title: “How well are the physical needs of people with mental health problems looked after?” But, according to the downloaded metadata, the indicator is based only on data for people with psychosis, not mental health problems as a whole, and the data only measures screening tests performed on people who are physically well, not looking after the physical needs of people who are ill. The title is deliberately misleading. On the map, Newcastle, Wigan, Bath and the Isle of Wight are all shown as average.
The message of the website boils down to this: if you are seriously ill then you might die as a result. Who knew?
Drilling down into more detail: if you have a serious illness, then you might die from the illness itself, or you might die from complications. In the case of serious mental illness you might die from suicide as a result of the illness itself, or from complications like heart disease, etc.
But then the website goes off into fantasy. Perhaps the reason these very ill people die is because the doctors treating them haven’t heard of complications. So the doctors don’t realise their patients might die, or don’t care. They deliberately ignore the complications that might kill them.
The fantasy solution to this is that these seriously ill people need to pull their socks up and take responsibility for the complications of their illness. It’s classic blame-the-victim.
In this fantasy world healthcare has no cost. Obviously no analysis of any cost data is needed. The most important factor to consider, apart from seriously ill people failing to pull their socks up, is the postcode lottery.
To disguise the fantasy there are some pretty bar charts and maps to make it all look sciency. But the data analysis is a hoax along the lines of the children’s game that ends, “Take away the number you first thought of.” The answer was known from the start. We don’t have enlightenment here: all we have is 21st century superstition.