In his blog Ben discusses the claims of "evidence" made by Andrew Lansley to support NHS reforms he is proposing as Secretary of State for Health in the UK. In contrast to the impression given by Andrew Lansley it seems there is no such evidence. In fact the few relevant studies that exist provide some (very weak) evidence that Lansley's reforms will have a negative impact on the health service.
Andrew Lansley is either deliberately lying or grossly incompetent at his job, which let me remind you is to review the evidence and make the best policy decisions in accordance with it and the general wishes of the public. Liar or incompetent, either way he should be sacked. These decisions can drastically affect many real lives, it may seem boring to think about NHS structures for delivering care, but get it wrong and valuable resources which could be used to save lives may be squandered.
Indeed for democracy to lead to good decisions it is essential that the public are informed properly about the evidence, rather than misled by false claims. Given a lack of expert knowledge and indeed actual interest, with little time to think about the issues in their busy lives, from the majority of the public I am often amazed at how well democracy performs relative to other systems. Yet the richest countries as measured by GDP per capita are mostly democracies.
While this may mix up causation with correlation I do believe democracies in the rich western countries have brought both political stability and prosperity to their populace. I mostly attribute this success to an independent press holding the leaders to account and reducing corruption, and technical advisers providing experience, knowledge and an intelligent review of the evidence. This, however, is not to say democracies are well run in absolute terms.
As Andrew Lansley illustrates politicians statements about evidence, and statistics are frequently poorly matched to reality and malformed policies often result. This is not an isolated case, for example in the run up to the recent UK general election another Conservative Chris Grayling, currently Minister of State in the Department for Work and Pensions, repeatedly made false claims by misrepresenting statistics about crime. And it is not only the tories, under Labour that the classification of cannabis was changed back to B, justified with misleading statements about the evidence of harm and against the repeated scientific advice the government received. Later an adviser to the government was sacked for discussing evidence the government didn't like. The Labour controlled county council of Durham even funded fundamentally flawed "tests" to see if fish oil pills could improve childrens' performance in GCSE exams.
These are not examples of politicians holding different opinions to me, they are examples of clear objective evidence being ignored, misunderstood or abused. The politicians involved should not be allowed to remain as MPs let alone hold onto important jobs in the government, as they are either unable to perform their job competently or are simply deceiving the public. Their contribution distorts the information the public (and the politicians) need to make informed decisions about how to achieve the outcomes we want. The desired outcomes and values are subjective and may differ from person to person, but the evidence and statistics have a clear interpretation and should not be abused. All of the politicians involved in the stories I've mentioned should be hounded by the media until they are sacked.
Sadly, disgracefully in fact, the media never holds them to account for this. While if any muddled or slightly dodgy expenses claim for a few thousand pounds comes to light the media, right and left, hound the politician relentlessly, on evidence about serious policies, politicians are free to continue to mislead the public, distorting the public debate, with potentially devastating results if flawed health policy is implemented nationwide. Perhaps the problem is that the media also likes to play this game of distorting evidence and statistics to sell newspapers or push ideology. Or possibly, simply through a lack of basic understanding of science and statistics, they actually believe statistics and scientific evidence really can mean anything and all opinions are equally valid.
Ben Goldacre is one of the few journalists who goes against this trend and instead exposes the dodgy claims of newspapers and politicians alike. However even if they attain a very high profile it is dangerous when only a small number of people can be relied upon to give a good overview of the evidence, because noone is perfect and the effect of any occasional error they make will be exaggerated. When I read Ben's blog on Andrew Lansley I initially really liked it, and broadly I remain in agreement with it, his exposure of Andrew Lansley's lack of evidence is excellent. Unfortunately, when I read the paragraph,
Working from first principles, markets where people compete on price as well as quality will probably make quality worse, because prices are easy to measure, while quality is not.The evidence seems to support this theory. The introduction of variable price competition in New Jersey in the 1990s was associated with a worsening death rate from heart attacks, while in the UK, stopping variable price competition was associated with improvement. It’s hard to measure either way, but despite his using the word repeatedly, again, the “evidence” does not support Lansley here.
things got a little murkier. There is a problem with the first sentence. "From first principles" implies to me that someone has really calculated what will happen based on some basic (presumably economic) postulates. I was surprised because it seems very tricky to judge what might happen. Ben's hand waved logic might apply, but I could imagine the lack of evidence on quality driving the hospitals to advertise procedures and safeguards, or hire "big name" consultants to assure people that they will provide high quality care. Perhaps the things they do improve quality perhaps they don't, but a clear answer from theory seems tricky. I am not an economist though, so I don't know.
However Ben links to an article on the words " Working from first principles" so I started to read it, only to discover it's a paper discussing the evidence, which is very relevant, just not at that place. Now normally actual evidence >> theory, but the problem here is Ben acknowledges the evidence is not that conclusive because "It’s hard to measure either way" in which case our prior prejudice based on theory becomes more important. It doesn't change the central theme of the article, that Lansley referred to non-existing evidence, but to me it does exaggerate the case against introducing a market on price and quality. This makes me uncomfortable praising it in my blog without adding this. Indeed, while a much more minor error, this statement also has the power to distort the public debate if most people interpret it the way I do.
Maybe I'm missing something though, but it seems to me that even the best examples of journalism fall short of getting it right every time. However that is a rare slip, while the real problem is that misleading statements about evidence from politicians and the media are the norm, not the exception, and there is no accountability.
Until this changes and the media take a positive role in informing the public about evidence, and thereby pressure politicians to do the same, we will be unable to transform to a society where policies are formed based on the best available evidence and people make informed decisions about how we are governed, and how it should affect our daily lives.

hi
ReplyDeletei linked that paper for the "first principles" part of my post because although it's reporting primary research, i think its introduction section gives the best summary of the first principles argument, out of all the papers i could find, counting only the ones that were publicly available without an academic subscription.
Hmm.... thanks Ben I thought I read the through introduction of that already, but will look again. Apologies if I missed something.
ReplyDeleteSome good points - we do appear to be living in a democracy where the overwhelming majority of people (voters, anyway) either don't realize just how statistically & scientifically illiterate their representatives are, or they do realize and simply don't care.
ReplyDeleteUnfortunately, the sector that really needs to be on the ball to correct this is the media, and they are not just a little bit worse, but much *much* worse. Ben Goldacre is one of very few beacons of light in a morass of scientific illiteracy; the overwhelming majority of the news media seems incapable of discerning appropriate use of statistics & evidence, preferring instead to take whatever position coincides with their prejudices, and pretend that the evidence supports this. Much like politicians.
Sigh.
OK I found some market theory discussion, buried in section 3, page 8, so I think I have a better idea where Ben is coming from now.
ReplyDeleteIf I'm understanding the jargon correctly they discuss a paper that shows quantitatively how pressure on quality varies depending on how sensitive the market is to quality and price and qualitatively, if it is far more sensitive to price, quality will be driven down as they reduce price, which is clear.
But my issue was that difficulty in measuring quality doesn't necessarily mean low sensitivity to it, as inventive indicators may be used to convey that this is a priority. I just can't imagine that you can much confidence either way in advance.
@anonymous: just saw ur post, thanks. Sighing is indeed one response, getting angry and jumping around to some machine head or gojira is another. The third would be to get angry and actually do soimething about the problem, but I never really get past the second option, too much fun.
ReplyDelete