Many individuals who work in healthcare, and perhaps quite a few who don’t, will have noticed a general thrust in the direction of what is known as ‘evidence-based medicine’. I have serious reservations about evidence-based medicine, myself. For a start, although evidence-based practice was originally billed as: ‘The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research,’ [1] it’s amazing to me just how often that first bit is forgotten. Having said that, the individuals who forget the importance of clinical experience do often seem to be academics who don’t see patients. I suppose it’s only natural if you don’t have much clinical experience then you’re not going to give that much credence to it.
And even if we focus on the science and the studies, evidence-based medicine is still fraught with difficulty. And just one factor here is what is known as ‘publication bias’ ” which describes the phenomenon when ‘positive’ studies tend to be more readily published than ‘negative’ ones. Such shenanigans are well known in medical research, and can give a very skewed impression of drug effectiveness and its risk-benefit profile.
This week’s New England Journal of Medicine carried an interesting article which sought to identify publication bias in the area of antidepressant medication [2]. The researchers assessed a total of 74 studies that had been registered with the FDA (Food and Drug Administration) in the USA. Some of these studies had been published, but many (details below). The researchers obtained the unpublished studies via various means including the invoking of the freedom of information act.
Analysing the 74 studies, the researchers found that:
38 had positive results, and all but one of these had been published.
36 had negative results, and 22 of these had not been published at all.
Of the 36 negative studies, 11 had been published, but in a way that conveyed a positive outcome (this is not ‘publication bias’ by the way, just plain ‘bias’).
This meant that of all the published studies, 94 per cent appeared to have positive findings.
However, FDA analysis revealed that 51 per cent of studies were genuinely positive.
Overall, publication bias meant that the drugs appeared about a third more effective than they are in reality (if all trials are taken into consideration).
The lead author of this study, Dr Erick Turner, is reported as saying: The bottom line for people considering an antidepressant, I think, is that they should be more circumspect about taking it.� That sounds like good advice to me. But I’d add that this data also suggests that doctors might be a bit more circumspect about prescribing them in the first place.
This is not the first time that there’s been evidence of publication bias in the area of antidepressants. Previous analysis found the same situation seems to have gone on regarding the use of antidepressants in adolescents [3]. This Lancet review found that while published studies support the use of a variety of various antidepressants in childhood depression, unpublished data shows that, in the main, risks of treatment such as an enhanced tendency towards suicidal behaviour seem to have been significantly underplayed. All this stuff on selective publication of data on antidepressant medication makes pretty depressing reading, I reckon.
References:
1. Sackett DL, et al. Evidence based medicine: what it is and what it isn’t: It’s about integrating individual clinical expertise and the best external evidence. BMJ 1996;312(7023):71-72.
2. Turner EH, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252-60.
3. Whittington CJ, et al. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet. 2004;363(9418):1341-5
