While many doctors and health professionals like to believe they practice ‘evidence-based’ medicine, the reality is that much of healthcare practice is simply not supported by good science. This isn’t necessarily a problem, of course, as there are other means that practitioners might use to base their practice decisions. One of these, as I pointed out recently, is clinical experience. Unfortunately, some proponents of ‘evidence-based medicine’ seem to have forgotten this component. Though, I have to say my experience has been that such individuals are usually academically-minded, and don’t actually see patients. This might help to explain why some of these individuals seem not to value clinical experience (they don’t have much or perhaps any at all).
However, there are other reasons why it does not necessarily make sense to base clinical decisions on science: the science can be unreliable. It appears that the conclusions study authors come to can be influence by the source of funding. For example, I previously reported on evidence which found that for blood pressure lowering medications, drug company funded studies were 4-5 times more likely to report positive findings compared to those that had been independently funded.
What this study found was that results from industry-funded and non-industry funded studies was essentially the same. It was the way in which the results were interpreted and reported that was different. Such so-called bias can give a very distorted picture regarding the effectiveness of a treatment, particularly when busy doctors feel they only have time to read the conclusion part of the summary of an article (known as the abstract).
The authors of a study published in this week’s British Medical Journal make this point as they set about determining whether bias is a factor in the publication of studies relating to flu vaccination. They chose flu vaccination as their subject partly because this practice is advocated on a near-global basis. Yet, as they point out, the evidence supporting this intervention is generally poor in quality and contradictory in nature. Not so long ago I reported on a study which calls into question the veracity of the oft-quoted ‘fact’ that flu vaccination reduces mortality in the elderly.
Having reviewed the literature in the area, the authors of the BMJ review concluded that (as with studies on blood pressure lowering medication) there is often a discrepancy between the results of a study and the conclusions drawn by the authors. However, here, there did not seem to be any relationship between the source of the funding and the discrepancy. In other words, industry-funded studies were no more likely to report inappropriately positive conclusions than other studies. However, as the authors of the review point out, almost a quarter of the studies they assessed had no declared source of funding, and without the full facts it’s not possible to know for sure if source of funding led to bias in terms of how the results were interpreted and reported.
What the results of this review also showed was that most (70 per cent) of the studies were of poor quality and drew generally positive conclusions simply not supported by the data. Industry-funded studies, though no better in quality, were more likely to be published in prestigious medical journals and therefore had greater visibility.
The authors of the review state that they cannot say for certain why industry-funded studies are seemingly more attractive to the more prestigious journals. However, it seems they are suspicious that money might have something to do with it when the recommend that As a measure of transparency for readers and authors we recommend that once a year editors and publishers should post all sources of income relating to the running of the journal.
The authors end their review by pointing again to the discordance between study results and conclusions. In most cases, what you see is not necessarily what you get, the authors lament. There is reason to believe that flu vaccination policy may be based on misreporting and misrepresentation of the evidence. In other words, it may not be based on the evidence at all.
References:
Jefferson T, et al. Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review. BMJ 2009;338:b354, doi: 10.1136/bmj.b354
