It’s often said that the ‘gold standard’ test of a medical intervention is the ‘randomised controlled trial’. If it was a drug being tested, this normally means randomly assigning half of a group of people to the drug, while the other half gets a placebo. In ‘double-blind’ studies neither the researchers nor the study subjects supposedly know who is taking what until the ‘code is cracked’ at the end of the study. These studies look like a pretty watertight and very dependable way of assessing treatments, but they are not necessarily as reliable as we might imagine.
To begin with, it’s sometimes possible for researchers to ‘work out’ who is likely to be taking what, which can jeopardise the results of a study. For example, let’s say the drug being tested is a statin (cholesterol-lowering drug). Researchers may have access to the study subjects’ cholesterol levels and assume that those whose cholesterol levels fall significantly are taking the statin while those whose levels stay the same are not.
The real issue here is that this may then affect, even unconsciously, decisions the researchers make which ultimately may affect study results. For example, researchers may recommend that patients on the placebo are at greater risk and might be more inclined to recommend that they undergo more aggressive treatments such as angioplasty (expansion of a coronary artery with a balloon) or insertion of a stent (termed ‘revascularisation’ procedures’).
Another problem is that these revascularisation procedures also increase the risk of complications including heart attack. The end result of all this is that those on placebo may end up ending worse outcomes and the drug now looks better than it is in reality.
Another example of bias concerns study length. Usually, how long a study will be continued for is decided in advance. Sometimes though, the ‘code is cracked’ before the end, and if the results favour the active treatment it can be deemed unethical to continue and allow not to have access to a treatment that might benefit them (the active drug). However, it is well known that early termination of trials tend to exaggerate the benefits of a treatment, and downplay the potential harms.
Overall, such biases favour the drug being tested, so sometimes researchers attempt to take this into account when assessing studies.
I recently came across a relevant review of statin therapy published in Therapeutics Letter produced by researchers at the University of British Columbia [1]. The researchers were specifically looking at the effect of statins in ‘primary prevention’ (in individuals without a prior history of cardiovascular issues such as heart attack or stroke). Previous reviews had found mixed results, it seems at least in part because some of these reviews included studies which themselves included individuals who did have a previous history of cardiovascular problems. The benefits of statins are generally greater in those with pre-existing disease and their inclusion in primary prevention studies biases the results in favour of statins.
The authors of the Therapeutics Letter review took out such studies, and also removed studies that seemed most subject to other forms of bias. Their final analysis revealed that while statin therapy, overall, led to a 21 per cent reduced risk of serious heart disease-related problems (e.g. fatal or non-fatal heart attack), overall risk of death was not reduced at all. Neither, it turns out, was risk of total ‘serious adverse events’ (this includes not just heart attacks, but ‘side-effects’ such as muscle damage or liver damage).
The authors point out that if the heart benefits of statins do not translate into a reduction in total serious adverse events, then statins must be increasing the risk of other (adverse) events to an extent which completely negates their positive heart effects. This fact is reflected in the last line of their conclusions which reads:
Statins do not have a proven net health benefit in primary prevention populations and thus when used in that setting do not represent good use of scarce health care resources.
References:
1. Do statins have a role in primary prevention? An update. Therapeutics Letter March-April 2010. Therapeutics Initiative University of British Columbia
