Frequency of Treatment-Effect Modification Affecting Indirect Comparisons: A Systematic Review

A key assumption of indirect comparisons is similarity, which means that, in the face of differences in patient characteristics or study methods, there is no treatment-effect modification across sides of the indirect comparison. We therefore conducted a systematic review of MEDLINE and EMBASE from inception to November 2009 to summarize currently available information about how frequently, on average, treatment-effect modification occurs across trials that might be used on different sides of an indirect comparison. Although similarity is a key assumption, there is currently no published evidence specifically for indirect comparisons about how frequently treatment-effect modification occurs. Six analyses were identified that assessed treatment-effect modification across studies included in direct head-to-head meta-analyses. Such analyses are relevant to indirect comparisons because the phenomenon being investigated would occur with similar frequency. They provide important information because lack of treatment-effect modification across sides of an indirect comparison cannot be directly assessed statistically; this is in contrast to direct head-to-head meta-analyses where Cochrane's Q statistic or I can be used. For ratio measures such as the odds ratio and relative risk, treatment-effect modification occurred for 10-33% of meta-analyses. For the risk difference (an arithmetic measure), the range was 15-46%. It is not prudent to assume similarity in an indirect comparison, based only on the result that ratio measures such as the odds ratio are reasonably robust to treatment-effect modification. All indirect comparisons should include a thorough narrative comparison of differences in patient characteristics and study methods. This will provide end users with the best evidence with which to make an assessment of the likelihood of treatment-effect modification and the plausibility of the similarity assumption.