J Clin Epidemiol 2019: , ISBN 1878-5921 (Electronic)
0895-4356 (Linking) (Journal)
Schandelmaier S., Chang Y., Devasenapathy N., Devji T., Kwong J. S., Colunga Lozano L. E., Lee Y., Agarwal A., Bhatnagar N., Ewald H., Zhang Y., Sun X., Thabane L., Walsh M., Briel M., Guyatt G. H.
OBJECTIVE: To systematically survey the methodological literature and collect suggested criteria for assessing the credibility of effect modification and associated rationales. STUDY DESIGN: We searched MEDLINE, Embase, textbook chapters to March 2018 for publications providing guidance for assessing the credibility of effect modification identified in randomized trials or meta-analyses. Teams of two investigators independently identified eligible publications and extracted credibility criteria and authors' rationale, reaching consensus through discussion. We created a taxonomy of criteria that we iteratively refined during data abstraction. RESULTS: We identified 150 eligible publications that provided 36 criteria and associated rationales. Frequent criteria included: significant test for interaction (n=54); a priori hypothesis (n=49); providing a causal explanation (n=47); accounting for multiplicity (n=45); testing a small number of effect modifiers (n=38); and pre-specification of analytic details (n=39). For some criteria, we found more than one rationale; some criteria were connected through a common rationale. For some criteria, experts disagreed regarding their suitability (e.g. added value of stratified randomization; trustworthiness of biologic rationales). CONCLUSION: Methodologists have expended substantial intellectual energy providing criteria for critical appraisal of apparent effect modification. Our survey highlights popular criteria, expert agreement and disagreement, and where more work is needed, including testing criteria in practice.