J Clin Epidemiol 2020: , ISBN 1878-5921 (Electronic)
0895-4356 (Linking) (Journal)
Ewald H., Klerings I., Wagner G., Heise T. L., Dobrescu A. I., Armijo-Olivo S., Stratil J. M., Lhachimi S. K., Mittermayr T., Gartlehner G., Nussbaumer-Streit B., Hemkens L. G.
OBJECTIVE: Assessing the agreement of treatment effect estimates from meta-analyses based on abbreviated or comprehensive literature searches. STUDY DESIGN AND SETTING: Meta-epidemiological study. We abbreviated 47 comprehensive Cochrane review searches and searched MEDLINE/Embase/CENTRAL alone, in combination, with/without checking references (658 new searches). We compared one meta-analysis from each review with recalculated ones based on abbreviated searches. RESULTS: The 47 original meta-analyses included 444 trials (median 6 per review [IQR 3-11]) with 360045 participants (median 1371 per review [IQR 685-8041]). Depending on the search approach, abbreviated searches led to identical effect estimates in 34%-79% of meta-analyses, to different effect estimates with the same direction and level of statistical significance in 15%-51%, and to opposite effects (or effects could not be estimated anymore) in 6%-13%. The deviation of effect sizes was zero in 50% of the meta-analyses and in 75% not larger than 1.07-fold. Effect estimates of abbreviated searches were not consistently smaller or larger (median ratio of odds ratio 1 [IQR 1-1.01]) but more imprecise (1.02-1.06-fold larger standard errors). CONCLUSION: Abbreviated literature searches often led to identical or very similar effect estimates as comprehensive searches with slightly increased confidence intervals. Relevant deviations may occur.