This website requires JavaScript.

Contra-Analysis for Determining Negligible Effect Size in Scientific Research

Bruce A. CorlissYaotian WangHeman ShakeriPhilip E. Bourne
Mar 2023
Scientific experiments study interventions that show evidence of an effectsize that is meaningfully large, negligibly small, or inconclusively broad.Previously, we proposed contra-analysis as a decision-making process to helpdetermine which interventions have a meaningfully large effect by using contraplots to compare effect size across broadly related experiments. Here, weextend the use of contra plots to determine which results have evidence ofnegligible (near-zero) effect size. Determining if an effect size is negligibleis important for eliminating alternative scientific explanations andidentifying approximate independence between an intervention and the variablemeasured. We illustrate that contra plots can score negligible effect sizeacross studies, inform the selection of a threshold for negligible effect basedon broadly related results, and determine which results have evidence ofnegligible effect with a hypothesis test. No other data visualization can carryout all three of these tasks for analyzing negligible effect size. Wedemonstrate this analysis technique on real data from biomedical research. Thisnew application of contra plots can differentiate statistically insignificantresults with high strength (narrow and near-zero interval estimate of effectsize) from those with low strength (broad interval estimate of effect size).Such a designation could help resolve the File Drawer problem in science, wherestatistically insignificant results are underreported because theirinterpretation is ambiguous and nonstandard. With our proposed procedure,results designated with negligible effect will be considered strong andpublishable evidence of near-zero effect size.