Reconstruction of a Train Wreck: How Priming Research Went off the Rails | Replicability-Index

Reconstruction of a Train Wreck: How Priming Research Went off the Rails | Replicability-Index: Schimmack and Brunner (2015) developed an alternative method for the estimation of replicability.� This method takes into account that power can vary across studies. It also provides 95% confidence intervals for the replicability estimate.� The results of this method are presented in the Figure above.�The replicability estimate is similar to the R-Index, with 14% replicability.� However, due to the small set of studies, the 95% confidence interval is wide and includes values above 50%. This does not mean that we can trust the published results, but it does suggest that some of the published results might be replicable in larger replication studies with more power to detect small effects.� At the same time, the graph shows clear evidence for a selection effect.� That is, published studies in these articles do not provide a representative picture of all the studies that were conducted.� The powergraph shows that there should have been a lot more non-significant results than were reported in the published articles.� The selective reporting of studies that worked is at the core of the replicability crisis in social psychology (Sterling, 1959, Sterling et al., 1995; Schimmack, 2012).� To clean up their act and to regain trust in published results, social psychologists have to conduct studies with larger samples that have more than 50% power (Tversky & Kahneman, 1971) and they have to stop reporting only significant results.� We can only hope that social psychologists will learn from the train wreck of social priming research and improve their research practices.