The “Reproducibility Project: Psychology” by the Open Science Collaboration caused some stir among psychologists, methodologists as well as scientists, since less than half of the replicated studies succeeded in reproducing the results of the original ones. The APA has attributed this result to hidden moderators that rendered the replications ineffective. Also publication bias and low power have been identified as possible sources for such mismatch. While some analysts have provided formal confirmation for the plausibility of such explanations (Etz and Vandekerkhove, 2016), others have further insisted on the problem of noisy data and suggested that “to resolve the replication crisis in science we may need to consider each individual study in the context of an implicit meta-analysis” (Andrew Gelman).

I investigate these positions through the lenses of Bayesian epistemology, and in particular of recent results on the Variety of Evidence Thesis.