From: Alex Frost <[log in to unmask]> Date: Mon, 26 Sep 2016 09:24:20 -0400 I think publicizing an interest in negative results may be more impactful than the outcome of the announced pilot, and maybe that’s the short-term goal... Under typical conditions in most fields the strongest filter likely arises from researchers who never bother to write up and submit negative results. This means that there is a relatively low percent of manuscripts submitted with negative results in the first place. Bias by reviewers could be consequential (and is worthwhile to study), but they can’t have a biased view of manuscripts they never see. If we agree that manuscripts with negative results are less likely to be submitted because of concern by the authors that they’re not “publishable”, then there will be other dynamics/biases: -- under typical conditions authors may be more likely to submit manuscripts with negative results if they believe the topic or rest of the manuscript is especially strong or “publishable” -- in the announced BMC Psych trial, there will be potentially competing biases because of self-selection by manuscript authors who seek to participate (BMC Psych has announced the trial and is asking for opt-ins) These dynamics complicate the interpretation of any study (let alone a very small pilot), but I’m happy to see BMC Psych get something started and would be interested in other data or opinions on the topic. Alex > On Sep 25, 2016, at 6:58 PM, LIBLICENSE <[log in to unmask]> wrote: > > From: Ann Shumelda Okerson <[log in to unmask]> > Date: Sun, 25 Sep 2016 18:55:01 -0400 > > Any reactions to the underlying premise (that knowing results > increases publication bias)? > > How big a pilot would be needed to test the premise? > > "Results-free peer review is getting its day in court, thanks to BMC > Psychology, an open-access journal that is launching the first-ever > randomized controlled trial of the process. Results-free peer review > puts manuscripts before reviewers without divulging results or > discussion sections until the end of the process. In theory, this > alteration to traditional peer review decreases publication bias by > forcing reviewers to focus solely on methodology and approach to an > experimental question." > > http://www.the-scientist.com/?articles.view/articleNo/47081/title/Reviewing-Results-Free-Manuscripts/