From: Alex Frost <[log in to unmask]>
Date: Mon, 26 Sep 2016 09:24:20 -0400

I think publicizing an interest in negative results may be more
impactful than the outcome of the announced pilot, and maybe that’s
the short-term goal...

Under typical conditions in most fields the strongest filter likely
arises from researchers who never bother to write up and submit
negative results. This means that there is a relatively low percent of
manuscripts submitted with negative results in the first place. Bias
by reviewers could be consequential (and is worthwhile to study), but
they can’t have a biased view of manuscripts they never see.

If we agree that manuscripts with negative results are less likely to
be submitted because of concern by the authors that they’re not
“publishable”, then there will be other dynamics/biases:

-- under typical conditions authors may be more likely to submit
manuscripts with negative results if they believe the topic or rest of
the manuscript is especially strong or “publishable”

-- in the announced BMC Psych trial, there will be potentially
competing biases because of self-selection by manuscript authors who
seek to participate (BMC Psych has announced the trial and is asking
for opt-ins)

These dynamics complicate the interpretation of any study (let alone a
very small pilot), but I’m happy to see BMC Psych get something
started and would be interested in other data or opinions on the
topic.

Alex


> On Sep 25, 2016, at 6:58 PM, LIBLICENSE <[log in to unmask]> wrote:
>
> From: Ann Shumelda Okerson <[log in to unmask]>
> Date: Sun, 25 Sep 2016 18:55:01 -0400
>
> Any reactions to the underlying premise (that knowing results
> increases publication bias)?
>
> How big a pilot would be needed to test the premise?
>
> "Results-free peer review is getting its day in court, thanks to BMC
> Psychology, an open-access journal that is launching the first-ever
> randomized controlled trial of the process. Results-free peer review
> puts manuscripts before reviewers without divulging results or
> discussion sections until the end of the process. In theory, this
> alteration to traditional peer review decreases publication bias by
> forcing reviewers to focus solely on methodology and approach to an
> experimental question."
>
> http://www.the-scientist.com/?articles.view/articleNo/47081/title/Reviewing-Results-Free-Manuscripts/