From: Stevan Harnad <[log in to unmask]>
Date: Thu, 21 Aug 2014 15:18:31 -0400

Harnad, S. (2014) Crowd-Sourced Peer Review: Substitute or supplement
for the current outdated system? LSE Impact Blog 8/21

EXCERPT:

If, as rumoured, google builds a platform for depositing unrefereed
research papers for “peer-reviewing” viacrowd-sourcing, can this
create a substitute for classical peer-review or will it merely
supplement classical peer review with crowd-sourcing?

... no one knows whether crowd-sourced peer-review, even if it could
work, would be scaleable or sustainable.

The key questions are hence:

1. Would all (most? many?) authors be willing to post their unrefereed
papers publicly (and in place of submitting them to journals!)?

2. Would all (most? many?) of the posted papers attract referees?
competent experts?

3. Who/what decides whether the refereeing is competent, and whether
the author has adequately complied? (Relying on a Wikipedia-style
cadre of 2nd-order crowd-sourcers who gain authority recursively in
proportion to how much 1st-order crowd-sourcing they have done —
rather than on the basis of expertise —  sounds like a way to generate
Wikipedia quality, but not peer-reviewed quality…)

4. If any of this actually happens on any scale, will it be sustainable?

5. Would this make the landscape (unrefereed preprints, referee
comments, revised postprints) as navigable and useful as classical
peer review, or not?

My own prediction (based on nearly a quarter century of umpiring both
classical peer review and open peer commentary) is that crowdsourcing
will provide an excellent supplement to classical peer review but not
a substitute for it. Radical implementations will simply end up
re-inventing classical peer review, but on a much faster and more
efficient PostGutenberg platform. We will not realize this, however,
until all of the peer-reviewed literature has first been made open
access. And for that it is not sufficient for Google merely to provide
a platform for authors to put their unrefereed papers, because most
authors don’t even put their refereed papers in their institutional
repositories until it is mandated by their institutions and funders.

http://openaccess.eprints.org/index.php?/archives/1121-Crowd-Sourced-Peer-Review-Substitute-or-Supplement.html