From: Laval Hunsucker <[log in to unmask]>
Date: Thu, 5 Apr 2012 15:21:09 -0700

At first glance, this pilot may indeed look like something appropriate
to a, in Elsevier's words, "project aimed at improving the article
value in all areas of its presentation, content, and context".  But,
thinking it over, I began to have my doubts.  To what extent is such
"review report transparency" really something that journal readers or
authors or reviewers or editors, themselves, are inclined to want to
see implemented, and something from which they and the scientific
communication system will actually significantly benefit?

I do read and use lots of articles from Elsevier journals for my own
purposes, and have for a pretty long time, though admittedly not in
the area of agricultural and forestry ecology, but can't recall ever
regretting that I didn't have access to the pre-review version of an
article or to the reviewer's report, and don't think it ever actually
occurred to me that such access would result in "improving the article
value" in terms of content and context, or that the presentation was
somehow unsatisfactory without those elements. Surely in certain cases
they might well have improved the article value for me somewhat, but
in most I would guess that their publication would just have amounted
to useless effort and wasted space. What is it, after all, that we've
actually been missing all this time?

Numerous are indeed the questions that come immediately to the mind of
a reader of this rather cursory press announcement. Obvious questions,
perhaps.

What does it mean exactly that Elsevier is in this case 'testing transparency'?

And will there also be transparency concerning the grounds on which
the pilot will eventually have been judged to be "successful" or
otherwise?

Who will be judging whether or not success is achieved? -- will the
disciplinary community, or even all readers,
be able to play a role?

Further on down, beyond the part which was quoted by Ann, I now read :
 "Review reports will only be published if they meet certain quality
standards."  Does this mean in the pilot phase, or does this indicate
the intended procedure after official implementation, or both?  In any
case, how does this relate with Tanke's statement that "Making the
reviews visible to all readers of the article will ultimately *lead
to* [my emphasis] higher quality review reports"?

Also in any case :  Will these standards (presumably established, and
to be applied, in the first instance, by A&FM's editor? -- or by
Elsevier ?) also be made clear to the outside world, so that also the
readers can judge what is going on?

To what extent will they ( later ) vary according to participating discipline?

And in fact, should an article even be published at all if (or so long
as) the review report is of substandard quality?  If the review report
is in some instances omitted as implied, isn't this then a signal to
the reader that the contribution as published is potentially of lower
quality, since the reviewer's performance was judged to be
qualitatively substandard?  Isn't it much better, *and* more
transparent, to include review reports with all articles -- or
otherwise with none, as now?

Or, so far from it being ( just ) a matter of quality, is there
perchance (also) an idea here that a journal might less easily attract
(enough) authors if those authors don't have the possibility to opt
out of having published, besides the final version of their
contribution, also their "originally submitted manuscript" (which
likewise is to be published as part of this scheme, according to the
full announcement) and the judgments passed on it by one or more
referees?  Is this not a pragmatic factor of some significance?

Insisting on publication of review reports and original manuscripts
for all articles would lose a journal some potential authors, I would
think, but shouldn't it also have the overall effect of improving the
quality of the authors who submit, and of their initial submissions?
The announcement emphasizes the objective of improving the quality of
reviewers and of their reports, but has nothing to say about the
quality of submissions. ( Note, too, that reviewers will -- oddly ? --
be allowed to opt out of being identified, the authors naturally not.)
 And, is a report which the reviewer knows will be made openly
available certain, or likely, to be as effective as one which he knows
will only be seen by the editor and (anonymously ) by the author ?

Beyond that:  What about the author's written responses to review
reports -- also now unavailable to readers (and often even to
reviewers?). Their availability would seem essential if the reader is
properly to understand the relationship of the originally submitted
manuscript to the version of record and if Elsevier's objective is
indeed, to use the words of Dr. Lee, to "provide context to the
article by giving multiple perspectives on the quality of the study".
I saw nothing here about publishing
those author's responses as well.

Note, also, that in the case of the particular journal being used for
this pilot, submitting authors themselves are, at least at this point,
required also to submit names of potential reviewers, and are
furthermore allowed to name persons whom they "do not want to review
[their] manuscript". How does that fit in with goals such as "giving
multiple perspectives on the quality of the study" and enhancing peer
review quality?

As I said: numerous questions, and perhaps obvious questions. But I'd
for one be quite interested to hear any further comments anyone might
wish to make on such matters.

Nonetheless :  interesting that Elsevier has decided to try this out
and, like Ann, I look forward to some eventual follow-through and
evaluation.

- Laval Hunsucker


----- Original Message -----
> From: Ann Shumelda Okerson <[log in to unmask]>
> Date: Wed, 4 Apr 2012 13:34:56 -0400
>
> This will be interesting to follow.  Hope there will be a report at
> the end, that we can read.
>
> ***************
>
> Review Reports Published With Articles Should Enhance Peer Review Quality
>
> Elsevier, a world-leading provider of scientific, technical, and
> medical information products and services, has launched a pilot
> project aimed at improving the quality of peer review. Review reports
> for articles in Agricultural and Forest Meteorology will be published
> alongside the article on SciVerse ScienceDirect. The pilot will last
> until the end of 2012. If successful, the initiative will also be
> applied to other Elsevier journals.....
>
> http://www.dailymarkets.com/stock/2012/04/02/elsevier-tests-peer-review-report-transparency/