From: Sandy Thatcher <[log in to unmask]>
Date: Fri, 8 Jun 2012 12:50:56 -0500

Kathleen Fitzpatrick is someone else who has addressed the problems of
traditional peer review and promoted alternative approaches.  Here is
a section of my review of her book "Planned Obsolescence" (NYU, 2011)
forthcoming in the October issue of the Journal of Scholarly
Publishing:

The critique of traditional peer review in Chapter 1 is especially
thought-provoking. Fitzpatrick scores some solid hits against the kind
of peer-review process that still dominates the evaluation of
scholarship today. For instance, as she observes, when only one review
raises a criticism, one doesn't know whether this is representative of
a general problem or merely reflects an idiosyncratic reaction of one
reader. In sketching an alternative system, she emphasizes that its
success depends on "prioritizing members' work on behalf of the
community" (p. 43). Reputation in such a system requires some way of
reviewing the reviewers. The ability to publish might therefore be
based on measurements of how "helpful" a scholar is in participating
in group discussion.  A system like this will "require a phenomenal
amount of labor" as well as a new set of metrics for reviewing the
reviewers, but if "reviewing were a prerequisite for publishing, we'd
likely see more scholars become better reviewers, which will in turn
allow for a greater diversity of opinion and a greater distribution of
the labor involved" (p. 46).  This is definitely a promising approach
as it directly addresses the chief challenge to a system of crowd
review, namely, the incentives for participation and the quality of
comments offered. 1

Sandy Thatcher

*************

From: Sean Andrews <[log in to unmask]>
Date: Thu, 7 Jun 2012 10:09:44 -0500

I came across this article today, by Cultural Studies scholar Ted
Striphas, which discusses this problem at length and mentions F1000
and Fitzgerald's CommentPress and Media Commons as possible solutions
to the problem you mention.

Striphas, T. 2011 Oct 26. Academic Labor: The Visible College.
International Journal of Communication [Online] 5:0.
Available:http://ijoc.org/ojs/index.php/ijoc/article/view/1339/648

It is well worth the read.  And if you don't have time for the whole
thing, here is one of the key paragraphs:

> The benefits of turning the social work of academic publishing inside out could exceed matters of  openness.  In fact, we might end up broadening the definition of higher education along the way.  One of  the things I most enjoy about being a professor is the ongoing interaction I get to share with colleagues-smart people with whom I'm in dialogue and from whom I'm constantly learning.  However important the  traditional student-teacher relationship may be, it's not the sum total of the education that occurs in university settings.  We professors also learn from our peers, teaching and mentoring one another about the specifics of some subject matter and about the broader ambit of the profession.  Scholarly communication is no exception to this.  Sure, it's about gatekeeping-often frustratingly so-but it fulfills  an indispensable phatic function as well.  The so-called "invisible college" of academic life is real, manifesting itself in a citation someone might casually share with you in an office hallway, a trusted friend's written feedback on a draft  essay, readers' reports, an editor's feedback, and any number of  other-usually  private-exchanges.    To  call  this work  "service"  is  to  do  it  a  disservice,  for  it's what makes higher education dynamic, relevant, and about so much more than the pursuit of degrees.

Sean Johnson Andrews
[log in to unmask]
Assistant Professor of Cultural Studies
Columbia College, Chicago
2011-2013 ACLS Public Fellow
Program Officer
The National Institute for Technology in Liberal Education
http://www.nitle.org | tel. 703-597-6948 | fax 512 819-7684
iChat: [log in to unmask] | skype: jnskolja | twitter: @skja76


On Wed, Jun 6, 2012 at 7:02 PM, LIBLICENSE <[log in to unmask]> wrote:

> From: <[log in to unmask]>
> Date: Wed, 6 Jun 2012 10:41:08 +0000
>
> Dear Jim,
>
> As it happens we are currently launching a journal across biology and
> Medicine called F1000 Research (http://f1000research.com) which will
> indeed experiment with completely opening up the formal peer review
> process to avoid many problems with peer review, including the one you
> eluded to below of not knowing whether an article has been rejected
> numerous times before finally being accepted in a 'peer-reviewed'
> journal.
>
> The idea behind F1000 Research is to provide scientists with a
> far-reaching alternative to existing journal publishing models and
> will address four significant issues:
>
> 1. Speed. We will offer immediate publication, following a brief
> internal review. All submissions will initially be labelled as
> 'awaiting peer review'.
>
> 2. Peer review. Submissions will go into a rapid, formal, and
> completely open peer review process post-publication. Our intention
> here is to eliminate the delays and injustices that too often arise in
> the standard closed peer review system.
>
> 3. Research types. A wide variety of types of research findings will
> be accepted, from traditional articles to null/negative findings, to
> replicate/refutation findings.
>
> 4. Primary data. We will promote sharing, publication and refereeing
> of datasets, in the form of separate data papers, to foster
> collaboration and accelerate scientific discovery.
>
> Although we plan to formally launch towards the end of the year, we do
> plan to start publishing a few articles starting in the next few weeks
> to test some of these novel processes. There are more details of our
> initial plan at
> http://f1000research.com/2012/01/30/f1000-research-join-us-and-shape-the-future-of-scholarly-communication-2/,
> and further information will be posted shortly at
> Http://f1000research.com.  We have received extensive support from all
> areas of biology and medicine, including our large prestigious
> Advisory Panel: http://f1000research.com/advisory-panel/.
>
> We are aiming to develop this initiative very much in consultation
> with the relevant communities and so any thoughts or comments on the
> plans would be very much welcomed.
>
> Kind regards
> Rebecca
>
> Rebecca Lawrence, PhD
> Publisher, F1000 Research
> FACULTY of 1000
> http://F1000.com