From: Sandy Thatcher <[log in to unmask]> Date: Fri, 8 Jun 2012 12:50:56 -0500 Kathleen Fitzpatrick is someone else who has addressed the problems of traditional peer review and promoted alternative approaches. Here is a section of my review of her book "Planned Obsolescence" (NYU, 2011) forthcoming in the October issue of the Journal of Scholarly Publishing: The critique of traditional peer review in Chapter 1 is especially thought-provoking. Fitzpatrick scores some solid hits against the kind of peer-review process that still dominates the evaluation of scholarship today. For instance, as she observes, when only one review raises a criticism, one doesn't know whether this is representative of a general problem or merely reflects an idiosyncratic reaction of one reader. In sketching an alternative system, she emphasizes that its success depends on "prioritizing members' work on behalf of the community" (p. 43). Reputation in such a system requires some way of reviewing the reviewers. The ability to publish might therefore be based on measurements of how "helpful" a scholar is in participating in group discussion. A system like this will "require a phenomenal amount of labor" as well as a new set of metrics for reviewing the reviewers, but if "reviewing were a prerequisite for publishing, we'd likely see more scholars become better reviewers, which will in turn allow for a greater diversity of opinion and a greater distribution of the labor involved" (p. 46). This is definitely a promising approach as it directly addresses the chief challenge to a system of crowd review, namely, the incentives for participation and the quality of comments offered. 1 Sandy Thatcher ************* From: Sean Andrews <[log in to unmask]> Date: Thu, 7 Jun 2012 10:09:44 -0500 I came across this article today, by Cultural Studies scholar Ted Striphas, which discusses this problem at length and mentions F1000 and Fitzgerald's CommentPress and Media Commons as possible solutions to the problem you mention. Striphas, T. 2011 Oct 26. Academic Labor: The Visible College. International Journal of Communication [Online] 5:0. Available:http://ijoc.org/ojs/index.php/ijoc/article/view/1339/648 It is well worth the read. And if you don't have time for the whole thing, here is one of the key paragraphs: > The benefits of turning the social work of academic publishing inside out could exceed matters of openness. In fact, we might end up broadening the definition of higher education along the way. One of the things I most enjoy about being a professor is the ongoing interaction I get to share with colleagues-smart people with whom I'm in dialogue and from whom I'm constantly learning. However important the traditional student-teacher relationship may be, it's not the sum total of the education that occurs in university settings. We professors also learn from our peers, teaching and mentoring one another about the specifics of some subject matter and about the broader ambit of the profession. Scholarly communication is no exception to this. Sure, it's about gatekeeping-often frustratingly so-but it fulfills an indispensable phatic function as well. The so-called "invisible college" of academic life is real, manifesting itself in a citation someone might casually share with you in an office hallway, a trusted friend's written feedback on a draft essay, readers' reports, an editor's feedback, and any number of other-usually private-exchanges. To call this work "service" is to do it a disservice, for it's what makes higher education dynamic, relevant, and about so much more than the pursuit of degrees. Sean Johnson Andrews [log in to unmask] Assistant Professor of Cultural Studies Columbia College, Chicago 2011-2013 ACLS Public Fellow Program Officer The National Institute for Technology in Liberal Education http://www.nitle.org | tel. 703-597-6948 | fax 512 819-7684 iChat: [log in to unmask] | skype: jnskolja | twitter: @skja76 On Wed, Jun 6, 2012 at 7:02 PM, LIBLICENSE <[log in to unmask]> wrote: > From: <[log in to unmask]> > Date: Wed, 6 Jun 2012 10:41:08 +0000 > > Dear Jim, > > As it happens we are currently launching a journal across biology and > Medicine called F1000 Research (http://f1000research.com) which will > indeed experiment with completely opening up the formal peer review > process to avoid many problems with peer review, including the one you > eluded to below of not knowing whether an article has been rejected > numerous times before finally being accepted in a 'peer-reviewed' > journal. > > The idea behind F1000 Research is to provide scientists with a > far-reaching alternative to existing journal publishing models and > will address four significant issues: > > 1. Speed. We will offer immediate publication, following a brief > internal review. All submissions will initially be labelled as > 'awaiting peer review'. > > 2. Peer review. Submissions will go into a rapid, formal, and > completely open peer review process post-publication. Our intention > here is to eliminate the delays and injustices that too often arise in > the standard closed peer review system. > > 3. Research types. A wide variety of types of research findings will > be accepted, from traditional articles to null/negative findings, to > replicate/refutation findings. > > 4. Primary data. We will promote sharing, publication and refereeing > of datasets, in the form of separate data papers, to foster > collaboration and accelerate scientific discovery. > > Although we plan to formally launch towards the end of the year, we do > plan to start publishing a few articles starting in the next few weeks > to test some of these novel processes. There are more details of our > initial plan at > http://f1000research.com/2012/01/30/f1000-research-join-us-and-shape-the-future-of-scholarly-communication-2/, > and further information will be posted shortly at > Http://f1000research.com. We have received extensive support from all > areas of biology and medicine, including our large prestigious > Advisory Panel: http://f1000research.com/advisory-panel/. > > We are aiming to develop this initiative very much in consultation > with the relevant communities and so any thoughts or comments on the > plans would be very much welcomed. > > Kind regards > Rebecca > > Rebecca Lawrence, PhD > Publisher, F1000 Research > FACULTY of 1000 > http://F1000.com