From: Sean Andrews <[log in to unmask]> Date: Tue, 5 Jun 2012 09:30:48 -0500 I can see the utility of this information - and in some ways this transparency is, in principle, what a more open peer review process allows, something along the lines of the post publication peer review (or Peer-to-peer review) that Kathleen Fitzpatrick talks about http://www.tandfonline.com/doi/abs/10.1080/02691728.2010.498929 and the Kent Anderson finds problematic http://scholarlykitchen.sspnet.org/2012/03/26/the-problems-with-calling-comments-post-publication-peer-review/ In a sense what this would show is the conversation that went on between the reviewers, the editor and the author, and, therefore, the process through which that knowledge was produced. In some cases, you may not want to see the sausage being made; in others it gives a window into the other possible ways an idea could have been developed. This is what I find useful about authors who use footnotes for side conversations: it often shows tangents that wouldn't fit into the current articulation of the paper, but which might have appeared in another tack. But the other thing I hear you saying is that it would be useful to have some metric of what those acceptance and rejection rates mean - some more qualitative information that would give some insight into what the process of peer review means in each case. In a way I can see these two converging in an interesting framework which might help make sense of what is an emerging diversity of peer review processes - both within journal editorial boards and on the open web. The language of "badges" is a bit overdone, but using something like a certification of peer review would allow readers to know something about the processes that go on behind the scenes - and to begin to have an alternative metric for items that are mostly subject to "post publication peer review." I don't know how the authority could be established to undertake this, but it might be something individual disciplines would try to do. For journals, this might mean allowing an audit of their processes. Not only information about acceptance and rejection, but actually digging into the files on the editors' hard drive or Scholar One account, using some metric to give general snapshot of what the process looks like. So while it might not tell you what any particular article went through (e.g. the whether, how often and by whom a piece had been rejected) it would give you a sense of the legitimacy of this process. In establishing what the guidelines are for peer review in journals - i.e. what would merit a certification - this could then be transferred to other forms of scholarship, like that on the open web (or in emergent open access publications). If the metric was that it should be looked at by at least two major people in the field, this could more easily measure up to the standard Anderson poses for comments as peer review. In many cases, an acceptance is not all that much longer than a lengthy comment on a blog post. Rejection or revise/resubmit might be a longer comment and a required revision of the post to meet some of those critiques - or at least a lengthy defense by the author. Likewise, if an author was interested in seeking certification, they could solicit comments from disciplinary listserves or other spaces to ensure that they got quality feedback and spurred some conversation. In any case, once an author felt they had fulfilled the peer review process requirements, they could then apply for some sort of certification by the peer review board of their discipline - or something like this. Maybe this is overdoing it - since the main issue for peer review is often it's place in tenure and promotion procedures, having the certification be something more internal to departments or colleges might make more sense. Either way, it is an interesting question and I think it points to the emergent disintegration of systems of legitimacy in scholarly communication - and the need to shore them up in creative ways that take into account the diversity of work and venues in which it is being produced. And perhaps I am totally misunderstanding your question. best, Sean Johnson Andrews [log in to unmask] Assistant Professor of Cultural Studies Columbia College, Chicago 2011-2013 ACLS Public Fellow Program Officer The National Institute for Technology in Liberal Education http://www.nitle.org | tel. 703-597-6948 | fax 512 819-7684 On Sun, Jun 3, 2012 at 6:36 PM, LIBLICENSE <[log in to unmask]> wrote: > From: "James J. O'Donnell" <[log in to unmask]> > Date: Sun, 3 Jun 2012 14:39:56 -0400 > > There is one experiment with transparency in scholarly communication > that I have not seen. I'd be glad to hear if there are any cases where it > has been tried and to hear comments on the possibility. > > The most confidential part of the process of "public"ation is peer > review. An author submits an article to a journal and it is accepted > or rejected; if rejected, the author goes elsewhere and repeats the > effort to win acceptance. Journals boast of their acceptance (i.e., > rejection) rates. Something I would like to know - but now cannot > find out, when I read an article - is whether and how often and by whom > the same piece has been rejected. Many editors would be glad to have > that information about individual items and "average prior > rejections/article" would be an interesting metric of the quality of a > journal. > > Publishing this information would also allow for validation of the > peer review system: articles with high citation counts and multiple > rejections would be interesting in one way, but it's likely in most > fields that the reverse would be the near-universal norm. Who > would not benefit from such transparency? If we are to mandate > access to results of research -- is this not one of the results? > > Jim O'Donnell > Georgetown