Peerage of Science

[shadowclaw]

Has anyone tried Peerage of Science and had success with it? POS is touted as a new way to do peer review (you can check out the website at http://www.peerageofscience.org). Basically, it's supposed to shorten the time your paper spends in review and help prevent those cruddy reviews where it seems like your reviewer didn't even read the paper. The basic process is that you first submit your manuscript and set deadlines for when you'd like it reviewed by, when you'd like peer review of your peer review done, and how long you'll have to make revisions. Next, your paper gets reviewed by two people using a specific format that is supposed to help show you what parts of your paper need most revision. After the review stage, the review actually gets reviewed. So if your reviewer did a lousy job, they will get called out on it by another reviewer. Finally, you use your reviews to make revisions. At this point, you can submit the whole history of the manuscript and peer review to participating journals (which may or may not require further peer review). Participating journals also have the option of viewing your manuscript anytime during the process and can make publishing offers to you.

It sounded like a good idea when one of my committee members from my masters program suggested we use it. However, our paper only got one review (and the review is currently under review), and it wasn't very good... not that the reviewer didn't try to do a good job, but they really didn't understand how the stats worked and that's what most of their comments were on.

Anyone else try this and have a good experience?

[rising_star]

I've never heard of it before! It seems like such a good idea though. I hope others chime in with their experiences, as I'd love to read more about this.

[shadowclaw]

Based on what I've read and experienced, I think it might really take off once more people hear of it. When you submit a manuscript, you can suggest reviewers and it lets you know if the person is already registered in the system or not. If he/she isn't, you can invite him/her to POS. None of the reviewers I suggested were registered already  Hopefully more people will find out about this and take part! The professor who suggested we use POS heard about while at a conference in France this summer. I don't know if that means it's more popular in Europe at this point.

[TakeruK]

Hmmm, this is an interesting concept. If I was a reviewer, my main concern would be that I would not be certain that my effort and time spent reviewing this paper would be useful/effective. If I am reviewing for a journal then I know that the editor will be able to judge my review and then require the author to actually take action on it. However, it sounds like if I do this for POS, it's not necessarily true that my comments will directly affect the manuscript. After all, the journal might then require further peer review and then it's likely the authors would prioritize these required edits rather than my suggestions.

In my field, we get the same effect as POS by sending our manuscript to people we know in the field to read and give us their feedback. We try to send it to people who would actually be interested in the paper and our results and they will sometimes give feedback (if it's relevant). Another thing our field does is to just upload the submitted version of the manuscript to a pre-print server and then get community feedback at the same time as we are waiting for the journal's peer-review process. 

In these cases, I think it's a little different than POS because the authors directly solicit reviews, not a third party and the reviewers comments go directly to the author, not any other third party. In this way, there is far less pressure to do an in-depth review if you don't have time and also less pressure to do a review if you don't want to. Also, ideally, in the cases I describe, the "volunteer reviewers" are people that will read the article when it does get published anyways!