Group items tagged

"If, as rumoured, google builds a platform for depositing unrefereed research papers for "peer-reviewing" via crowd-sourcing, can this create a substitute for classical peer-review or will it merely supplement classical peer review with crowd-sourcing?"

Two facts that makes me think, peer-reviewing via crowd-sourcing, at best would supplement the traditional peer-review process. Fact one, there are already open access repositories that allow "deposit first; review later", but those repositories have not taken over other journals. Fact two, Wikipedia is an example in that, though theoretically anyone can contribute and edit the articles, there is definite number of people who would do it. Therefore, I don't see crowd sourcing peer review would really substitute the traditional route.

I appreciated that this source was framed outside of dichotomous thinking by not pitting more traditional and open access peer review models directly against one another, carrying the assumption that a particular publishing process must choose one or another. Although, I think I would challenge Harnad to take this thought process further. Rather than supplementing or complementing one another, traditional and open peer review models are distinct enough to also be applicable in different types of contexts, without necessarily needing to rely on one another. That is not to disagree with Harnad that the two do not "substitute" one another, but precisely because they cannot substitute one another indicates that they serve different purposes and could thus be useful in different contexts…. Or, as Harnad suggest, supplement each other in the same context. I think this very well parallels the context of taxonomies and folksonomies.

Publicado el 7/5/2012 This is a brief overview of several Open Peer Review Models, including ETAI, Nature, ACP, PLoS One and EJCBS. It is recorded based on a Prezi Presentation first developed for Open Access Week 2011 at UBC.

I found this presentation, in particular the visual representations, to be very useful in understanding just how diverse open peer review models can be. Several things stuck out to me throughout the presentation. First, I was surprised that many of the open peer review models either maintained anonymity of the reviewers or self-identification was optional. For example, PLOSone and the Atmospheric Chemistry and Physics journals did not require self-identification. This raises a tension for me in that it does enable more people to participate in the publishing/review process, but it still inherently indicates context does not matter, which is something I disagree with. That is, if, for example, a paper is on student-faculty partnerships or feminism, it seems to me that crucial insights pertain to the particularities of the people reviewing an article. The other aspect that stuck out to me was how crucial it is for a journal to be intentional about implementing, integrating, and valuing an open peer review process. The Nature experiment is a good example of this. While I am sure they spent a great deal of time figuring out how to construct and enable an open peer review process, it was not necessarily emphasized as important by the journal nor well integrated into people's current practice. In contrast, the ETAI did this by permanently archiving the peer comments rather than deleting them unannounced, and editors also sent notifications to people that articles were ready rather than assuming people would seek out articles themselves.

"Think of your meanest high school mean girl at her most gleefully, underminingly vicious. Now give her a doctorate in your discipline, and a modicum of power over your future. That's peer review."

Great article. Funny, descriptive and useful. Anyone non-academic who works with academics should understand the pressure of peer review and this article introduces it wonderfully. I want to dig deeper into this idea of open peer review.

See my bookmark to: 'Open peer review is a welcome step towards transparency ...'

Online discussions and post-publication analyses are catching mistakes that sneak past editorial review. This article describes the pitfalls with editorial review and pre-publication peer review, and advocates for post-publication crowd-sourced reviewing through social media platforms.

I think having a comments section is a great way to provide feedback on the information provided. Often when I read articles the comments section allows me to understand different perspectives and interpretations of the information.

This article, while not necessarily explicitly, managed to hint at what I find to be a source of problematic practices/outcomes in the academy, publishing, etc. That is, it is not necessarily that traditional peer review processes are ineffective at finding errors or misconduct, but rather it is when our processes and practices become so systematized that we can mindlessly or effortlessly engage in and reproduce them without our full, critical attention that they can produce problems. While I think there are good reasons to critique the notion of peer and "expert" culture within traditional peer review processes, an additional and separate critique is the problems that arise with systematization. The article implicitly addressed this when the author commented that current post-publication environments "provide a public space that is not under the control of journal editors and conference organizers." Yet, as White indicates, there exists skepticism of the value of post-publication reviews along with a simultaneous effort to build post-publication systems that have standards that put those questioning it at ease. The National Institutes of Health establishing requirements that potential post-publication reviewers must meet demonstrated this. That is, they are trying to figure out how to systematize post-publication. For me, what this article indicates is that we ought to figure out how to keep our academic and publishing processes "fresh," so to speak. This way we don't become so comfortable with our methods and practices that they allow us to simply go through the motions without fostering innovative and critical inquiry.

From Book Censorship to Academic Peer Review Mario Biagioli Together with tenure, peer review is probably the most distinctive feature of the modern academic system. Peer review, we are told, sets academia apart from all other professions by construing value through peer judgment, not market dynamics. Given the remarkable epistemological and symbolic burden placed on peer review, it is surprising to find that so little research has analyzed it either empirically (in its actual daily practices) or philosophically (as one of the conditions of possibility of academic knowledge).

"Instead of calling for more money to prop up a traditional model that was never particularly viable in the first place, we need to embrace a variety of alternatives. Academic librarians are well positioned to lead the way here, both because of their long history of managing change and because they often hold the purse strings."

This article provides a brief historical contextualization of different forms of peer review. It does well to highlight not only pros and cons of the various processes, but it also discusses the positions from which these pros and cons come form. For example, gender is discussed, revealing that women frequently advocate for anonymous peer review due to the sexism they encounter leading to their work not being published. Another aspect that the authors engage with is how the technologies available shape the forms that peer review takes.

Of the 255 papers that underwent the entire editing process to acceptance or rejection, about 60% of the final decisions occurred with no sign of peer review. For rejections, that's good news: It means that the journal's quality control was high enough that the editor examined the paper and declined it rather than send it out for review.

This article is certainly controversial, and I believe in some way did a service to the Open Access community by highlighting the practice of predatory journals. However, the irony of Bohannon's article, being an example of the kind of "bad science" he describes in his own article is inescapable. First, there is no randomization of his "experimental group", and there is no control group; second, there was elimination of non-responders; third, there was no application of the intention to treat principle in the analysis; and finally there were no inferential statistics and no references! Using his own standard, there is nothing that can be concluded from his study. For the criticism regarding Bohannon's targeting of OA journals exclusively, it is important to note that this experiment has been done before with 'traditional' journals as well- and many of them failed the test of peer review. http://www.slate.com/blogs/future_tense/2014/02/27/how_nonsense_papers_ended_up_in_respected_scientific_journals.html

I think Bohannens "study" should be considered more "investigative journalism" than scientific study. While it may have some flaws if held against the standards of a scientific study, as a journalistic piece it goes a long way to justify its central accusation that there are predatory open access journals. He does not claim that there are no or evwen less predatory journals in the tradional sector (although it seems reasonable to believe that it might seem easier to predatory publishers to dupe unsuspecting scientists rather than subscription paying librarians). It demonstrates that open access is not a cure for all the problems besetting acacemic publishing. I think more deeply about it, it shows that author fees for publication may create a buisiness model just as open to abouse as the traditional subscription system. One answer might be to make the peer-review process more transparent, i.e. name the reviewers But that of course has other drawbacks.

The concept of student peer review has gained increased attention in higher education in recent years, in line with the growing focus on peer and collaborative learning (Falchikov, 2005; Van den Berg et al.; 2006). Peer review is a form of peer assessment that has been used for over three decades in a wide range of disciplines from Architecture to Computer Science to Music, just to name a few.

Every now and then a scholarly journal retracts an article because of errors or outright fraud. In academic circles, and sometimes beyond, each retraction is a big deal. Now comes word of a journal retracting 60 articles at once.

Thank you for sharing! The article highlights a problem of Internet where one can create any number of digital identities and use them for various purposes hoping that this will go unnoticed. This time the activity takes place in the settings of a scientific journal peer review.

This provides blog post provides an interesting comparison of the post-publication review process of F1000 Research to the traditional peer review model. What I really found interesting is the author's hypothesis that the anonymity of traditional peer review might benefit science.

Un post (en inglés) en el que se hace una crítica sobre el proceso de revisión por pares para las publicaciones científicas.

En realidad, muchas veces el sistema de revisión queda un poco en entredicho cuando el peso de uno de los autores puede hacer que el artículo se publique casi sólo. Otras veces, al proponer a revisores, lógicamente la propuesta se hace a revisores amigos que pueden ser algo más subjetivos a la hora de revisar tu artículo. Sin embargo, cuando empiezas de cero, y tu nombre no tiene aún ningún peso, tienes que tener un trabajo muy muy bueno para que pueda ser aceptado. Aún así, parece ser el método menos malo que conocemos para garantizar la objetividad. Esto no quita a que hay revisores muy buenos, de los cuáles se aprende y el artículo enviado se ve enriquecido. Y, como digo, también es positivo la rapidez a la hora de evaluar un artículo, teniendo en cuenta que el trabajo de revisión es siempre desinteresado (yo intento revisar en las fechas que me imponen con el objetivo de que cuando yo mande un nuevo trabajo susceptible de ser publicado, pueda ser revisado también en los plazos estipulados por la revista). Saludos. Antonio.

Peer review citation ring exposed in major scholarly publisher SAGE.

Sad and interesting situation here, scientist publishes his research, and in the comments, his work is called into question and it costs him a job. Brings into question the integrity of pre-publication peer review, and casts a shadow on the future of comments. Frequently, the comments are made by anonymous users, which itself should be a problem, even if the comment brings to light errors in the research. Now it seems, entire websites have sprung up with the sole purpose of debunking the hard work of others, sometimes needlessly. It's a good read.

Thank you for sharing. The situation with PubPeer actually encounters problems which are common on internet. As long as one comments anonymously, there are many off-topic and offensive comments. As soon as only registered comments are welcome, there are very few of them or not at all. Besides, one can presume that some scientific communities, even world-wide, are quite small and because of different reasons people do not want to comment under their real names... Another interesting resource to explore.

A digital journal of learning, teaching and technology. You can read interesting articles on MOOCs, digital writing, peer review and other relevant topics, as well as future of open knowledge.

This is a useful guide for peer reviewers.

DESIDOC Journal of Library & Information Technology (DJLIT), is an international, peer-reviewed, open access jounal that endeavours to bring recent developments in information technology, as applicable to library and information science. It is meant for librarians, documentation and information professionals, researchers students and others interested in the field. It is published bimonthly. It was formerly known as 'DESIDOC Bulletin of Information Technology (DBIT)'.

This article employs queer theory and challenges the notion of fitting emergent open access practices within current frameworks of academic success. While I was partially surprised by some of the assertions made early on in the article regarding open access journals being perceived as not as valuable as more traditional journal models, I think in part I may just hang out in academic circles that gravitate towards open access (hence... this course). But, I am very compelled by the conclusions made by Gurfinkel. That is, rather than trying to figure out how to systematize open access models to be respected within current academic standards, open access (as informed by queer theory, in this article) challenges us to investigate and question our standards in a more radical way. For example, in open access peer review models or post-publication review, the notion of a "peer" and thus who are considered credible and worthy sources of knowledge--and consequentially, what "knowledge" is-- are put into question. So, more than trying to figure out how to systematize and make more "legitimate" open access models, Gurfinkel wants us to ask what about the academy currently excludes open access models from being meaningful and legitimate practices in the first place.