Group items tagged

"If, as rumoured, google builds a platform for depositing unrefereed research papers for "peer-reviewing" via crowd-sourcing, can this create a substitute for classical peer-review or will it merely supplement classical peer review with crowd-sourcing?"

Two facts that makes me think, peer-reviewing via crowd-sourcing, at best would supplement the traditional peer-review process. Fact one, there are already open access repositories that allow "deposit first; review later", but those repositories have not taken over other journals. Fact two, Wikipedia is an example in that, though theoretically anyone can contribute and edit the articles, there is definite number of people who would do it. Therefore, I don't see crowd sourcing peer review would really substitute the traditional route.

I appreciated that this source was framed outside of dichotomous thinking by not pitting more traditional and open access peer review models directly against one another, carrying the assumption that a particular publishing process must choose one or another. Although, I think I would challenge Harnad to take this thought process further. Rather than supplementing or complementing one another, traditional and open peer review models are distinct enough to also be applicable in different types of contexts, without necessarily needing to rely on one another. That is not to disagree with Harnad that the two do not "substitute" one another, but precisely because they cannot substitute one another indicates that they serve different purposes and could thus be useful in different contexts…. Or, as Harnad suggest, supplement each other in the same context. I think this very well parallels the context of taxonomies and folksonomies.

Online discussions and post-publication analyses are catching mistakes that sneak past editorial review. This article describes the pitfalls with editorial review and pre-publication peer review, and advocates for post-publication crowd-sourced reviewing through social media platforms.

I think having a comments section is a great way to provide feedback on the information provided. Often when I read articles the comments section allows me to understand different perspectives and interpretations of the information.

This article, while not necessarily explicitly, managed to hint at what I find to be a source of problematic practices/outcomes in the academy, publishing, etc. That is, it is not necessarily that traditional peer review processes are ineffective at finding errors or misconduct, but rather it is when our processes and practices become so systematized that we can mindlessly or effortlessly engage in and reproduce them without our full, critical attention that they can produce problems. While I think there are good reasons to critique the notion of peer and "expert" culture within traditional peer review processes, an additional and separate critique is the problems that arise with systematization. The article implicitly addressed this when the author commented that current post-publication environments "provide a public space that is not under the control of journal editors and conference organizers." Yet, as White indicates, there exists skepticism of the value of post-publication reviews along with a simultaneous effort to build post-publication systems that have standards that put those questioning it at ease. The National Institutes of Health establishing requirements that potential post-publication reviewers must meet demonstrated this. That is, they are trying to figure out how to systematize post-publication. For me, what this article indicates is that we ought to figure out how to keep our academic and publishing processes "fresh," so to speak. This way we don't become so comfortable with our methods and practices that they allow us to simply go through the motions without fostering innovative and critical inquiry.

Publicado el 7/5/2012 This is a brief overview of several Open Peer Review Models, including ETAI, Nature, ACP, PLoS One and EJCBS. It is recorded based on a Prezi Presentation first developed for Open Access Week 2011 at UBC.

I found this presentation, in particular the visual representations, to be very useful in understanding just how diverse open peer review models can be. Several things stuck out to me throughout the presentation. First, I was surprised that many of the open peer review models either maintained anonymity of the reviewers or self-identification was optional. For example, PLOSone and the Atmospheric Chemistry and Physics journals did not require self-identification. This raises a tension for me in that it does enable more people to participate in the publishing/review process, but it still inherently indicates context does not matter, which is something I disagree with. That is, if, for example, a paper is on student-faculty partnerships or feminism, it seems to me that crucial insights pertain to the particularities of the people reviewing an article. The other aspect that stuck out to me was how crucial it is for a journal to be intentional about implementing, integrating, and valuing an open peer review process. The Nature experiment is a good example of this. While I am sure they spent a great deal of time figuring out how to construct and enable an open peer review process, it was not necessarily emphasized as important by the journal nor well integrated into people's current practice. In contrast, the ETAI did this by permanently archiving the peer comments rather than deleting them unannounced, and editors also sent notifications to people that articles were ready rather than assuming people would seek out articles themselves.

The issue of subjectivity in peer reviewing an open access journal article where the name of the author is disclosed.

What I appreciated most about Costa's account of her first time experiences with an open peer review where author/reviewer are known to one other is that the changes it invoked in her behavior ought to have been possible in under traditional peer review. There is another article in the Diigo, Is Social Media Saving Science?, where I discuss this a bit, but what Costa's comment highlight is that traditional peer review processes are partially problematic simply because we've become too comfortable with the process, enabling us to take shortcuts. That is, we know what our responsibilities and duties are to one another as peers, but we are not fulfilling them because there are not external pressures. I agree with Costa's insights. Simultaneously, I find it concerning that there is a need for "peer pressure," in a sense, for us to fulfill our responsibilities. It makes me question how we can change our practices in a way that make us actually want to do our best, regardless of external pressures. For me, this raises very big picture questions regarding how we can change the meaning of work so that it doesn't invoke us to cut corners because we are not wholly invested and/or enjoying how we are spending our time.

"Think of your meanest high school mean girl at her most gleefully, underminingly vicious. Now give her a doctorate in your discipline, and a modicum of power over your future. That's peer review."

Great article. Funny, descriptive and useful. Anyone non-academic who works with academics should understand the pressure of peer review and this article introduces it wonderfully. I want to dig deeper into this idea of open peer review.

See my bookmark to: 'Open peer review is a welcome step towards transparency ...'

Of the 255 papers that underwent the entire editing process to acceptance or rejection, about 60% of the final decisions occurred with no sign of peer review. For rejections, that's good news: It means that the journal's quality control was high enough that the editor examined the paper and declined it rather than send it out for review.

This article is certainly controversial, and I believe in some way did a service to the Open Access community by highlighting the practice of predatory journals. However, the irony of Bohannon's article, being an example of the kind of "bad science" he describes in his own article is inescapable. First, there is no randomization of his "experimental group", and there is no control group; second, there was elimination of non-responders; third, there was no application of the intention to treat principle in the analysis; and finally there were no inferential statistics and no references! Using his own standard, there is nothing that can be concluded from his study. For the criticism regarding Bohannon's targeting of OA journals exclusively, it is important to note that this experiment has been done before with 'traditional' journals as well- and many of them failed the test of peer review. http://www.slate.com/blogs/future_tense/2014/02/27/how_nonsense_papers_ended_up_in_respected_scientific_journals.html

I think Bohannens "study" should be considered more "investigative journalism" than scientific study. While it may have some flaws if held against the standards of a scientific study, as a journalistic piece it goes a long way to justify its central accusation that there are predatory open access journals. He does not claim that there are no or evwen less predatory journals in the tradional sector (although it seems reasonable to believe that it might seem easier to predatory publishers to dupe unsuspecting scientists rather than subscription paying librarians). It demonstrates that open access is not a cure for all the problems besetting acacemic publishing. I think more deeply about it, it shows that author fees for publication may create a buisiness model just as open to abouse as the traditional subscription system. One answer might be to make the peer-review process more transparent, i.e. name the reviewers But that of course has other drawbacks.

This provides blog post provides an interesting comparison of the post-publication review process of F1000 Research to the traditional peer review model. What I really found interesting is the author's hypothesis that the anonymity of traditional peer review might benefit science.

From Book Censorship to Academic Peer Review Mario Biagioli Together with tenure, peer review is probably the most distinctive feature of the modern academic system. Peer review, we are told, sets academia apart from all other professions by construing value through peer judgment, not market dynamics. Given the remarkable epistemological and symbolic burden placed on peer review, it is surprising to find that so little research has analyzed it either empirically (in its actual daily practices) or philosophically (as one of the conditions of possibility of academic knowledge).

This article provides a brief historical contextualization of different forms of peer review. It does well to highlight not only pros and cons of the various processes, but it also discusses the positions from which these pros and cons come form. For example, gender is discussed, revealing that women frequently advocate for anonymous peer review due to the sexism they encounter leading to their work not being published. Another aspect that the authors engage with is how the technologies available shape the forms that peer review takes.

From the Guardian: "How open-access journals are changing the field of peer-reviewed science"

that's an experience that can lead the way to scientific publishing after that of open archiving:http://scoap3.org/

I think there are real benefits to make the research available to everybody. Because. most of those research works are financed with tax payers money directly or indirectly. So why should there be this private monopolies milking the society and the scientists and blocking the knowledge to be spread?!!!

"SAGE Open is a peer-reviewed, "Gold" open access journal from SAGE that publishes original research and review articles in an interactive, open access format. Articles may span the full spectrum of the social and behavioral sciences and the humanities."

A Review of Cybertypes: Race, Ethnicity, and Identity on the Internet Lisa NakamuraLondon: Routledge, 2002 ISBN: 0415938376 $18.95 pp. 192 Review by Samantha Blackmon Purdue University Lisa Nakamura's Cybertypes: Race, Ethnicity, and Identity on the Internet aims to interrogate how the Internet shapes and reshapes our perceptions of race, ethnicity, and identity.

This is a useful guide for peer reviewers.

Every now and then a scholarly journal retracts an article because of errors or outright fraud. In academic circles, and sometimes beyond, each retraction is a big deal. Now comes word of a journal retracting 60 articles at once.

Thank you for sharing! The article highlights a problem of Internet where one can create any number of digital identities and use them for various purposes hoping that this will go unnoticed. This time the activity takes place in the settings of a scientific journal peer review.

An online review and ratings of the Terms of Service from different social media sites, including Facebook, Twitter, and more. "Terms of service are often too long to read, but it's important to understand what's in them. Your rights online depend on them. We hope that our ratings can help you get informed about your rights. "

I think people really need this online review because most of people do not have patience to read the long list of Terms of service but it does not mean they do not care about their rights online. This tool is great, I would like to use it and share to all of my friends. Hope more people could know that such a great tool exists.

This snapshot of terms of service for the various social media sites is excellent. Bookmarked!

DOAJ is an online directory that indexes and provides access to quality open access, peer-reviewed journals.

DOAJ is an online directory that indexes and provides access to quality open access, peer-reviewed journals.

I like the directory as it will allow access to the type of information that the world needs at no cost at all. Well done!

"Instead of calling for more money to prop up a traditional model that was never particularly viable in the first place, we need to embrace a variety of alternatives. Academic librarians are well positioned to lead the way here, both because of their long history of managing change and because they often hold the purse strings."

Un post (en inglés) en el que se hace una crítica sobre el proceso de revisión por pares para las publicaciones científicas.

En realidad, muchas veces el sistema de revisión queda un poco en entredicho cuando el peso de uno de los autores puede hacer que el artículo se publique casi sólo. Otras veces, al proponer a revisores, lógicamente la propuesta se hace a revisores amigos que pueden ser algo más subjetivos a la hora de revisar tu artículo. Sin embargo, cuando empiezas de cero, y tu nombre no tiene aún ningún peso, tienes que tener un trabajo muy muy bueno para que pueda ser aceptado. Aún así, parece ser el método menos malo que conocemos para garantizar la objetividad. Esto no quita a que hay revisores muy buenos, de los cuáles se aprende y el artículo enviado se ve enriquecido. Y, como digo, también es positivo la rapidez a la hora de evaluar un artículo, teniendo en cuenta que el trabajo de revisión es siempre desinteresado (yo intento revisar en las fechas que me imponen con el objetivo de que cuando yo mande un nuevo trabajo susceptible de ser publicado, pueda ser revisado también en los plazos estipulados por la revista). Saludos. Antonio.

The concept of student peer review has gained increased attention in higher education in recent years, in line with the growing focus on peer and collaborative learning (Falchikov, 2005; Van den Berg et al.; 2006). Peer review is a form of peer assessment that has been used for over three decades in a wide range of disciplines from Architecture to Computer Science to Music, just to name a few.