Group items tagged

...The RQF differs from existing international research assessment methods by considering res- earch impact in addition to the more conven- tional quality measures normally used in the academic community. This inclusion has created some controversy. Detractors argue that the inclusion of impact devalues the assessment process by moving beyond the scholarly domain, and that there may be undue emphasis on research that can demonstrably show shorter- term economic or other gains. An alternative view is that the absence of an assessment of impact seriously unbalances the evaluation of research and its importance to national and global priorities....

Science-Metrix is an independent research evaluation firm specializing in the assessment of science and technology (S&T) organizations and activities. Our services enable evidence-based decision-making, strategic planning, and outcome assessments.

"Long-term preservation and stewardship of scientific data and research-related information are vitally important to future science and scholarship. Scientific data archives can offer capabilities for managing and preserving disciplinary and interdisciplinary data for research, education, and decision-making activities of future communities of users. Meeting the requirements for a trusted digital repository will help to ensure that today's collections of scientific data will be available in the future. A continuing self-assessment of a long-term archive for interdisciplinary scientific data is being conducted to identify the additional steps needed for it to become a trustworthy repository. Recommendations include a strategy for collaborative organizational sustainability, a model for submission and workflow to ingest interdisciplinary scientific data into a repository, and a plan for facilitating intra-organizational transfer between repositories."

A model from the Becker Medical Library at Washington University in St. Louis, to aid in tracking and documenting biomedical research impact through a series of impacts not limited to publication, with additional resources including a glossary, strategies for improving impact and a reference library.

"The combination of Elsevier's and Collexis' knowledge discovery technologies will create a powerful set of applications designed to support scientific management and assessment." Publishers are offering more and more tools, e.g. Thomson Reuter's InCites and SciVal.

A strategy used for scientific data curation needs assessment used at the University of oregon is demonstrated.

The Public Library of Science (PLoS) is the first publisher to place transparent and comprehensive information about the usage and reach of published articles onto the articles themselves, so that the entire academic community can assess their value. We call these measures for evaluating articles 'Article-Level Metrics', and they are distinct from the journal-level measures of research quality that have traditionally been made available until now.

An excellent overview of the profusion of measures and metrics, among other topics.

Now some humanities scholars have begun to challenge the monopoly that peer review has on admission to career-making journals and, as a consequence, to the charmed circle of tenured academe. They argue that in an era of digital media there is a better way to assess the quality of work.

IU Bloomington School of Informatics and Computing associate professor Johan Bollen and the National Information Standards Organization (NISO) will share the Mellon Foundation grant designed to build upon the Metrics from Scholarly Usage of Resources (MESUR) project that Bollen began in 2006

In February 2009, the Higher Education Funding Council for England (HEFCE) commissioned RAND Europe to review approaches to evaluating the impact of research as part of their wider work programme to develop new arrangements for the assessment and funding of research - referred to as the Research Excellence Framework (REF).

Bibliometric methods can be used to assess many types of impact. See the example from Science-Metrics and the Univeristy of Leiden Book of Abstracts.

"Cloud computing comes in several different forms and this article documents how service, platform, and infrastructure forms of cloud computing have been used to serve library needs. Following an overview of these uses the article discusses the experience of one library in migrating IT infrastructure to a cloud environment and concludes with a model for assessing cloud computing."

Some now predict that PLoS ONE's model will become the dominant one for scholarly journals. OA advocate Cameron Neylon, for instance, predicted that in future "most scholarly publishing will be in publication venues that place no value on a subjective assessment of 'importance'

"The data underlying scientific papers should be accessible to researchers both now and in the future, but how best can we ensure that these data are available? Here we examine the effectiveness of four approaches to data archiving: no stated archiving policy, recommending (but not requiring) archiving, and two versions of mandating data deposition at acceptance. We control for differences between data types by trying to obtain data from papers that use a single, widespread population genetic analysis, STRUCTURE. At one extreme, we found that mandated data archiving policies that require the inclusion of a data availability statement in the manuscript improve the odds of finding the data online almost a thousand-fold compared to having no policy. However, archiving rates at journals with less stringent policies were only very slightly higher than those with no policy at all. At one extreme, we found that mandated data archiving policies that require the inclusion of a data availability statement in the manuscript improve the odds of finding the data online almost a thousand fold compared to having no policy. However, archiving rates at journals with less stringent policies were only very slightly higher than those with no policy at all. We also assessed the effectiveness of asking for data directly from authors and obtained over half of the requested datasets, albeit with about 8 days delay and some disagreement with authors. Given the long term benefits of data accessibility to the academic community, we believe that journal based mandatory data archiving policies and mandatory data availability statements should be more widely adopted."

"The term "altmetrics" is short for "alternative metrics." These are a range of nontraditional metrics that can be used to assess the impact that scholars have on research in their areas of study. They can include the number of article downloads, citation of research in online news/social media sources, Mendeley bookmarks (a web-based system for sharing and extracting information from PDFs and other electronic documents), and nontraditional forms of scholarship. "

from the abstract: "assessing the influence of spatial proximity between scientists is crucial to promote efficient collaboration strategies and, ultimately, to improve the quality of science. Here we present a systematic analysis of citation and collaboration streams between cities and countries. By assigning papers to the geographic locations of their authors' affiliations, we construct weighted networks of citations and collaborations. The citation flows as well as the collaboration strengths between cities decrease with the distance between them and follow gravity laws with exponents close to 1. Moreover, for a given number of authors, the diversity of affiliations increases the number of citations, especially when many countries are represented. In addition, the total research impact of a country grows linearly with the amount of national funding for research & development. However, the average impact reveals a peculiar threshold effect: the scientific output of a country may reach an impact larger than the world average only if the country invests more than 120,000 US $ per researcher annually. Our results reveal the overall structure of scientific research by showing the correlation between collaboration, citation, geography and funding, and could provide valuable inputs in shaping the future science policies."

A review of a declaration signed by ACSB and other participants which urges consideration of research metrics beyond the journal impact factor.

(Note: See the .pdf attached to this media release for the various models suggested for analyzing impact) It is often stated that the most difficult task in any research project is not the achievement of results, but the accurate analysis of the benefits that those outcomes reap. The Australian Government's Cooperative Research Centres (CRC) Programme has been a leading light in the discovery and industry-focused application of new technologies and techniques. Now, with the release of a landmark guidebook by the CRC Association, scientists are armed with tools to greatly improve their ability to accurately assess the benefit to the economy of their outputs.