Group items tagged

The impact of scientific publications has traditionally been expressed in terms of citation counts. However, scientific activity has moved online over the past decade. To better capture scientific impact in the digital era, a variety of new impact measure

A model from the Becker Medical Library at Washington University in St. Louis, to aid in tracking and documenting biomedical research impact through a series of impacts not limited to publication, with additional resources including a glossary, strategies for improving impact and a reference library.

* A common reason for measuring the impact of research is to demonstrate accountability, but results of measuring can also be used to guide improvements in research and programming. * Health research impacts generally include: knowledge production; research capacity-building; informed decision-making; health and health sector benefits; and economic benefits. * Among some of the widely used methods for measuring the benefits from research are bibliometric analysis, economic rate of return, peer review, case studies, logic modelling, and benchmarking. Taking a multi-indicator, multi-method approach is advised.

Formerly, the impact of authors and their scientific production was measured by the average citation frequencies of journals publishing their research: the Journal Impact Factor (JIF), calculated by the Institute for Scientific Information (ISI) in the United States and published annually in the Journal Citation Reports (JCR)-the most frequently used quantitative indicator to measure the quality/value/impact of research works published in the core international journals. It has been suggested that, by calculating the number of webpages pointing to a given site, analogously, a Web Impact Factor can be calculated as a way of comparing the attractiveness of sites or domains on the World Wide Web.

...The RQF differs from existing international research assessment methods by considering res- earch impact in addition to the more conven- tional quality measures normally used in the academic community. This inclusion has created some controversy. Detractors argue that the inclusion of impact devalues the assessment process by moving beyond the scholarly domain, and that there may be undue emphasis on research that can demonstrably show shorter- term economic or other gains. An alternative view is that the absence of an assessment of impact seriously unbalances the evaluation of research and its importance to national and global priorities....

Case studies proven to be the best method in demonstrating impact outside of academia; impact will now be an integral part of the REF Program.

'Maximizing The Impacts Of Your Research: A Handbook For Social Scientists' now available to download as a PDF

Open Society Foundations funded project that aims to facilitate a conversation between researchers, their funders, and developers about what we mean by the "impact" of research and how we can make its measurement more reliable, more useful, and more accepted by the research community

Abstract: "Traditionally, scholarly impact and visibility have been measured by counting publications and citations in the scholarly literature. However, increasingly scholars are also visible on the Web, establishing presences in a growing variety of social ecosystems. But how wide and established is this presence, and how do measures of social Web impact relate to their more traditional counterparts? To answer this, we sampled 57 presenters from the 2010 Leiden STI Conference, gathering publication and citations counts as well as data from the presenters' Web "footprints." We found Web presence widespread and diverse: 84% of scholars had homepages, 70% were on LinkedIn, 23% had public Google Scholar profiles, and 16% were on Twitter. For sampled scholars' publications, social reference manager bookmarks were compared to Scopus and Web of Science citations; we found that Mendeley covers more than 80% of sampled articles, and that Mendeley bookmarks are significantly correlated (r=.45) to Scopus citation counts. " "Accepted to 17th International Conference on Science and Technology Indicators, Montreal, Canada, 5-8 Sept. 2012."

From the abstract: ", we propose a social framework based on crowdsourced annotations of scholars, designed to keep up with the rapidly evolving disciplinary and interdisciplinary landscape. We describe a system called Scholarometer, which provides a service to scholars by computing citation-based impact measures. This creates an incentive for users to provide disciplinary annotations of authors, which in turn can be used to compute disciplinary metrics. We first present the system architecture and several heuristics to deal with noisy bibliographic and annotation data. We report on data sharing and interactive visualization services enabled by Scholarometer. Usage statistics, illustrating the data collected and shared through the framework, suggest that the proposed crowdsourcing approach can be successful. Secondly, we illustrate how the disciplinary bibliometric indicators elicited by Scholarometer allow us to implement for the first time a universal impact measure proposed in the literature. Our evaluation suggests that this metric provides an effective means for comparing scholarly impact across disciplinary boundaries."

(Note: See the .pdf attached to this media release for the various models suggested for analyzing impact) It is often stated that the most difficult task in any research project is not the achievement of results, but the accurate analysis of the benefits that those outcomes reap. The Australian Government's Cooperative Research Centres (CRC) Programme has been a leading light in the discovery and industry-focused application of new technologies and techniques. Now, with the release of a landmark guidebook by the CRC Association, scientists are armed with tools to greatly improve their ability to accurately assess the benefit to the economy of their outputs.

ReaderMeter: Crowdsourcing research impact September 22nd, 2010

manipulation of impact factors?

from the abstract: "assessing the influence of spatial proximity between scientists is crucial to promote efficient collaboration strategies and, ultimately, to improve the quality of science. Here we present a systematic analysis of citation and collaboration streams between cities and countries. By assigning papers to the geographic locations of their authors' affiliations, we construct weighted networks of citations and collaborations. The citation flows as well as the collaboration strengths between cities decrease with the distance between them and follow gravity laws with exponents close to 1. Moreover, for a given number of authors, the diversity of affiliations increases the number of citations, especially when many countries are represented. In addition, the total research impact of a country grows linearly with the amount of national funding for research & development. However, the average impact reveals a peculiar threshold effect: the scientific output of a country may reach an impact larger than the world average only if the country invests more than 120,000 US $ per researcher annually. Our results reveal the overall structure of scientific research by showing the correlation between collaboration, citation, geography and funding, and could provide valuable inputs in shaping the future science policies."

Scholarometer(beta) is a social tool to facilitate citation analysis and help evaluate the impact of an author's publications. Developed at Indiana University School of Informatics and Computing

OpenAccess Statistics: Alternative Impact Measures for Open Access documents? An examination how to generate interoperable usage information from distributed Open Access services

Gargouri Y, Hajjem C, Larivière V, Gingras Y, Carr L, Brody T, Harnad S. Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research. PLoS ONE. 2010;5(10):e13636+. doi:10.1371/journal.pone.0013636.

In February 2009, the Higher Education Funding Council for England (HEFCE) commissioned RAND Europe to review approaches to evaluating the impact of research as part of their wider work programme to develop new arrangements for the assessment and funding of research - referred to as the Research Excellence Framework (REF).

"Recent studies have begun to show that open access increases impact" "Also lists the web tools available to measure impact"