Overview
Are digital resources succeeding at reaching their intended users, or are they being missed because of poor information seeking skills among the target users?
There are a number of challenges in assessing the use and impact of online digital resources: these include new methods, shifts in the way that people access resources, new audiences, and new forms of information-seeking behaviour among different audiences.
Quantitative and Qualitative Methods
This project combined quantitative and qualitative indicators to measure the impact of online scholarly resources. Quantitative measures included webometrics, log file analysis, scientometric (or bibliometric) analysis, and content analysis. These were complemented by an array of qualitative measures (stakeholder interviews, resource surveys, user feedback, focus groups, and questionnnaires) that captured information about the whole cycle of usage and impact.
The qualitative measures allow us to examine the impact of projects from the point of view of various stakeholders, starting with the host institutions such as libraries and archives, the personnel at the host institutions responsible for implementation (including the developers and engineers of the systems and curators and archivists of the collections), and stretching all the way to the various types of end users and the uses they represent. Measures have ben included that involve programme funders, who are often overlooked as stakeholders.
The Toolkit for the Impact of Digitised Scholarly Resources (TIDSR)
We have developed a best practices toolkit, the Toolkit for the Impact of Digitised Scholarly Resources (TIDSR), for the assessment of the impact of digitisation projects. This toolkit includes a number of elements which will aid other researchers and funding bodies to assess the impact of digitisation projects. The toolkit contains measures that have been judged to be effective measures of impact. The toolkit will help funding bodies, and others, make recommendations (on the basis of our qualitative and quantitative findings) for the construction of digital project sites, precisely in order to enhance the measurability of impact.
Testing the TIDSR toolkit
The digitisation projects that served as the test of the best practices toolkit were:
The potential impact of this project extends beyond these specific collections to a broader context. The evaluation of online scholarship is a moving target, and therefore a flexible set of measures and practices has been used: the toolkit doesn’t consist of a single software solution, but a set of recommendations for best practices. It also includes historical change data developed as part of the JISC-funded OII project: World Wide Web of Humanities.
Support
This project was funded by the Joint Information Systems Committee (JISC).
