Benching for Health Literature Searches

Email from Paula Clark, Health Services Research Librarian, Institute for Clinical Evaluative Sciences, Toronto, Ontario

I received many requests to post the responses for benchmarking literature searches in the health sciences, and I’m happy to share.

I didn’t receive or find a specific guidelines for establishing timelines/hours for a search, no doubt because of the variability of any search topic.

However PRESS (Peer Review of Electronic Search Strategies) is a tool recently developed by a team at CADTH (Canadian Agency for Drugs and Technology in Health) that provides a checklist to assess the quality of search strategies: http://www.cadth.ca/index.php/en/publication/781

Here are the responses:

- there may be a few studies that could be helpful. At one time I did what ship radio shack people do, and wrote down every 5 and 15 minute intervals what I did. (ship radio operators log every minute and every 5 minutes what they hear on the shortwaves.) Then produced a report as to what kinds of work I did, what the percentage was for the week of each activity. Also by having the time logged as well, you get a good idea of the heaviest times of the day (well, when the most requests come in.)
http://www.annals.org/content/144/10/742.full
http://www.rcjournal.com/contents/05.10/05.10.0578.pdf

- Both MLA and VALNET have done benchmarking. VALNET never differentiated in types of searches. We just counted searches.

- [re: MLA Benchmarking project] The last Benchmarking survey was published in 2008 and we would normally do another one in 2011. The Board will be discussing this at their meeting in October, but for now we have the 2008 data.

- I don’t know that you can benchmark actual hours spent/required, but you might be able to benchmark timelines. I work just on orthopaedic practice guidelines—fairly well-defined, right? But it is all over the map depending on the topic selected, limits, scope, etc. each work group decides on. However, for establishing the guideline development timetable, we regularly budget at least one month for the search and one month for the abstract review and article retrieval. Within that timeline, I might spend more or fewer hours each guideline, depending on the search parameters

And here is a list of resources I came across that were helpful/interesting:

MLA Benchmarking Network:

Website: http://www.mlanet.org/resources/benchmark07/definitions.html

Article: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1435860/

Library Activity Benchmarking Table : http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1435842/table/i1536-5050-094-02-0118-t01/
(taken from this report on the results of the MLA Benchmarking Network: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1435842)

Standards :

Standards for Hospital Libraries 2007:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2268237/

Articles :

The Value of Hospital Library Benchmarking
http://www.informaworld.com/smpp/1021028687-7766837/content~db=all~content=a903922501~frm=titlelink

The Buck Starts Here : Using In-House Reports to Market your Library
http://www.informaworld.com/smpp/content~db=all~content=a902276607

Using standards to make your case: examples from the medical library community
http://www.emeraldinsight.com/journals.htm?articleid=1728032&show=abstract&