Comparing Google to Ask-a-Librarian Service for Answering Factual and Topical Questions

Abstract

This paper evaluates to which extent Google retrieved correct answers as responses to queries inferred from factual and topical requests in a digital Ask-a-Librarian service. 100 factual and 100 topical questions were picked from a digital reference service run by public libraries. The queries inferred simulated average Web queries. The top 10 retrieval results were observed for the answer. The inspection was stopped when the first correct answer was identified. Google retrieved correct answers to 42 % of the topical questions and 29 % of factual questions. Results concerning the characteristics of queries and retrieval effectiveness are also presented. Evaluations indicate that public libraries’ reference services answer correctly 55 % of the questions. Thus, Google is not outperforming Ask-a-Librarian service, although it seems to perform relatively satisfactory in retrieving answers to topical questions.