Examination of work task and criteria choices for the relevance judgment process

Purpose – The purpose of this paper is to better understand the dynamic nature of the relevance judgment process and the influence of work task on that process. Design/methodology/approach – The empirical study reported here examined the information seeking behavior of a group of undergraduate college students assigned a set of research assignments (work tasks). Subjects recorded their selection of documents used for an assignment and the criteria used to judge those documents relevant. Statistical analysis was used to associate relevance judgments and the criteria used to make those judgments with work tasks. Findings – Findings indicate a strong statistical association between work task and criteria used to judge relevance. Findings also include identification of specific criteria used to judge relevance and the relative importance of those criteria based on frequency of selection of criteria for a work task. Research limitations/implications – Findings provide additional insights into the dynamic nature of the relevance judgment process. Relevance judgment influences revealed in these findings in the form of criteria used to make relevance judgments further explicate the relevance judgment process and provide suggestions for the improvement of information retrieval systems and information literacy efforts. Originality/value – Understanding the relevance judgment process is critical to understanding information behavior in general. Few studies have examined relevance criteria selections as part of the relevance judgment process and fewer still have studied these selections in relation to work tasks. A better understanding of this relationship is an essential part of understanding the dynamic nature of the relevance judgment process and its influences.