Examination of relevance criteria choices and the information search process

Purpose – The purpose of this paper is to examine changes in relevance assessments, specifically the selection of relevance criteria by subjects as they move through the information search process. Design/methodology/approach – The paper examines the relevance criteria choices of 39 subjects in relation to search stage. Subjects were assigned a specific search task in a controlled test. Statistics were collected and analyzed using descriptive statistics and the chi‐square goodness‐of‐fit tests. Findings – The statistically significant findings identified a number of commonly reported relevance criteria, which varied over an information search process for relevant and partially relevant judgments. These results provide statistical confirmations of previous studies, and extend these findings identifying specific criteria for both relevant and partially relevant judgments. Research limitations/implications – The study only examines a short duration search process and since the convenience sample of subjects were from similar backgrounds and were assigned similar tasks, the study did not explicitly examine the impact of contextual factors such as user experience, background or task in relation to relevance criteria choices. Practical implications – The paper has implications for the development of search systems which are adaptive and recognize the cognitive changes which occur during the information search process. Examining and identifying relevance criteria beyond topicality and the importance of those criteria to a user can help in the generation of better search queries. Originality/value – The paper adds more rigorous statistical analysis to the study of relevance criteria and the information search process.