How Relevant is Fieldwork to DSS Design-Science Research?

Recent reviews of DSS research have indicated a need to improve its quality and relevance. Design science has an important role to play in this improvement as this research strategy can engage industry and the profession in intellectually important projects. DSS has a long history of design science, although most of this research was not conducted under this rubric. Recent publications have clarified what is required for quality outcomes in IS design science. A central issue in these publications is the evaluation of the design artifact, especially the use of the artifact in a field setting. Successful fieldwork is perhaps the most difficult, and most rewarding, aspect of design science. This paper presents research that used bibliometric content analysis to examine evaluation strategies, especially fieldwork, in a representative sample of 362 DSS design-science papers in 14 journals. The analysis found that DSS design-science papers whose artifacts are used in actual field environments are of significantly higher quality, have significantly higher organizational impact, and have significantly higher relevance with both IS professionals and managers. The major conclusion is that rigorous fieldwork should be the ultimate goal of DSS design-science evaluation.