Studying the Accuracy of Software Process Elicitation: The User Articulated Model

Process models are often the basis for demonstrating improvement and compliance in software engineering organizations. A descriptive model is a type of process model describing the human activities in software development that actually occur. The purpose of a descriptive model is to provide a documented baseline for further process improvement analysis. Organizations may hire experts or train employees to create a descriptive model but the costs are often exorbitant. As a result, some companies skip steps necessary for ensuring that employees are equipped to deal with the challenges of creating a descriptive model. Unfortunately, a descriptive model created by inexperienced practitioners usually does not contain information necessary for informed decision making. Ideally, a descriptive model provides an explicit representation of human centric software processes. However, if the model does not represent how a process is actually performed, subsequent recommendations for improvement may be based upon activities that are depicted in the model but that do not actually take place. Similarly, a descriptive model may omit important features and characteristics that are centrally relevant for an organization's process improvement goals. The accuracy of software process elicitation is an important topic for study given this context. Accuracy, is the degree to which a descriptive model reflects the actual process. Prior studies address accuracy from the perspective of an expert working with a group of process performers. However, given the problems in practice, it is surprising that there is no identified research studying accuracy from the perspective of the user. This dissertation provides a synthesis of arguments from related literature that inform a research approach for studying accuracy from the perspective of the user creating a descriptive model. A series of experiments are conducted with test subjects in professional training settings. Hypothesis testing reveals that subjects who use a conceptual model to articulate process knowledge perceive more error between description and performance then those who do not. The results also suggest that written text can be captured to provide insight into a new important property of a process model, Coherence.