An Experimental Investigation into the Efficacy of Visual Interactive Simulation

The use of a Visual Interactive Simulation (VIS) model for experimental analysis, where the user initiates runs and gathers information as desired without necessarily any respect for formal analysis, is encouraged by some proponents of VIS and VIS software packages. Proponents of formal output analysis view this approach as dangerous and irresponsible. We designed and executed a laboratory experiment in which 51 subjects solved a case study based around the allocation of trucks in a mining operation in order to investigate the efficacy of VIS to model experimentation. Subjects were provided with a VIS, developed by the authors, which contained a terminating simulation of the system and two different displays: an animation and a dynamically changing histogram. The user could halt execution of the model and change the truck allocation at any time. We found that subjects performed badly relative to a known solution obtained through detailed formal experimentation but performed well compared to solutions they provided prior to use of the model. Use of the animated display was not associated with correct solutions but was associated with more efficient use of the VIS. Subjects who obtained a correct solution investigated fewer alternatives and used fewer interactions than those obtaining incorrect solutions. Finally, we found a significant difference in the process used between subjects providing correct and incorrect solutions.