Employing a transitivity violation detection algorithm to assess best-worst scaling designs

When choosing a partial factorial design for best-worst scaling (BWS) or other discrete choice experiment, researchers are faced with design size choices. This work investigates differences between two case 1 (object) BWS choice experiment designs that varied in choice set size and number of questions. Using a random-parameter logit model, preference shares were determined and statistically compared between models. The number of transitivity violations occurring between the experimental designs was analyzed employing a newly developed directed graph algorithm. The relative importance consumers placed on dairy milk attributes differed between the designs studied. The design presenting fewer attributes per choice set forced novel tradeoffs more often and yielded no increase in transitivity violations. In general, if one seeks to establish rank among variables and force tradeoffs, smaller designs should be considered.