Validity Assessment of an Experimental e-Shopping Interface and Survey in a Multicultural Context

Within a context of rapidly expanding international Internet penetration, organizations must offer multicultural versions of their information systems. In multicultural countries such as Canada, enterprises recognize the benefits of conducting e-business in a multilingual environment. As well, more and more IS research is conducted within a multicultural context. However, previous research has highlighted the difficulties of IS internationalization: differences in verbal style and word interpretation greatly increase the risk of miscommunication. In the case of multicultural IS research, miscommunication problems threaten the validity of the results. Internet-based shopping interfaces offer the opportunity of selling products and services across national and geographical boundaries. However, these interfaces must be adapted to multicultural settings. A research about the cognitive fit of two e-shopping interfaces was conducted in a multicultural context. For each of eight experimental conditions, two equivalent versions, in English and in French, were prepared. The subjects shopped online for a television set then answered a survey about their shopping experience. 561 consumers participated in the first study: 399 answered in English and 162 in French. The second study, a protocol analysis, required 39 consumers (13 English-speaking and 26 French-speaking) to complete the same online experiment while performing a think-aloud protocol. To ensure that results could be pooled across the two cultural contexts, great care was taken to ensure that a) each cultural version of the interface was valid and controlled for confounding variables, b) each cultural version of the survey met strict criteria of convergent and discriminating validity, but also c) that the two cultural versions were in every respect equivalent to each other. This third point was critical to the success of the experiment. To achieve this goal, 8 pre-tests were conducted. This paper presents the methodology used to ensure experimental interface and survey validity as well as their perfect equivalence. This research methodology represents a contribution to IS researchers as well as practitioners who internationalize web-based applications and must ensure their cultural context equivalence