Organizations make large investments in Information Systems (IS) expecting positive impacts to the organisation and its functions. Yet, there exists much controversy surrounding the 'potential' impacts of these systems, with some studies reporting broadly positive impacts of IS across organizations (Barua, Kriebel and Mukhopadhyay 1995; Barua and Lee 1997; Brynjolfsson and Hitt 1996; Lehr and Lichtenberg 1999; Mukherjee, Ray and Miller 2001), while others have shown nil or detrimental impacts (Attewell and Rule 1984; Brynjolfsson and Yang 1996; Cameron and Quinn 1988; Wilson 1993). Various authors have suggested that these conflicting results may be due to poor measurement - E.g. incomplete or inappropriate measures of success (DeLone and McLean 1992; Gable 1996; Melone 1990), lack of theoretical grounding and hence agreement on appropriate measures of success (Bonner 1995; Myers, Kappelman and Prybutok 1998), myopic focus on financial performance indicators (Ballantine, Bonner, Levy, Martin, Munro and Powell 1996; Kaplan and Norton 1996), weaknesses in survey instruments employed (Gable, Sedera and Chan 2003) (e.g., constructs lacking in validity), or (5) inappropriate data collection approach (Seddon, Staples, Patnayakuni and Bowtell 1999; Sedera and Gable 2004) (e.g., asking the wrong people, unrepresentative sample). Enterprise Systems (ES) have over the past decade emerged to be one of the most important developments in the corporate use of information technology. Anecdotal evidence reveals discontent with these large application software packages. Yet Enterprise System investments are seldom systematically evaluated post-implementation; the review process and measures typically being idiosyncratic and lacking credibility. Impacts resulting from 'Enterprise Systems' are particularly difficult to measure, with an Enterprise System entailing many users ranging from top executives to data entry operators; many applications that span the organization; and a diversity of capabilities and functionality. Despite the substantial investments made by organizations and the anecdotal evidence of discontent, systematic attempts to measure their success have been few. The primary objective of this research is to develop and test a standardized instrument for measuring ES-Success. Other related objectives of this research include: (1) to identify the dimensions and measures of ES-Success, (2) to validate a maximally generalizable measurement model and survey instrument for gauging ES-Success; (3) to develop an understanding of the state of Enterprise Systems using descriptive/comparative statistics, and (4) to identify and test an antecedent of ES-Success. With the above objectives, and in attention to the weaknesses identified in past IS-success research, this study follows and extends the 'research cycle' guidelines of Mackenzie and House (1979) and McGrath (1979). The research cycle entails two main phases: (1) an exploratory phase to develop the hypothesized measurement model, and (2) a confirmatory phase, to test the hypothesized measurement model against new data. The two surveys (termed as identification-survey and specification-survey) conducted in the exploratory phase of this research go beyond the activities recommended by Mackenzie and House (1979) and McGrath (1979). A third "confirmation-survey" was completed in the confirmatory phase of the research cycle. The three surveys gathered and analyzed data from six hundred (600) respondents.The purpose of the identification-survey was to discover the salient ES-Success dimensions and measures to include in an a-priori ES-Success model. Data from 137 respondents representing 27 Australian State Government Agencies that had implemented SAP R/3 in the late 1990s were analyzed. The analysis of identification-survey data yielded an a-priori model with 41 measures of 5 dimensions of ES-Success that provide a holistic view across the organization from strategic to operational levels. The specification-survey was employed to validate the a-priori ES-Success measurement model derived in the preceding identification-survey. Employing 310 responses from the same 27 public sector organizations, exploratory data analysis validated 27 measures of success pertaining to the 4 dimensions: information quality, system quality, individual impact and organizational impact. Data for testing the influence of an antecedent of ES-Success was simultaneously gathered during the specification-survey. This analysis, based on the Adaptive Structuration Theory (AST), investigated the influence of Knowledge Management Structures Adequacy (KMSA) on ES-Success. Preliminary results indicate a strong relationship between the Knowledge Management Structures Adequacy and ES-Success. The purpose of the confirmation-survey was to further validate the dimensions and measures of the ES-Success model, using new data, employing confirmatory statistical techniques. Data was gathered from 153 respondents across a large University that had implemented the Oracle Enterprise System, which facilitated further construct validity of the ES-Success measurement instrument was further established using Structural Equation Modeling (SEM).
