Determining Success: An Exploration of the Critical Success Criteria for the Design, Production and Implementation of Computer Based Education and Training

Abstract

This study is aimed at providing insight into the reasons for the uneven adoption of the use of computer based training in education and training. To achieve this goal, applied research directed at identifying the critical success criteria required for the introduction and sustained use of computer based training (CBT) and education (CBE) was undertaken. The research methodology for the study made use of the Analytical Hierarchy Process (AHP) after Saaty (1990) and involved the construction and examination of a three-level hierarchical matrix of criteria associated with the introduction, and sustained usage of CBT. These three levels were comprised of a factor level (economics, policy, technology, pedagogy, organisation), an activities level (design, production, implementation, evaluation) and an interactive components level (hardware, software, personnel). The first phase of this study, involving 44 GET professionals, produced profiles detailing what these practitioners believed to be the critical criteria for the successful introduction and use of GET. These profiles were derived for three hypothetical scenarios representing successful small, medium and large scale developers/users of CBT. These profiles (defined as scenario matrix dominance profiles) showed a distinct evolutionary pattern from small scale CBT activities dominated by economic factors, to large scale GET operations dominated by organisational and policy factors. In the second phase of the study, interviews were conducted involving key individuals from the 19 organisations involved in the studyTs first phase of interviews. Individuals were asked to examine the CBT success criteria with respect to what influenced the development and use of CBT in their organisations. The data collected from these interviews was used to produce additional sets of actual matrix dominance profiles for the participating organisations. Each organisation's actual matrix dominance profile (representing the existing mix of influential criteria) was then compared with the corresponding derived scenario matrix dominance profiles (representing the ideal mix of critical criteria) from the first round of interviews. The examination and interpretation of the scenario matrix dominance profiles, and the subsequent comparisons with the actual matrix dominance profiles, provided a detailed insight into both individual success criteria and their inter-relationships with other criteria in the hierarchy. At the system level, this comparative analysis revealed that there were five key criteria identified as being important in contributing to the success of CBT. These five criteria are: At the factor level: 1. Organisation - commitment at all levels of an organisation, 2.Economics - identifiable economic returns, 3.Policy - identifiable policy dictates concerning instructional technologies. At the activity level: 4. Adherence to instructional systems design principles with particular importance placed on Design and Implementation. At the component level: 5. Personnel - skilled professional staff. The results of this study would suggest that it is the lack of attention to (or the inability to address) these key elements that is inhibiting the introduction and use of CBT technologies on a much wider scale. Further, it would appear that these critical success criteria, in association with the remaining identified criteria, need to be addressed in a multidisciplinary framework ( eg. instructional design, human resource management, training and economic management) in order to significantly increase the use of CBT as a teaching and training strategy. The study concludes with the discussion of potential research areas which could expand upon the findings of this examination of GET success criteria and promote the more effective use of CBT in industry and education.