Making summative online assessmenmt more secure: investigating time, content, and question type

Abstract

To date, online assessment has been widely used in the formative capacity, wherein the results of the assessment are not graded and the assessment may be retaken a specified number of times. In this capacity, formative online assessment has been used successfully as a diagnostic tool to provide for prompt feedback and interaction on assessments such as self-tests and pre-tests. In these applications, the concern over security is minimal since there is no compelling motivation for students to cheat. While sharing some of these positive features with online formative assessment, summative online assessment does have certain drawbacks, with the problem of student identity and the use of prohibited materials during the assessment among the major concerns. In terms of the latter concern, the time available to complete the assessment is one factor that may impact security; shorter exam times may result in less time to access prohibited materials. Another factor is the type of question type used in the assessment. For example, a multiple choice question may be more readily answered without the need to turn to prohibited materials than a true/false question. In addition, the inclusion of additional content in the exam such as a list of relevant terms again may factor into the need to reference prohibited materials. This paper describes the investigation of the effects of these three factors on performance for an online Astronomy course with the goal of establishing a potentially more secure summative online testing environment while not negatively impacting assessment performance. The results of the investigation are then compared to those obtained for identical exams taken in a proctored environment with the conclusion that the exam question type and content are the critical factors in determining exam performance and may be used to increase the security of an online testing environment.