This is the second in an article series explaining the certification process. The first, "OSH Certifications: Behind the Exams" (PS, July 2017, pp. 44–48), addresses the overall process. This article focuses on how questions are created and included on the exam.
Developing test questions follows a rigorous process to ensure that a certification exam is valid, reliable, fair and practical.
The article is intended to help OSH professionals understand this rigor and how properly developed and scrutinized exam questions help to measure the mark of excellence in the OSH field.
For more than 40 years, the multiple-choice examination has been the standardized assessment tool used in the certification process of OSH professionals (Wright, Turnbeaugh, Weldon, et al., 2015). The use of a multiple-choice exam to award a credential, however, has been criticized by many OSH professionals. This may be primarily due to a perception that relates to their previous academic experience with multiple-choice exams and a misunderstanding of the science behind the development of such exams.
The use of standardized tests clearly ensures a consistent and rapid method of scoring, but the use of such tests is legally defensible only if the test is developed through a systematic, psychometric process that objectively measures the relevant skills and knowledge of the individuals being assessed (Wright, et al., 2015). These exams are not, as many perceive, developed solely by individual certificants intending to make the test questions as hard or as trivial as possible.
The process of establishing and delivering a high-quality certification examination involves several steps and many subject matter experts (SMEs), as well as extensive statistical evaluation. The process must generate an examination that is valid, reliable, fair and practical. Each component plays a role in the development of a high-quality examination for the certification process (Figure 1, p. 46).