Abstract
Cognitive diagnostic computerized adaptive testing (CD-CAT) has been suggested by
researchers as a diagnostic tool for assessment and evaluation. Although
model-based CD-CAT is relatively well researched in the context of large-scale
assessment systems, this type of system has not received the same degree of
research and development in small-scale settings, such as at the course-based
level, where this system would be the most useful. The main obstacle is that the
statistical estimation techniques that are successfully applied within the
context of a large-scale assessment require large samples to guarantee reliable
calibration of the item parameters and an accurate estimation of the examinees’
proficiency class membership. Such samples are simply not obtainable in
course-based settings. Therefore, the nonparametric item selection (NPS) method
that does not require any parameter calibration, and thus, can be used in small
educational programs is proposed in the study. The proposed nonparametric CD-CAT
uses the nonparametric classification (NPC) method to estimate an examinee’s
attribute profile and based on the examinee’s item responses, the item that can
best discriminate the estimated attribute profile and the other attribute
profiles is then selected. The simulation results show that the NPS method
outperformed the compared parametric CD-CAT algorithms and the differences were
substantial when the calibration samples were small.