Effects of multimedia on psychometric characteristics of cognitive tests: A comparison between technology-based and paper-based modalities

The study aims to investigate the effects of delivery modalities on psychometric characteristics and student  performance on cognitive tests. A first study assessed the inductive reasoning ability of 715 students under the  supervision of teachers. A second study examined 731 students’ performance on the application of the control-of variables strategy in basic physics but without teacher supervision due to the COVID-19 pandemic. Rasch  measurement showed that the online format fitted to the data better in the unidimensional model across two  conditions. Under teacher supervision, paper-based testing was better than online testing in terms of reliability  and total scores, but contradictory findings were found in turn without teacher supervision. Although measurement invariance was confirmed between two versions at item level, the differential bundle functioning  analysis supported the online groups on the item bundles constructed of figure-related materials. Response time  was also discussed as an advantage of technology-based assessment for test development.