reflectR: Automatic Scoring of the Cognitive Reflection Test
A tool for researchers and psychologists to automatically code open-ended responses to the Cognitive Reflection Test (CRT), a widely used class of tests in cognitive science and psychology for assessing an individual's propensity to override an incorrect gut response and engage in further reflection to find a correct answer. This package facilitates the standardization of Cognitive Reflection Test responses analysis across large datasets in cognitive psychology, decision-making, and related fields. By automating the coding process, it not only reduces manual effort but also aims to reduce the variability introduced by subjective interpretation of open-ended responses, contributing to a more consistent and reliable analysis. 'reflectR' supports automatic coding and machine scoring for the original English-language version of CRT (Frederick, 2005) <doi:10.1257/089533005775196732>, as well as for CRT4 and CRT7, 4- and 7-item versions, respectively (Toplak et al., 2014) <doi:10.1080/13546783.2013.844729>, for the CRT-long version built via Item Response Theory by Primi and colleagues (2016) <doi:10.1002/bdm.1883>, and for CRT-2 by Thomson & Oppenheimer (2016) <doi:10.1017/s1930297500007622>.
Note: While 'reflectR' draws inspiration from the principles and scientific literature underlying the different versions of the Cognitive Reflection Test, it has been independently developed and does not hold any affiliation with any of the original authors.
The development of this package benefited significantly from the kind insight and suggestion provided by Dr. Keela Thomson, whose contribution is gratefully acknowledged.
Additional gratitude is extended to Dr. Paolo Giovanni Cicirelli, Prof. Marinella Paciello, Dr. Carmela Sportelli, and Prof. Francesca D'Errico, who not only contributed to the manual multi-rater coding of CRT-2 items but also profoundly influenced the understanding of the importance and practical relevance of cognitive reflection within personality, social, and cognitive psychology research.
Acknowledgment is also due to the European project STERHEOTYPES (STudying European Racial Hoaxes and sterEOTYPES) for funding the data collection that produced the datasets initially used for manual multi-rater coding of CRT-2 items.
Documentation:
Downloads:
Linking:
Please use the canonical form
https://CRAN.R-project.org/package=reflectR
to link to this page.