Paths of improvement and reports on cognitive diagnosis with the “rule space” method
No. 1 (2017-05-01)Author(s)
-
Álvaro Artavia MedranoUniversidad de Costa Rica, Montes de Oca, San José, Costa Rica (alvartavia@gmail.com)
Abstract
Given that the development of models of cognitive diagnosis has expanded in recent years, there is increasingly a need to communicate the results to different publics in an effective way. Hence, this article sets forth the use of the “rule space” model in the cognitive diagnosis of structures of knowledge and skills in mathematics, shown in a representative sample of students in Costa Rica who took the National High School Diploma Test in that subject. With the aim of presenting the analyses pertaining to the chosen cognitive psychometric method, the article reports on empirical evidence for the validity of the test, as well as the formulation of the incidence matrix and the classification rate of the students. It concludes that people with the same scores in the test show differences in their probabilities in the mastery of attributes, which is why there is a need for both individual and collective reports which take the differences in the mastery of cognitive attributes into account, so they can be employed to create strategies focused on the strengthening of skills. And likewise, ensure that the information of a diagnostic nature which is obtained from such models is comprehensible and easy to interpret in terms of descriptions of cognitive attributes, in order to trace out learning routes, according to the probabilities of the mastery of skills which reveal the strengths and weaknesses of the student body, and in turn, come up with remedial educational measures.
References
Artavia-Medrano, A. (2014). Evaluación cognitiva diagnóstica en Matemática: modelo elaborado con el método rule space para estudiantes costarricenses de undécimo año. Disertación doctoral no publicada, Universidad de Costa Rica, Costa Rica.
Artavia-Medrano, A. (2015). Interpretación y análisis de pruebas educativas y psicológicas con el método rule space. Actualidades en Psicología, 29 (119), 63-77.
Artavia-Medrano, A. y Larreamendy-Joerns, J. (2012). Información cognitiva a partir de pruebas de gran escala: el método de representación del espacio de reglas. Universitas Psychologica, 11 (2), 599-610.
Birenbaum, M. y Tatsuoka, K. (1982). On the dimensionality of achievement test data. Journal of Educational Measurement, 19 (4), 259-266.
Birenbaum, M. y Tatsuoka, K. (1993). Applying an IRT-based cognitive diagnostic model to diagnose students’ knowledge states in multiplication and division with exponents. Applied Measurement in Education, 6 (4), 255-268.
Bond, T. G. y Fox, C. M. (2007). Applying the Rasch model: Fundamental Measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
Buck, G., VanEssen, T., Tatsuoka, K., Kostin, I., Lutz, D. y Phelps, M. (1998). Development, selection and validation of a set of cognitive and linguistic attributes for the SAT I verbal: Sentence completion section. (Technical Report RR-98-23). Princeton, NJ: Educational Testing Service.
de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76 (2), 179-199.
de la Torre, J. y Douglas, J. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69 (3), 333-353.
Embretson, S. E. (1997). Multicomponent response models. En W. J.van der Linden y R. K.Hambleton (eds.), Handbook of modern item response theory (pp. 305-321). Nueva York: Springer.
Ericsson, K. y Simon, H. (1993). Protocol analysis: Verbal reports as data. Massachusetts: The MIT Press.
Ferrer, G. (2006). Sistemas de evaluación de aprendizajes en América Latina: balance y desafíos. Santiago: Alfabeta Artes Gráficas.
Fischer, G. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359-374.
Fischer, G. (1997). Unidimensional linear logistic Rasch models. En W. J.van der Linden y R. K.Hambleton (eds.), Handbook of modern item response theory (pp. 225-243). Nueva York: Springer.
Junker, B. y Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric IRT. Applied Psychological Measurement, 25 (3), 211-220.
Leighton, J., Gierl, M. y Hunka, S. (2002, abril). The attribute hierarchy model for cognitive assessment. Louisiana: Artículo presentado en Annual Meeting of the National Council on Measurement in Education (NCME). http://www.education.ualberta.ca/educ/psych/crame
Leighton, J., Gierl, M. y Hunka, S. (2004). The attribute hierarchy model: An approach for integrating cognitive theory with assessment practice. Journal of Educational Measurement, 41 (3), 205-236.
Linacre, M. (2012). WINSTEPS (Versión 3.74.0) [Software de cómputo]. Chicago, IL: Winsteps Rasch Measurement.
Mislevy, R., Steinberg, L. y Almond, R. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1 (1), 3-62.
Roberts, M. (2012). Developing and evaluating score reports for cognitive diagnostic assessment. Disertación doctoral no publicada, University of Alberta, Canadá.
Roberts, M. y Gierl, M. (2010). Developing score reports for cognitive diagnostic assessments. Educational Measurement: Issues and Practice, 29 (3), 25-38.
Roberts, M. y Gierl, M. (2011, abril). Developing and evaluating score reports for a diagnostic mathematics assessment. Artículo presentado en Annual meeting of the American Educational Research Association (AERA), New Orleans, LA.
Tatsuoka, K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20 (4), 345-354.
Tatsuoka, K. (1985). A probabilistic model for diagnosing misconceptions by the pattern classification approach. Journal of Educational Statistics, 10 (1), 55-73.
Tatsuoka, K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. En N.Frederiksen, R.Glaser, A.Lesgold y M.Shafto (eds.), Diagnostic monitoring of skills and knowledge acquisition (pp. 453-488). Nueva Jersey: Erlbaum.
Tatsuoka, K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. En P. D.Nichols, S. F.Chipman y R. L.Brennan (eds.), Cognitively diagnostic assessment (pp. 327-359). Hillsdale, NJ: Erlbaum.
Tatsuoka, K. (2009). Cognitive assessment: An introduction to the rule space method. Nueva York: Routledge Taylor y Francis Group.
Tatsuoka, K., Birenbaum, M., Lewis, C. y Sheehan, K. (1993). Proficiency scaling based on conditional probability functions for attributes. (Technical Report RR-93-50-ONR). Nueva Jersey: Educational Testing Service.
Tatsuoka, K., Varadi, F. & Tatsuoka, C. (2004). PMAIN-RULE SPACE. [Software de cómputo no publicado]. Trenton, NJ, EE.UU.: Tanar.