Yıl 2018, Cilt 5, Sayı 1, Sayfalar 105 - 118 2017-11-25

Use of Full Hierarchy Consistency Index to Assess Response Consistency

Lokman Akbay [1] , Mustafa Kılınç [2]

40 31

Measurement models need to properly delineate the real aspect of examinees’ response processes for measurement accuracy purposes. To avoid invalid inferences, fit of examinees’ response data to the model is studied through person-fit statistics. Misfit between the examinee response data and measurement model may be due to invalid models and/or examinee’s aberrant response behavior such as cheating, creative responding, and random responding. Hierarchy consistency index (HCI) was introduced as a person-fit statistics to assess classification reliability of particular cognitive diagnosis models. This study examines the HCI in terms of its usefulness under nonhierarchical attribute conditions and under different item types. Moreover, current form of HCI formulation only considers the information based on correct answers only. We argue and demonstrate that more information could be obtained by incorporating the information that may be obtained from incorrect responses. Therefore, this study considers the full-version of the HCI (i.e., FHCI). Results indicate that current form of HCI is sensitive to misfitting item types (i.e., basic or more complex) and examinee attribute patterns. In other words, HCI is affected by the attribute pattern an examinee has as well as by the item s/he aberrantly responded. Yet, FHCI is not severely affected by item types under any examinee attribute pattern.

Person-fit,Attribute Hierarchy Index,Cognitive Diagnosis
  • Cui, Y., & Leighton, J. P. (2009). The hierarchy consistency index: Evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46, 429-449.
  • de la Torre, J. (2009b). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115-130.
  • Doornik, J. A. (2011). Object-oriented matrix programming using Ox (Version 6.20). London: Timberlake Consultants Press.
  • Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
  • Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuokas rule-space approach. Journal of Educational Measurement, 41, 205-237.
  • Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187-212.
  • Robitzsch, A., Kiefer, T., George, A. C., & Uenlue, A. (2014). CDM: Cognitive diagnosis modeling. R package version 5.8-9. https://CRAN.R-project.org/package=CDM
  • Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345-354.
  • Templin, J. L., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317-339.
Konular Eğitim Bilimleri
Yayımlanma Tarihi January
Dergi Bölümü Makaleler
Yazarlar

Yazar: Lokman Akbay
E-posta: lokmanakbay@gmail.com
Kurum: Mehmet Akif Ersoy University
Ülke: Turkey


Orcid: orcid.org/0000-0002-2759-4916
Yazar: Mustafa Kılınç
E-posta: mkilinc@mehmetakif.edu.tr
Kurum: Mehmet Akif Ersoy University
Ülke: Turkey


Bibtex @araştırma makalesi { ijate350499, journal = {International Journal of Assessment Tools in Education}, issn = {}, address = {İzzet KARA}, year = {2017}, volume = {5}, pages = {105 - 118}, doi = {10.21449/ijate.350499}, title = {Use of Full Hierarchy Consistency Index to Assess Response Consistency}, key = {cite}, author = {Kılınç, Mustafa and Akbay, Lokman} }
APA Akbay, L , Kılınç, M . (2017). Use of Full Hierarchy Consistency Index to Assess Response Consistency. International Journal of Assessment Tools in Education, 5 (1), 105-118. DOI: 10.21449/ijate.350499
MLA Akbay, L , Kılınç, M . "Use of Full Hierarchy Consistency Index to Assess Response Consistency". International Journal of Assessment Tools in Education 5 (2017): 105-118 <http://dergipark.gov.tr/ijate/issue/30630/350499>
Chicago Akbay, L , Kılınç, M . "Use of Full Hierarchy Consistency Index to Assess Response Consistency". International Journal of Assessment Tools in Education 5 (2017): 105-118
RIS TY - JOUR T1 - Use of Full Hierarchy Consistency Index to Assess Response Consistency AU - Lokman Akbay , Mustafa Kılınç Y1 - 2017 PY - 2017 N1 - doi: 10.21449/ijate.350499 DO - 10.21449/ijate.350499 T2 - International Journal of Assessment Tools in Education JF - Journal JO - JOR SP - 105 EP - 118 VL - 5 IS - 1 SN - -2148-7456 M3 - doi: 10.21449/ijate.350499 UR - http://dx.doi.org/10.21449/ijate.350499 Y2 - 2017 ER -
EndNote %0 International Journal of Assessment Tools in Education Use of Full Hierarchy Consistency Index to Assess Response Consistency %A Lokman Akbay , Mustafa Kılınç %T Use of Full Hierarchy Consistency Index to Assess Response Consistency %D 2017 %J International Journal of Assessment Tools in Education %P -2148-7456 %V 5 %N 1 %R doi: 10.21449/ijate.350499 %U 10.21449/ijate.350499