Yıl 2018, Cilt 5, Sayı 1, Sayfalar 1 - 14 2017-08-01

Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning

Hüseyin Selvi [1] , Devrim Özdemir Alıcı [2]

177 502

In this study, it is aimed to investigate the impact of different missing data handling methods on the detection of Differential Item Functioning methods (Mantel Haenszel and Standardization methods based on Classical Test Theory and Likelihood Ratio Test method based on Item Response Theory). In this regard, on the data acquired from 1046 candidates who entered to Foreign National Student Exam (FNSE) held in year 2016 by Mersin University (MEU) and answered Basic Skills subtest, using different missing data handling methods, differential item functioning analyses with Mantel Haenszel, Standardization and Likelihood Ratio Test methods are performed. Basic Skills test consists of 80 multiple choice items. The items are all binary scored (1-0) items. Among the participants 523 are female and 523 are male. The findings showed that the number of items flagged as DIF has changed with the used missing data handling methods. The DIF detection methods based on Classical Test Theory are more consistent within themselves compared to DIF detection method based on Item Response Theory, whereas the used missing data handling methods differentiate the DIF detected items and this difference reaches a significant level for Mantel Haenszel method

Differential Item Functioning DIF, Test and Item Bias, Missing Values; Imputation of Missing Data, Mantel Haenszel, Likelihood Ratio Test
  • Abedlazeez, N. (2010). Exploring DIF: Comparison of CTT and IRT methods. International Journal of Sustainable Development, 7(1), 11-46.
  • Allison, P. D. (2002). Missing data. California: Sage Publication Inc.
  • Alpar, R. (2011). Uygulamalı çok değişkenli istatistiksel yöntemler. Ankara: Detay Yayıncılık.
  • Angoff, W.H. (1993). Perspectives on differential item functioning methodology. In Holland & Wainer (Ed.), Differential Item Functioning. New Jersey: Lawrence Erlbaum Associates Publishers.
  • Banks, K., & Walker, C. (2006). Performance of SIBTEST when focal group examinees have missing data. San Francisco: National Council of Measurement in Education.
  • Banks, K. (2015). An introduction to missing data in the context of differential item functioning. Practical Assessment, Research & Evaluation. 20(12).
  • Bennett, D. A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25, 464–469.
  • Bernhard, J., Celia, D.F., &Coates, A.S. (1998). Missing quality of life data in cancer clinical trials: Serious problems and challenges. Statistics in Medicine, 17, 517-532.
  • Camili, G., & Shepard, L.A. (1994). Methods for identifying biased test items. London: Sage Publication.
  • Demir, E., & Parlak, B. (2012). Türkiye’de eğitim araştırmalarında kayıp veri sorunu. Journal of Measurement and Evaluation in Education and Psychology 3(1), 230-241.
  • Demir, E. (2013). Kayıp verilerin varlığında çoktan seçmeli testlerde madde ve test parametrelerinin kestirilmesi: SBS örneği [Item and test parameters estimations for multiple choice tests in the presence of missing data: The case of SBS]. Journal of Educational Sciences Research, 3(2), 47–68.
  • Dişçi, R. (2012). Temel ve klinik biyoistatistik. İstanbul: Tıp Kitapevi.
  • Doğan, N., & Öğretmen, T. (2008). Değişen Madde Fonksiyonunu belirlemede Mantel–Haenszel, Ki-Kare ve Lojistik Regresyon tekniklerinin karşılaştırılması. Education and Science, 33(148).
  • Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. London: Lawrence Erlbaum Associates.
  • Emenogu, B. C., Falenchuck, O., & Childs, R. A. (2010). The effect of missing data treatment on Mantel-Haenszel DIF detection. The Alberta Journal of Educational Research, 56(4), 459-469.
  • Falenchuk, O., & Herbert, M. (2009). Investigation of differential non-response as a factor affecting the results of Mantel-Haenszel DIF detection California: American Educational Research Association.
  • Finch, W.H. (2011). The impact of missing data on the detection of nonuniform differential ıtem functioning. Educational and Psychological Measurement, 71(4) 663–683.
  • Garrett, P. L. (2009). A monte carlo study investigating missing data, differential item functioning, and effect size. Georgia State University, Unpublished doctoral dissertation.
  • Gelin, M.N. & Zumbo, B.D. (2003). Differential item functioning results may change depending on how an item is scored: an illustration with the center for epidemiologic studies depression scale. Educational and Psychological Measurement, X(X) DOI: 10.1177/0013164402239317.
  • Gierl, M.J., Jodoin, M.G., & Ackerman, T.A. (2000). Performance of Mantel-Haenszel, Simultaneous Item Bias Test, and Logistic Regression when the proportion of DIF items is large. American Educational Research Association.
  • Gonzales, A., Padilla, J.L., Dolores, H., Gomez-Benito, J., & Benitez, I. (2010). EASY-DIF: Software for analyzing differential item functioning using the Mantel-Haenszel and Standardization procedures. Applied Psychological Measurement. doi:10.1177/0146621610381489.
  • Graham, J.W. (2009). Missing Data Analysis: Making it work in the real world. Annual Review of Psychology, 60(4), 549-576.
  • Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675.
  • Gözen Çıtak, G. (2007). Klasik test ve madde-tepki kuramlarına göre çoktan seçmeli testlerde farklı puanlama yöntemlerinin karşılaştırılması. Doktora Tezi, Ankara Üniversitesi, Ankara
  • Hambletton, R.K. & Swaminathan, H. (1985). Item Response Theory: Principles and applications. Boston: Kluwer-Nijhoff Publishing.
  • Harwell, M. Stone, C. A., Hsu, T.C., & Kirisci, L. (1996). Monte carlo studies in item response theory. Applied Psychological Measurement, 20, 101-125.
  • Hohensinn, C. & Kubinger K. D. (2011). On the impact of missing values on item fit and the model validness of the Rasch model. Psychological Test and Assessment Modeling, 53, 380-393.
  • Kan, A., Sünbül, Ö., Ömür, S. (2013). 6.- 8. sınıf seviye belirleme sınavları alt testlerinin çeşitli yöntemlere göre değişen madde fonksiyonlarının incelenmesi. Mersin University Journal of the Faculty of Education, 9(2), 207-222.
  • Kothari, C.R. (2004). Research methodology: Methods and techniques (Second Revised Edition). New Delhi: New Age Int. Ltd.
  • Kristanjansonn E., R. Aylesworth, I. McDowell & B.D. Zumbo (2005). A Comparison of four methods for detecting differential item functioning in ordered response model. Educational and Psychological Measurement. 65(6), 935-953.
  • Little, R. J. A & Rubin, D. B. (1987). Statistical analysis with missing data (2nd ed.). New York: John Wiley & Sons, Inc.
  • Lord, F. M. (1974). Estimation of latent ability and item parameters when there are omitted responses. Psychometrika, 39, 247-264.
  • Lord, F. M. (1980). Applications of item response theory to practical testing problems. New Jersey: Lawrence Erlbaum Associates.
  • Molenberghs, G., & Kenward, M.G. (2007). Missing data in clinical studie (1 st ed.). England: John Wiley&Sons.
  • Narayanan, P., & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and Simultaneous Item Bias procedures for detecting differential ıtem functioning, Applied Psychological Measurement, 18(4).
  • Osterlind, S.J. (1983). Test item bias. London: Sage Publication.
  • Padilla, J.L., Hidalgo, J.L., Benitez, I., & Gomez-Benito, J. (2012). Comparison of three software programs for evaluating DIF by means of the Mantel-Haenszel procedure; EASY DIF, DIFAS and EZDIF, Psicologica, 33,135-156.
  • Peng, C.Y.J., Harwell, M., Liou, S.M., & Ehman, L. H. (2006). Advances in missing data methods and implications for educational research. In S. Sawilowsky (Ed.), Greenwich: Real data analysis.
  • Peng, C. J., & Zhu, J. (2008). Comparison of two approaches for handling missing covariates in logistic regression. Educational and Psychological Measurement, 68(1), 58-77.
  • Pigott, T.D. (2001). A review of methods for missing data. Educational Research and Evaluation, 7(4); 353-383.
  • Robitzsch, A, & Rupp, A.A. (2009). Impact of missing data on the detection of differential item functioning the case of mantel-haenszel and logistic regression analysis. Educational and Psychological Measurement, 69(1): 18-34.
  • Rousseau, M., Bertrand, R., & Boiteau, N. (2006, April). Impact of missing data treatment on the efficiency of DIF methods. California: National Council on Measurement in Education.
  • Royce, S., Straits, B.C., & Straits, M.M. (1993). Approaches to social research (2nd ed.). New York: Oxford University Press.
  • Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581-592.
  • Schafer, J. L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research, (8), 3-15.
  • Sedivy, S. K., Zhang, B., & Traxel, N. M. (2006). Detection of differential item functioning with polytomous items in the presence of missing data. California: National Council of Measurement in Education.
  • Selvi, H. (2013). Klasik test ve madde tepki kuramlarına dayalı değişen madde fonksiyonu belirleme tekniklerinin farklı puanlama durumlarında incelenmesi. Yayınlanmamış Doktora Tezi. Mersin Üniversitesi Eğitim Bilimleri Enstitüsü.
  • Singh, Y.K. (2006). Fundamental of research methodology and statistics. New Delhi: New Age Int. Ltd.
  • Spray, J., & Miller, T. (1994). Identifying nonuniform DIF in polytomously scored test items. American College Testing Research Report Series 94-1. Iowa City, IA: American College Testing Program.
  • Ward, W.C., & Bennett, R.E. (2012). Construction versus choice in cognitive measurement: issues in constructed response, performance testing, and portfolio assessment. London and New York: Routledge, Taylor & Francis Group.
  • Woodward, M., Smith, W.C., & Tunstall Pedoe H. (1991). Bias from missing values: Sex differences in implication of failed venepuncture for the Scottish Health Study.Int J. Epidemiol.
  • Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research & Evaluation, 12(3), 1-26.
  • Zumbo, B. D. (1999). A Handbook on the theory and methods of Differential Item Functioning (DIF): Logistic Regression modeling as a unitary framework for binary and likert-type (ordinal) item scores. Ottawa ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.
Konular Eğitim Bilimleri
Yayımlanma Tarihi January
Dergi Bölümü Makaleler
Yazarlar

Orcid: https://orcid.org/0000-0002-3513-0003
Yazar: Hüseyin Selvi
E-posta: hsyn_selvi@yahoo.com.tr
Kurum: Mersin University
Ülke: Turkey


Orcid: https://orcid.org/0000-0001-5542-0609
Yazar: Devrim Özdemir Alıcı
E-posta: devrimo.alici@gmail.com
Kurum: Mersin University
Ülke: Turkey


Bibtex @araştırma makalesi { ijate330885, journal = {International Journal of Assessment Tools in Education}, issn = {}, address = {İzzet KARA}, year = {2017}, volume = {5}, pages = {1 - 14}, doi = {10.21449/ijate.330885}, title = {Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning}, key = {cite}, author = {Selvi, Hüseyin and Özdemir Alıcı, Devrim} }
APA Selvi, H , Özdemir Alıcı, D . (2017). Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning. International Journal of Assessment Tools in Education, 5 (1), 1-14. DOI: 10.21449/ijate.330885
MLA Selvi, H , Özdemir Alıcı, D . "Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning". International Journal of Assessment Tools in Education 5 (2017): 1-14 <http://dergipark.gov.tr/ijate/issue/30630/330885>
Chicago Selvi, H , Özdemir Alıcı, D . "Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning". International Journal of Assessment Tools in Education 5 (2017): 1-14
RIS TY - JOUR T1 - Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning AU - Hüseyin Selvi , Devrim Özdemir Alıcı Y1 - 2017 PY - 2017 N1 - doi: 10.21449/ijate.330885 DO - 10.21449/ijate.330885 T2 - International Journal of Assessment Tools in Education JF - Journal JO - JOR SP - 1 EP - 14 VL - 5 IS - 1 SN - -2148-7456 M3 - doi: 10.21449/ijate.330885 UR - http://dx.doi.org/10.21449/ijate.330885 Y2 - 2017 ER -
EndNote %0 International Journal of Assessment Tools in Education Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning %A Hüseyin Selvi , Devrim Özdemir Alıcı %T Investigating the Impact of Missing Data Handling Methods on the Detection of Differential Item Functioning %D 2017 %J International Journal of Assessment Tools in Education %P -2148-7456 %V 5 %N 1 %R doi: 10.21449/ijate.330885 %U 10.21449/ijate.330885