Cilt 6, Sayı 4, Sayfalar 143 - 157 2017-10-10

Three, four, and none of the above options in multiple-choice items
Çoktan seçmeli sorularda üç, dört ve hiçbiri seçenekleri

Erkan Atalmış [1] , Neal Kingston [2]

82 90

High-quality multiple-choice (MC) items are essential for creating efficient, valid assessments. Haladyna, Downing, and Rodriguez (2002) suggested that using plausible distractors is crucial to achieving this goal, although distractor creation can be time-consuming and challenging. Haladyna et al. (2002) provided two related test development guidelines: #18, “Write as many plausible distractors as you can,” and #25, “Use carefully None of the above.” This research aims to test the impact of these two guidelines on item difficulty (p), item discrimination (r), and test reliability for mathematics items empirically. The research findings have revealed that item discrimination and test reliability were not statistically different across MC items with four options, three options, and NOTA options while the means of item difficulty of four-option MC items was not statistically different from those of three-option and NOTA-option MC items. However, the mean of item difficulty of NOTA-option MC items was statistically lower than those of three-option.

Etkili ve geçerli değerlendirmeler yapabilmek için yüksek kaliteli çoktan seçmeli sorular (maddeler) oluşturmak gereklidir. Bu amaç doğrultusunda Haladyna, Downing, and Rodriguez (2002) çoktan seçmeli madde oluşturma sürecinde uygun çeldiricilerin kullanılmasının önemli olduğunu vurgulamaktadırlar. Ancak bu çeldirilerin oluşturulması zaman alıcı ve zahmetli bir süreçtir. Bu bağlamda Haladyna ve diğerleri (2002) test geliştirme ile ilişkili olan "Olabildiği kadar uygun ve mantıklı çeldiricileri yazınız" ve "Hiçbiri seçeneğini dikkatli kullanınız" ilkelerini rapor etmektedirler. Bu araştırmanın amacı bu iki ilke kullanımının matematik sorularınn zorluk derecesine, ayırt edicilik derecesine ve test güvenirliğine etkisini test etmektir. Araştırma sonucunda, madde ayırt edicilik derecesi ve test güvenirliği dört seçenekli, üç seçenekli, ve “hiçbiri” seçeneğine sahip maddeler arasında istatistiksel olarak anlamlı bir farklılık göstermediği bulunmuştur. Benzer şekilde, dört seçenekli maddelerin madde zorluk dereceleri, üç seçenekli ve “hiçbiri” seçeneğine sahip maddelerin madde zorluk derecelerinden istatistiksel olarak anlamlı bir farklılık göstermediği bulunurken, “hiçbiri” seçeneğine sahip maddelerin madde zorluk dereceleri üç seçenekli maddelerin madde zorluk derecelerinden istatistiksel olarak düşük olduğu elde edilmiştir.

  • Abad, F. J., Olea, J., & Ponsoda, V. (2001). Analysis of the optimum number of alternatives from the Item Response Theory. Psicothema, 13(1), 152-158.
  • Baghaei, P., & Amrahi, N. (2011). The effects of the number of options on the psychometric characteristics of multiple choice items. Psychological Test and Assessment Modeling, 53(2), 192-211.
  • Burton, S.J., Sudweeks, R. R., Merrill, P. F., & Wood, B. (1991). How to prepare better multiple-choice test items: Guidelines for university faculty. Brigham Young University Department of Instructional Science. Retrieved from http://testing.byu.edu/info/handbooks/betteritems.pdf
  • Cizek, G. J., & O'Day, D. M. (1994). Further investigation of nonfunctioning options in multiple-choice test items. Educational and Psychological Measurement, 54(4), 861–872. doi:10.1177/0013164494054004002
  • Common Core State Standards Initiative (CCSSI). (2010). Common Core state standards for mathematics. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers.
  • Crehan, K. D., & Haladyna, T. M. (1991).The validity of two item-writing rules. The Journal of Experimental Education, 59(2), 183–192. doi:10.1080/00220973.1991.10806560
  • Crehan, K. D., Haladyna, T. M., & Brewer, B. W. (1993). Use of an inclusive option and the optimal number of options for multiple-choice items. Educational and Psychological Measurement, 53(1), 241-247. doi:10.1177/0013164493053001027
  • Dehnad, A., Nasser, H., & Hosseini, A. F. (2014). A comparison between three-and four-option multiple choice questions. Procedia-Social and Behavioral Sciences, 98, 398–403. doi:10.1016/j.sbspro.2014.03.432
  • Delgado, A. R., & Prieto, G. (1998). Further evidence favoring three-option items in multiple-choice tests. European Journal of Psychological Assessment, 14(3), 197–201. doi:10.1027/1015-5759.14.3.197
  • DiBattista, D., Sinnige-Egger, J., & Fortuna, G. (2014). The “none of the above” option in multiple-choice testing: An experimental study. The Journal of Experimental Education, 82(2), 168-183. doi:10.1080/00220973.2013.795127
  • Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in health sciences education, 10(2), 133-143.doi:10.1007/s10459-004-4019-5
  • Duhachek, A., & Iacobucci, D. (2004). Alpha's standard error (ASE): an accurate and precise confidence interval estimate. Journal of Applied Psychology, 89(5), 792–808. doi:10.1037/0021-9010.89.5.792
  • Frary, R. B. (1991). The none-of-the-above option: An empirical study. Applied Measurement in Education, 4(2), 115–124. doi:10.1207/s15324818ame0402_2
  • Frey, B.B., Petersen, S., Edwards, L. M., Pedrotti, J.T., & Peyton, V. (2005). Item-writing rules: Collective wisdom. Teaching and Teacher Education, 21(4), 357–364.doi:10.1016/j.tate.2005.01.008
  • Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37–50. doi:10.1207/s15324818ame0201_3
  • Haladyna, T. M., & Downing, S. M. (1993). How many options is enough for a multiple-choice test item? Educational and psychological measurement, 53(4), 999–1010.doi:10.1177/0013164493053004013
  • Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choiceitem-writing guidelines for classroom assessment. Applied Measurement in Education,15(3), 309–334. doi:10.1207/S15324818AME1503_5
  • Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. New York, NY: Routledge.
  • Hansen, J. D., & Dexter, L. (1997). Quality multiple-choice test questions: Item-writing guidelines and an analysis of auditing testbanks. Journal of Education for Business 73(2), 94–97. doi:10.1080/08832329709601623
  • IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.
  • Kingston, N.M., & Kramer, L.B. (2013). Highstakes test construction and test use. In T.D. Little (Ed.),The Oxford handbook of quantitative methods: Vol. 1 foundations (pp. 189–205). New York, NY: Oxford University Press.
  • Kingston, N.M., Scheuring, S.T., & Kramer, L.B. (2013). Test Development Strategies. In Kurt Geisinger (Ed.) APA Handbook of Testing and Assessment in Psychology. Washington, DC: APA Books.
  • Knowles, S. L.,&Welch, C. A. (1992). A meta-analytic review of item discrimination and difficulty in multiple-choice items using "none-of-the-above.” Educational and Psychological Measurement, 52(3), 571–577. doi:10.1177/0013164492052003006
  • Kolstad, R. K., & Kolstad, R. A. (1991). The option “none of these” improves multiple-choice test items. Journal of Dental Education, 55(2), 161–163.
  • Landrum, R. E., Cashin, J. R., & Theis, K. S. (1993). More evidence in favor of three-option multiple-choice tests. Educational and Psychological Measurement, 53(3), 771–778. doi:10.1177/0013164493053003021
  • Lee, H., & Winke, P. (2013). The differences among three-, four-, and five-option-item formats in the context of a high-stakes English-language listening test. Language Testing, 30(1), 99-123.
  • McCoubrie, P. (2004). Improving the fairness of multiple-choice questions: a literature review. Medical Teacher, 26(8), 709–712. doi:10.1080/01421590400013495
  • Odegard, T. N., & Koen, J. D. (2007). “None of the above” as a correct and incorrect alternative on a multiple-choice test: Implications for the testing effect. Memory, 15(8), 873–885. doi:10.1080/09658210701746621
  • Rich, C. E., & Johanson, G. A. (1990, April).An item-level analysis of “none of the above.”Paper presented at the annual meeting of the American Educational Research Association, Boston, MA. Retrieved from http://files.eric.ed.gov/fulltext/ED400299.pdf
  • Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysisof 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3-13. doi:0.1111/j.1745-3992.2005.00006.x
  • Rogers, W. T., & Harley, D. (1999). An empirical comparison of three-and four-choice items and tests: Susceptibility to testwiseness and internal consistency reliability. Educational and Psychological Measurement, 59(2), 234–247. doi:10.1177/00131649921969820
  • StataCorp (2013). Stata: release 13 - statistical software. College Station, TX: StataCorp LP
  • Shizuka, T., Takeuchi, O., Yashima, T. & Yoshizawa, K. (2006). A comparison of three-and four-option English tests for university entrance selection purposes in Japan. Language Testing, 23(1), 35–57. doi:10.1191/0265532206lt319oa
  • Tarrant, M., Ware, J., & Mohammed, A. M. (2009). An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis.BMC Medical Education, 9(1), 40. doi:10.1186/1472-6920-9-40
  • Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse Education Today, 30(6), 539–543. doi:10.1016/j.nedt.2009.11.002
  • Tollefson, N. (1987). A comparison of the item difficulty and item discrimination of multiple-choice items using the"none of the above" and one correct response options. Educational and Psychological Measurement, 47(2), 377–383.doi:10.1177/0013164487472010
  • Trevisan, M. S., Sax, G., & Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51(4),829–837. doi:10.1177/001316449105100404
Konular Eğitim Bilimleri
Dergi Bölümü Araştırma Makaleleri
Yazarlar

Orcid: orcid.org/0000-0001-9610-491X
Yazar: Erkan Atalmış
E-posta: erkanatalmis@gmail.com
Ülke: Turkey


Orcid: orcid.org/0000-0003-1870-309X
Yazar: Neal Kingston
E-posta: nkingsto@ku.edu
Ülke: United States


Bibtex @araştırma makalesi { turje333687, journal = {Turkish Journal of Education}, issn = {}, address = {Mehmet TEKEREK}, year = {2017}, volume = {6}, pages = {143 - 157}, doi = {10.19128/turje.333687}, title = {Three, four, and none of the above options in multiple-choice items}, language = {en}, key = {cite}, author = {Kingston, Neal and Atalmış, Erkan} } @araştırma makalesi { turje333687, journal = {Turkish Journal of Education}, issn = {}, address = {Mehmet TEKEREK}, year = {2017}, volume = {6}, pages = {143 - 157}, doi = {10.19128/turje.333687}, title = {Çoktan seçmeli sorularda üç, dört ve hiçbiri seçenekleri}, language = {tr}, key = {cite}, author = {Kingston, Neal and Atalmış, Erkan} }
APA Atalmış, E , Kingston, N . (2017). Three, four, and none of the above options in multiple-choice items. Turkish Journal of Education, 6 (4), 143-157. DOI: 10.19128/turje.333687
MLA Atalmış, E , Kingston, N . "Three, four, and none of the above options in multiple-choice items". Turkish Journal of Education 6 (2017): 143-157 <http://dergipark.gov.tr/turje/issue/31357/333687>
Chicago Atalmış, E , Kingston, N . "Three, four, and none of the above options in multiple-choice items". Turkish Journal of Education 6 (2017): 143-157
RIS TY - JOUR T1 - Çoktan seçmeli sorularda üç, dört ve hiçbiri seçenekleri AU - Erkan Atalmış , Neal Kingston Y1 - 2017 PY - 2017 N1 - doi: 10.19128/turje.333687 DO - 10.19128/turje.333687 T2 - Turkish Journal of Education JF - Journal JO - JOR SP - 143 EP - 157 VL - 6 IS - 4 SN - -2147-2858 M3 - doi: 10.19128/turje.333687 UR - http://dx.doi.org/10.19128/turje.333687 Y2 - 2017 ER -
EndNote %0 Turkish Journal of Education Çoktan seçmeli sorularda üç, dört ve hiçbiri seçenekleri %A Erkan Atalmış , Neal Kingston %T Çoktan seçmeli sorularda üç, dört ve hiçbiri seçenekleri %D 2017 %J Turkish Journal of Education %P -2147-2858 %V 6 %N 4 %R doi: 10.19128/turje.333687 %U 10.19128/turje.333687