Yıl 2018, Cilt 7, Sayı 4, Sayfalar 186 - 195 2018-10-31

Does equating matter in value-added models?
Katma-değerli değerlendirme modellerinde eşitleme önemli mi?

Sedat Şen [1] , Ragıp Terzi [2] , İbrahim YILDIRIM [3] , Allan Cohen [4]

73 117

The purpose of this study was to examine the effect of equated and non-equated data on value-added assessment analyses. Several models have been proposed in the literature to apply the value-added assessment approach. This study compared two different value-added models: the unadjusted hierarchical linear model and the generalized persistence model. The former model assumes equated tests while the latter one relaxes this assumption. Two different data sets (equated and non-equated) were analyzed with both models. Value-added estimates for both models based on a statewide examination (equated) and a countrywide examination (non-equated) data were generally consistent. School rankings showed differences between the two models. The practical implication of this study is that although there were small differences in school rankings, a model requiring an equating assumption can be applied to a non-equated data set in a case when equating between test forms is not possible.

Bu çalışmanın amacı, eşitlenmiş ve eşitlenmemiş verilerin katma-değerli değerlendirme analizlerine etkisini incelemektir. Katma-değerli değerlendirme yaklaşımını uygulayabilmek için literatürde birçok model önerilmiştir. Bu çalışma, iki farklı katma değerli değerlendirme modeli karşılaştırmıştır: düzeltilmemiş hiyerarşik doğrusal model (UHLMM) ve genelleştirilmiş süreklilik (GP) modeli. Birinci model eşitlenmiş testler için kullanılırken, ikincisi bu varsayımı esnetir. Her iki modelde iki farklı veri seti (eşitlenmiş ve eşitlenmemiş) analiz edildi. Her iki model için eyalet çapında yapılan bir sınav (eşitlenmiş) ve ülke çapında yapılan bir sınav (eşitlenmemiş) verilerine dayanan katma-değer kestirimleri genellikle tutarlı bulundu. Okul sıralamalarında iki model arasında bazı farklılıklar gözlendi. Bu çalışmanın pratik çıkarımı, okul sıralamasında küçük farklılıklar olmasına rağmen, test formları arasında eşitlemenin mümkün olmadığı durumlarda eşitlenmemiş bir veri seti gerektiren bir modelin uygulanabileceğidir.

  • Aitkin, M., & Longford, N. (1986). Statistical modeling in school effectiveness studies. Journal of the Royal Statistical Society, Series A, 149, 1–43.
  • Balcı, A. (1988). Etkili okul. Eğitim ve Bilim, 12(70), 21-30.
  • Ballou, D. (2009). Test scaling and value-added measurement. Education Finance and Policy, 4, 351–383.
  • Ballou, D., Sanders, W. L., & Wright, P. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29, 37–66.
  • Beardsley, A. A. (2008). Methodological concerns about the education value-added assessment system. Educational Researcher, 37(2), 65–75.
  • Bessent, A. M., & Bessent, E. W. (1980). Determining the comparative efficiency of schools through data envelopment analysis. Educational Administration Quarterly, 16(2), 57–75.
  • Boran, A., Atalmis, E. H., Sagir, E. (2015). Özel öğretim kurs merkezi öğretmenleri ve çalışma koşulları [Private tutoring centers and their working conditions]. Turkish Journal of Education 4(4), 17–29.
  • Braun, H. (2005). Value-added modeling: What does due diligence require? In R.W. Lissitz (Ed.), Value-added models in education: Theory and application (pp. 19–40). Maple Grove, MN: JAM Press.
  • Braun, H., & Wainer, H. (2007). Value added modeling. In C.R. Rao & S. Sinharay (Eds.) Handbook of Statistics, Vol. 26. Amsterdam: Elsevier.
  • Briggs, D. C., & Domingue, B. (2013). The gains from vertical scaling. Journal of Educational and Behavioral Statistics, 38(6), 551–576.
  • Briggs, D. C., & Weeks, J. P. (2009). The sensitivity of value-added modeling to the creation of a vertical score scale. Education Finance and Policy, 4, 384–414. doi:10.1162/edfp.2009.4.4.384
  • Briggs, D. C., Weeks, J. P., & Wiley, E. (2008, April). Vertical scaling in value-added models for student learning. National Conference on Value-Added Modeling, Madison, WI.
  • Broatch, J., & Lohr, S. (2012). Multidimensional assessment of value added by teachers to real-world outcomes. Journal of Educational and Behavioral Statistics, 37, 256–277.
  • Doran, H. C., & Cohen, J. (2005). The confounding effect of linking bias on gains estimated from value-added models. In R. Lissitz (Ed.), Value-added models in education: Theory and application (pp. 80–104). Maple Grove, MN: JAM Press.
  • Ercikan, K. (2006). Development in assessment of student learning. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 929–952). Mahwah, NJ: Erlbaum.
  • Hanushek, E. A. (1972). Education and race: An analysis of the educational production process. Lexington, MA: Lexington Books.
  • Hill, H. C. (2009). Evaluating value-added models: A validity argument approach. Journal of Policy Analysis and Management, 28, 700–709. doi:10.1002/pam.20463
  • Karl, A. T., Yang, Y., & Lohr, S. (2012). GPvam: maximum likelihood estimation of multiple membership mixed models used in value-added modeling. R Package Version 2.0-0.
  • Lockwood, J. R., McCaffrey, D. F., Hamilton, L. S., Stecher, B., Le, V. N., & Martinez, J. F. (2007). The sensitivity of value‐added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement, 44(1), 47–67.
  • Mariano, L. T., McCaffrey, D. F., & Lockwood, J. R. (2010). A model for teacher effects from longitudinal data without assuming vertical scaling. Journal of Educational and Behavioral Statistics, 35(3), 253–279.
  • Martineau, J. (2006). Distorting value-added: The use of longitudinal, vertically scaled student achievement data for value-added accountability. Journal of Educational and Behavioral Statistics, 31, 35–62.
  • Marzano, R. J. (2003). What works in schools: Translating research into action? Alexandria, VA: ASCD.
  • McCaffrey, D., Lockwood, J. R., Koretz, D., & Hamilton, L. (2003). Evaluating value-added models for teacher accountability. Washington, DC: RAND.
  • McCaffrey, D., Lockwood, J., Koretz, D., Louis, T., & Hamilton, L. (2004). Models for value added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29, 67–101.
  • Murnane, R. J. (1975). The impact of school resources on the learning of children. Cambridge, MA: Ballinger Publishing.
  • Raudenbush, S. & Bryk, A. S. (1986). A hierarchical model for studying school effects. Sociology of Education, 59, 1–17.
  • Reckase, M. D. (2004). The real world is more complicated than we would like. Journal of Educational and Behavioral Statistics, 29, 117–120.
  • Sanders, W. L. & Horn, S. P. (1994). The Tennessee Value-Added Assessment System (TVAAS): Mixed model methodology in educational assessment. Journal of Personnel Evaluation in Education, 8, 299–311.
  • Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on future student academic achievement. Knoxville: University of Tennessee, Value-Added Research and Assessment Center.
  • Sanders, W. L., Saxton, A. M., & Horn, S. P. (1997). The Tennessee Value-Added Assessment System: A quantitative, outcomes-based approach to educational assessment. In J. Millman, (Ed.), Grading teachers, grading schools. Is student achievement a valid evaluation measure? (pp. 137–162). Thousand Oaks, CA:Corwin.
  • Sanders, W. L., Saxton, A., Schneider, J., Dearden, B., Wright, S. P., & Horn, S. (2002). Effects of building change on indicators of student achievement growth: Tennessee Value-Added Assessment System. Knoxville, TN: University of Tennessee Value-Added Research and Assessment Center.
  • Shaw, L. H. (2012). Incorporating latent variable outcomes in value-added assessment: An evaluation of univariate and multivariate measurement model structures. (Unpublished doctoral dissertation). University of Nebraska, Digital Commons at the University of Nebraska-Lincoln.
  • Şen, S., Kim, S.-H., & Cohen, A. S. (2017). Comparative analysis of common statistical models used for value-added assessment of school performance. Journal of Measurement and Evaluation in Education and Psychology, 8(3), 303–320.
  • Tekwe, C. D., Carter, R. L., Ma, C., Algina, J., Lucas, M. E., Roth, J., … Resnick, M. B. (2004). An empirical comparison of statistical models for value-added assessment of school performance. Journal of Educational and Behavioral Statistics, 29, 11–36.
  • Wright, S. P., Horn, S. P., & Sanders, W. L. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 1(1), 57–67.
  • Yıldırım, İ., & Şen, S. (2018). Katma-değerli değerlendirme modellerinde test eşitleme durumunun incelenmesi. In S. Dinçer, (Ed.), Değişen dünyada eğitim (pp. 125–134). Ankara: Pegem Akademi.
Birincil Dil en
Konular Eğitim, Bilimsel Disiplinler
Dergi Bölümü Araştırma Makaleleri
Yazarlar

Orcid: 0000-0001-6962-4960
Yazar: Sedat Şen (Sorumlu Yazar)
Kurum: Harran University
Ülke: Turkey


Orcid: 0000-0003-3976-5054
Yazar: Ragıp Terzi
Kurum: Harran University
Ülke: Turkey


Orcid: 0000-0002-4137-2025
Yazar: İbrahim YILDIRIM
Kurum: Harran University
Ülke: Turkey


Orcid: 0000-0002-8776-9378
Yazar: Allan Cohen
Kurum: University of Georgia
Ülke: United States


Bibtex @araştırma makalesi { turje456656, journal = {Turkish Journal of Education}, issn = {}, eissn = {2147-2858}, address = {Mehmet TEKEREK}, year = {2018}, volume = {7}, pages = {186 - 195}, doi = {10.19128/turje.456656}, title = {Does equating matter in value-added models?}, key = {cite}, author = {YILDIRIM, İbrahim and Şen, Sedat and Terzi, Ragıp and Cohen, Allan} }
APA Şen, S , Terzi, R , YILDIRIM, İ , Cohen, A . (2018). Does equating matter in value-added models?. Turkish Journal of Education, 7 (4), 186-195. DOI: 10.19128/turje.456656
MLA Şen, S , Terzi, R , YILDIRIM, İ , Cohen, A . "Does equating matter in value-added models?". Turkish Journal of Education 7 (2018): 186-195 <http://dergipark.gov.tr/turje/issue/39139/456656>
Chicago Şen, S , Terzi, R , YILDIRIM, İ , Cohen, A . "Does equating matter in value-added models?". Turkish Journal of Education 7 (2018): 186-195
RIS TY - JOUR T1 - Does equating matter in value-added models? AU - Sedat Şen , Ragıp Terzi , İbrahim YILDIRIM , Allan Cohen Y1 - 2018 PY - 2018 N1 - doi: 10.19128/turje.456656 DO - 10.19128/turje.456656 T2 - Turkish Journal of Education JF - Journal JO - JOR SP - 186 EP - 195 VL - 7 IS - 4 SN - -2147-2858 M3 - doi: 10.19128/turje.456656 UR - http://dx.doi.org/10.19128/turje.456656 Y2 - 2018 ER -
EndNote %0 Turkish Journal of Education Does equating matter in value-added models? %A Sedat Şen , Ragıp Terzi , İbrahim YILDIRIM , Allan Cohen %T Does equating matter in value-added models? %D 2018 %J Turkish Journal of Education %P -2147-2858 %V 7 %N 4 %R doi: 10.19128/turje.456656 %U 10.19128/turje.456656
ISNAD Şen, Sedat , Terzi, Ragıp , YILDIRIM, İbrahim , Cohen, Allan . "Katma-değerli değerlendirme modellerinde eşitleme önemli mi?". Turkish Journal of Education 7 / 4 (Ekim 2018): 186-195. http://dx.doi.org/10.19128/turje.456656