Determining the Measurement Quality of a Montessori High School Teacher Evaluation Survey
DOI:
https://doi.org/10.17161/jomr.v3i1.5871Palavras-chave:
Erdkinder, Rasch Modeling, Survey ResearchResumo
The purpose of this study was to conduct a psychometric validation of a course evaluation instrument, known as a student evaluation of teaching (SET), implemented in a Montessori high school. The authors demonstrate to the Montessori community how to rigorously examine the measurement and assessment quality of instruments used within Montessori schools. The Montessori high school community needs an SET that has been rigorously examined for measurement issues. The examined SET was developed by a Montessori high school, and the sample data were collected from Montessori high school students. Using a Rasch partial credit model, the results of the analysis identified several measurement issues, including multidimensionality, misfit items, and inappropriate item difficulty levels. A revised version of the SET underwent the same analysis procedure, and the results indicated that measurement issues persisted. The authors suggest several ways to improve the overall measurement quality of the instrument while keeping the Montessori foundation. Additional validation studies with a revised version of the SET will be needed before the instrument can be endorsed for full implementation in a Montessori setting.
Referências
American Montessori Society. (2015). Montessori schools. Retrieved from http://amshq.org/Montessori-Education/Introduction-to-Montessori/Montessori-Schools
Barker, D. (2011). A historical look at Montessori’s Erdkinder. Communications, 1-2, 96–112.
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). New York, NY: Lawrence Erlbaum Associates.
Bradley, K. D., Sampson, S. O., & Royal, K. D. (2006). Applying the Rasch rating scale model to gain insights into students’ conceptualisation of quality mathematics instruction. Mathematics Education Research Journal, 18, 11–26. doi:10.1007/BF03217433
de Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: The Guilford Press.
Donahoe, M., Cichucki, P., Coad-Bernard, S., Coe, B., & Scholtz, B. (2013). Best practices in Montessori secondary programs. Montessori Life, 25(2), 16–23.
Feldman, K. A. (1976). The superior college teacher from the students’ view. Research in Higher Education, 5, 243–288.
Kahn, D., & Pendleton, D. R. (2007). The whole-school Montessori handbook for teachers and administrators. Burton, OH: North American Montessori Teachers’ Association.
Kahn, D. (2011). Eight pictures at an exhibition: A Montessori retrospective on the discovery of the adolescent. Communications, 1-2, 15–41.
Kulik, J. A. (2001). Student ratings: Validity, utility, and controversy. New Directions for Institutional Research, 109, 9–25. doi:10.1002/ir.1
Linacre, J. M. (2015). A user guide to Winsteps Ministeps Rasch-model computer programs: Program manual 3.90.0. Available from Winsteps http://www.winsteps.com/manuals.htm
Linacre, J. M. (2016). Winsteps [Computer software]. Available from Winsteps http://www.winsteps.com
Marsh, H. W., & Dunkin, M. J. (1992). Students’ evaluations of university teaching: A multidimensional perspective. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 8, pp. 143–233). New York, NY: Agathon Press.
Mayes, C., & Williams, E. (2013). Nurturing the whole student: Five dimensions of teaching and learning. Lanham, MD: Rowman & Littlefield Education.
Miller, J. P. (2010). Whole child education. Canada: University of Toronto Press.
Miller, R. (1990). What are schools for? Holistic education in American culture. Brandon, VT: Holistic Education Press.
Montessori, M. (1973). From childhood to adolescence: Including Erdkinder and the function of the university. New York, NY: Schocken Books.
Montessori, M. (2011). Principles and practice in education. Communications, 1-2, 50–60. (Reprinted from First Lecture, Institute of Medical Psychology, London, November 10, 1936.)
Nardi, P. M. (2006). Doing survey research: A guide to quantitative methods. Boston, MA: Pearson Education.
National Center for Montessori in the Public Sector. (2014). USA Montessori census. Available from http://www.montessoricensus.org
North American Montessori Teachers’ Association. (2015). Curriculum downloads. Available from http://www.montessori-namta.org/Curriculum-Downloads
Royal, K. D., & Elahi, F. (2011). Psychometric properties of the Death Anxiety Scale (DAS) among terminally ill cancer patients. Journal of Psychosocial Oncology, 29, 359–371. http://www.tandfonline.com/doi/abs/10.1080/07347332.2011.582639
Toland, M. D. (2014). Practical guide to conducting an item response theory analysis. The Journal of Early Adolescence, 34, 120–151. doi:10.1177/0272431613511332
Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago, IL: MESA Press.
Wright, B. D. (1993). “Logits”? Rasch Measurement Transactions, 7, 228. Retrieved from https://www.rasch.org/rmt/rmt72e.htm
Wright, S. L., & Jenkins-Guarnieri, M. A. (2012). Student evaluations of teaching: Combining the meta-analyses and demonstrating further evidence for effective use. Assessment & Evaluation in Higher Education, 37, 683–699. http://dx.doi.org/10.1080/02602938.2011.563279
