Abstracts
Abstract
Despite the widespread adoption of data-based decision making (DBDM) policies in schools around the world, there is limited understanding of how teachers use DBDM in K-12 classrooms and the impact of DBDM training on teacher practices and student outcomes. This scoping review aims to provide an overview of the existing literature on the uses of DBDM by teachers globally and identify gaps in the field. The findings (a) highlight a geographical and temporal clustering, with a notable emphasis on studies conducted in the United States and the Netherlands and published in 2016–2017 and 2020–2022; (b) identify a gap in the literature, particularly in the context of online and secondary schools, where the predominant focus has been on elementary and in-person settings; and (c) suggest that although DBDM interventions have been found helpful in altering teacher practices and student outcomes, there is still a need for more sustainable support to enhance DBDM implementation. The study concludes with recommendations for future DBDM research, building on implications from previous interventions.
Keywords:
- K-12 Education,
- Teacher Practices,
- student outcomes,
- data-based decision making
Résumé
Malgré l'adoption généralisée des politiques de prise de décision fondée sur les données probantes (PDDP) dans les écoles à travers le monde, peu d’information est disponible au sujet de l’utilisation de la PDDP par les enseignants oeuvrant aux paliers primaire et secondaire, ainsi que sur l'impact de la formation en PDDP sur le comportement des enseignants et les résultats scolaires. Cette recension exploratoire vise à fournir un aperçu des écrits actuels sur les usages de la PDDP par les enseignants à l'échelle mondiale et à identifier les lacunes dans le domaine. Les résultats mettent en évidence les points suivants : (a) les études réalisées jusqu’à présent peuvent être groupées de manière géographique et temporelle, et ont surtout été réalisées aux États-Unis et aux Pays-Bas; de plus la majorité des études ont été publiées en 2016-2017 et 2020-2022 ; (b) il existe des lacunes importantes dans les écrits actuels, notamment par rapport au contexte des écoles en ligne et secondaires - les études actuelles reflètent davantage un intérêt pour les écoles élémentaires et les contextes d’études en présentiel ; et (c) les études recensées suggèrent que, bien que les interventions relatives à la PDDP se soient révélées utiles pour modifier les pratiques des enseignants et les résultats scolaires, les enseignants ont besoin d’un soutien plus durable pour améliorer la mise en oeuvre de la PDDP. Enfin, l'article fournit des recommandations pour la recherche sur la PDDP, en s'appuyant sur les conclusions des interventions précédentes.
Mots-clés :
- prise de décision fondée sur les données probantes (PDDP),
- Éducation primaire et secondaire,
- Pratiques enseignantes,
- Résultats des élèves
Appendices
Bibliography
- Abdusyakur, I., & Poortman, C. L. (2019). Study on data use in Indonesian primary schools. Journal of Professional Capital and Community, 4(3), 198–215. https://doi.org/10.1108/JPCC-11-2018-0029
- Admiraal, W., Vermeulen, J., & Bulterman-Bos, J. (2020). Teaching with learning analytics: How to connect computer-based assessment data with classroom instruction? Technology, Pedagogy and Education, 29(5), 577–591. https://doi.org/10.1080/1475939X.2020.1825992
- Andersen, I. G. (2020). What went wrong? Examining teachers’ data use and instructional decision making through a bottom-up data intervention in Denmark. International Journal of Educational Research, 102, 101585. https://doi.org/10.1016/j.ijer.2020.101585
- Behrens, J., Piety, P., DiCerbo, K., & Mislevy, R. (2018). Inferential foundations for learning analytics in the digital ocean. In D. Niemi, R. D. Pea, B. Saxberg, & R. E. Clark (Eds.), Learning analytics in education (pp. 1–48). Information Age Publishing Inc.
- Campos, F. C., Ahn, J., DiGiacomo, D. K., Nguyen, H., & Hays, M. (2021). Making sense of sensemaking: Understanding how K-12 teachers and coaches react to visual analytics. Journal of Learning Analytics, 8(3), 60–80.
- Carlson, D., Borman, G. D., & Robinson, M. (2011). A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33(3), 378–398. https://doi.org/10.3102/0162373711412765
- Cheng, L. (1999). Changing assessment: Washback on teacher perspectives and actions. Teaching and Teacher Education, 15(3), 253–271. https://doi.org/10.1016/S0742-051X(98)00046-8
- Cheng, L., & Curtis, A. (2004). Washback or backwash: A review of the impact of testing on teaching and learning. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 3–17). Lawrence Erlbaum.
- Christman, J. B., Ebby, C. B., & Edmunds, K. A. (2016). Data use practices for improved mathematics teaching and learning: The importance of productive dissonance and recurring feedback cycles. Teachers College Record, 118(11), 1–32. https://doi.org/10.1177/016146811611801101
- Copp, D. T. (2017). Policy incentives in Canadian large-scale assessment: How policy levers influence teacher decisions about instructional change. Education Policy Analysis Archives, 25, 115. https://doi.org/10.14507/epaa.25.3299
- Curry, K. A., Mwavita, M., Holter, A., & Harris, E. (2016). Getting assessment right at the classroom level: Using formative assessment for decision making. Educational Assessment, Evaluation and Accountability, 28, 89–104. https://doi.org/10.1007/s11092-015-9226-5
- Datnow, A., & Hubbard, L. (2015). Teachers’ use of assessment data to inform instruction: Lessons from the past and prospects for the future. Teachers College Record, 117(4), 1–26. https://doi.org/10.1177/016146811511700408
- Datnow, A., Choi, B., Park, V., & John, E. S. (2018). Teacher talk about student ability and achievement in the era of data-driven decision making. Teachers College Record, 120(4), 1–34. https://doi.org/10.1177/016146811812000408
- Datnow, A., Lockton, M., & Weddle, H. (2021). Capacity building to bridge data use and instructional improvement through evidence on student thinking. Studies in Educational Evaluation, 69, 100869. https://doi.org/10.1016/j.stueduc.2020.100869
- De Simone, J. J. (2020). The roles of collaborative professional development, self-efficacy, and positive affect in encouraging educator data use to aid student learning. Teacher Development, 24(4), 443–465. https://doi.org/10.1080/13664530.2020.1780302
- Dunn, K. E., Airola, D. T., Lo, W. J., & Garrison, M. (2013). What teachers think about what they can do with data: Development and validation of the data driven decision-making efficacy and anxiety inventory. Contemporary Educational Psychology, 38(1), 87–98. https://doi.org/10.1016/j.cedpsych.2012.11.002
- Ebbeler, J., Poortman, C. L., Schildkamp, K., & Pieters, J. M. (2016). Effects of a data use intervention on educators’ use of knowledge and skills. Studies in Educational Evaluation, 48, 19–31. https://doi.org/10.1016/j.stueduc.2015.11.002
- Faber, J., Glas, C., & Visscher, A. J. (2018). Differentiated instruction in a data-based decision-making context. School Effectiveness and School Improvement, 29(1), 43–63. https://doi.org/10.1080/09243453.2017.1366342
- Farley-Ripple, E. N., Jennings, A. S., & Buttram, J. (2019). Toward a framework for classifying teachers’ use of assessment data. AERA Open, 5(4), 2332858419883571. https://doi.org/10.1177/2332858419883571
- Farrell, C. C., & Marsh, J. A. (2016a). Metrics matter: How properties and perceptions of data shape teachers’ instructional responses. Educational Administration Quarterly, 52(3), 423–462. https://doi.org/10.1177/0013161X16638429
- Farrell, C. C., & Marsh, J. A. (2016b). Contributing conditions: A qualitative comparative analysis of teachers’ instructional responses to data. Teaching and Teacher Education, 60, 398–412. https://doi.org/10.1016/j.tate.2016.07.010
- Filderman, M. J., Austin, C. R., & Toste, J. R. (2019). Data-based decision making for struggling readers in the secondary grades. Intervention in School and Clinic, 55(1), 3–12. https://doi.org/10.1177/1053451219832991
- Gelderblom, G., Schildkamp, K., Pieters, J., & Ehren, M. (2016). Data-based decision making for instructional improvement in primary education. International Journal of Educational Research, 80, 1–14. https://doi.org/10.1016/j.ijer.2016.07.004
- Hebbecker, K., Förster, N., Forthmann, B., & Souvignier, E. (2022). Data-based decision-making in schools: Examining the process and effects of teacher support. Journal of Educational Psychology, 114(7), 1695. https://psycnet.apa.org/doi/10.1037/edu0000530
- Heinrich, C., & Good, A. (2018). Research-informed practice improvements: Exploring linkages between school district use of research evidence and educational outcomes over time. School Effectiveness and School Improvement, 29(3), 418–445. https://doi.org/10.1080/09243453.2018.1445116
- Ho, J. E. (2022). What counts? The critical role of qualitative data in teachers’ decision making. Evaluation and Program Planning, 91, 102046. https://doi.org/10.1016/j.evalprogplan.2021.102046
- Hoover, N. R., & Abrams, L. M. (2013). Teachers’ instructional use of summative student assessment data. Applied Measurement in Education, 26(3), 219–231. https://doi.org/10.1080/08957347.2013.793187
- Kempf, A. (2015). The school as factory farm: All testing all the time. In The pedagogy of standardized testing (pp. 13–28). Palgrave Macmillan US. https://doi.org/10.1057/9781137486653_2
- Keuning, T., Geel, M. V., Visscher, A., Fox, J.-P., & Moolenaar, N. M. (2016). The transformation of schools’ social networks during a data-based decision making reform. Teachers College Record, 118(9), 1–33. https://doi.org/10.1177/016146811611800908
- Lai, M. K., & McNaughton, S. (2016). The impact of data use professional development on student achievement. Teaching and Teacher Education, 60, 434–443. https://doi.org/10.1016/j.tate.2016.07.005
- Lai, M. K., Wilson, A., McNaughton, S., & Hsiao, S. (2014). Improving achievement in secondary schools: Impact of a literacy project on reading comprehension and secondary school qualifications. Reading Research Quarterly, 49(3), 305–334. https://doi.org/10.1002/rrq.73
- Maier, U. (2010). Accountability policies and teachers’ acceptance and usage of school performance feedback – A comparative study. School Effectiveness and School Improvement, 21(2), 145–165. https://doi.org/10.1080/09243450903354913
- Marsh, J. A. (2012). Interventions promoting educators’ use of data: Research insights and gaps. Teachers College Record, 114(11), 1–48. https://doi.org/10.1177/016146811211401106
- Marsh, J. A., Bertrand, M., & Huguet, A. (2015). Using data to alter instructional practice: The mediating role of coaches and professional learning communities. Teachers College Record, 117(4), 1–40. https://doi.org/10.1177/016146811511700411
- Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006, Nov 7). Making sense of data-driven decision making in education [Occasional Paper]. RAND Corporation.
- Michaud, R. (2016). The Nature of Teacher Learning in Collaborative Data Teams. Qualitative Report, 21(3), 529-545. https://doi.org/10.46743/2160-3715/2016.231.
- Park, V., & Datnow, A. (2017). Ability grouping and differentiated instruction in an era of data-driven decision making. American Journal of Education, 123(2), 281–306. https://doi.org/10.1086/689930
- Peters, M. T., Förster, N., Hebbecker, K., Forthmann, B., & Souvignier, E. (2021). Effects of data-based decision-making on low-performing readers in general education classrooms: Cumulative evidence from six intervention studies. Journal of Learning Disabilities, 54(5), 334–348. https://doi.org/10.1177/00222194211011580
- Rangel, V. S., Bell, E. R., & Monroy, C. (2017). A descriptive analysis of instructional coaches’ data use in science. School Effectiveness and School Improvement, 28(2), 217–241. https://doi.org/10.1080/09243453.2016.1255232
- Reed, D. K. (2015). Middle level teachers’ perceptions of interim reading assessments: An exploratory study of data-based decision making. RMLE Online: Research in Middle Level Education, 38(6), 1–13. https://doi.org/10.1080/19404476.2015.11462119
- Regan, K., Evmenova, A. S., Mergen, R. L., Verbiest, C., Hutchison, A., Murnan, R., Field, S., & Gafurov, B. (2023). The feasibility of using virtual professional development to support teachers in making data‐based decisions to improve students’ writing. Learning Disabilities Research & Practice, 38(1), 40–56. https://doi.org/10.1111/ldrp.12301
- Rodríguez‐Martínez, J. A., González‐Calero, J. A., del Olmo‐Muñoz, J., Arnau, D., & Tirado‐Olivares, S. (2023). Building personalised homework from a learning analytics based formative assessment: Effect on fifth‐grade students’ understanding of fractions. British Journal of Educational Technology, 54(1), 76–97. https://doi.org/10.1111/bjet.13292
- Schifter, C., Natarajan, U., Ketelhut, D. J., & Kirchgessner, A. (2014). Data-driven decision-making: Facilitating teacher use of student data to inform classroom instruction. Contemporary Issues in Technology and Teacher Education, 14(4), 419–432.
- Schildkamp, K., & Ehren, M. (2013). From “intuition” to “data”-based decision making in Dutch secondary schools? In K. Schildkamp, M. K. Lai, & L. Earl (Eds.), Data-based decision making in Education (pp. 49–67). Springer Netherlands. https://doi.org/10.1007/978-94-007-4816-3_4
- Schildkamp, K., Poortman, C., & Handelzalts, A. (2016). Data teams for school improvement. School Effectiveness and School Improvement, 27(2), 228–254. https://doi.org/10.1080/09243453.2015.1056192
- Schildkamp, K., Poortman, C., Luyten, H., & Ebbeler, J. (2017). Factors promoting and hindering data-based decision making in schools. School Effectiveness and School Improvement, 28(2), 242–258. https://doi.org/10.1080/09243453.2016.1256901
- Shirley, D. (2017). The new imperatives of educational change: Achievement with integrity. Routledge. https://doi.org/10.4324/9781315682907
- Staman, L., Timmermans, A. C., & Visscher, A. J. (2017). Effects of a data-based decision making intervention on student achievement. Studies in Educational Evaluation, 55, 58–67. https://doi.org/10.1016/j.stueduc.2017.07.002
- Staman, L., Visscher, A. J., & Luyten, H. (2014). The effects of professional development on the attitudes, knowledge and skills for data-driven decision making. Studies in Educational Evaluation, 42, 79–90. https://doi.org/10.1016/j.stueduc.2013.11.002
- Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018). PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–485. https://doi.org/10.7326/M18-0850
- Tsai, Y., Poquet, O., Gašević, D., Dawson, S., & Pardo, A. (2019). Complexity leadership in learning analytics: Drivers, challenges and opportunities. British Journal of Educational Technology, 50(6), 2839–2854. https://doi.org/10.1111/bjet.12846
- Truckenmiller, A. J., Cho, E., & Troia, G. A. (2022). Expanding assessment to instructionally relevant writing components in middle school. Journal of School Psychology, 94, 28–48. https://doi.org/10.1016/j.jsp.2022.07.002
- van den Bosch, R. M., Espin, C. A., Chung, S., & Saab, N. (2017). Data‐based decision‐making: Teachers’ comprehension of curriculum‐based measurement progress‐monitoring graphs. Learning Disabilities Research & Practice, 32(1), 46–60. https://doi.org/10.1111/ldrp.12122
- van Geel, M., Keuning, T., Visscher, A. J., & Fox, J.-P. (2016). Assessing the Effects of a School-Wide Data-Based Decision-Making Intervention on Student Achievement Growth in Primary Schools. American Educational Research Journal, 53(2), 360–394. https://doi.org/10.3102/0002831216637346
- van der Scheer, E. A., & Visscher, A. J. (2016). Effects of an intensive data-based decision making intervention on teacher efficacy. Teaching and Teacher Education, 60, 34–43. https://doi.org/10.1016/j.tate.2016.07.025
- van der Scheer, E. A., Glas, C. A., & Visscher, A. J. (2017). Changes in teachers’ instructional skills during an intensive data-based decision making intervention. Teaching and Teacher Education, 65, 171–182. https://doi.org/10.1016/j.tate.2017.02.018
- Ylimaki, B., & Brunderman, L., (2019). School development in culturally diverse U.S. schools: Balancing evidence-based policies and education values. Education Sciences, 9(84), 1–15. https://doi.org/10.3390/educsci9020084