Abstracts
Abstract
The importance of teachers in online learning is widely acknowledged to effectively support and stimulate learners. With the increasing availability of learning analytics data, online teachers might be able to use learning analytics dashboards to facilitate learners with different learning needs. However, deployment of learning analytics visualisations by teachers also requires buy-in from teachers. Using the principles of technology acceptance model, in this embedded case-study, we explored teachers’ readiness for learning analytics visualisations amongst 95 experienced teaching staff at one of the largest distance learning universities by using an innovative training method called Analytics4Action Workshop. The findings indicated that participants appreciated the interactive and hands-on approach, but at the same time were skeptical about the perceived ease of use of learning analytics tools they were offered. Most teachers indicated a need for additional training and follow-up support for working with learning analytics tools. Our results highlight a need for institutions to provide effective professional development opportunities for learning analytics.
Keywords:
- learning analytics,
- information visualisation,
- learning dashboards,
- distance education
Appendices
Bibliography
- Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211. doi: 10.1016/0749-5978(91)90020-T
- Charleer, S., Klerkx, J., Duval, E., De Laet, T., & Verbert, K. (2016, September 13-16). Creating effective learning analytics dashboards: Lessons learnt. In K. Verbert, M. Sharples, & T. Klobučar (Eds.). Adaptive and adaptable learning. Paper presented at thee 11th European Conference on Technology Enhanced Learning, EC-TEL 2016 (pp. 42-56). Lyon, France: Springer International Publishing.
- Daley, S. G., Hillaire, G., & Sutherland, L. M. (2016). Beyond performance data: Improving student help seeking by collecting and displaying influential data in an online middle-school science curriculum. British Journal of Educational Technology, 47(1), 121-134. doi: 0.1111/bjet.12221
- Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340.
- Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1002.
- Ferguson, R., Brasher, A., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B.... Vuorikari, R. (2016). Research evidence of the use of learning analytics: Implications for education policy. In R. Vuorikari & J. Castano-Munoz (Eds.), A European framework for action on learning analytics (pp. 1-152). Luxembourg: Joint Research Centre Science for Policy Report.
- Fynn, A. (2016). Ethical considerations in the practical application of the Unisa socio-critical model of student success. International Review of Research in Open and Distributed Learning, 17(6). doi: 10.19173/irrodl.v17i6.2812
- Heath, J., & Fulcher, D. (2017). From the trenches: Factors that affected learning analytics success with an institution-wide implementation. Paper presented at the 7th International learning analytics & knowledge conference (LAK17; pp. 29-35). Vancouver, BC: Practitioner Track.
- Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., Hlosta, M., & Naydenova, G. (2017). Implementing predictive learning analytics on a large scale: The teacher's perspective. Paper presented at the Proceedings of the seventh international learning analytics & knowledge conference (pp. 267-271). Vancouver, British Columbia: ACM. Canada.
- Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z., & Wolff, A. (2015). OU Analyse: Analysing at-risk students at The Open University. Learning Analytics Review, LAK15-1, 1-16. Retrieved from http://www.laceproject.eu/publications/analysing-at-risk-students-at-open-university.pdf
- Jindal-Snape, D., & Topping, K. J. (2010). Observational analysis within case-study design. In S. Rodrigues (Ed.), Using analytical frameworks for classroom research collecting data and analysing narrative (pp. 19-37). Dordecht: Routledge.
- Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to evaluate: preparing learning analytics dashboards for educational practice. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge - LAK'18 (pp. 31-40). Sydney, New South Wales, Australia: ACM Press. doi: 10.1145/3170358.3170421
- Lawless, K. A., & Pellegrino, J. W. (2007). Professional development in integrating technology into teaching and learning: Knowns, unknowns, and ways to pursue better questions and answers. Review of Educational Research, 77(4), 575-614. doi: 10.3102/0034654307309921
- McKenney, S., & Mor, Y. (2015). Supporting teachers in data-informed educational design. British Journal of Educational Technology, 46(2), 265-279. doi: 10.1111/bjet.12262
- Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054.
- Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221-229. doi: 10.1111/bjet.12273
- Muñoz Carril, P. C., González Sanmamed, M., & Hernández Sellés, N. (2013). Pedagogical roles and competencies of university teachers practicing in the e-learning environment. International Review of Research in Open and Distributed Learning, 14(3). doi: 10.19173/irrodl.v14i3.1477
- Papamitsiou, Z., & Economides, A. (2016). Learning analytics for smart learning environments: A meta-analysis of empirical research results from 2009 to 2015. In J. M. Spector, B. B. Lockee, & D. M. Childress (Eds.), Learning, design, and technology: An international compendium of theory, research, practice, and policy (pp. 1-23). Cham: Springer International Publishing.
- Pynoo, B., Devolder, P., Tondeur, J., van Braak, J., Duyck, W., & Duyck, P. (2011). Predicting secondary school teachers' acceptance and use of a digital learning environment: A cross-sectional study. Computers in Human Behavior, 27(1), 568-575. doi: 10.1016/j.chb.2010.10.005
- Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action evaluation framework: A review of evidence-based learning analytics interventions at Open University UK. Journal of Interactive Media in Education, 1(2), 1-12. doi: 10.5334/jime.394
- Rienties, B., Brouwer, N., & Lygo-Baker, S. (2013). The effects of online professional development on higher education teachers' beliefs and intentions towards learning facilitation and technology. Teaching and Teacher Education, 29, 122-131. doi: 10.1016/j.tate.2012.09.002
- Rienties, B., Cross, S., & Zdrahal, Z. (2016). Implementing a learning analytics intervention and evaluation framework: what works? In B. Kei Daniel (Ed.), Big data and learning analytics in Higher Education (pp. 147-166). Cham: Springer International Publishing. doi: 10.1007/978-3-319-06520-5_10
- Rienties, B., Giesbers, S., Lygo-Baker, S., Ma, S., & Rees, R. (2016). Why some teachers easily learn to use a new Virtual Learning Environment: a Technology Acceptance perspective. Interactive Learning Environments, 24(3), 539-552. doi: 10.1080/10494820.2014.881394
- Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: a cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333-341. doi: 10.1016/j.chb.2016.02.074
- Sanchez-Franco, M. J. (2010). WebCT - The quasimoderating effect of perceived affective quality on an extending Technology Acceptance Model. Computers & Education, 54(1), 37-46. doi: 10.1016/j.compedu.2009.07.005
- Schwendimann, B. A., Rodríguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., ... Dillenbourg, P. (2017). Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30-41. doi: 10.1109/TLT.2016.2599522
- Shattuck, J., & Anderson, T. (2013). Using a design-based research study to identify principles for training instructors to teach online. International Review of Research in Open and Distributed Learning, 14(5). doi: 10.19173/irrodl.v14i5.1626
- Shattuck, J., Dubins, B., & Zilberman, D. (2011). Maryland online's inter-institutional project to train higher education adjunct faculty to teach online. International Review of Research in Open and Distributed Learning, 12(2). doi: 10.19173/irrodl.v12i2.933
- Slade, S., & Boroowa, A. (2014). Policy on ethical use of student data for learning analytics. Milton Keynes: Open University UK.
- Stenbom, S., Jansson, M., & Hulkko, A. (2016). Revising the community of inquiry framework for the analysis of one-to-one online learning relationships. The International Review of Research in Open and Distributed Learning, 17(3). doi: 10.19173/irrodl.v17i3.2068
- Šumak, B., Heričko, M., & Pušnik, M. (2011). A meta-analysis of e-learning technology acceptance: The role of user types and e-learning technology types. Computers in Human Behavior, 27(6), 2067-2077. doi: 10.1016/j.chb.2011.08.005
- Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157-167. doi: 10.1016/j.chb.2014.05.038
- Teo, T. (2010). A path analysis of pre-service teachers' attitudes to computer use: applying and extending the technology acceptance model in an educational context. Interactive Learning Environments, 18(1), 65-79. doi: 10.1080/10494820802231327
- Teo, T., & Zhou, M. (2016). The influence of teachers' conceptions of teaching and learning on their technology acceptance. Interactive Learning Environments, 25(4), 1-15. doi: 10.1080/10494820.2016.1143844
- van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2015). Teacher regulation of cognitive activities during student collaboration: Effects of learning analytics. Computers & Education, 90, 80-94. doi: 10.1016/j.compedu.2015.09.006
- Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning Analytics Dashboard Applications. American Behavioral Scientist, 57(10), 1500-1509. doi: 10.1177/0002764213479363
- Yin, R. K. (2009). Case-study research: Design and methods (5th ed.). Thousand Oaks: Sage.