Evaluation Model of Virtual Learning Environments: A Pilot Study

Alane de Almeida Ferreira; Neide dos Santos; Rosa Maria E. M. da Costa; Fernando Antonio  de A. Carneiro; Vera Maria B. Werneck

doi:10.31686/ijier.vol8.iss8.2569

Authors

Alane de Almeida Ferreira
Neide dos Santos
Rosa Maria E. M. da Costa

State University of Rio de Janeiro
Fernando Antonio de A. Carneiro
Vera Maria B. Werneck

State University of Rio de Janeiro

DOI:

https://doi.org/10.31686/ijier.vol8.iss8.2569

Keywords:

Evaluation Models, Learning Environments, Learning

Abstract

Virtual learning environments (VLE) have frequently been used in educational practices, and the evaluation of their effectiveness as instruments to support learning gains must consider several dimensions. This paper presents an evaluation model for VLE, called MA-AVA (Model for the Evaluation of VLE), built after a review of the literature and focused on verifying students' learning gains. The MA-AVA evaluation model was applied in a pilot study to an undergraduate engineering class, using a VLE, Educ-MAS-GA, in the discipline of Analytical Geometry. The results indicate that, although students' perception of learning in VLE is relevant, the knowledge acquired is more subtle and difficult to assess. Therefore, a VLE learning evaluation model should include different dimensions of learning, such as the students’ perceptions and their measures of learning gain.

References

V. Agredo-Delgado, J. D.Pinto-Corredor, C. A. Collazos, P.H. Ruiz & H. M. Fardoun, (2019). “Structure of a Guide for Usability Evaluation in Virtual Learning Environments”, In: Ruiz P., Agredo-Delgado V. (eds) Human-Computer Interaction. HCI-COLLAB 2019. Communications in Computer and Information Science, 1114. Springer, Cham. Retrieved from https://doi.org/10.1007/978-3-030-37386-3_26 DOI: https://doi.org/10.1007/978-3-030-37386-3_26

M. Aldredge, L. Dubois, D. Mobley, P. Abington, & M. Vienne, (2019). “Maintaining Quality in Online Learning Environments-Issues and Chalengers”, International Journal for Innovation Education and Research, 7(12), pp. 361-367. Retrieved from https://doi.org/10.31686/ijier.vol7.iss12.2077 DOI: https://doi.org/10.31686/ijier.vol7.iss12.2077

Anonymous, 2016.

B. S. Bloom, (1956). Taxonomy of educational objectives, Handbook I: Cognitive Domain. 2nd ed. New York: Addison-Wesley Longman Ltd.

J. L. Bruso & J. E. Stefaniak. (2016). “The use of self-regulated learning measure questionnaires as a predictor of academic success”. TechTrends, 60(6), pp. 577-584. Retrieved from https://doi.org/10.1007/s11528-016-0096-6 DOI: https://doi.org/10.1007/s11528-016-0096-6

M. Gustafsson, C. Englund & G. Gallego. (2017). “The description and evaluation of virtual worlds in clinical pharmacy education in Northern Sweden”, Currents in Pharmacy Teaching and Learning, 9(5), pp. 887-892. Retrieved from https://doi.org/10.1016/j.cptl.2017.06.002 DOI: https://doi.org/10.1016/j.cptl.2017.06.002

J. M. Keller, (2009). “The Arcs model of motivational design. Motivational design for learning and performance: The ARCS model approach”. Springer Science & Business Media, USA. doi: 10.107/978-1-4419-1250-3

D. L. Kirkpatrick, (1996). “Techniques for evaluating training programs”. In: Ely D, Plomp T, eds. Classic Writings on Instructional Technology. Volume 1. Englewood, CO: Libraries Unlimited, pp. 119-142.

C. Lin, E. Z. Liu, Y. Chen, P. Liou,M. Chang, C. Wu, & S. Yuan. (2013). “Game-Based Remedial Instruction in Mastery Learning for Upper-Primary School Students”. Educational Technology & Society, 16(2), pp. 271-281. Retrieved from https://www.jstor.org/stable/jeductechsoci.16.2.271

T. Markova,I. Glazkova, & E. Zaborova. (2017). “Quality issues of online distance learning”. Procedia-Social and Behavioral Sciences, 237, pp. 685-691. doi: 10.1016/j.sbspro.2017.02.043 DOI: https://doi.org/10.1016/j.sbspro.2017.02.043

D. Moody, G. Sindre. (2002). “Evaluating the Effectiveness of Learning Interventions: An Information Systems Case Study”, In: 11th European Conference on Information Systems, Italy. Retrieved from https://aisel.aisnet.org/ecis2003/80

L. Nepomuceno, A. Silva, D. Xavier, J. Barbosa, A. Araújo, H. Borges Neto & A. Torres. (2019). “TeleMeios as a Virtual Environment and their possibilities in Hybrid education”, International Journal for Innovation Education and Research v. 7, n. 11, pp. 1330-1340. Retrieved from https://doi.org/10.31686/ijier.vol7.iss11.2007 DOI: https://doi.org/10.31686/ijier.vol7.iss11.2007

O. Pastushenko, T. Hruska & J. Zendulka. (2018). “Increasing students' motivation by using virtual learning environments based on gamification mechanics: Implementation and evaluation of gamified assignments for students”. In: Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality. pp. 755-760. Retrieved from https://doi.org/10.1145/3284179.3284310 DOI: https://doi.org/10.1145/3284179.3284310

Portal do Professor. (2011) Retrieved from http://portaldoprofessor.mec.gov.br/index.html

E. Rahimi, J. Van Den Berg & W. Veen (2015). “A learning model for enhancing the student's control in educational process using Web 2.0 personal learning environments”. British Journal of Educational Technology, 46(4), pp. 780-792. Retrieved from https://doi.org/10.1111/bjet.12170 DOI: https://doi.org/10.1111/bjet.12170

J. L. Sabourin, L. R. Shores,B. W. Mott & J. C. Lester. (2013). “Understanding and predicting student self-regulated learning strategies in game-based learning environments”, International Journal of Artificial Intelligence in Education, 23(1-4), pp. 94-114. Retrieved from https://doi.org/10.1007/s40593-013-0004-6 DOI: https://doi.org/10.1007/s40593-013-0004-6

J. Sandars, N. Lafferty. (2010). “Twelve tips on usability testing to develop effective e-learning in medical education”. Medical teacher, 32(12). 956-960. Retrieved from https://doi.org/10.3109/0142159X.2010.507709 DOI: https://doi.org/10.3109/0142159X.2010.507709

R. Savi, R., C. G. von Wangenheim & A. F. Borgatto. (2011). “A model for the evaluation of educational games for teaching software engineering”. Proccedings on XXV Simpósio Brasileiro de Engenharia de Software, pp. 194-203. São Paulo, Brasil. doi: 10.1109/SBES.2011.27 DOI: https://doi.org/10.1109/SBES.2011.27

R. Sousa, G. Percú, P. Pinto, O. Bernardo Filho & V. M. Werneck. (2012). “Avaliação Diagnóstica Fuzzy no Educ-MAS GA”. In: Anais do II Congresso Brasileiro de Sistemas Fuzzy. Retrieved from http://www.dimap.ufrn.br/~cbsf/pub/anais/2012/10001031.pdf (in Portuguese).

C. Von Wangenhein, M. Thiry & D. Kochanski. (2009). “Empirical evaluation of an educational game on software measurement”. Empirical Software Engineering, 14(4), pp. 418-452. Retrieved from https://doi.org/10.1007/s10664-008-9092-6 DOI: https://doi.org/10.1007/s10664-008-9092-6

C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell & A. Wesslén. (2012), Experimentation in Software Engineering, First Edition, Springer, Berlin, Heidelberg. doi: 10/1007/978-3642-29044-2 DOI: https://doi.org/10.1007/978-3-642-29044-2