Utilization of ChatGPT in educational environmental research: Assessing teachers’ evaluation skills on AI-generated data for educational environmental research

Maria Christoforaki; Evangelia Mavrikaki; Apostolia Galani

doi:10.29329/journalted.54

Authors

Maria Christoforaki National and Kapodistrian University of Athens, Greece https://orcid.org/0009-0003-3859-3470
Evangelia Mavrikaki National and Kapodistrian University of Athens, Greece https://orcid.org/0000-0001-9029-6340
Apostolia Galani National and Kapodistrian University of Athens, Greece https://orcid.org/0000-0002-2742-7392

DOI:

https://doi.org/10.29329/journalted.54

Keywords:

AI-generated data, Digital literacy, Evaluation Skills, Teachers

Abstract

The increasing use of Artificial Intelligence (AI) in education presents new challenges, particularly in evaluating the credibility of AI-generated content. This study explores the attitudes and evaluative skills of 279 in-service primary and secondary teachers in assessing AI-generated information on urban heatwaves, as a climate-related issue. Using a quantitative pre-and-post design, data were collected through a digital questionnaire administered before and after a targeted training intervention. The instrument, informed by the CRAAP test, assessed teachers’ ability to critically appraise AI-generated content. Findings revealed initial concerns about students’ reliance on AI and difficulties in evaluating the validity and reliability of AI-generated data. Post-training results demonstrated marked improvements in participants’ evaluative competencies across all CRAAP dimensions, particularly in assessing the currency and relevance of AI-generated content. These outcomes underscore the effectiveness of targeted training and reinforce the ongoing need to strengthen digital and data literacy among educators.

References

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922

Bozkurt, A., & Sharma, R. C. (2023). Challenging the status quo and exploring the new boundaries in the age of algorithms: Reimagining the role of generative AI in distance education and online learning. Asian Journal of Distance Education, 18(1). https://asianjde.com/ojs/index.php/AsianJDE/article/view/714

Brown, M., Bisbee, J., Lai, A., Bonneau, R., Nagler, J., & Tucker, J. A. (2022). Echo chambers, rabbit holes, and algorithmic bias: How YouTube recommends content to real users. SSRN. https://doi.org/10.2139/ssrn.4114905

Cheng, J., Xu, Z., Bambrick, H., Su, H., Tong, S., & Hu, W. (2018). Heatwave and elderly mortality: An evaluation of death burden and health costs considering short-term mortality displacement. Environment International, 115, 334–342. https://doi.org/10.1016/j.envint.2018.03.041

Jemetz, M., Dolezal, D., & Motschnig, R. (2025). Secondary teachers’ self-perceived AI competences in relation to renowned European digital competence frameworks. In Z. Pluhár & B. Gaál (Eds.), Informatics in schools. Innovative approaches to computer science teaching and learning. ISSEP 2024 (Lecture Notes in Computer Science, Vol. 15228, pp. 3–15). Springer. https://doi.org/10.1007/978-3-031-73474-8_1

Kong, S. C., Yang, Y., & Hou, C. (2024). Examining teachers’ behavioural intention of using generative artificial intelligence tools for teaching and learning based on the extended technology acceptance model. Computers and Education: Artificial Intelligence, 7, 100328. https://doi.org/10.1016/j.caeai.2024.100328

Lombardi, D., & Bailey, J. M. (2023). Science learning and teaching. In Handbook of educational psychology (pp. 531–552). Routledge.

Meriam Library, California State University, Chico. (n.d.). Evaluating information – Applying the CRAAP Test. https://guides.lib.chico.edu/c.php?g=414315&p=2828764

Naqvi, W. M., Ganjoo, R., Rowe, M., & Pashine, A. A. (2025). Critical thinking in the age of generative AI: Implications for health sciences education. Frontiers in Artificial Intelligence. (in press)

Nguyen, A., Ngo, H. N., Hong, Y., et al. (2023). Ethical principles for artificial intelligence in education. Education and Information Technologies, 28, 4221–4241. https://doi.org/10.1007/s10639-022-11316-w

Nunnally, J. C. (1978). An overview of psychological measurement. In Clinical diagnosis of mental disorders: A handbook (pp. 97–146). Springer. https://doi.org/10.1007/978-1-4684-2490-4_4

Oates, A., & Johnson, D. (2025). ChatGPT in the classroom: Evaluating its role in fostering critical evaluation skills. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-024-00452-8

Ottenbreit-Leftwich, A., Liao, J. Y. C., Sadik, O., & Ertmer, P. A. (2018). Evolution of teachers’ technology integration knowledge, beliefs, and practices: How can we support beginning teachers’ use of technology? Journal of Research on Technology in Education, 50(4), 282–304. https://doi.org/10.1080/15391523.2018.1487350

Selvarajan, G. (2025). Enhancing transparency in data analytics through explainable artificial intelligence. Journal of Computational Analysis and Applications. (in press)

Selwyn, N., Hillman, T., Eynon, R., Ferreira, G., Knox, J., Macgilchrist, F., & Sancho-Gil, J. M. (2019). What’s next for ed-tech? Critical hopes and concerns for the 2020s. Learning, Media and Technology, 45(1), 1–6. https://doi.org/10.1080/17439884.2020.1694945

Strubell, E., Ganesh, A., & McCallum, A. (2020). Energy and policy considerations for modern deep learning research. Proceedings of the AAAI Conference on Artificial Intelligence, 34(9), 13693–13696. https://doi.org/10.1609/aaai.v34i09.7123

Wibawa, A. P., Handayani, A. N., Anugrah, P., & Utama, A. B. P. (2024). A novel “conceive, design, implement, operate (CDIO)” framework for evaluating artificial intelligence–generated scholarly manuscripts. Science Editing, 12(1), 70–75. https://doi.org/10.6087/kcse.348

Wineburg, S., & McGrew, S. (2019). Learning more when evaluating digital information. Teachers College Record, 121(11), 1–40. https://doi.org/10.1177/016146811912101102

Wu, C.-J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Aga, F., Huang, J., Bai, C., Gschwind, M., Gupta, A., Ott, M., Melnikov, A., Candido, S., Brooks, D., Chauhan, G., Lee, B., Lee, H.-H., Akyildiz, B., Balandat, M., Spisak, J., Jain, R., Rabbat, M., & Hazelwood, K. (2022). Sustainable AI: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4, 795–813. https://doi.org/10.48550/arXiv.2111.00364

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – Where are the educators? International Journal of Educational Technology in Higher Education, 16, 39. https://doi.org/10.1186/s41239-019-0171-0