Designing AI-Generated and Human-Taught Short-Video Mandarin Lessons: Learner Needs and Instructional Design Quality
DOI:
https://doi.org/10.14456/rc-sdj.2026.2Keywords:
Human-taught instruction, Short-video learning, Mandarin as a foreign language, AI-assisted language learning, Media instructional design qualityAbstract
Background and Objective: The growing use of short-video learning and artificial intelligence (AI) in education has intensified interest in scalable instructional formats, yet the quality of instructional design in AI-generated materials remains underexamined. Existing studies often evaluate learning outcomes or technology acceptance without establishing whether the instructional content itself is pedagogically sound.
Methodology: This study adopts a design-oriented approach to examine how AI-taught and human-taught short-video Mandarin lessons can be systematically developed and evaluated. A sequential design was employed, including learner needs analysis (N = 180), controlled instructional design, and expert evaluation (N = 5). Two sets of videos were developed using identical content to isolate the effect of delivery mode. Instructional quality was assessed in terms of clarity and suitability.
Results: Expert evaluation showed that human-taught videos achieved a higher overall quality rating (M = 4.31, SD = 0.57) than AI-generated videos (M = 4.04, SD = 0.96). Human-taught videos were particularly strong in clarity of explanation and naturalness of language delivery (M = 4.80, SD = 0.45), whereas AI-generated videos excelled in suitability for the short-video format (M = 5.00, SD = 0.00) and pronunciation consistency (M = 4.80, SD = 0.45). The findings suggest that instructional quality is shaped primarily by systematic design rather than delivery mode alone, and that the two approaches offer complementary pedagogical strengths.
Discussion: This study advances a design validation perspective in AI-supported learning by demonstrating that instructional effectiveness depends on how learning materials are designed, not merely on how they are delivered.
References
Bakker, A. (2018). Design research in education: A practical guide for early career researchers (1st ed.). Routledge. https://doi.org/10.4324/9780203701010
Bond, M., Bedenlier, S., Marín, V. I., & Händel, M. (2021). Emergency remote teaching in higher education: Mapping the first global online semester. International Journal of Educational Technology in Higher Education, 18, 50. https://doi.org/10.1186/s41239-021-00282-x
Cao, M., Pavlik, P. I., & Bidelman, G. M. (2024). Enhancing lexical tone learning for second language speakers: Effects of acoustic properties in Mandarin tone perception. Frontiers in Psychology, 15, 1403816. https://doi.org/10.3389/fpsyg.2024.1403816
Conde-Caballero, D., Castillo-Sarmiento, C. A., Ballesteros-Yánez, I., Rivero-Jiménez, B., & Mariano-Juárez, L. (2024). Microlearning through TikTok in higher education: An evaluation of uses and potentials. Education and Information Technologies, 29(2), 2365–2385. https://doi.org/10.1007/s10639-023-11904-4
Deng, R., & Gao, Y. (2023). Using learner reviews to inform instructional video design in MOOCs. Behavioral Sciences, 13(4), 330. https://doi.org/10.3390/bs13040330
Galimova, E. G., Oborsky, A. Y., Khvatova, M. A., Astakhov, D. V., Orlova, E. V., & Andryushchenko, I. S. (2024). Mapping the interconnections: A systematic review and network analysis of factors influencing teachers' technology acceptance. Frontiers in Education, 9, 1436724. https://doi.org/10.3389/feduc.2024.1436724
Granić, A. (2022). Educational technology adoption: A systematic review. Education and Information Technologies, 27(7), 9725–9744. https://doi.org/10.1007/s10639-022-10951-7
Holmes, W., Bialik, M., & Fadel, C. (2022). Artificial intelligence in education: Promises and implications for teaching and learning (Updated ed.). Center for Curriculum Redesign. https://curriculumredesign.org/our-work/artificial-intelligence-in-education/
Kohnke, L., Zou, D., & Su, F. (2025). Exploring the potential of GenAI for personalised English teaching: Learners' experiences and perceptions. Computers and Education: Artificial Intelligence, 8, 100371. https://doi.org/10.1016/j.caeai.2025.100371
Law, L. (2024). Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature review. Computers and Education Open, 6, 100174. https://doi.org/10.1016/j.caeo.2024.100174
Liu, C., Wang, Y., Evans, M., & Correia, A.-P. (2024). Critical antecedents of mobile learning acceptance and moderation effects: A meta-analysis on technology acceptance model. Education and Information Technologies, 29(15), 20351–20382. https://doi.org/10.1007/s10639-024-12645-8
Mayer, R. E. (2021). Multimedia learning (3rd ed.). Cambridge University Press. https://doi.org/10.1017/9781316941355
McKenney, S., & Reeves, T. (2018). Conducting Educational Design Research (2nd ed.). Routledge. https://doi.org/10.4324/9781315105642
Montero Perez, M. (2022). Second or foreign language learning through watching audio-visual input and the role of on-screen text. Language Teaching, 55(2), 163–192. https://doi.org/10.1017/S0261444821000501
Niekrenz, L., & Spreckelsen, C. (2024). How to design effective educational videos for teaching evidence-based medicine to undergraduate learners. Medical Education Online, 29(1), 2339569. https://doi.org/10.1080/10872981.2024.2339569
Nieveen, N., & Folmer, E. (2013). Formative evaluation in educational design research. In T. Plomp & N. Nieveen (Eds.), Educational design research (pp. 152–169). SLO Netherlands Institute for Curriculum Development. https://www.slo.nl/publish/pages/2904/educational-design-research-part-a.pdf
Or, C. (2024). Thirty-five years of the technology acceptance model: Insights from meta-analytic structural equation modelling. The Open Technology in Education, Society, and Scholarship Association Journal, 4(3), 1–26. https://doi.org/10.18357/otessaj.2024.4.3.66
Plomp, T. (2013). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), Educational design research (pp. 10–51). SLO Netherlands Institute for Curriculum Development. https://www.slo.nl/publish/pages/2904/educational-design-research-part-a.pdf
Seidel, N. (2024). Short, long, and segmented learning videos: From YouTube practice to enhanced video players. Technology, Knowledge and Learning, 29(4), 1965–1991. https://doi.org/10.1007/s10758-024-09745-2
Sweller, J. (2023). The development of cognitive load theory: Replication crises and incorporation of other theories can lead to theory expansion. Educational Psychology Review, 35(4), 95. https://doi.org/10.1007/s10648-023-09817-2
Tan, S., Abd Samad, A., & Ismail, L. (2022). Systematic literature review on audio-visual multimodal input in listening comprehension. Frontiers in Psychology, 13, 980133. https://doi.org/10.3389/fpsyg.2022.980133
Teng, M. F. (2024). Incidental vocabulary learning from listening, reading, and viewing captioned videos: Frequency and prior vocabulary knowledge. Applied Linguistics Review, 15(4), 1607–1633. https://doi.org/10.1515/applirev-2023-0106
Tinoca, L., Piedade, J., Santos, S., Pedro, A., & Gomes, S. (2022). Design-based research in the educational field: A systematic literature review. Education Sciences, 12(6), 410. https://doi.org/10.3390/educsci12060410
Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.). (2006). Educational Design Research (1st ed.). Routledge. https://doi.org/10.4324/9780203088364
Xue, L., Mahat, J., & Ghazali, N. (2026). Technology acceptance model in artificial intelligence in education: A meta-analysis. SAGE Open, 16(1). https://doi.org/10.1177/21582440251409441
Zakariya, Y. F. (2022). Cronbach's alpha in mathematics education research: Its appropriateness, overuse, and alternatives in estimating scale reliability. Frontiers in Psychology, 13, 1074430. https://doi.org/10.3389/fpsyg.2022.1074430
Zawacki-Richter, O., Bai, J. Y. H., Lee, K., Slagter van Tryon, P. J., & Prinsloo, P. (2024). New advances in artificial intelligence applications in higher education?. International Journal of Educational Technology in Higher Education, 21(1), 32. https://doi.org/10.1186/s41239-024-00464-3
