Designing AI-Generated and Human-Taught Short-Video Mandarin Lessons: Learner Needs and Instructional Design Quality - Zhou, Ning; Youngmee, Kotchaphan (Research Community and Social Development Journal, 2026)
Background and Objective: The growing use of short-video learning and artificial intelligence (AI) in education has intensified interest in scalable instructional formats, yet the quality of instructional design in AI-generated materials remains underexamined. Existing studies often evaluate learning outcomes or technology acceptance without establishing whether the instructional content itself is pedagogically sound. Methodology: This study adopts a design-oriented approach to examine how AI-taught and human-taught short-video Mandarin lessons can be systematically developed and evaluated. A sequential design was employed, including learner needs analysis (N = 180), controlled instructional design, and expert evaluation (N = 5). Two sets of videos were developed using identical content to isolate the effect of delivery mode. Instructional quality was assessed in terms of clarity and suitability. Results: Expert evaluation showed that human-taught videos achieved a higher overall quality rating (M = 4.31, SD = 0.57) than AI-generated videos (M = 4.04, SD = 0.96). Human-taught videos were particularly strong in clarity of explanation and naturalness of language delivery (M = 4.80, SD = 0.45), whereas AI-generated videos excelled in suitability for the short-video format (M = 5.00, SD = 0.00) and pronunciation consistency (M = 4.80, SD = 0.45). The findings suggest that instructional quality is shaped primarily by systematic design rather than delivery mode alone, and that the two approaches offer complementary pedagogical strengths. Discussion: This study advances a design validation perspective in AI-supported learning by demonstrating that instructional effectiveness depends on how learning materials are designed, not merely on how they are delivered.