Readability of Reading Passages in the General Aptitude Test (GAT) and English Textbooks for the Upper Secondary Level in Thailand by Using Coh-Metrix
Article Sidebar
Main Article Content
Abstract
This study aimed to compare the readability levels of the reading passages in the General Aptitude Test (GAT) and those of English textbooks for the Upper Secondary Level in Thailand. The corpus of the General Aptitude Tests (C-GAT) was compiled from 70 reading passages on GAT test papers from 2009-2021, and the corpus of English textbooks (C-ET) was compiled from 66 English textbook series. The data were quantitatively analyzed to investigate the readability levels of the reading passages in the two sets of corpora. The instrument of this study was Coh-Metrix - a common core text ease and readability assessor. The data were analyzed based on 11 categories of language characteristics divided into 106 indices. The results showed that the reading passages in the C-ET were easier than those in the C-GAT in 10 out of 11 categories with significant mean values in 4 categories: text easability principal component scores, referential cohesion, connectives, and word information. As a result, it can be concluded that the readability levels of the teaching and testing materials were incongruent. Thus, English textbook designers, English teachers, and test developers need to concentrate on the relationship among reading passages used in both national tests and English textbooks in order to achieve the right balance between students’ English reading skills and results of the tests.
Article Details
References
Abdollahi-Guilani, M. (2022). Readability index and reading complexity in high school EFL textbooks. Foreign Language Research Journal, 12(2), 37-51. https://jflr.ut.ac.ir/jufile?ar_sfile=1372261
Ağçam, R., & Babanoğlu, M. (2018). A comparative study on EFL textbooks in Turkish and German secondary public schools. International Journal of Eurasia Social Sciences, 9(32), 948-959. https://www.researchgate.net/profile/M-Babanoglu/publication/327594574_A_comparative_study_on_EFL_textbooks_in_Turkish_and_German_secondary_public_schools/links/5b98f45592851c4ba813e93c/A-comparative-study-on-EFL-textbooks-in-Turkish-and-German-secondary-public-schools.pdf
Anwar, A.S. (2017). Lexical density of reading material in English textbooks for the tenth graders: Interlanguage and pathway to English [Skripsi Sarjana Thesis]. Universitas Muria Kudus.
Apairach, S. (2023). Going the extra miles in a reading lesson: Insights from a Thai EFL classroom. LEARN Journal: Language Education and Acquisition Research Network, 16(1), 320–338. https://so04.tci-thaijo.org/index.php/LEARN/article/view/263444
Boghrati, R., Hoover, J., Johnson, K. M., Garten, J., & Dehghani, M. (2018). Conversation level syntax similarity metric. Behavior research methods, 50, 1055-1073. https://doi.org/10.3758/s13428-017-0926-2
Cain, K., & Nash, H. M. (2011). The influence of connectives on young readers' processing and comprehension of text. Journal of Educational Psychology, 103(2), 429–441. htttps://doi.org/10.1037/a0022824
Cheng, Y. S., & Chang, S. C. (2022). Do senior high school English textbooks sufficiently prepare students for the high-stakes college entrance examinations? A corpus-based analysis of text difficulty. Journal of Textbook Research, 15(1), 43-80. http://dx.doi.org/10.6481/JTR.202204_15(1).02
Chon, Yuah V., & Shin, Dongkwang. (2020). Direct writing, translated writing, and machine translated writing: A text level analysis with Coh-Metrix. English Teaching, 75 (1), 25-48. https://files.eric.ed.gov/fulltext/EJ1274324.pdf
Crossley, S. A., Skalicky, S., & Dascalu, M. (2019). Moving beyond classic readability formulas: new methods and new models. Journal of Research in Reading, 1-21. htttps://doi.org/10.1111/1467-9817.12283
Dang, T. C. T., & Seals, C. (2018). An evaluation of primary English textbooks in Vietnam: A sociolinguistic perspective. TESOL Journal, 9(1), 93-113. http://dx.doi.org/10.1002/tesj.309
Ebrahim, K., & Roghayeh, M. (2017). Evaluating two high intermediate EFL and ESL textbooks: A comparative study based on readability indices. Sociolinguistic International Journal, 1(3), 93-102. http://dx.doi.org/10.15406/sij.2017.01.00016
Faramarzi, A., & Janfeshan, K., & Artium, A. (2021). An investigation into ecolinguistics representation in Iranian high school English textbooks. Ars Artium: An International Refereed Research Journal of English Studies and Culture, 9, 70-85.
Gashaye, S., & Manyazewal, G. (2020). Investigating the appropriateness of reading passages of grade seven English textbooks. Theory and Practice in Language Studies, 10(4), 372-378. http://dx.doi.org/10.17507/tpls.1004.04
Gizatulina, D., Ismaeva, F., Solnyshkina, M., Martynova, E., & Yarmakeev, I. (2020). Fluctuations of text complexity, the case of basic state examination in English. ISHS Web of Conferences, 6(03008), 1-5. https://doi.org/10.1051/shsconf/20208802001
Gopal, R., Maniam, M., Madzlan, N. A., B., Shukor, S. S., & Neelamegam, K. (2021). Readability formulas: An analysis into reading index of prose forms. Studies in English Language and Education, 8(3), 972-985. http://dx.doi.org/10.24815/siele.v8i3.20373
Graesser, A. C., & McNamara, D.S. (2011). Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science, 3(2), 371–398. http://dx.doi.org/10.1111/j.1756-8765.2010.01081.x
Graesser, A. C., McNamara, D.S., Cai, Z., Conley, M., Li, H., & Pennebaker, J. (2014). Coh-Metrix measures text characteristics at multiple levels of language and discourse. The Elementary School Journal, 115(2), 210-229. https://doi.org/10.1086/678293
Graesser, A.C., McNamara, D.S., Louwerse, M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36, 193-202. https://link.springer.com/article/10.3758/BF03195564
Hakim, A. A., Setyaningsih, E., & Cahyaningrum, D. (2021). Examining the readability level of reading texts in English textbook for Indonesian senior high school. Journal of English Language Studies, 6(1), 18-35. http://dx.doi.org/10.30870/jels.v6i1.8898
Hanifah, M., D.S., Wahyuni, & S., Haryati (2022). The readability of reading texts in English textbook for Indonesian senior high school. English Education Journal, 10(2), 81-82. https://jurnal.uns.ac.id/englishedu/article/view/58638/34502
Hidayatulloh, M. (2019). Lexical density and lexical variation in EFL junior high school textbook reading materials. [Thesis, The University Institutional Repository of Majapahit Islamic University]. http://repository.unim.ac.id/92/
Howie, A. (2022). Analysis of the Correlation between the Lexical Profile and Coh-Metrix 3.0 Text Easability and Readability Indices of the Korean CSAT from 1994–2022. Central Washington University. https://digitalcommons.cwu.edu/etd/1784
Huang, P. (2019). Textbook interaction: A study of the language and cultural contextualization of English learning textbooks. Learning, Culture and Social Interaction, 21, 87-99. http://dx.doi.org/10.1016/j.lcsi.2019.02.006
Imsa-ard, P. (2020). Voices from Thai EFL teachers: Perceptions and beliefs towards the English test in the national examination in Thailand. LEARN Journal: Language Education and Acquisition Research Network, 13(2), 269–287. https://so04.tci-thaijo.org/index.php/LEARN/ article/view/243713
Jukic, M. L., & Glasnovic, G. D. (2020). How do teacher guides give support to mathematics teachers? Analysis of a teacher guide and exploration of its use in teachers' practices. Research in Mathematics Education, 23(1), 1-20. http://dx.doi.org/10.1080/14794802.2019.1710554
Keith, B., Sloas, S., Mooney, M., & Norris, T. (2014). Readability of PTA textbooks and reading comprehension rates of PTA students. IARTEM E-Journal, 6(1), 37-47. https://doi.org/10.21344/iartem.v6i1.763
Lee, H. Y., & Kang, M. K. (2021). Text difficulty of English textbooks for senior high school and the college scholastic ability test in South Korea. Asia-Pacific Journal of Educational Management Research, 6(1), 55-62. http://dx.doi.org/10.21742/ajemr.2021.6.1.05
Lendo, N., Liando, N., & Olii, S. (2021). An analysis of readability of reading texts on English national examination on junior high school. Journal of English Culture, Language, Literature and Education, 9(2), 128-143. https://doi.org/10.53682/eclue.v9i2.3013
Li, Lijuan. (2014). Textbook reading strategies and its relationship to reading test performance. Gema Online Journal of Language Studies, 1, 1-18. htttps://doi.org/10.17576/GEMA-2014-1403-01.
Liao, L. (2020). A comparability study of text difficulty and task characteristics of parallel academic IELTS reading tests. English Language Teaching, 13(1), 31-42. htttps://doi.org/10.5539/elt.v13n1p31
Mallipa, I., & Murianty, R. (2019). English teachers’ decision in utilizing textbook in their classroom. Linguistic, English Education and Art (LEEA) Journal, 2(2), 116-131. https://doi.org/10.31539/leea.v2i2.588
McCarthy, M. P., Lightman, J. E., Dufty, F. D., & McNamara, S. D. (2019). Using Coh-Metrix to assess cohesion and difficulty in high-school textbooks. Proceedings of the Annual Meeting of the Cognitive Science Society, 28, 2556. htttps://doi.org/10.1.1.530.7721
McNamara, D. S., & Graesser, A. C. (2012). Coh-Metrix: An automated tool for theoretical and applied natural language processing. In P. McCarthy & C. Boonthum-Denecke (Eds.), Applied natural language processing: Identification, investigation and resolution (pp. 188-205). IGI Global.
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Coh-Metrix indices norms: Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.
McNamara, D. S., & Kintsch, W. (2009). Learning from texts: Effects of prior knowledge and text coherence. Discourse Processes, 22(3), 247-288. htttps://doi.org/10.1080/01638539609544975
Mirshojaee, S. B., & Sahragard, R. (2015). Reading comprehension passages of Iranian general English books and MA reading comprehension tests: A corpus analysis. Journal of Modern Research in English Language Studies, 2(2), 77-98. https://www.academia.edu/39817106/Reading_ Comprehension_Passages_of_Iranian_General_English_Books_and_MA_ Reading_Comprehension_Tests_A_Corpus_Analysis
National Institute of Educational Testing Service. (2021, November 25). NIETS data catalog. https://catalog.niets.or.th/group/1education
Nguyen, C.D. (2020). Lexical features of reading passages in English-language textbooks for Vietnamese high-school students: Do they foster both content and vocabulary gain? RELC Journal, 1(14), 1-14. https://doi.org/10.1177/0033688219895045
Nipakornkitti, N., & Adunyarittigun, D. (2018). An investigation of content validity in O-NET (English Subject) for the upper secondary level (Matthayom 6). Journal of Liberal Arts, 18(1), 56-75. https://so03.tci-thaijo.org/index.php/liberalarts/article/download/93733/90689/305479
Noviyenti, L. (2021). Analysis of English Alive textbook in terms of genres and lexical density. (2021). AL-ISHLAH: Jurnal Pendidikan, 13(2), 799-812. https://doi.org/10.35445/alishlah.v13i2.587
Odo, D. M. (2018). Evaluating SLA textbook comprehensibility: Comparing Coh-Metrix results with EFL teachers’ ratings. Modern English Education Journal, 15(3), 750-765. https://doi.org/10.18095/mee.2018.19.2.02
Oslund, E. L., Clemens, N. H., Simmons, D. C., & Simmons, L. E. (2017). The direct and indirect effects of word reading and vocabulary on adolescents’ reading comprehension: Comparing struggling and adequate comprehenders. Read Writ, 31, 355–379. https://doi.org/10.1007/s11145-017-9788-3
Parlindungan, F., Rifai, I., & Safriani, A. (2018). The representation of Indonesian cultural diversity in middle school English textbooks. Indonesian Journal of Applied Linguistics, 8(2), 289-302. http://dx.doi.org/10.17509/ijal.v8i2.13276
Rahmawati, L. (2018). A content analysis of the English textbook: Primary English as a second language”. [Thesis, UIN Sunan Ampel Surabaya]. http://digilib.uinsby.ac.id/23517/
Rami, A. S., & Abdalla. K. (2020). Analyzing the readability level of the English textbooks used in Jordan. Research in English Language Pedagogy, 8, 334-348. http://dx.doi.org/10.30486/relp.2020.1895290.1191.
Rimkeeratikul, S. (2022). Needs analysis on an English textbook entitled English for international business. LEARN Journal: Language Education and Acquisition Research Network, 15(1), 748–774. https://so04.tci-thaijo.org/index.php/LEARN/article/view/ 256745
Ryu, J., & Jeon, M. (2020). An analysis of text difficulty across grades in Korean middle school English textbooks using Coh-Metrix. Journal of Asia TEFL, 17(3), 921-936. https://doi.org/10.18823/asiatefl.2020.17.3.11.921
Satriawan, J. (2018). Lexical density of reading texts in English on Sky (VIII) textbook published by Erlangga [Thesis (Diploma)], UIN Raden Fatah Palembang]. http://repository.radenfatah.ac.id/5517/
Srisunakrua, T. (2018). The readability of reading passages in prescribed English textbooks and Thai national English tests [Doctor’s thesis, Thammasat University]. http://dx.doi.org/10.14457/TU.the.2018.338
Srisunakrua, T., & Chumworatayee, T. (2019). Readability of reading passages in English textbooks and the Thai national education English test: A comparative study. Arab World English Journal (AWEJ), 10(2), 257-269. http://dx.doi.org/10.2139/ssrn.3418124
Tapiero, I. (2007). Situation models and levels of coherence: Toward a definition of comprehension. Lawrence Erlbaum Associates Publishers.
Unaka NI, Statile A, Haney J, Beck AF, Brady PW, Jerardi KE. Assessment of readability, understandability, and completeness of pediatric hospital medicine discharge instructions. J Hosp Med, 12(2), 98-101. https://doi.org/10.12788/jhm.2688.
Uri, N. F. M., & Abd, A. M. S. (2018). Assessing readability of a national exam reading texts in Malaysia. European Journal of English Language Teaching, 4(1), 149-164. https://doi.org/10.5281/zenodo.1463061
Wood, C. L., Bustamante, K. N., Schatschneider, C., & Hart, S. (2018). Relationship between children’s lexical diversity in written narratives and performance on a standardized reading vocabulary measure. Assessment for Effective Intervention, 153450841774987, 1-11. https://doi.org/10.1177/1534508417749872
Wulan, J. (2017). The readability between reading texts in English textbook and reading texts in national examination of the ninth grade of SMP Negeri 27 Medan. [Thesis, State University of Medan]. http://digilib.unimed.ac.id/ 29303/
Zamanian, M., & Heydari, P. (2012). Readability of texts: State of the art. Theory & Practice in Language Studies, 2(1), 43-53. http:// https://doi.org/10.4304/tpls.2.1.