Key Defining Linguistic Features in the Writing Performance of First-Year University Students Across Different Language Proficiency Levels
Article Sidebar
Main Article Content
Abstract
This study sought to investigate the key determining characteristics in the writing performance of first-year Chulalongkorn University students across language proficiency levels as measured by CU-TEP. The focus was on both syntactic and lexical complexity components. The sample comprised the writings from a corpus of 4,812 first-year students divided into four CEFR levels (C1, B2, B1, and A2), using CU-TEP and corresponding CEFR levels as the strata. The sample size of all four groups was identical comprising 50 students each, totaling 200 students. Multiple computational tools were utilized for data analysis. The findings revealed that the distinctive features typifying the most proficient writers include the production of longer as well as more clausally and phrasally complex sentences. They also demonstrated high lexical richness through the use of wide-ranging vocabulary and rare or sophisticated academic words. These features were also discovered in other less proficient groups but to a lesser extent at decreasing proficiency levels. It was also found that the syntactic complexity measures that better differentiated proficiency levels were: mean length of sentence, mean length of T-unit, and mean length of clause while all three lexical complexity indices were proven to be good predictors of L2 writing quality.
Article Details
References
Banerjee, J., franceschina, F., & Smith, A. M. (2007). 5. Documenting features of written language production typical at different IELTS band score levels (5). British Council. https://www.ielts.org/-/media/research-reports/ielts_rr_volume07_report5.ashx
Bayazaidi, A., Ansarin, A.-A., & Mohammadnia, Z. (2020). Lexical complexity as a function of task type and proficiency level in the speech monologs of Iranian EFL Learners. Journal of Modern Research in English Language Studies, 7(1), 29-44. https://doi.org/10.30479/jmrels.2019.11406.1418
Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45(1), 5-35. https://doi.org/https://doi.org/10.5054/tq.2011.244483
Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37(5), 639-668. https://doi.org/10.1093/applin/amu059
Biber, D., Gray, B., Staples, S., & Egbert, J. (2020). Investigating grammatical complexity in L2 English writing research: Linguistic description versus predictive measurement. Journal of English for Academic Purposes, 46, Article 100869. https://doi.org/10.1016/j.jeap.2020.100869
Bulté, B., & Housen, A. (2015). Evaluating short-term changes in L2 complexity development. Círculo de Lingüística Aplicada a la Comunicación, 63, 42-76. http://dx.doi.org/10.5209/rev_CLAC.2015.v63.50169
Celce-Murcia, M., & Olshtain, E. (2000). Discourse and context in language teaching: A guide for language teachers. Cambridge University Press.
Chen, H., Xu, J., & He, B. (2014). Automated essay scoring by capturing relative writing quality. The Computer Journal, 57(9), 1318-1330. https://doi.org/10.1093/comjnl/bxt117
Chuenchaichon, Y. (2011). The development of paragraph writing for EFL writers through the use of a reading into writing method [Ph.D. thesis, University of Reading].
Chulalongkorn University Academic Testing Center. (2007). CU-TEP. http://www.atc.chula.ac.th/en_html/en_tep.html
Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34(2), 213-238. https://doi.org/10.2307/3587951
Fatemi, M. A. (2008). The relationship between writing competence, language proficiency and grammatical errors in the writing of Iranian TEFL sophomores [Ph.D. thesis, Universiti Sains Malaysia, Malaysia]. https://core.ac.uk/download/pdf/11933914.pdf
Glen, S. (n.d.). Box Plot (Box and Whiskers): How to Read One & How to Make One in Excel, TI-83, SPSS. StatisticsHowTo.com: Elementary Statistics for the rest of us! https://www.statisticshowto.com/probability-and-statistics/descriptive-statistics/box-plot/
Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123-145. https://doi.org/10.1016/S1060-3743(00)00019-9
Gustin, S. (2019). Differences in syntactic complexity in the writing of EL1 and ELL civil engineering students [Master thesis, Portland State University].
Hawkey, R., & Barker, F. (2004). Developing a common scale for the assessment of writing. Assessing Writing, 9(2), 122-159. https://doi.org/10.1016/j.asw.2004.06.001
Housen, A., & Kuiken, F. (2009). Complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30(4), 461-473. https://doi.org/10.1093/applin/amp048
Jarvis, S., Grant, L., Bikowski, D., & Ferris, D. (2003). Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing, 12(4), 377-403. https://doi.org/10.1016/j.jslw.2003.09.001
Johansson, V. (2008). Lexical diversity and lexical density in speech and writing: A developmental perspective. Lund University, Department of Linguistics and Phonetics Working Papers, 53, 61-79. https://journals.lub.lu.se/LWPL/article/view/2273/1848
Kampookaew, P. (2020). An analysis of grammatical errors made by Thai EFL university students in an EAP writing class: Issues and recommendations. rEFLections, 27(2), 246-273. https://so05.tci-thaijo.org/index.php/reflections/article/view/248862
Kim, J. (2014). Predicting L2 writing proficiency using linguistic complexity measures: A corpus-based study. English Teaching, 69(4), 27-51. https://doi.org/10.15858/engtea.69.4.201412.27
Kovacevic, E. (2018). The relationship between lexical and syntactic complexity measures in a learner corpus. In S. Gudurić & B. Radić-Bojanić (Eds.), Jezici i Kulture u Vremenu i Prostoru VII/2 (pp. 469-479). University of Novi Sad. https://www.academia.edu/37808815/The_Relationship_Between_Lexical_and_Syntactic_Complexity_Measures_in_a_Learner_Corpus
Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication. Dissertation, Georgia State University. doi:https://doi.org/10.57709/8501051
Kyle, K., & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. The Modern Language Journal, 102(2), 333-349. https://doi.org/10.1111/modl.12468
Larsen-Freeman, D. (1978). An ESL index of development. TESOL Quarterly, 12(4), 439-448. https://doi.org/10.2307/3586142
Larsson, T., & Kaatari, H. (2020). Syntactic complexity across registers: Investigating (in)formality in second-language writing. Journal of English for Academic Purposes, 45, Article 100850. https://doi.org/10.1016/j.jeap.2020.100850
Liu, L., & Li, L. (2016). Noun phrase complexity in EFL academic writing: A corpus-based study of postgraduate academic writing. The Journal of Asia TEFL, 13(1), 1-71. https://doi.org/10.18823/asiatefl.2016.13.1.4.48
Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4), 474-496. https://doi.org/10.1075/ijcl.15.4.02lu
Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers' language development. TESOL Quarterly, 45(1), 36-62. https://doi.org/10.5054/tq.2011.240859
Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96(2), 190-208. https://doi.org/10.1111/j.1540-4781.2011.01232_1.x
Lu, X. (2014). Computational methods for corpus annotation analysis. Springer. https://doi.org/10.1007/978-94-017-8645-4
Malvern, D., & Richards, B. (2002). Investigating accommodation in language proficiency interviews using a new measure of lexical diversity. Language Testing, 19(1), 85-104. https://doi.org/10.1191/0265532202lt221oa
McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2), 381-392. https://doi.org/10.3758/BRM.42.2.381
Michel, M. (2017). Complexity, accuracy, and fluency in L2 production. In S. Loewen & M. Sato (Eds.), The Routledge handbook of instructed second language acquisition (pp. 50-68). Routledge. https://doi.org/10.4324/9781315676968
Nasseri, M., & Lu, X. (2020). Updated LCA-AW for Python 3, Lexical Complexity Analyzer for Academic Writing, version 2.2. Zenodo. https://doi.org/10.5281/zenodo.4147000
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50(3), 417-528. https://doi.org/10.1111/0023-8333.00136
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college‐level L2 writing. Applied Linguistics, 24(4), 492-518. https://doi.org/10.1093/applin/24.4.492
Padgate, W. (2008). Beliefs and opinions about English writing of students at a Thai university. PASAA Journal, 42, 31-53. https://www.culi.chula.ac.th/publicationsonline/files/article/lAUywNh8sITue82813.pdf
Polio, C. (2017). Second language writing development: A research agenda. Language Teaching, 50(2), 261-275. https://doi.org/10.1017/S0261444817000015
Read, J. (2000). Assessing Vocabulary. Cambridge University Press. https://doi.org/10.1017/CBO9780511732942
Richards, J. C., & Renandya, W. A. (Eds.). (2002). Methodology in language teaching: An anthology of current practice. Cambridge University Press.
Sermsook, K., Liamnimitr, J., & Pochakorn, R. (2017). An analysis of errors in written English sentences: A case study of Thai EFL students. English Language Teaching, 10(3), 101-110. https://doi.org/10.5539/elt.v10n3p101
Skehan, P. (2009). Lexical performance by native and non-native speakers on language-learning tasks. In B. Richards, M. H. Daller, D. D. Malvern, P. Meara, J. Milton, & J. Treffers-Daller (Eds.), Vocabulary studies in first and second language acquisition: The interface between theory and application (pp. 107-124). Palgrave Macmilla. https://doi.org/10.1057/9780230242258_7
Syarif, H., & Putri, R. E. (2018). How lexical density reveals students' ability in writing academic text. Lingua Didaktika: Jurnal Bahasa dan Pembelajaran Bahasa, 12(2), 86-94. https://doi.org/10.24036/ld.v12i1.10408
Tan, K. E., & Manochphinyo, A. (2017). Improving grammatical accuracy in Thai learners' writing: Comparing direct and indirect written corrective feedback. The Journal of Asia TEFL, 14(3), 430-442. https://doi.org/10.18823/asiatefl.2017.4.3.4.430
Text Inspector. (2022). Measure lexical diversity. Weblingua Ltd. https://textinspector.com/help/lexical-diversity/
Thongyoi, K., & Poonpon, K. (2020). Phrasal complexity measures as predictors of EFL university students’ English academic writing proficiency. rEFLections, 27(1), 44-61. https://so05.tci-thaijo.org/index.php/reflections/article/view/241750
To, V., Fan, S., & Thomas, D. (2013). Lexical density and readability: A case study of English textbooks. Internet Journal of Language, Culture and Society(37), 61-71. http://aaref.com.au/en/publications/journal/journal-articles/issue-37-2013/
Verspoor, M., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21(3), 239-263. https://doi.org/10.1016/j.jslw.2012.03.007
Weigle, S. C. (2002). Assessing writing. Cambridge University Press. https://doi.org/10.1017/CBO9780511732997
Wolf-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy & complexity. University of Hawai'i, Second Language Teaching & Curriculum Center.
Zhang, H., Chen, M., & Li, Z. (2021). Developmental features of lexical richness in English writings by Chinese beginner learners. Frontiers in Psychology, 12, Article 665988. https://doi.org/10.3389/fpsyg.2021.665988
Zhang, S., Zhang, H., & Zhang. C. (2022). A dynamic systems study on complexity, accuracy, and fluency in English writing development by Chinese university students. Frontiers in Psychology, 13 Article 787710.
Zhang, X., & Lu, X. (2022). Revisiting the predictive power of traditional vs. fine-grained syntactic complexity indices for L2 writing quality: The case of two genres. Assessing Writing, 51, Article 100597. https://doi.org/10.1016/j.asw.2021.100597
Zhu, H. (2013). Developmental features of lexical richness in English writings by Chinese EFL students. Shanghai Foreign Language Education Press.