The Effects of Lexical-Features Changes in ChatGPT-Generated Content Using N-Gram Approach

Abstract

This article aims 1) to analyze changes in word types appearing in content generated by ChatGPT through n-gram analysis and 2) to examine the lexical features of revised content modified by ChatGPT at the third-gram position. The research adopts a quantitative approach, employing n-gram analysis focusing on the third-gram position as the framework for evaluating changes in word types and lexical features resulting from word modifications. The study is document-based, with the sample comprising eight children’s stories selected through purposive sampling. The research utilized two tools: 1) the ChatGPT program and 2) Python via Google Collaboratory. The data were analyzed using basic statistical methods. The findings revealed that 1. Regarding the first objective, modifications at the third-gram position influenced the realism and coherence of the content. Nouns were the most frequently affected word type, followed by verbs, determiners, pronouns, adjectives, conjunctions, adverbs, numerals, and interjections, respectively. 2. For the second objective, eight types of lexical relationships were identified: 1) Antonym, 2) Co-hyponym, 3) Co-hypernym, 4) Hyponym, 5) Hypernym, 6) Synonym, 7) Meronym, and 8) Holonym. A new pattern, Word embedding similarity, was also observed as the most frequently occurring relationship, reflecting the AI’s capability to maintain contextual relevance and content appropriateness. The knowledge derived from this research enhances the integration of linguistics and artificial intelligence (AI). The user can apply this model to develop improved methods for evaluating AI-generated content, refining prompts for better performance, and designing suitable teaching materials. Furthermore, this study advocates for responsible, transparent, and ethical AI use, contributing to advancements in academia, industry, and society