Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation

Da Yan* School of Foreign Languages, Xinyang Agriculture and Forestry University, Xinyang 464000, China

• Correspondence: [Address same as affiliation] email: yanda1987@126.com Orcid: https://orcid.org/0000-0002-1265-9772

Abstract

Technology-enhanced language learning has exerted positive effects on the performance and engagement of L2 learners. Since the advent of tools based on recent advancement in artificial intelligence (AI), educators have made major strides in applying state-of-the-art technologies to writing classrooms. In November 2022, an AI- powered chatbot named ChatGPT capable of automatic text generation was introduced to the public. The study tried to apply ChatGPT’s text generation feature in a one-week L2 writing practicum. The study adopted a qualitative approach to investigate students’ behaviors and reflections in their exposure to ChatGPT in writing classrooms. The developmental features in learning activities and reflective perceptions were triangulated for the piloting evaluation of the impact of ChatGPT on L2 writing learners. The findings revealed the affordance and potential applicability of the tool in

POST-PRINT VERSION

L2 writing pedagogy. Additionally, the tool also showcasedan automatic workflow that could maximize the efficiency in composing writing. However, participants generally expressed their concern with its threats to academic honesty and educational equity. The study impelled the reconceptualization of plagiarism in the new era, development

FOR WEBSITE VIEWER ONLY

of regulatory policies and pedagogical guidance to regulate proper utilization of the tool. Being a pioneering effort, the study accentuated future research directions for more

RESPECT PUBLISHER RIGHTS

insights into the application of ChatGPT in L2 learning, and the establishment of corresponding pedagogical adjustments.

Keywords: ChatGPT, technology-enhanced writing, automatic text generation, plagiarism, AI-enhanced education

1. Introduction

Improvement in the availability and affordance of technology have augmented the utilization of technology-enhanced learning strategies in L2 writing (Mannion et al.,

2019). In the contemporary era, many aspects in the L2 writing have been digitalized in pedagogy, e.g., implementing web-based collaborative activities in L2 writing classrooms (Bikowski & Vithanage, 2016), using interactive e-feedback for students’ writing (Saeed & Al Qunayeer, 2022), incorporating corpus-assisted error resolution to improve writing quality (Crosthwaite et al., 2020), etc. Particularly, the introduction of tools and applications based on artificial intelligence (AI) have brought about paradigmatical changes to technology-enhance L2 writing. Tools such as Grammarly (Koltovskaia, 2020), Quillbot (Kurniati & Fithriani, 2022), Google Translate (Cancino & Panes, 2021) have significantly automated the workflow of writing, assessment, proofreading, etc. In November 2022, OpenAI released ChatGPT, its latest chatbot powered by generative artificial intelligence. Despite its recent appearance, ChatGPT has attracted the attention of educators from various backgrounds. Preliminary research demonstrates the great potential of ChatGPT’s text generation abilities in completing POST-PRINT VERSION writing tasks (Stokel-Walker, 2022). However, the plagiarism issue raised by the fully automatic workflow is aggravated by the low plagiarism detection rate of ChatGPT’s output combined with human modification or other AI-based rephrasing tools (Gao et

FOR WEBSITE VIEWER ONLY

al., 2022; Haque et al., 2022). Paradoxical to its popularity across the globe, empirical evidence for the affordance and impact of the application of ChatGPT in authentic educational settings is insubstantial. The potential and hazard underneath the mighty strength of ChatGPT leaves much space for researchers to delve into.

Against the above backdrops, the study ventures into investigating the impact of

ChatGPT on L2 learners’ attitudes and learning behaviors through piloting application. Classroom observations, learning log analysis and interview data are triangulated to provide insights into the potential and threat posed by ChatGPT. The study is significant for its conceptual and pedagogical contribution to L2 writing in the contemporary era. The study expands the understanding of plagiarism and the impact of latest technologies on language education in the age of AI. For pedagogical practice, the study reveals the strength of ChatGPT in undergraduate L2 writing classrooms and warned practitioners against its impending threats to academic honesty and educational equity.

2. Literature Review

2.1. Technology-enhanced L2 Writing: new trends in the age of AI

Under the umbrella of computer-assisted language learning, modern technologies played a role of growing importance in the pedagogy of L2 writing (Adams & Chuah,

2022). For decades, the inclusion of technology-enhanced strategies in L2 writing has become comprehensive, e.g., using e-feedback for revisions (Tuzi, 2004), applying wiki-based collaboration (Hsu, 2019), utilizing corpus-based tools as writing assistant (Yoon & Hirvela, 2004), focusing on students’ behavior in using computer for writing (Miller et al., 2008), writing with social media (Lee, 2020), etc. Focusing on different aspects in L2 writing, the application of technology was asserted to be effective in its own domain. According to the meta-analysis of Seyyedrezaei et al. (2022), a significant POST-PRINT VERSION effect size (g=1.00) was reported regarding the impact of technology-enhance language learning on EFL writing performance. Keeping abreast of the state-of-the-art technologies, researchers and educators FOR WEBSITE VIEWER ONLYintroduced AI-based tools to L2 writing in recent years (Nazari et al., 2021; Wu et al.,

2021). The first trend in recent research examined the effects of replacing existing technologies with AI -based alternatives. AI-based machine translation application has replaced conventional digital lexicographic resources and writing assistant. In a study situated in a Chilean high school EFL program, the researchers endorsed the affordance

of Google Translate in elevating students’ syntactic complexity and writing accuracy (Cancino & Panes, 2021). Similarly, Tsai’s (2019) findings pinpointed the strength of Google Translate in reducing grammatical errors and improving overall length of composition (Tsai, 2019). In a similar vein, the feedback on writing quality began to adopt AI-based automated written corrective feedback (AWCF) applications such as Grammarly. Dizon & Gayed (2021) asserted that Grammarly users outperformed their peers in L2 writing tasks. Nazari et al. (2021) confirmed the positive effect of AI-based written feedback on students’ motivation and self-efficacy for L2 writing. Conversely, a relatively conservative finding was reported from a similar setting, in which students experienced limited improvement in their cognitive engagement after learning with Grammarly (Koltovskaia, 2020).

The second trend in research focused on the augmentation of students’ roles and involvement in L2 writing. AI-based plagiarism detection applications were applied in L2 classrooms as a channel for feedback and quality evaluation. From a traditional perspective, plagiarism detection was generally beyond the reach of learners. However, in a study using the plagiarism detector Turnitin for peer-feedback in an ESL writing setting, the researchers identified the positive role of the application in augmenting students’ ability to produce accurate and holistic feedback on the quality of peers ’ L2 writing (Jinrong & Mimi, 2018). Similarly, researchers paid growing attention to the

POST-PRINT VERSION

emergence of AI-based paraphrasing or rephrasing applications. Quillbot was believed to be applicable for L2 writing with a variety of functions and a balance between technology and human intelligence (Fitria, 2022). In a recent study examining the

FOR WEBSITE VIEWER ONLY effects of QuillBot on post-graduate students’ performance in academic writing,

students expressed positive attitude towards the adoption of the application as an assistant for high quality writingRESPECT PUBLISHER RIGHTS(Kurniati & Fithriani, 2022).

In a nutshell, the existed a shortage of research in applying the state -of-the-art technologies of generative AI in the field of language education. For example, the

advancement in large scale language modeling and AI expedited the development interactive text generators that can respond to user’s prompts (Hagendorff et al., 2022). Nonetheless, the implementation of such tools in L2 writing remained untouched and unacknowledged. Consequently, we are facing lacunae in implementing state-of-the-art technologies in the pedagogy of L2 writing.

2.2. Using ChatGPT in education: a game changer

Prior to the advent of ChatGPT, conversational chatbots have been actively applied in multiple domains of education in recent years (Okonkwo & Ade-Ibijola, 2021), e.g., writing practice (Kılıçkaya, 2020), dynamic assessment (Jeon, 2021), and language pedagogy (Kohnke, 2022). Furthermore, a plethora of studies has contributed to the consensus in the effectiveness of chatbot-based systems. In an Iranian study, Ebadi and Amini (2022) confirmed the positive effects of AI-based chatbot on learners’ affective status. Similarly, pilot efforts to implement a chatbot-based approach to scaffold learners’ L2 writing received well-acclaimed results (Guo et al., 2022). For the effects of chatbot-based writing systems on specific aspects of writing performance, Nagata et al. (2020) argued that the most prominent advantage of such system were reflected in word usage and self-correction.

ChatGPT was released by OpenAI in November 2022 as a new generation chatbot

POST-PRINT VERSION built on top of GPT-3.5 family of large-scale language models (OpenAI, 2022). Unlike

its predecessors, ChatGPT has grasped the attention of researchers and the public for its ability to respond smartly, rapidly, and multilingually. At present stage, a few studies were published to explore the potential and strength of using ChatGPT’s features in

FOR WEBSITE VIEWER ONLY

education, e.g., enhancing creativity and critical thinking (Zhai, 2022), sitting for an medical exam (Gilson et al., 2022), and empowering learners in engineering education (Qadir, 2022). A consensus in the power of ChatGPT for education was reached in piloting studies in which its equivalence to an early-stage learners was articulated (Gilson et al., 2022; Qadir, 2022).

Among its versatile utilization, the power of ChatGPT’s automatic text generation attracted interest of researchers increasingly. Wenzlaff and Spaeth (2022) affirmed that ChatGPT was basically equivalent to human in composing explanatory answers. ChatGPT-generated texts were praised for overall quality (Gao et al., 2022), originality in contents (Yeadon et al., 2022), and good performance in writing literature reviews (Aydın & Karaarslan, 2022). Apart from the promising effects in writing, thought- provoking concerns emerged for its potential to elude plagiarism detection (Yeadon et al., 2022). Haque et al. (2022) pointed out that social media users were more concerned for ChatGPT’s application in education than other field. In a similar fashion, researchers highlighted the potential threats posed by ChatGPT to academic integrity in tests and exams (Susnjak, 2022). For academic writing, diversified opinions were heard about whether using automatically generated text was plagiarizing (Frye, 2022; Jabotinsky & Sarel, 2022; Yeadon et al., 2022).

To date, we are in a paucity of empirical studies on the application of ChatGPT in education. Most existing literature piloted the application of ChatGPT in a setting out of the context of pedagogy. Furthermore, students’ involvement in both practice and reflection about ChatGPT were trivial. Consequently, we are now facing a knowledge and practical gap in understanding the impact of ChatGPT on education.

POST-PRINT VERSION 2.3. Plagiarism in Writing: a challenged definition

According to Pecorari (2001), plagiarism was defined as “material that has been taken from some source by someone, without acknowledgment and with/without intention to deceive” (p. 235). Plagiarism in writing was a complex and

FOR WEBSITE VIEWER ONLY

multidisciplinary practice. According to Goh (2013), four types of plagiarism were identified from existing literature: 1) patchwork plagiarism, 2) copying directly from peers, 3) self-plagiarism and 4) buying articles from the Internet. Researchers have advocated the inclusion of all “cheating behaviors” under the umbrella of plagiarism for their harm to academic integrity (Jamieson & Howard, 2019).

The plagiarizing behaviors have been sternly criticized for being a “heinous crime” (Pecorari, 2003) and “intellectual theft” (Traniello & Bakker, 2016). In actual educational settings, plagiarism was often intertwined with other aspects other than academic dishonesty. In a study to investigate writer’s intention in plagiarizing, a contradiction in perception of plagiarism between learners and educators was discovered (Pecorari, 2003). Furthermore, students resorted to strategies such as paraphrasing and patchwriting to avoid explicit plagiarism (Liu et al., 2018). Educators have complained that such behaviors were hard to detect (Pecorari, 2022), even with latest detection systems (N. E. J. A. Bowen & Nanni, 2021).

For decades, researchers have produced prolific evidence for the perception (Chien, 2017; Hu & Lei, 2012; Merkel, 2020), practice (Keck, 2006; Neumann et al.,

2019), detection (Hayes & Introna, 2005; Stapleton, 2012) and countermeasures (Keck, 2010; Yeh, 2021) of plagiarism in writing. However, the conceptualization and contextualization of plagiarism has always been subject to the development of technology and changes in pedagogical environments. When the Internet prevailed as a major source of information for learners, scholars has warned that “unconventional and interactive” sources of information should be scrutinized in plagiarism detection (Li & Casanave, 2012, p. 166). Subsequently, an array of research was undertaken to reconceptualize plagiarism and suggest coping strategies (Davies & Howard, 2016; POST-PRINT VERSION Flowerdew & Li, 2007). The issue emerged anew amid the advancement of AI. Applications such as ChatGPT and Quillbot offered multiple strategies to avoid plagiarism detection (Gao et al., 2022; Haque et al., 2022; Susnjak, 2022). Additionally,

FOR WEBSITE VIEWER ONLY the emergence of new technologies obsoleted the definition and classifications of

plagiarism. As a result, the existing gap in understanding the potential plagiarism with AI-based tools andRESPECT PUBLISHER RIGHTSlearners’ perceptions in pedagogy should be imperatively investigated.

2.4. The Study

The study set out to examine the impact of ChatGPT’s automatic text generation on undergraduate students’ behavior and perceptions in learning L2 writing skills. The theoretical underpinning of the study was grounded in peer scaffolding (Taheri & Nazmi, 2021) and reflective learning (Morris, 2020; Şener & Mede, 2022) theories. Adhering to Kolb’s experiential learning cycles (Kolb & Kolb, 2017), the study embarked on the investigation through a series of collaborative and reflective practices in a practicum with the theme of applying ChatGPT in L2 writing. In the experiential learning sessions, students were encouraged to explore the features of ChatGPT with few restrictions; while in reflective learning sessions, participants were requested to introspect and analyze the values and threats of using ChatGPT in L2 writing learning. Specifically, the study addressed the following two research questions: RQ1: How do participants in the practicum develop skills in applying ChatGPT to complete undergraduate L2 writing tasks? RQ2: How do participants in the practicum perceive the application of ChatGPT in L2 Writing?

3. Methods and Materials

3.1. Design and Participants

The study adopted a multi-method qualitative approach (Davis et al., 2011) to acquire in-depth understanding regarding the practice of using ChatGPT’ text

POST-PRINT VERSION

generation function in completing L2 writing tasks and students’ perception thereof. Specifically, the study encompassed two strands, through which both research questions were answered respectively. Result and findings from each strand were collectively analyzed to construct a holistic interpretation.

FOR WEBSITE VIEWER ONLY

The study took place in an undergraduate EFL major program in a Chinese university. A special practicum was developed for learning and practicing introductory knowledge in applying ChatGPT for L2 writing. To recruit participants for the study, a pool of 35 participants candidates was formed based on students’ voluntary application. All participants were Chinese undergraduate EFL majors with an average of 19.4 years.

For students’ perceptions of the application of ChatGPT in L2 writing, eight participants were finally recruited out of the pool through case-by-case observation and screening (see Table 1). The criteria for selecting the participants included: 1) grades in a precedent summative test in L2 writing; 2) experiences in computer-assisted language learning; 3) interest in the project, and 4) learning behavior and motivation in classroom activities. Students attaining grades higher than 80 in the summative L2 writing test were believed to be apt learners for the undergraduate EFL level and the mean score of all test-takers is 71. Student’ interest in and prior experiences for the project were collected through pre-project surveys. For students’ learning behavior and motivation in L2 writing classrooms, video-based classroom recordings were referred- to and the opinions and recommendations from lecturers were respected.

Table 1. Information of Participants

Note: Grades refers to the grades in a precedent summative test in L2 writing

POST-PRINT VERSION

All participants were acknowledged of the purpose and designof the study. Written informed consents were obtained from all participants prior to the data collection procedures. All data, i.e., classroom behaviors, learning artifacts and perceptions of the participants, from the study were used for the sole purpose of academic research and

FOR WEBSITE VIEWER ONLY

were retained in anonymity or under pseudonyms.

3.2.RESPECT PUBLISHER RIGHTSPracticum Design

From December 12th to December 18th, 2022, 116 EFL majors in an undergraduate program participated in a one-week practicum about applying the text generation

feature of ChatGPT in L2 writing. According to the curriculum settings of the program, all the instructional sessions of other courses were suspended during the practicum. For each day, two instructional sessions (each of 45 minutes) and six practical sessions were arranged. See Table 2 for the syllabus of the practicum.

Table 2. Syllabus of the practicum

POST-PRINT VERSION FOR WEBSITE VIEWER ONLY

From Monday to Wednesday, a series of video about the application of ChatGPT in general writing scenarios were provided to students. The videos were collected from Youtube and Bilibili, two major video-sharing social media websites. A series of document detailing the procedures for text auto -generation were available as supplementary didactic materials with which students could become skillful at utilizing ChatGPT in L2 writing. Most of the videos and materials were produced and published within two weeks after the release of ChatGPT. The contents of the materials were about using the text generated in ChatGPT’s interactive chat flows for generic writing tasks,

e.g., casual essay, blog posts, and fictions. Nevertheless, we provided several videos created by Chinese content producers whose foci included high-stakes tests and academic writings. A self-evaluation of textual quality was arranged on Wednesday. On Thursday, student exchanged with and learned from peer learners for advanced tips and techniques discovered during their practices. A brainstorm approach was adopted for augmentation of effectiveness and expansion in the scope of application of ChaptGPT in L2 writing. A flipped-classroom approach was adopted for the instructional sessions in which students voluntarily showcased their “tricks” in using ChatGPT’s advanced features to the class. The outcomes of the brainstorm were further consolidated in the peer-feedback POST-PRINT VERSION session held on Saturday, in which students provided judgmental feedback regarding the textual quality of peer learners. On Friday and Sunday, two sessions of collaborative activities were prescribed in which students honed their skills through extensive practices. Students’ learning artifacts were automatically graded by Microsoft Aim

FOR WEBSITE VIEWER ONLY

Writing on Friday.

With the arrangement of the practicum, students could focus on learning and practicing relevant skills and techniques required by developer of the practicum. In pre- practicum training sessions, the lecturers imparted necessary prior knowledge in

computer-assisted language learning and protocols for academic honesty to participants. During the practicum, students were encouraged to take an exploratory perspective towards the behavior of using auto-generated text for completing L2 writing tasks. Academic integrity and honesty were upheld throughout the practicum and critical reflection was encouraged.

3.3. Procedures

Different data collection strategies were employed in both research strands, i.e., observation method for students’ behavior in classroom (Jamshed, 2014), document analysis for learning log (G. A. Bowen, 2009) and thematic analysis for the interview (Braun & Clarke, 2012).

First, lecturers were required to observe students’ performance and learning behaviors. Multiple methods were employed for observation, e.g., screen monitoring, videorecording, and paper-and-pen notes. Students were asked to submit a daily learning log (see Appendix 1) of their experiences inclusive of the methods to apply ChatGPT in L2 writing. Upon the completion of the practicum, all learning logs and classroom observations were collected and screened by researcher based on the relevance to the topic. Eventually, 159 learning log items and 89 pieces of classroom observation details were selected for analysis.

POST-PRINT VERSION

Second, the eight participants were invited for three sessions of in-depth interview. Students were required to follow the instruction of the moderator to answer and discuss about questions from the interview protocol (see Appendix 2). Each interview session lasted for about 45 to 60 minutes. The interview was audio recorded and transcribed

FOR WEBSITE VIEWER ONLY

verbatim.

Upon the completion of the interview session RESPECT PUBLISHER RIGHTS s, member checking was conducted to ensure that the respondents agreed with the credibility of the results. Specifically, the method and procedures for member checking recommended by Doyle (2007) were

followed in the study. Additionally, the overall interview process was overseen by an expert panel whose members are deans and deputy deans of the research site. Such expert judgement was recommend by researchers as a measure to assess and ensure content validity of qualitative research (Fernández-Gómez et al., 2020; Sánchez- Guardiola Paredes et al., 2021).

3.4. Analysis

Document analysis, adhering to the four-step systematic approach advocated by Daglish et al. (2020), was applied to the analysis of classroom observation and learning logs. For the analysis of the interview, the thematic analysis strictly followed the six- step procedures suggest by Braun and Clarke (2012). Two additional lecturers were recruited to assist the researchers in coding and theme extraction. A joint discussion was convened to settle disagreements among coders. When the data analyses were finalized, all findings were converged and triangulated for the major findings of the study.

4. Findings

4.1. Developmental Stages of Student’s Practices in the Practicum

To answer the first research question, students’ practical experiences in using ChatGPT through classroom observation and learning logs were analyzed. As a result, three developmental stages were identified: 1) familiarization with ChatGPT, 2)

POST-PRINT VERSION

experimenting the basic usage, and 3) exploration for advanced techniques. Fig. 1 shows a visual representation of three stages.

FOR WEBSITE VIEWER ONLY

Fig. 1 Developmental Stages of Using ChatGPT in L2 Writing

During the familiarization stage, students began playing around in ChatGPT individually after learning from introductory videos. Most of the students were able to effectively prompts ChatGPT to generate text for a given topic. However, most of the practices at this stage remained one-time performances, i.e., students were unable to formatively amend the prompt for regeneration of texts until satisfaction. The students were able to post-edit the generated texts for improvement in the choice of words, grammatical structures and syntax. Based on the self-evaluation, students were generally satisfied with the quality of the generated text in language accuracy, syntactic coherence, grammatical correctness, language styles and the compliance with academic requirements. Riley’s account of her procedures (excerpted from the learning log on

POST-PRINT VERSION

December 13) epitomized the typical workflows of participants in writing at this stage: Prompt in ChatGPT: Write an essay discussing whether all college students should take part in public speaking.

FOR WEBSITE VIEWER ONLYWriting Procedures:

1. Generate the composition from ChatGPT. 2. Copy and paste to Microsoft word.

RESPECT PUBLISHER RIGHTS3. Use electronic dictionaries to paraphrase and edit certain phrases and

sentences. After the initial introduction and familiarization, students practiced the obtained

skills for multiple writing tasks. At this stage, students reported that they gradually learned how to utilize the interactive features of ChatGPT for improvement in text quality and relevance to the task requirements. According to the learning logs, students began to seek external sources for hints and assistance. Through the practice sessions, learners gradually comprehend the internal mechanism of ChatGPT’s regeneration of answers to amended prompts. According to the results of automatic grading, the overall quality of the writing has significantly improved after two or three rounds of “prompting for regeneration”. Minor improvements of the writing, e.g., inclusion of more citation or examples, change of language style, replacement of examples within the writing, were generally attained after one iteration of the “prompting-regeneration” cycle. See Fig. 2 for a comparison in automatically graded scores of an original version of ChatGPT’s text generation and an improved version with explicit prompt for a change to “academic language style”.

POST-PRINT VERSION FOR WEBSITE VIEWER ONLY

Fig. 2 Automatic grading of direct output (A) from ChatGPT’s text generation and an

improved version (B)

Note: The holistic grading was composed of three dimensions: vocabulary-level quality,

sentence-making abilities and overall quality.

In the final stage during the practicum, students began to finetune textual quality

POST-PRINT VERSION

with other automatic text processing tools. Based on students’ experiences, four major types of software or application were employed by students during the practicum: 1) text generators, which were basically similar solutions as ChatGPT; 2) Paraphrasers,

FOR WEBSITE VIEWER ONLY

which provided functions to automatic rewrite, paraphrase, or patch-write the original writing for improvement of textual quality or reduction in vulnerabilities to plagiarism detection; 3) grammar checkers, which provided automatic grammatical checking and suggestions for amendments and corrections; and 4) summarizers, which helped students turn the text from a redundant and repetitive style to a more concise one.

Students proactively exchanged their experiences in using “software combinations” to automate the workflow in L2 writing and attaining better textual quality through AI- based rephrasing and amendments. As a result, relevant tips and techniques for automatic editing spread rapidly among learners. For example, Benjamin’s approach (reported and shared on December 17) to combine multiple AI-based writing assistants and iteratively elicit ChatGPT’s power in text generation were especially popular: Prompts in ChatGPT: Round 1: Write an essay discussing whether it is good for senior managers to have higher salaries than other workers in a company. Round 2:Suppose you are an academic writer, write an essay on whether it is good for senior managers to have higher salaries than other workers in a company, take an academic and critical stance. Round 3: Regenerate the essay with more detailed examples in the context of China. Writing and Editing Procedures:

1. Iteratively generate and compare the outputs of ChatGPT. 2. Use automatic essay grader to select the most appealing version. 3. Use Quillbot to paraphrase and edit the desired output. 4. Use Grammarly to check possible grammatical and syntactical errors. As the students reported in learning logs, the further modification and rewriting not only lead to better writing quality but also remarkably reduced the possibilities to be dictated as “cheating or plagiarizing” through extant means of detection. Upon completion of the practicum, most students showed sufficient mastery in using

POST-PRINT VERSION

ChatGPT and relevant AI-based tools for completing writing tasks of average difficulties.

4.2. Perceptions and Reflections from Students

FOR WEBSITE VIEWER ONLYTo answer the second research question, students’ perceptions and reflections

obtained from the in-depth interview sessions were synthesized. Therefore, the following themes were identified: RESPECT PUBLISHER RIGHTS 1) the Power of ChatGPT, 2) the potential threats posed by ChatGPT for learners, and 3) suggestions for proper use of ChatGPT in L2 writing. Fig. 3 shows a thematic map illustrating the themes and key contents of the

interviews.

Fig. 3 Thematic map of the in-depth interview

4.2.1 The Power of ChatGPT

When asked about the effects of the application of ChatGPT on L2 writing, students expressed their appreciation ofthe tool in generating text for multilingual writing tasks. Students reached a consensus in the overwhelming speed of text generation by ChatGPT. As argued by a respondent:

[ChatGPT] has been really fast in generating text. When you give it a topic,

POST-PRINT VERSION

the application will return with a short essay of about 300 words in no time. (Julia, Session #2).

In addition to its speedy performance, students argued that the quality of FOR WEBSITE VIEWER ONLY ChatGPT’s writing was at “above average or at least acceptable levels”. To the students, the generated text was idiomatic and well-structured. In addition, ChatGPT was believed to be “good at giving examples” from “many disciplines and fields”. RESPECT PUBLISHER RIGHTS Most students have tried to check the grammar and textual quality in applications such as Grammarly. The overall quality in grammar, vocabulary, sentence structure was

believed to be stable and error-free in most cases. As a student remarked:

[Generated texts] are of good structure. Based on my experiences, they often include an opening paragraph to lead in, several detailed paragraphs to fill in and a concluding one to finish. I can’t say it’s perfect, but it is at least complete, formal, consistent and reader friendly. (David, Session #1)

Additionally, the respondents expressed their astonishment in ChatGPT’s obedience to the standards of academic writing. Users could prompt the application to include legitimate in-text citation and references list. Furthermore, users could wield the flexibility provided by the platform to request for a change in linguistics tones or flavors. As a student reported:

I was shocked to play with the different styles of language [ChatGPT] generated. For example, when you ask it to produce a text with “academic tone” or “casual style”, it would generate very different writings. [This means that] it is an all-rounder for different tasks. (Olivia, Session #2)

In regard to the automatic workflow of ChatGPT, students argued that better results could be obtained through the interaction with the platform. Users could ask ChatGPT to regenerate the text if dissatisfied with the present version. As a student

POST-PRINT VERSION metaphorically argued, ChatGPT served as a “powerful yet obedient servant”

(Lydia, Session #2).

4.2.2 Potential threats

Pertinent to the potential threats posed by ChatGPT, students generally showed

FOR WEBSITE VIEWER ONLY

concern about inequity. To many student participants in the practicum, the power of ChatGPT to generate a piece of writing “in the blink of an eye” violated the basic principle of educational equity. Students affirmed that the knowledge and experiences of using ChatGPT brought students “enormous advantages” to outperform their peers.

According to the view of the students, ChatGPT could be deemed as a shortcut for writing, literature reading and data gathering. The complex process of “reading- writing-revision” could be simplified into “text-generation and post-editing” which demanded for an apparent lower level of language competence and writing skills. According to one of the respondents:

I found it utterly unfair for students using traditional ways learning … For us, we needed to spend hours reading and searching for the specific piece of information. They don’t even need to write by themselves. (Benjamin, Session #3)

Furthermore, students argued that the chances for the generated document to be detected by existing plagiarism detection software were minimal. According to a number of respondents, the general “plagiarism rates” arbitrated by software were approximately 10%-15%, which were “distinctly far away from the danger zone”. As argued by the students, ChatGPT was not the first widely used tool for plagiarism, patchwriting and paraphrasing. However, its ability to elude detection and recognition was shockingly impactful on learners. As a student put it, ChatGPT “depreciated her efforts” (Riley, Session #3).

Last but not least, the spread of “tips and tricks” of using ChatGPT for plagiarism was viral on major social media platforms. To the dismay of many respondents, the

POST-PRINT VERSION

popularity of such know-hows contravened equity and impaired learners’ motivation. As argued by a respondent, the potential competition with AI-driven application in high-stakes tests “made him sad and helpless” (Benjamin, Session #3). In addition,

FOR WEBSITE VIEWER ONLY

students reported that users were actively advocating a workflow to “automate composition” without “any actual writing at all”. According to a respondent:

The combination of ChatGPT and tools such as quillbot was a blessing for those seeking shortcuts. The whole workflow was automatic, all you need to do is just copying and pasting a few keywords and a few clicks for final production. From

the perspective of ideal plagiarism, it seems perfect. (Olivia, Session #3)

4.2.3 Suggestions for proper use

Based on the observed strength and threats of the utilization of ChatGPT in L2 writing, students made several recommendations and suggestions for regulation.

First and foremost, students claimed that the usage of ChatGPT in L2 writing should be strictly limited. It was advised by the respondents that ChatGPT should be primarily used in self-learning scenario. The practices to use ChatGPT directly in completing writing tasks and tests should be prohibited and punished. Specifically, students argued that ChatGPT could be used only as a supplementary channel of didactic materials. As the students put it, it would be beneficial to use ChatGPT to provide “extra materials for studying and comparison” (Lucas, Session #2) against the documents provided by lecturers. As a student remarked in the interview:

[It’s natural] that a tool’s value is justified by its proper usage. I am angry that everyone is talking about ChatGPT as a ‘shortcut’ or ‘secret recipe’. If it is not controlled and limited, the true learners would be hurt eventually. (Riley, Session #2)

Furthermore, students took a relatively pessimistic view on the plagiarism

POST-PRINT VERSION

detection of ChatGPT. Based on their understanding and experiences during the practicum, the chance for lecturers (with or without plagiarism detection technologies) to discover the traces of plagiarizing with ChatGPT remained minimal at current stage. Students expressed their concerns that the development of AI-based plagiarism FOR WEBSITE VIEWER ONLYdetection lagged behind the “leaps and bounds in software and applications such as ChatGPT and Quillbot” (David, Session #1). Additionally, suggestions were made to improve lecturers’ understanding of RESPECT PUBLISHER RIGHTS the mechanism of ChatGPT and corresponding abilities to “deal with such behaviors” (Julia, Session #3).

For the pedagogy of L2 writing, students emphasized the urgency for developing

regulations and guidance regarding the use of ChatGPT and similar tools. Students believed that the continuous exposure to AI-based tools would be a “new normal”. In lieu of rejecting and underplaying technology-based plagiarism, such behaviors should be transformed from a hazard to a lever and production booster for L2 learners. Unfortunately, the prerequisites for such transformation, i.e., regulations, guidance, and protocols for practice, were nonexistent. Additionally, students believed that their awareness of the ethical and academic danger resulted from plagiarism with state-of- the-art technologies should be raised through proper education.

5. Discussions

Through the one-week practicum designed to let students get familiar with ChatGPT’s text generation functions and apply in L2 writing, the following major findings were reached: students could easily grasp the basic skills to use ChatGPT in writing and improved their proficiency and capabilities through collaborative activities; students acknowledge the strength of ChatGPT and expressed more concern for educational equity. In the following sections, the findings and implications are discussed in light of relevant research and theories from previous literature.

5.1. The potential of generative artificial intelligence in L2 writing

According to the experiences and the quality of learning artifacts in the practicum,

POST-PRINT VERSION

the strength of applications powered by generative artificial intelligence was fully displayed. The acceptable quality of direct output observed in the study was in tandem with the promising results reported in study by Aydın and Karaarslan (2022). Students’ experiences from the self-evaluation and automatic grading systems supported the

FOR WEBSITE VIEWER ONLY

claim from other studies that ChatGPT’s affordance in composing writing was equivalent to that of an average learners (Wenzlaff & Spaeth, 2022). Additionally, ChatGPT’s performance in L2 writing was a testimony to its versatility for variousRESPECT PUBLISHER RIGHTS educational domains. From the viewpoints of the participants, ChatGPT performed well in generating evidence from a varied scope of disciplines and fields. This was in tandem

with the diversity of educational settings in which ChatGPT has proved its potential and applicability (Gilson et al., 2022; Qadir, 2022).

The study was in agreement with the finding from a previous study that students wrote with fewer grammatical errors and more lexical diversities with technologies (Dizon & Gayed, 2021). Similarly, the proven strength of chatbot-based writing systems in promoting self-revision from the work of Nagata et al. (2020) was echoed in the comparison between direct and modified outputs of ChatGPT. However, due to the variances in internal mechanism, the direct comparison between ChatGPT and precedented writing assistants would be impossible. By the same token, the researcher have observed a similar improvement in learning engagement as in a study utilizing Grammarly as a feedback platform (Koltovskaia, 2020). The similarities revealed that fact that ChatGPT’s strength in assisting L2 writing was significant, yet it offered a totally different way to write.

However, it was inappropriate to infer that ChatGPT was capable of all writing tasks. For demanding tasks such as postgraduate-level academic writing, using ChatGPT as a major information source was risky. Gao et al. (2022) have criticized that ChatGPT’s production lacked in-depth insights which were required for high quality academic writing. Furthermore, participants in the study didn’t encounter issues

POST-PRINT VERSION

existing in other studies (Haque et al., 2022; M. Alshater, 2022) such as the lack of domain expertise and limited abilities to generate original text. Therefore, the relatively outstanding performance observed in the study could be explained by the requirement of L2 writing at undergraduate level. The major foci of undergraduate L2 writing tasks

FOR WEBSITE VIEWER ONLY

included composition structure, grammaticality and semantics accuracy (Munoz-Luna,

2015), which happened to be the strength of ChatGPT.

5.2. A new AI-powered workflow of writing

Unlike software and applications implemented in previous documents, ChatGPT offered an “all-in-one” solution for users. In the past, the technology -based tools were

basically used for a specific aspect in the writing process, e.g., grammar checking (Dizon & Gayed, 2021), rephrasing (Fitria, 2022), feedback (Koltovskaia, 2020), etc. In the study, students can take advantage of ChatGPT’s interaction functionalities for almost all the tasks conceivable. The difference could be attributed to the fact that ChatGPT is innately designed to be a chatbot that will respond to all kinds of requests. Furthermore, ChatGPT provided a linear workflow to writing. For example, in the cases using Grammarly, students are required to write by themselves and use the tool to check the textual quality of the composition (Dizon & Gayed, 2021). The grammar check is a one-time action with constant evaluation as long as the source text remains unchanged. However, for the workflow of ChatGPT, users can always prompt the AI-powered system to regenerate information according to amended requirements.

The experiences from students were in alignment with the articulation of Benzon (2023, p. 1) that ChatGPT possessed “sophisticated discourse skills”. Through iterations of conversation with the users, ChatGPT could adjust its text generation strategies to better suit the explicit demands. This could be deemed as one of ChatGPT’s advantage over precedent text generators. In addition, the iterative and interactive feature happened to be the actual connotation of the word “chat” in its name. At present stage, researchers actively explored possibilities to exploit ChatGPT’s potential through re-prompting the system incrementally. In a piloting effort to use ChatGPT’s answer in

POST-PRINT VERSION

the bar exam, the authors purposefully used a “re-prompt and regeneration” strategy for eliciting answers from ChatGPT (Bommarito II & Katz, 2022). The findings from the present study indicated that learners could attain significant improvement in textual

FOR WEBSITE VIEWER ONLY quality through interaction with the system. Nevertheless, the elicitation of satisfactory

responses from ChatGPT was a craft in need of special expertise and repeated practices.

5.3.

ChatGPT’s threats outweigh its merits

In the study, participants generally showed more concern rather than satisfaction towards the unrestricted application of ChatGPT in L2 writing. This was the unexpected

yet understandable finding of the study. In the first place, the relatively conservative and negative attitudes were beyond the expectations in designing the practicum. In previous literature pertinent to technology-enhanced L2 writing, students were generally satisfied with the enhancing effects of technology on learning outcomes (Seyyedrezaei et al., 2022; Tuzi, 2004). Furthermore, conventional technology- enhanced writing strategies, such as conversational chatbots, generally resulted in improved learning motivation and eagerness (Ebadi & Amini, 2022). In this study, the strength and versatility of ChatGPT led to students’ agreement in its danger to academic honesty. The unexpected aspect of the finding could be attributed to the design of the practicum. In pre-practicum training sessions, we have emphasized academic integrity and the avoidance of explicit plagiarism. As a result, participants generally adopted a critical stance on the application of ChatGPT in L2 writing.

From another perspective, students’ concerns and worries were reasonable. The fully automated workflow of ChatGPT naturally made undergraduate EFL learners, whose exposure to latest technologies was limited, concerned about the position of human in L2 writing pedagogy. In literature applying ChatGPT in education, similar concerns were heard from participants and scholars (Haque et al., 2022; Susnjak, 2022). Furthermore, researchers have explicated warned against ChatGPT’s potential unethical application (Gao et al., 2022). The concerted concerns pinpointed that

POST-PRINT VERSION

ChatGPT could be a double-edged sword for students and educators. However, the concerns didn’t necessarily result in a pessimistic attitude towards the application of ChatGPT in education. As argued by Zhai (2022), instructional and judgmental

FOR WEBSITE VIEWER ONLY adjustments were mandatory for education to embrace AI-based tools. The issues

encountered at present stage would be mediated through corresponding improvements in assessment strategies,RESPECT PUBLISHER RIGHTSlearning objectives, development of learning tasks and teacher education.

5.4. Implications

The study has implications for conceptualization in research and pedagogical practices regarding the application of ChatGPT or similar AI-based text generators in language education.

The advancements of AI and language models have ushered in paradigmatic changes to the conceptualization of plagiarism and academic honesty. Based on the observation of students’ behaviors and analyses of their reflections, an extension to Goh’s (2013) tetrachotomous taxonomy of plagiarism is proposed. The original classification was incapacitated confronting the practice of using AI generated text for writing tasks. Direct or minorly amended utilization of AI-generated text should be classified under plagiarism as the writing process is based on the collective intelligence in the database but void of the practitioner’s contribution. Automatic paraphrasing with AI-based tools should also be included as a plagiarizing practice, in accordance with the recent finding in plagiarism detection (Roe & Perkins, 2022; Wahle et al., 2022). Moreover, according to Pecorari & Petrić (2014), the shortage and misuse of quotation and citation could be deemed as plagiarism as well. A visual representation of the updated conceptual model is shown in Fig. 4.

POST-PRINT VERSION FOR WEBSITE VIEWER ONLY

Fig. 4 Conceptual model of the updated plagiarism typologies

For pedagogy, extra attention should be paid to the application of tools such as ChatGPT in authentic learning environment. Pedagogical guidance for proper

utilization of state-of-the-art technologies should be implemented, regulatory policies in educational institutions should be reformulated, and plagiarism detection literacy of lecturers should be reinforced. For an undergraduate EFL program, ChatGPT’s performance is comparable to that of an average or above-average student in composing writing for a given topic. The research suggests educators to adopt a rational perspective towards the emergence of ChatGPT and follow-up applications taking advantage of the powerful GPT-3.5 language model. Instead of obstructing students’ exposure and access to such tools, lecturers should develop and implement a protocol of application to make proper use of the technology.

6. Conclusions

In a one-week practicum, a group of undergraduate EFL learners were exposed to ChatGPT’s automatic text generation features. Their practices in applying the tool and corresponding reflections were collected and analyzed. The findings revealed that the tool and the revolutionary workflow around it were powerful and potentially applicable for L2 writing learners who were more perturbed than jubilant for its presence. The study is significant for its contribution to expand our knowledge in plagiarism and to forewarn educators of the looming menaces from ChatGPT.

The study faced the limitation in the duration and design of the practicum.

POST-PRINT VERSION Specifically, a one-week condensed practicum would be insufficient for students to

fully grasp and attain proficiency in the features of ChatGPT. Moreover, the impact of ChatGPT on other aspects in L2 writing were not measured and investigated. However, the study had the merits of being a pioneering effort to evaluate the impact of the AI-

FOR WEBSITE VIEWER ONLY

based tool on learning behaviors and attitudes in a L2 writing classroom. In follow-up studies, researchers could expand the scope of the research in the following directions:

1. lecturers’ attitude towards ChaGPT’s application; 2) rater’s plagiarism detection literacy facing AI-based tools; and 3) the longitudinal effects of ChatGPT on learning achievements, etc.

References

Adams, D., & Chuah, K.-M. (2022). Artificial Intelligence-Based Tools in Research Writing. In P. P. Churi, S. Joshi, M. Elhoseny, & A. Omrane (Eds.), Artificial Intelligence in Higher Education: A Practical Approach (pp. 169–184). CRC Press. Aydın, Ö., & Karaarslan, E. (2022). OpenAI ChatGPT Generated Literature Review: Digital Twin in Healthcare. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4308687 Benzon, W. L. (2023). Discursive Competence in ChatGPT, Part 1: Talking with Dragons Version 2. SSRN Scholarly Paper. https://ssrn.com/abstract=4318832 Bikowski, D., & Vithanage, R. (2016). Effects of web-based collaborative writing on individual L2 writing development. Language Learning & Technology, 20(1), 79–99. Bommarito II, M., & Katz, D. M. (2022). GPT Takes the Bar Exam (arXiv:2212.14402). arXiv. https://doi.org/10.48550/arXiv.2212.14402 Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40. https://doi.org/10.3316/QRJ0902027 Bowen, N. E. J. A., & Nanni, A. (2021). Piracy, playing the system, or poor policies? Perspectives on plagiarism in Thailand. Journal of English for Academic Purposes, 51, 100992. https://doi.org/10.1016/j.jeap.2021.100992 Braun, V., & Clarke, V. (2012). Thematic analysis. In APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association. https://doi.org/10.1037/13620-004 Cancino, M., & Panes, J. (2021). The impact of Google Translate on L2 writing quality measures: Evidence from Chilean EFL high school learners. System, 98, 102464. https://doi.org/10.1016/j.system.2021.102464 Chien, S.-C. (2017). Taiwanese College Students’ Perceptions of Plagiarism: Cultural and Educational Considerations. Ethics & Behavior, 27(2), 118–139. https://doi.org/10.1080/10508422.2015.1136219 Crosthwaite, P., Storch, N., & Schweinberger, M. (2020). Less is more? The impact of written corrective feedback on corpus-assisted L2 error resolution. Journal of Second

POST-PRINT VERSION

Language Writing, 49, 100729. https://doi.org/10.1016/j.jslw.2020.100729 Dalglish, S. L., Khalid, H., & McMahon, S. A. (2020). Document analysis in health policy research: The READ approach. Health Policy and Planning, 35(10), 1424–1431. https://doi.org/10.1093/heapol/czaa064 Davies, L. J. P., & Howard, R. M. (2016). Plagiarism and the Internet: Fears, Facts, and

FOR WEBSITE VIEWER ONLYPedagogies. In T. Bretag (Ed.), Handbook of Academic Integrity (pp. 591–606).

Springer. https://doi.org/10.1007/978-981-287-098-8_16 Davis, D. F., Golicic, S. L., & Boerstler, C. N. (2011). Benefits and challenges of conducting

RESPECT PUBLISHER RIGHTS multiple methods research in marketing. Journal of the Academy of Marketing Science,

39(3), 467–479. https://doi.org/10.1007/s11747-010-0204-7 Dizon, G., & Gayed, J. M. (2021). Examining the Impact of Grammarly on the Quality of Mobile L2 Writing. JALT CALL Journal, 17(2), 74–92.

https://doi.org/10.29140/jaltcall.v17n2.336 Doyle, S. (2007). Member checking with older women: A framework for negotiating meaning. Health Care for Women International, 28(10), 888–908. https://doi.org/10.1080/07399330701615325 Ebadi, S., & Amini, A. (2022). Examining the roles of social presence and human-likeness on Iranian EFL learners’ motivation using artificial intelligence technology: A case of CSIEC chatbot. Interactive Learning Environments, 0(0), 1–19. https://doi.org/10.1080/10494820.2022.2096638 Fernández-Gómez, E., Martín-Salvador, A., Luque-Vara, T., Sánchez-Ojeda, M. A., Navarro- Prado, S., & Enrique-Mirón, C. (2020). Content Validation through Expert Judgement of an Instrument on the Nutritional Knowledge, Beliefs, and Habits of Pregnant Women. Nutrients, 12(4), 1136. https://doi.org/10.3390/nu12041136 Fitria, T. N. (2022). Avoiding Plagiarism of Students’ Scientific Writing by Using the QuillBot Paraphraser. Elsya : Journal of English Language Studies, 4(3), Article 3. https://doi.org/10.31849/elsya.v4i3.9917 Flowerdew, J., & Li, Y. (2007). Plagiarism and second language writing in an electronic age. Annual Review of Applied Linguistics, 27, 161–183. https://doi.org/10.1017/S0267190508070086 Frye, B. L. (2022). Should Using an AI Text Generator to Produce Academic Writing Be Plagiarism? SSRN Scholarly Paper. https://ssrn.com/abstract=4292283 Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers (p. 2022.12.23.521610). bioRxiv. https://doi.org/10.1101/2022.12.23.521610 Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2022). How Does ChatGPT Perform on the Medical Licensing Exams? The Implications of Large Language Models for Medical Education and Knowledge Assessment (p.

2022.12.23.22283901). medRxiv. https://doi.org/10.1101/2022.12.23.22283901

POST-PRINT VERSION

Goh, E. (2013). Plagiarism Behavior Among Undergraduate Students in Hospitality and Tourism Education. Journal of Teaching in Travel & Tourism, 13(4), 307–322. https://doi.org/10.1080/15313220.2013.839295 Guo, K., Wang, J., & Chu, S. K. W. (2022). Using chatbots to scaffold EFL students’ argumentative writing. Assessing Writing, 54, 100666.

FOR WEBSITE VIEWER ONLYhttps://doi.org/10.1016/j.asw.2022.100666

Hagendorff, T., Fabi, S., & Kosinski, M. (2022). Machine intuition: Uncovering human-like intuitive decision-making in GPT-3.5 (arXiv:2212.05206). arXiv.

RESPECT PUBLISHER RIGHTS https://doi.org/10.48550/arXiv.2212.05206

Haque, M. U., Dharmadasa, I., Sworna, Z. T., Rajapakse, R. N., & Ahmad, H. (2022). ‘I think this is the most disruptive technology’: Exploring Sentiments of ChatGPT Early Adopters using Twitter Data (arXiv:2212.05856). arXiv.

https://doi.org/10.48550/arXiv.2212.05856 Hayes, N., & Introna, L. (2005). Systems for the Production of Plagiarists? The Implications Arising from the Use of Plagiarism Detection Systems in UK Universities for Asian Learners. Journal of Academic Ethics, 3(1), 55–73. https://doi.org/10.1007/s10805- 006-9006-4 Hsu, H.-C. (2019). Wiki-mediated collaboration and its association with L2 writing development: An exploratory study. Computer Assisted Language Learning, 32(8), 945–967. https://doi.org/10.1080/09588221.2018.1542407 Hu, G., & Lei, J. (2012). Investigating Chinese University Students’ Knowledge of and Attitudes Toward Plagiarism From an Integrated Perspective. Language Learning, 62(3), 813–850. https://doi.org/10.1111/j.1467-9922.2011.00650.x Jabotinsky, H. Y., & Sarel, R. (2022). Co-authoring with an AI? Ethical Dilemmas and Artificial Intelligence. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4303959 Jamieson, S., & Howard, R. (2019). Rethinking the relationship between plagiarism and academic integrity. Revue Internationale Des Technologies En Pédagogie Universitaire / International Journal of Technologies in Higher Education, 16(2), 69–

85. https://doi.org/10.18162/ritpu-2019-v16n2-07 Jamshed, S. (2014). Qualitative research method-interviewing and observation. Journal of Basic and Clinical Pharmacy, 5(4), 87–88. https://doi.org/10.4103/0976-0105.141942 Jeon, J. (2021). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning, 0(0), 1–27. https://doi.org/10.1080/09588221.2021.1987272 Jinrong, L., & Mimi, L. (2018). Turnitin and peer review in ESL academic writing classrooms. Language Learning & Technology, 22(1), 27–41. https://doi.org/10125/44576 Keck, C. (2006). The use of paraphrase in summary writing: A comparison of L1 and L2 writers. Journal of Second Language Writing, 15(4), 261–278. https://doi.org/10.1016/j.jslw.2006.09.006 Keck, C. (2010). How Do University Students Attempt to Avoid Plagiarism? A Grammatical Analysis of Undergraduate Paraphrasing Strategies. Writing and Pedagogy, 2(2), 193–

POST-PRINT VERSION

222. https://doi.org/10.1558/wap.v2i2.193 Kılıçkaya, F. (2020). Using a Chatbot, Replika, to Practice Writing Through Conversations in L2 English: A Case Study. In M. Kruk & M. Peterson (Eds.), Advances in Educational Technologies and Instructional Design (pp. 221–238). IGI Global. https://doi.org/10.4018/978-1-7998-2591-3.ch011

FOR WEBSITE VIEWER ONLYKohnke, L. (2022). A Pedagogical Chatbot: A Supplemental Language Learning Tool. RELC

Journal, 00336882211067054. https://doi.org/10.1177/00336882211067054 Kolb, A., & Kolb, D. (2017). Experiential Learning Theory as a Guide for Experiential

RESPECT PUBLISHER RIGHTS Educators in Higher Education. Experiential Learning & Teaching in Higher

Education, 1(1), 7–44. Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44,

100450. https://doi.org/10.1016/j.asw.2020.100450 Kurniati, E. Y., & Fithriani, R. (2022). Post-Graduate Students’ Perceptions of Quillbot Utilization in English Academic Writing Class. Journal of English Language Teaching and Linguistics, 7(3), 437–451. https://doi.org/10.21462/jeltl.v7i3.852 Lee, L. (2020). An Exploratory Study of Using Personal Blogs for L2 Writing in Fully Online Language Courses. In B. Zou & M. Thomas (Eds.), Recent Developments in Technology-Enhanced and Computer-Assisted Language Learning (pp. 145–163). Information Science Reference. https://doi.org/10.4018/978-1-7998-1282-1.ch007 Li, Y., & Casanave, C. P. (2012). Two first-year students’ strategies for writing from sources: Patchwriting or plagiarism? Journal of Second Language Writing, 21(2), 165–180. https://doi.org/10.1016/j.jslw.2012.03.002 Liu, G.-Z., Lu, H.-C., Lin, V., & Hsu, W.-C. (2018). Cultivating undergraduates’ plagiarism avoidance knowledge and skills with an online tutorial system. Journal of Computer Assisted Learning, 34(2), 150–161. https://doi.org/10.1111/jcal.12223

M. Alshater, M. (2022). Exploring the Role of Artificial Intelligence in Enhancing Academic Performance: A Case Study of ChatGPT. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4312358 Mannion, P., Siegel, M., Li, Z., Pham, Q. N., & Alshaikhi, A. (2019). Technology-Enhanced L2 Writing: A Systematic Literature Review Analysis and Synthesis. Journal of Foreign Language Education and Technology, 4(1), 127–150. Merkel, W. (2020). A case study of undergraduate L2 writers’ concerns with source-based writing and plagiarism. TESOL Journal, 11(3), e00503. https://doi.org/10.1002/tesj.503 Miller, K. S., Lindgren, E., & Sullivan, K. P. H. (2008). The Psycholinguistic Dimension in Second Language Writing: Opportunities for Research and Pedagogy Using Computer Keystroke Logging. TESOL Quarterly, 42(3), 433–454. https://doi.org/10.1002/j.1545- 7249.2008.tb00140.x Morris, T. H. (2020). Experiential Learning – a Systematic Review and Revision of Kolb’s Model. Interactive Learning Environments, 28(8), 1064–1077.

POST-PRINT VERSION

https://doi.org/10.1080/10494820.2019.1570279 Munoz-Luna, R. (2015). Main Ingredients for Success in L2 Academic Writing: Outlining, Drafting and Proofreading. PLOS ONE, 10(6), e0128309. https://doi.org/10.1371/journal.pone.0128309 Nagata, R., Hashiguchi, T., & Sadoun, D. (2020). Is the Simplest Chatbot Effective in English FOR WEBSITE VIEWER ONLYWriting Learning Assistance? In L.-M. Nguyen, X.-H. Phan, K. Hasida, & S. Tojo (Eds.), Computational Linguistics (pp. 245–256). Springer. https://doi.org/10.1007/978-981-15-6168-9_21 Nazari, N., Shabbir, M. S., & Setiawan, R. (2021). Application of Artificial Intelligence RESPECT PUBLISHER RIGHTS powered digital writing assistant in higher education: Randomized controlled trial. Heliyon, 7(5), e07014. https://doi.org/10.1016/j.heliyon.2021.e07014 Neumann, H., Leu, S., & McDonough, K. (2019). L2 writers’ use of outside sources and the

related challenges. Journal of English for Academic Purposes, 38, 106–120. https://doi.org/10.1016/j.jeap.2019.02.002 Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033 OpenAI. (2022, November 30). ChatGPT: Optimizing Language Models for Dialogue. OpenAI. https://openai.com/blog/chatgpt/ Pecorari, D. (2001). Plagiarism and International Students: How the English-Speaking University Responds. In D. D. Belcher & A. R. Hirvela (Eds.), Linking Literacies: Perspectives on L2 Reading-Writing Connections (pp. 229–245). University of Michigan Press. Pecorari, D. (2003). Good and original: Plagiarism and patchwriting in academic second- language writing. Journal of Second Language Writing, 12(4), 317–345. https://doi.org/10.1016/j.jslw.2003.08.004 Pecorari, D. (2022). Plagiarism and English for academic purposes: A research agenda. Language Teaching, 1–15. https://doi.org/10.1017/S0261444821000495 Pecorari, D., & Petrić, B. (2014). Plagiarism in second-language writing. Language Teaching, 47(3), 269–302. https://doi.org/10.1017/S0261444814000056 Qadir, J. (2022). Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education. TechRxiv. https://doi.org/10.36227/techrxiv.21789434.v1 Roe, J., & Perkins, M. (2022). What are Automated Paraphrasing Tools and how do we address them? A review of a growing threat to academic integrity. International Journal for Educational Integrity, 18(1), 15. https://doi.org/10.1007/s40979-022-00109-w Saeed, M. A., & Al Qunayeer, H. S. (2022). Exploring teacher interactive e-feedback on students’ writing through Google Docs: Factors promoting interactivity and potential for learning. The Language Learning Journal, 50(3), 360–377. https://doi.org/10.1080/09571736.2020.1786711 Sánchez-Guardiola Paredes, C., Aguaded Ramírez, E. M., & Rodríguez-Sabiote, C. (2021).

POST-PRINT VERSION

Content Validation of a Semi-Structured Interview to Analyze the Management of Suffering. International Journal of Environmental Research and Public Health, 18(21), 11393. https://doi.org/10.3390/ijerph182111393 Şener, B., & Mede, E. (2022). Promoting learner autonomy and improving reflective thinking skills through reflective practice and collaborative learning. Innovation in Language

FOR WEBSITE VIEWER ONLYLearning and Teaching, 1–16. https://doi.org/10.1080/17501229.2022.2047694

Seyyedrezaei, M. S., Amiryousefi, M., Gimeno-Sanz, A., & Tavakoli, M. (2022). A meta- analysis of the relative effectiveness of technology-enhanced language learning on

RESPECT PUBLISHER RIGHTS ESL/EFL writing performance: Retrospect and prospect. Computer Assisted Language

Learning, 1–34. https://doi.org/10.1080/09588221.2022.2118782 Stapleton, P. (2012). Gauging the effectiveness of anti-plagiarism software: An empirical study of second language graduate writers. Journal of English for Academic Purposes, 11(2),

125–133. https://doi.org/10.1016/j.jeap.2011.10.003 Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays—Should professors worry? Nature. https://doi.org/10.1038/d41586-022-04397-7 Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity? (arXiv:2212.09292). arXiv. https://doi.org/10.48550/arXiv.2212.09292 Taheri, P., & Nazmi, R. (2021). Improving EFL Learners’ Argumentative Writing Ability: Teacher vs. Peer Scaffolding. Teaching English Language, 15(2), 299–333. https://doi.org/10.22132/tel.2021.143348 Traniello, J. F. A., & Bakker, T. C. M. (2016). Intellectual theft: Pitfalls and consequences of plagiarism. Behavioral Ecology and Sociobiology, 70(11), 1789–1791. https://doi.org/10.1007/s00265-016-2207-y Tsai, S.-C. (2019). Using google translate in EFL drafts: A preliminary investigation. Computer Assisted Language Learning, 32(5–6), 510–526. https://doi.org/10.1080/09588221.2018.1527361 Tuzi, F. (2004). The impact of e-feedback on the revisions of L2 writers in an academic writing course. Computers and Composition, 21(2), 217–235. https://doi.org/10.1016/j.compcom.2004.02.003 Wahle, J. P., Ruas, T., Foltýnek, T., Meuschke, N., & Gipp, B. (2022). Identifying Machine- Paraphrased Plagiarism. In M. Smits (Ed.), Information for a Better World: Shaping the Global Future (pp. 393–413). Springer International Publishing. https://doi.org/10.1007/978-3-030-96957-8_34 Wenzlaff, K., & Spaeth, S. (2022). Smarter than Humans? Validating how OpenAI’s ChatGPT Model Explains Crowdfunding, Alternative Finance and Community Finance. SSRN Scholarly Paper. https://doi.org/10.2139/ssrn.4302443 Wu, L., Wu, Y., & Zhang, X. (2021). L2 Learner Cognitive Psychological Factors About Artificial Intelligence Writing Corrective Feedback. English Language Teaching, 14(10), Article 10. https://doi.org/10.5539/elt.v14n10p70 Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. (2022). The Death of the Short-Form Physics Essay in the Coming AI Revolution (arXiv:2212.11661). arXiv.

POST-PRINT VERSION

https://doi.org/10.48550/arXiv.2212.11661 Yeh, E. (2021). Intentional Plagiarism? Strategies for Teaching Language Learners Academic Integrity. Kappa Delta Pi Record, 57(3), 132–137. https://doi.org/10.1080/00228958.2021.1935506 Yoon, H., & Hirvela, A. (2004). ESL student attitudes toward corpus use in L2 writing. Journal

FOR WEBSITE VIEWER ONLYof Second Language Writing, 13(4), 257–283.

https://doi.org/10.1016/j.jslw.2004.06.002 Zhai, X. (2022). ChatGPT User Experience: Implications for Education. SSRN Scholarly

RESPECT PUBLISHER RIGHTS Paper. https://doi.org/10.2139/ssrn.4312418

Appendix 1.

Learning Log for the ChatGPT & Writing Practicum

Name: Matric No.: Session:

1. Daily experiences in using ChatGPT for writing: 2. Workflow used today:

3. Reflection and plan:

FOR WEBSITE VIEWER ONLY

4. Feedback and suggestions for the practicum:

Appendix 2. Interview Protocol

1. What are the effects of ChatGPT on your completion of writing tasks in the practicum?

2. What are the advantages of ChatGPT over its precedents and alternatives?

3. How do you evaluate the quality of textual output from ChatGPT?

4. What is the most striking feature of ChatGPT in your experience?

5. Does ChatGPT pose any threat to the learning and teaching of English writing?

6. If everyone is equipped with the knowledge of ChatGPT in the near future, how do you think about it?

7. How do you think about the rapid spread of ChatGPT in online communities?

8. Do you support or oppose the usage of ChatGPT in writing classrooms?

9. Do you think the official introduction of ChatGPT in language education will have a bright future?

10. If we are to use ChatGPT regularly in classrooms, what do you think we need to adjust most urgently?

POST-PRINT VERSION FOR WEBSITE VIEWER ONLY