Automated feedback complements online dialogic peer feedback: Fostering L2 writers’ feedback quality and self-regulatory writing

strategy use

Da “Alex” Yan

a, *

###, Chenjin Jia

b

###, and Feng Tian

c,d a School of Foreign Languages, Xinyang Agriculture and Forestry University, Xinyang, China. b School of Literature and Journalism, Yibin University, Yibin, China. c Center of Instructional Technology and Multimedia, University of Science, Malaysia, Penang, Malaysia. d School of Foreign Languages, Ningxia University, Yinchuan, China.

• Correspondence: [Postal address as affiliation]; email: alexyan1987@outlook.com This study examined the effects of the complementary integration of automated feedback provided by automated writing evaluation (AWE) systems and online dialogic peer feedback on the quality of feedback and the use of self-regulatory writing strategies among second language (L2) writers. The participants were 68 Chinese L2 students who were assigned to two experimental conditions: one using automated feedback to complement dialogic peer feedback, and the other relying solely on dialogic peer feedback. The participants engaged in dyadic peer feedback for three writing tasks during a 14-week writing course. After collecting peer feedback comments and administering the self-regulatory writing strategy POST-PRINT VERSIONquestionnaire, the researchers coded and statistically analyzed the data. Results indicate that integrating automated and peer feedback enhances the cognitive and constructive quality of peer feedback messages; however, there was no significant difference in the affective features of feedback between the two groups. Additionally, students in the integrated condition more frequently

FOR WEBSITE VIEWER ONLY

employed self-regulatory writing strategies during the text generation, self- monitoring, and revision phases, while their strategy use during the planning phase was comparable to that of students in the peer feedback-only condition.

RESPECT PUBLISHER RIGHTS

These findings have implications for educators and researchers aiming to advance peer feedback practices in L2 writing classrooms.

Keywords: online dialogic peer feedback, automated feedback, L2 writing, feedback quality, self-regulated learning

Introduction

Peer feedback has been widely implemented in second language (L2) writing classrooms as an instructional scaffold to support the development of writing competence (Lee, 2017). Recently, peer feedback has increasingly been integrated into online settings to address teachers’ high workload and time constraints when providing feedback in large classrooms (Er et al., 2021). While the efficacy of peer feedback in fostering writing development is widely recognized (Vuogan & Li, 2023), researchers have also identified challenges that students encounter when attempting to utilize peer feedback effectively. For instance, students, particularly those who struggle to provide constructive feedback without additional guidance and support, experienced significant cognitive barriers in peer feedback processes (Cheng et al., 2023; Kerman et al., 2024). Consequently, a growing body of research has explored various forms of support and scaffolding to enhance both the quality of peer-generated feedback and learners’ engagement with and use of feedback messages (Noroozi et al., 2023; Author, 2024). A supportive measure adopted by researchers and educators in the context of L2

POST-PRINT VERSION

writing is the application of automated feedback to complement peer feedback (Loncar et al., 2023; Tan et al., 2023). A review of related literature shows that such an approach to integrating feedback sources has proven effective in promoting learner mindset and

FOR WEBSITE VIEWER ONLYmotivation (Yao et al., 2021), improving writing performance (Kao & Reynolds, 2024),

reducing anxiety (Liu et al., 2023), and enhancing feedback engagement (Zhang &

RESPECT PUBLISHER RIGHTS

Hyland, 2022). However, further investigation into the effects of integrating automated and peer feedback is warranted. First, existing studies offer limited insights into the effects of the integration of feedback on the improvement of peer feedback quality. In

response to Panadero’s (2023) call that “feedback can be … intervention [or] … dependent variable” in research (p. 198), a closer examination of feedback quality could contribute to a more comprehensive evaluation of the integrative approach as both an intervention and an instructional scaffold. The importance of feedback quality cannot be overstated, as it is a prerequisite for active feedback uptake and successful, feedback-informed revisions (Er et al., 2021; Zong et al., 2021). Second, the current body of scholarship has largely neglected the effects of feedback integration on the development of higher-order capabilities, such as self -regulated learning (SRL). To achieve long- term effects of feedback on performance and learning, feedback messages must first be internalized and transformed into self -feedback through a process significantly influenced by students’ self-regulatory abilities (Nicol & Macfarlane‐Dick, 2006). Unfortunately, in the context of integrating automated and peer feedback in L2 writing classrooms, little is known about how students might strategically regulate this process and its subsequent impact on learning. Against the above-stated backdrops, the current study examined the effects of the complementary integration of automated and peer feedback on the quality of peer feedback messages and the use of self-regulatory writing strategies. Following current

POST-PRINT VERSION

understanding of effective feedback practices, for instance, utilization of student-centered (Reynolds & Zhang, 2023) and technology-enhanced feedback (Kao & Reynolds, 2024), this study was conducted in a 14-week L2 writing course at a Chinese FOR WEBSITE VIEWER ONLYuniversity. This study contributes to the literature on peer feedback and L2 writing and offers insights for researchers and educators interested in advancing contemporary formative feedback practices in L2 writing.RESPECT PUBLISHER RIGHTS

Literature review

Online dialogic peer feedback

The understanding of peer feedback continues to evolve. Peer feedback is defined as a reciprocal process whereby students provide and receive evaluative comments on their work or performance (Hattie & Timperley, 2007; Winstone et al., 2022). In recent years, various online learning environments have been implemented to support the peer feedback process and its impact on writing pedagogy (Latifi et al., 2021; Noroozi et al.,

2023). Compared to in-person peer feedback, online peer feedback leverages the advantages of reciprocal and anonymous performance evaluation, as well as a relatively continuous workflow (Noroozi et al., 2023). At the same time, researchers in the field of assessment and feedback have highlighted the added value of dialogic peer feedback, a collaborative meaning-making process in which students actively engage in dialog with peers to co-construct feedback and knowledge (Er et al., 2021). Wood (2022) argues that technology-mediated dialogic peer feedback enables students to collaboratively create, process, and utilize actionable feedback, leading to improved feedback literacy and uptake. Zhao et al. (2024) proposed a three-step model to explain the mechanism of dialogic peer feedback. The POST-PRINT VERSION model’s second phase, termed “feedback-on-feedback,” refers to the discussion between feedback receivers and providers following the initial feedback exchange. This phase is followed by a third stage of “re-feedback,” in which the original feedback providers

FOR WEBSITE VIEWER ONLY

refine or adjust their comments based on the outcomes of the feedback discussion. In summary, online dialogic peer feedback has been contended as a technology-

RESPECT PUBLISHER RIGHTS

mediated and interactional process of evaluation and knowledge co -construction. Therefore, the current study drew upon the above understandings and aimed to examine the effects of a feedback intervention in a dialogic online peer feedback setting. The

subsequent sections review two aspects of successful peer feedback practices: quality of and self-regulatory learning in peer feedback.

Peer feedback quality

A central challenge that hinders the effectiveness of peer feedback is its unsatisfactory quality. To begin with, peer feedback comments produced by students are frequently criticized for being cognitively superficial due to their limited domain knowledge (Latifi et al., 2023). In the realm of L2 writing, a widely recognized concern regarding peer feedback quality is its disproportionate focus on local issues within writing products (Cheng et al., 2023; Tan et al., 2024). Moreover, the quality challenges faced by peer feedback also have an affective dimension. Receivers’ skepticism toward feedback stems from their distrust in peers’ knowledge as well as environmental factors such as an inharmonious classroom atmosphere (Ibarra-Sáiz et al., 2020). Additionally, peer feedback has been found to lack constructive solutions and actionable suggestions for further improvements (Foo,

2021). Previous literature has attributed this shortcoming to students' limited higher-order thinking skills and evaluative judgment capabilities (Latifi & Noroozi, 2021). Taken together, t

POST-PRINT VERSION hese quality issues reduce the cognitive and affective engagement with

peer feedback, thereby diminishing its impact on learning improvements (Cheng et al.,

2014). To enhance peer feedback quality, existing studies have employed either

FOR WEBSITE VIEWER ONLY

cognitive or social scaffolds. The former enables students to co-construct meaningful evaluative judgmentsRESPECT PUBLISHER RIGHTSwhile the latter promotes deeper engagement with feedback processes (Bürgermeister et al., 2021). In the present study, we designed a feedback intervention by integrating automated feedback with online dialogic peer feedback in a complementary fashion. Following the aforementioned categorization, the intervention

exerts a double-faceted effect on writing development. First, it serves as a cognitive scaffold, expanding the scope of available feedback comments through the integration of multiple feedback sources. Second, it functions as a communicative facilitator, leveraging technology-based platforms to encourage dialogic interaction during peer feedback encounters.

Peer feedback and self-regulated learning in L2 writing context

Beyond its impact on learning/performance and motivation/affect, feedback serves a third purpose: supporting self -regulated learning (SRL), which refers to students’ ability to regulate their (meta)cognition, behavior, emotions, and motivation during learning processes (Panadero & Lipnevich, 2022). Defined as a process encompassing forethought, performance, and self -reflection, SRL is considered a key factor that comprehensively influences cognitive processes in writing development (Teng & Zhang, 2020). Effective self-regulation of the writing process requires L2 writers to proactively deploy strategies to achieve the learning goals by activating, sustaining, and adjusting the cognitive, affective, and behavioral aspects of learning (Zhang & Zhang, 2024). Since both SRL and the writing process are multi

POST-PRINT VERSION-phase constructs, existing SRL

models can be integrated into a framework aligned with the process-genre approach to L2 writing (Shen & Wang, 2024). Based on the integrated framework, we can identify the following four major types of self -regulatory writing strategies: (1) before writing,

FOR WEBSITE VIEWER ONLY

learners employ planning strategies to analyze task requirements, set goals, and generate ideasRESPECT PUBLISHER RIGHTS; (2) during writing, they retrieve relevant information from long-term memory, translate ideas into textual output through text-generating strategies; (3) learners also engage in self-monitoring to refine their work against initial plans during

writing; and (4) after writing, revising strategies are activated to identify and address problems in writing products (Bai et al., 2022). Formative feedback is linked to SRL due to its iterative nature and ability to elicit monitoring and self-reflection in learning (Hattie & Timperley, 2007; Nicol & Macfarlane‐Dick, 2006). Scientific literature indicates that teacher feedback influences EFL writers’ self-regulatory writing strategy use, especially their abilities to set goals, metacognitively monitor learning processes, process text, and handle learning resources (Yang et al., 2022, 2023). Similarly, portfolio feedback is also believed to be an effective practice for fostering L2 writers’ critical awareness in writing evaluation (Lam, 2015; Mak & Wong, 2018). Noticeably, researchers have highlighted the potential of self -assessment and peer feedback to effectively scaffold SRL in classroom settings (Panadero et al., 2016). Compared to other feedback forms, dialogic peer feedback has the following advantages in supporting SRL. First, its reciprocal nature can stimulate feedback recipients’ SRL development while simultaneously co -regulating feedback providers’ writing skills and evaluative judgment (Alemdag & Yildirim, 2022). Second, the dialogic discussion on feedback, also known as back -evaluation, fosters sustained, in-

POST-PRINT VERSION

depth, and critical reflection (Misiejuk & Wasson, 2021). Unfortunately, the use of dialogic peer feedback as an intervention to promote SRL development was scarcely found (Yang et al., 2023). Moreover, following the

FOR WEBSITE VIEWER ONLYaccounts on internal feedback generation (Lipnevich & Smith, 2022; Nicol &

Macfarlane‐Dick, 2006), effective translating from external feedback sources into learning interactions could substantially enhance learnersRESPECT PUBLISHER RIGHTS’ SRL capabilities. Therefore, the complementary use of both automated and peer feedback to foster L2 writers’ self-regulation warrants further investigation and offers valuable insights for the existing

literature. In the following subsection, empirical evidence on integrating automated and peer feedback is synthesized.

Automated feedback complementing peer feedback: empirical evidence

A few studies have examined the effects of combining automated and peer feedback in L2 writing settings. Zhang and Hyland (2022) conducted a case study in a writing class that enacted an integrated approach to combining feedback from automated writing evaluation (AWE) systems, peers, and the teacher. The study showed that the combination of multiple feedback sources enhanced students’ cognitive, affective, and behavioral engagement with feedback and promoted effective revision. Liu et al. (2023) consolidated and extended the above observation through an experimental study. The researchers contended that the peer-automated-combined-feedback intervention enables learners to outperform in writing performance, motivation, critical thinking, and anxiety reduction. Similarly, the factorial experimental study by Tan et al. (2023) also discovered that the combination of AWE and asynchronous peer feedback facilitated learners’ gains in quality of writing products. However, only the study by Yao et al. (2021) implemented the complementary integration of AWE and dialogic peer feedback,

POST-PRINT VERSION engaging participants in discussion

about peers’ writing products and Pigai-generated corrective feedback comments before asking the students to carry out revision based on the integrated feedback messages. The short-term longitudinal study revealed that the integration of automated and dialogic

FOR WEBSITE VIEWER ONLY

peer feedback promotes language mindset and motivation.

RESPECT PUBLISHER RIGHTS

In summary, while the complementary use of automated and peer feedback has shown potential in existing literature, the integration of these two feedback sources within an interactive learning environment requires further research to be systematically explored. In the current study, the advocacy for an integrative approach to automated

and peer feedback is grounded in the student-feedback interaction model proposed by Lipnevich and Smith (2022), which is reviewed in the following section.

The student-feedback interaction model as the theoretical framework

Lipnevich and Smith (2022) theoretically outlined the mechanism by which feedback enhances learning from a student-feedback interaction perspective. They emphasized that the effectiveness of feedback depends on how feedback is processed, rather than merely how it is provided. Their model highlights the need to focus not only on the feedback messages, but also on the agents (i.e., providers and recipients), the context in which the feedback occurs, and the interactions among these components. The model highlights several key feedback variables, for instance, feedback sources, feedback types, characteristics of the students. Internally, Lipnevich and Smith (2022) incorporated core tenets shared by contemporary feedback researchers. For instance, the route from (multiple) feedback sources to feedback messages reflected their broad definition of feedback as “any information about a performance” that can “come from any source” (p. 2). Their accounts on inclusive feedback sources justified the approaches adopted by previous researchers who attempted to employ automated feedback as a complementary source to human

POST-PRINT VERSION-constructed feedback.

Furthermore, the emphasis on feedback processing in the model aligns with our earlier conceptualization of online dialogic peer feedback. Specifically, the formative and iterative processes in online dialogic peer feedback involves student’s agentic

FOR WEBSITE VIEWER ONLY

interactions, use of technological and digital resources to self-generate feedback, collaborativeRESPECT PUBLISHER RIGHTSand dialogic sense-making of the feedback information, and active uptake of the feedback for decision-making about next learning steps. Thus, it is logical to position the feedback processing variable in the model as an umbrella construct that supports our latest takes on feedback agency and learner-centeredness, for example,

student-oriented perspective of feedback (Winstone et al., 2022) and the new feedback paradigm (de Kleijn, 2023). The current study examines the effects of complementing peer feedback with automated feedback on feedback quality and self -regulation. We closely followed the core principles of the student-feedback interaction model to operationalize online dialogic peer feedback, design the feedback intervention (see the intervention section below), and interpret the observed effects.

The current study

Based on the reviewed literature and the theoretical framework, the potential of the complementary use of automated and peer feedback in L2 writing settings is better understood. Nevertheless, an overall evaluation of automated and dialogic peer feedback integration is still needed and anticipated. Therefore, we followed the integration model of AWE-generated and peer feedback proposed by Yao et al. (2021) in our interventional design but shifted the research focus to feedback quality and SRL strategy use among L2 writers. To address the knowledge gap identified in the literature reviewed above, this study is guided by the following research questions (RQ):

POST-PRINT VERSION

(1) To what extent does completing online dialogic peer feedback with automated feedback affect the quality of feedback messages generated by L2 writers? (2) To what extent does completing online dialogic peer feedback with automated

FOR WEBSITE VIEWER ONLY

feedback affect L2 writers’ use of self-regulatory writing strategies?

Methods and materials

RESPECT PUBLISHER RIGHTS

Participants

This study took place in the School of Foreign Languages at a university in China.

Participants were recruited from four classes of a 1 4-week “Introductory English Writing II” course, taught by the same instructor and following the same curriculum. We applied two recruitment criteria during sampling: (1) participants should have prior experience with and an understanding of formative assessment, particularly dialogic and interactive peer feedback practices; and (2) participants should demonstrate at least an average level of English writing proficiency and sufficient feedback literacy to effectively provide and process peer feedback while regulating their learning behaviors throughout the study. Recruitment decisions were made at the researchers’ discretion following a case-by-case review of students’ profiles, which included self -reported details relevant to these criteria. Additionally, to preserve the naturally formed peer partnerships, only dyads in which both members qualified and agreed to participate were recruited. As a result, the researchers recruited a sample of 68 L2 students (Mage=18.7 years; 11 males and 57 females). These peer feedback dyads were randomly assigned to either a control group (CG; n=32), using online dialogic peer feedback without automated feedback, or an experimental group (EG; n=36), using the complementary integration of automated and online dialogic peer feedback. Pre-study evaluations

POST-PRINT VERSION

indicated that the two groups did not have statistically significant differences in writing proficiency measured by an official TEM-4 writing task [F(1, 66)=1.56; p=.217; η 2p =0.02]. Survey data on prior feedback experiences showed that students commonly FOR WEBSITE VIEWER ONLYengaged in peer feedback (M=3.76 on a five-point scale, SD=1.28); however, only six participants reported prior experience with the integrated use of multiple feedback sources in response to openRESPECT PUBLISHER RIGHTS-ended survey questions.

Intervention

We adopted a theory-informed approach to design the interventional conditions

implemented in this study (see theoretical framework). As illustrated in Figure 1, the two experimental conditions differed only in the feedback sources (i.e., the CG relied solely on peer feedback, whereas the EG utilized both automated and peer feedback) and in how the dyads engaged in dialogic peer feedback discussions to process the feedback. After the dialogic peer feedback sessions, students individually revised their work based on the feedback.

Figure 1. Experimental conditions.

The dialogic peer feedback processes designed for each condition followed the three-step model proposed by Zhao et al. (2024). For the CG, after the initial provision of peer feedback, the feedback dyads were required to engage in a feedback

POST-PRINT VERSION

-on- feedback process. The feedback-on-feedback processes involved (1) clarifying the intent of the writing task, (2) seeking clarification on ambiguous feedback points, (3)

FOR WEBSITE VIEWER ONLY

discussing and resolving disagreements between the feedback provider and recipient, and (4) nailing down alternative revision plans. In the re-feedback stage, the feedback provider refined RESPECT PUBLISHER RIGHTS the original feedback message based on the clarifications and agreements reached during the discussion.

Similarly, EG members engaged in these peer feedback processes but with an additional step: they also discussed and analyzed the automated feedback generated by Grammarly. In the re-feedback stage, EG feedback providers refined their feedback comments by incorporating insights gained from discussions about both peer and automated feedback.

Procedures

This study employed a short-term longitudinal experimental design. As shown in Figure 2, course orientation and dialogic peer feedback training were arranged in the first week. During instructional weeks, the teacher focused on core writing strategies and competence development. In the course, three writing tasks (T1-T3) following the official TEM-4 writing task requirements were assigned. An individual writing and feedback week is allocated for each writing task. Specifically, in each writing and feedback week, students were advised to spend one individual session (45 minutes) to prepare their draft writing, another session to engage in dialogic peer feedback with their partner, and finally one or two sessions to finish the revision based on the received feedback messages.

POST-PRINT VERSION FOR WEBSITE VIEWER ONLY

RESPECT PUBLISHER RIGHTS

Figure 2. Research procedures.

POST-PRINT VERSION

Dialogic peer feedback was conducted in written forms using Shimo.im, an online collaborative office suite. Feedback-on-feedback and re-feedback were appended to the original feedback messages as discussion threads, enabling a continuous and

FOR WEBSITE VIEWER ONLY visible workflow of dialogic and iterative knowledge co-construction. When the

dialogic peer feedback processes for a certain writing product concluded, the feedback dyads were requested to submit a finalized version of the feedback messages with which

RESPECT PUBLISHER RIGHTS

both students were content.

Peer feedback quality was assessed three times, following each writing task. The teacher collected the finalized version of the peer feedback messages for coding and analysis. Self -regulatory writing strategy use was measured through a pre-and post- questionnaire.

Measures

Feedback quality

To measure feedback quality, the coding scheme developed by Kerman et al. (2024) was applied. The coding scheme included three major dimensions of peer feedback quality: (1) affective, (2) cognitive, and (3) constructive. The cognitive category was further divided into three sub-categories: description, identification, and justification. Each of the five features was rated on a scale from 0 to 20, depending on its quality. The details of the coding scheme are shown in Appendix A. To ensure inter-rater reliability, the first author and five co -researchers independently coded 20% of the total peer feedback messages collected from the study. Inter-rater reliability was measured using Fleiss’ kappa: (1) affective (0.89), (2) description (0.84), (3) identification (0.79), (4) justification (0.92), and (5) constructive

POST-PRINT VERSION

(0.81). Discrepancies were addressed through discussion until consensus was reached. Then, the coders shared the remainder of the coding duty.

Self-regulatory writing strategy questionnaire

FOR WEBSITE VIEWER ONLY

We used the SRL writing strategy questionnaire developed by Bai et al. (2022) to assess students

RESPECT PUBLISHER RIGHTS

’ use of self-regulatory writing strategies. The questionnaire included four main dimensions and a total of 17 items. All responses were rated on 5-point Likert-type scale ranging from “never” to “always”. The four sections of the instrument respectively

measured students’ strategy use planning (4 items), text-generation (4 items), self-monitoring (5 items), and revision (4 items). A sample item under the self -monitoring dimension reads “I re-read the task requirements to generate new ideas.” For the research sample, all four sub-scales of the instrument demonstrated high reliability (Cronbach’s α = 0.82, 0.92, 0.88, and 0.84, respectively).

Analysis

To address RQ1, Mixed ANOVA tests were conducted using afex¹ and emmeans² packages in R software to compare the two conditions in terms of the quality of peer feedback. Before performing the Mixed ANOVA tests, we assessed the assumptions of normality (Shapiro-Wilk test; p> .05) and homogeneity of variance (Levene’s test; p>.05). The Greenhouse-Geisser correction was applied to within- subject factors that violated the sphericity assumption (i.e., p< .05 in Mauchly’s test). Partial eta squared (η 2p ) was used to assess the effect size in Mixed ANOVA, and values were interpreted based on the suggestions from Gignac and Szodorai (2016). Following the Mixed ANOVA, post-hoc comparisons were conducted to decompose the interaction effects across different time points. Bonferroni correction was used to adjust *p-*values and control the risk of a Type I error due to multiple

POST-PRINT VERSION

hypothesis tests across different time points. Significant pairwise comparisons were followed up with effect size calculations (using Cohen’s d indices) to assess the magnitude of the observed differences. The interpretation of the Cohen’s d followed the

FOR WEBSITE VIEWER ONLY

cutoffs recommended by Plonsky and Oswald (2014) for L2 research.

RESPECT PUBLISHER RIGHTS

https://cran.r-project.org/web/packages/afex/index.html https://cran.r-project.org/web/packages/emmeans/index.html

For RQ2, we performed One-way analysis of covariance (ANCOVA) using afex package to examine whether L2 writers’ use of self -regulatory writing strategies improved after the intervention. Prior to conducting ANCOVA, we examined the assumption of homogeneity of regression slopes to ensure that there were no statistically significant interactions between the intervention conditions (treatment; independent variable) and the covariates. The effect sizes for ANCOVA were assessed using partial eta squared (η 2p ) and interpreted following the same standard as in Mixed ANOVA.

Results

Peer feedback quality

Table 1 presents the descriptive statistics of feedback quality, and Table 2 shows the

results of the Mixed ANOVA tests. As indicated, statistically significant interactions and main effects (ps <.05) of the grouping and test time factors could be found for description, identification, justification, and constructive dimension of feedback quality. However, for the affective dimension, a significant interaction effect was found, but the grouping factor did not have a significant main effect [F(1,66) = 0.22; p = .64;

POST-PRINT VERSION

η 2p =.003].

Table 1. Descriptive statistics of feedback quality.

FOR WEBSITE VIEWER ONLY RESPECT PUBLISHER RIGHTS

Table 2. Results of Mixed ANOVA tests on peer feedback quality

Note: *** p <.001; df: Greenhouse-Geisser corrected degrees of freedom; SS: Sum of squares.

Coupled with the descriptive statistics of quality scores of each category across all tests (Table 1), between-subjects post-hoc pairwise comparison showed that: (1) for the affective feature, the CG and EG learners did not have significant differences across the three tasks [

POST-PRINT VERSIONps>.05]; (2) in terms of the description feature, the EG outperformed the

CG with large effect sizes in T1 [p<.001; Cohen’s d=2.24] and T3 [p<.001; Cohen’s d=2.51]; (3) regarding the identification feature, EG surpassed CG in T2 [p<.001; Cohen’s d=5.53] and T3 [p<.001; Cohen’s d=5.31] with large effect sizes; (4) for

FOR WEBSITE VIEWER ONLY

justification feature, EG outperformed CG [ps<.001] in all three tests with incrementally larger effect sizes [Cohen’s ds=4.02, 4.69, 6.05, respectively]; and (5)

RESPECT PUBLISHER RIGHTS

pertaining to the constructive feature, EG significantly [ps<.05] surpassed CG in all three tasks by incrementally larger margins [Cohen’s ds=2.94, 5.42, 7.76, respectively].

Note: ** p <.01; *** p <.001

Discussion

In answering RQ1, the study demonstrated that the complementary integration of automated and dialogic peer feedback supported students to construct feedback messages with more comprehensive and critical content and constructiveness. The findings were generally in alignment with previous understanding that the inclusion of multiple feedback sources in a technology-mediated language learning setting could improve the quality and usability of feedback (Loncar et al., 2023). Specifically, students’ improvements in content-related feedback quality can be attributed to their enhanced critical thinking abilities (Liu et al., 2023) and increased cognitive engagement in the feedback process (Zhang & Hyland, 2022). Moreover, with the relatively higher level of interactivity during peer feedback processes, the gains in higher-order cognitive skills enable students to provide more constructive comments

POST-PRINT VERSION

that facilitate recipients’ uptake and utilization (Noroozi et al., 2023). Thus, the findings from this study suggest that integrating automated and dialogic peer feedback can help address persistent criticisms of peer feedback, particularly its tendency toward

FOR WEBSITE VIEWER ONLY

superficial evaluation and lack of constructive suggestions for improvement (Latifi et al., 2023).

RESPECT PUBLISHER RIGHTS

However, the study also found that the intervention did not enhance the affective features of peer feedback. This is somewhat unexpected, as previous literature suggests that technology-mediated dialogs can fortify affective rapport among learners and

“lower relational and social-affective barriers” (Wood, 2022, p. 338). The finding also contrasts with Cheng et al. (2023), who reported that a dialogic peer feedback intervention improved students’ trust in peer feedback. In contrast, the present study revealed that, compared to the CG, EG members did not show noticeable improvement in commenting with affectively friendly tones and encouraging emotions. The discrepancies in research findings could be explained by the methodological differences. While Cheng et al. (2023) measured perceived trust in peer feedback through self-reported questionnaires, the current study analyzed the affective quality of actual feedback comments co-constructed by students in a naturalistic setting. This suggests that establishing “a supportive and secure learning environment” to foster trust and harmony in peer feedback (Lee, 2017, p. 90) may be a complex, time -consuming process that requires the synergy between students’ perception and behaviors during feedback processes. Regarding RQ2, this study found that integrating automated and dialogic peer feedback enhanced overall self -regulatory writing strategy use among L2 writers. This finding aligns with previous research on the effects of formative assessment on SRL development (Nicol & Macfarlane‐Dick, 2006; Panadero & Lipnevich, 2022).

POST-PRINT VERSION

Notably, EG learners reported higher levels of regulatory strategy use in the during-writing (i.e., text-generation and self -monitoring) and post-writing (i.e., revision) phases. Statistically significant differences between the two groups during writing could FOR WEBSITE VIEWER ONLYbe attributed to the belief that dialogic peer feedback can regulate feedback recipients’ writing performance and co-regulate feedback providers’ evaluative judgment skills (Alemdag & Yildirim, 2022)RESPECT PUBLISHER RIGHTS. Similarly, the significant changes in self-reported post- writing SRL strategy use of EG learners could be linked to their expanded opportunities to engage in feedback and feedback-on-feedback sessions, which fostered their self-

reflectivity and subsequent revision decisions (Misiejuk & Wasson, 2021). Overall, this study contributes to existing scholarship by emphasizing that incorporating automated and peer feedback sources significantly enhances self-regulatory writing strategy use. Unfortunately, the intervention did not significantly impact the use of pre- writing self-regulatory strategies (i.e., planning). This contrasts with the SRL and feedback model (Nicol & Macfarlane‐Dick, 2006) and several empirical studies in L2 writing contexts (e.g., Mak & Wong, 2018; Yang et al., 2022, 2023), which suggest that internalizing external feedback can influence the entire SRL process. The lack of statistical significance may be explained by Panadero et al.’s (2016) emphasis on the importance of discussing assessment criteria during the forethought phase of SRL. In this study, however, L2 writers engaging with either peer feedback or automated feedback received a brief training on peer feedback, but assessment criteria were not explicitly provided during the feedback processes. Given this limitation, the intervention could be enhanced by incorporating training or co -development of assessment criteria, as suggested by Bürgermeister et al. (2021) and Author (2024).

Conclusion, limitations and directions for future research

In this study,

POST-PRINT VERSIONwe investigated the effects of the complementary use of automated and

peer feedback on L2 writers’ feedback quality and self -regulatory writing strategy use. The findings revealed that the integration of automated and dialogic peer feedback led to improvements in the cognitive and constructive dimensions of feedback quality.

FOR WEBSITE VIEWER ONLY

Moreover, students exposed to the integrated feedback condition demonstrated enhanced use of selfRESPECT PUBLISHER RIGHTS-regulatory writing strategies across key stages of the writing process. Based on the results of the present study, the following pedagogical implications emerge. First, incorporating computer-generated feedback as a complementary source to

peer feedback should be scaled up in contemporary language classrooms. The study demonstrated that integrating automated and peer feedback improved the quality of student-constructed feedback comments and supported SRL development. However, this integrative approach remains less widely practiced compared to their separate applications. Thus, L2 educators are encouraged to adopt this model to scaffold students’ language competence and higher-order skills. Second, educators should pay more attention to students’ regulatory abilities. Undoubtedly, the benefits of expanding feedback sources come at the cost of additional time and cognitive demands. To help students effectively engage with these extended learning resources, educators should emphasize skills such as evaluative judgment, feedback literacy, and self-and co- regulation. Finally, educators should design and implement scaffolding measures that enhance students’ awareness of and ability to produce, interpret, engage with, and apply feedback, especially from a relational and socio -affective perspective. Inevitably, this research faced its own limitations. First, the study implemented an integrated approach combining dialogic peer feedback with AWE feedback. Given the emergence of more advanced technology such as generative artificial intelligence systems, AWE systems may no longer be the ideal complement for peer feedback,

POST-PRINT VERSION

particularly due to their black-box mechanisms regarding assessment criteria and user customizability. Therefore, future research should explore the possibilities of integrating newer technologies with peer feedback to enhance interactivity in feedback processes. FOR WEBSITE VIEWER ONLYSecond, the duration of the intervention and the sample size may still limit its validity and generalizability. Replicating the study with a different sample (in terms of size or origin) and a longer duration or more frequent feedback exposure could yield differentRESPECT PUBLISHER RIGHTS and interesting results. Finally, this study employed a quantitative approach to data collection and analysis. Future research could consider incorporating qualitative

approaches such as think-aloud protocols or content analysis to garner nuanced insights about peer feedback.

Additional information

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors