Intersections between Tutorial Engagement, Directive Feedback, and Critical Reflection

A handful of research studies have investigated the effect of writing centre tutorials on subsequent revisions. This classroom-based study adds to that research by reporting results from a collaborative study between a composition professor and a writing centre tutor. The aim of the study was to examine the influence of writing centre tutorials on immediate student revisions as well as final drafts. The analysis was extensively framed by the Vygotskyan sociocultural model of language and cognitive development with an emphasis on tutor-student engagement as reciprocal interaction which include directive feedback and consequential revision. This study employed a qualitative design with students in a sophomore-level core composition course. Participants attended a writing centre session concerning their major writing assignment. Data triangulation included analysis of assignment drafts, observation notes, and tutorial transcripts. Findings revealed that students attended to feedback that was directive and straightforward. Additionally, students did not attend to citation feedback unless it was direct and explicit. Furthermore, students sometimes overgeneralized and misapplied the feedback. The findings highlight the impact of individual learner factors as well as the results of directive feedback on revisions.


Introduction
Over the past several decades, the pedagogical practices of writing centre tutors and composition teachers in first or second language writing have undergone substantial changes. Researchers point out that writing centre tutors help learners write better compositions by providing students feedback on their writing during one-on-one tutorials (Bromley, Schonberg, & Northway, 2015;Eckstein, 2014;Guity, 2004;Uscinski, 2015).
One-on-one writing centre tutorials are intended to address the writer's various concerns which occur by "identifying a writer's specific writing strengths and challenges, offering a reader's response to a writer's work, discussing different revision strategies, rhetorical choices, approaches; and advising writers on issues of clarity, style, and argument structure" (Appleby-Ostroff, 2017, p. 69). Knowing when and how to give feedback on such issues that then lead to successful revision requires effective tutor training and comprehensible clarification on the part of the tutor during the tutorial.
Writing centre pedagogy distinguishes between direct and indirect tutoring strategies in giving feedback (Truesdell, 2007). Indirect or non-directive feedback involves targeted questioning and careful listening to guide the writer towards more active learning (Boyd & Haibeck, 2010).
Whether written or oral, indirect feedback is intended to encourage self-correction (Ellis, 2009, p. 14). Direct feedback is intended to "give explanations, examples, or the answer...planning what the student should do next" (Thompson, 2009, p. 427). Direct feedback means the tutor gives explicit instruction on what the writer should do to revise or correct some portion of the text. Both types of feedback can address global or higher order concerns (HOCs)-referring to discussions concerning rhetorical concepts-and local or lower order concerns (LOCs)-referring to discussions regarding sentence-level issues (Ryan & Zimmerelli, 2010, pp. 48-54).
The work of writing centres is closely aligned with the Vygotskyan sociocultural model of language and cognitive development. We adopted this framework because of its emphasis on how learners develop their linguistic and cognitive skills through participation in activities with others. Vygotsky (1962) highlights the importance of social interaction during teaching and learning. In writing centre scenarios, this interaction might take the form of a collaborative conferencing style.
to disrupt the goal of making "better writers not better writing" (North, 1984, p. 438) a handful of scholars have sought to investigate the connection between visits to the writing centre and grades.
Most of these studies look at overall GPA in connection to writing centre visits. Lerner's 1997 study investigated whether visits to the writing centre led to higher grades in first-year composition. He reported that, indeed, students who visited the writing centre had higher grades than the students who did not. In a second study investigating impact on grades, he also found "a pretty powerful relationship" between writing centre visits and grades (Lerner, 2003, p. 69). Thompson (2006) reports on data that compared composition grades and GPAs of first-year students who used the writing centre to those who did not. She found a correlation between those students who used the writing centre and "higher grades in composition [as well as] higher overall GPAs" at the end of their fall semester (p. 46). Though she did not distinguish whether or not students visited voluntarily or as a requirement, her study is encouraging in that it offers empirical evidence that writing centre interventions have positive academic outcomes. Archer (2008) examined textual features to see what improvements, whether actual or perceived, were made in student writing. Student improvement was assessed looking at three criteria: organization, voice and register, and language use. Archer found student improvement in all areas. However, in Archer's study, the "grades" were given by the researchers. Using a template, they graded the rough drafts and final drafts to draw their conclusions. Archer does not mention the actual grade given by the course instructor.
In sum, calls continue for research regarding what happens after a writing centre conference.
Though some expressed caution about making a connection between grades and writing centre visits, a listserv discussion on "WCENTRE" digest (April 2015) revealed the subject to be a research interest. However, such calls for research have been magnified by the need for that research to be "evidence-based" (Driscoll & Perdue, 2012, p. 16). The challenge of demonstrating tutorial impact, therefore, has been augmented by the need to investigate in a way that is replicable, aggregable, and data driven (RAD).
Combined calls for RAD research and evidence-based outcomes led to the following study.
Though not without limitations, it seeks to address this call by reporting on collaboration between two of the authors: one acting as the composition instructor (hereafter referred to as "the instructor") and the other as the writing centre tutor (hereafter referred to as "the tutor"). This study investigated student responses to writing centre feedback by examining the pre-tutorial draft, post-tutorial draft(s), and final draft along with tutorial transcripts. Rather than an attempt to assess so-called writing centre effectiveness, we present this research as an illustration of student response to a tutorial by analyzing subsequent drafts including the student's final graded draft. In other words, an analysis of the transcripts and drafts shows what the student did Canadian Journal for Studies in Discourse and Writing/Rédactologie Volume 28, 2018 http://journals.sfu.ca/cjsdw 250 following the tutorial. A look at the summative feedback of the final paper points toward the outcome of the session in terms of grades.
Thus, a non-directional research question guided this interpretive qualitative study: Which, if any, topics of the writing centre tutorial did students attend to 1 as they revised their papers following the tutorial?
The following section describes the methodology conducted in this study including a description of the participants, setting, and procedure.

Participants, Setting, and Procedure
This study reports on data from three students (David, Tom, and Alan). We chose these three because the transcripts revealed that each was engaged in the tutorial, but their final grades ranged from a D to an A. Also, they were all full-time students in the traditional student age range (18)(19)(20)(21)(22), which eliminated the variables of full-time student status and age. In each case, we mention their classification and major just as a means of providing some background. David was a first-year student majoring in Gaming and Animation. Alan was a sophomore biology major with future plans of attending medical school. Tom, a non-native English speaker, was a senior Computer Science major from Uganda. Though we did not investigate this aspect, it might be that a senior is a more mature "reviser" than a first-year student; it might be that a student's major and future plans influence attention to detail.
The study took place at a small, liberal arts university in the southwest region of the United States during a three-week summer intensive course where students met for three hours Monday to Friday. The course was a required sophomore-level English course with a focus on research, writing, and presentation. The students were required to write an 8-10 page persuasive research paper using scholarly sources. Ten students enrolled in the course. Following standard ethics review board protocol approved by the university, students were invited to participate in this study by signing an informed consent form. After grade submission at the end of the course, we learned that nine students had agreed to participate. However, due to length restrictions, the information presented here covers the tutorials and analysis of only the three students' papers.
These students were chosen because the transcripts showed a high level of engagement in the tutorial, but the grades on their final papers showed a great level of disparity-from an A to a D paper.
The composition instructor has a PhD in TESL with a secondary emphasis in Composition/Rhetoric and Writing Centre Pedagogy. She has taught composition in higher Canadian Journal for Studies in Discourse and Writing/Rédactologie Volume 28, 2018 http://journals.sfu.ca/cjsdw education for over two decades. The tutor is a post-baccalaureate English/TEFL major currently pursuing an MA in English. She had been a writing centre tutor during her undergraduate years at the university and was working as an ESL teacher with a private language school when data collection began in May 2012.
The tutorials took place at the beginning of week three after students had written their first rough draft. As part of the course design, all students signed-up for a 30-minute one-on-one tutorial in the writing centre followed immediately by a 30-minute minimum revision period.
They submitted copies of the drafts to the instructor before the tutorial and after the 30-minute minimum revision period. These drafts were labeled "Pre-tutorial Drafts" and "Post-tutorial Drafts" respectively.
The tutorials were not in any way scripted, but the tutor tended to follow the same pattern each time. She began by asking students to name the topic, identify the thesis, and explain the paper's organization. Thus, the tutor's style of tutoring was to ask questions that encouraged students to talk through their papers. During each individual 30-minute session, the tutor and student referred to the printed copy of the draft. After this oral walk-through of the paper, the tutor asked students to identify any specific concerns they wanted to address in the tutorial.
During the tutorial, the tutor refrained from writing comments on the students' drafts; she wrote comments on a peer feedback form used in the writing centre and encouraged the students to make notes on their drafts. The instructor noted the students' attendance at the tutorial on a chart and noted the length of time when a student spent longer than 30-minutes in revision. The instructor did not intervene during the post-tutorial revision, which was treated by many students as a silent revision time. For this immediate revision, students moved to a quiet corner of the writing centre.
Analysis involved isolating tutorial discussions of HOCs and LOCs. Although any bifurcation may be artificial, for purposes of analysis and coding, we operationalized HOCs as discussions concerning thesis, development, organization, and style; we operationalized LOCs as discussions regarding sentence-level issues including word choice, citation information, grammar, and mechanics. The taxonomy of revision changes suggested by Bell (2002) was adopted while operationalizing HOCs and LOCs.
The 30-minute tutorials were audio-recorded and then transcribed. After making multiple copies of each draft to preserve the originals, we independently read the drafts and transcripts and began a 4-step colour-coding process: 1. pink to highlight HOC discussions 2. yellow to highlight LOC discussions Canadian Journal for Studies in Discourse and Writing/Rédactologie Volume 28, 2018 http://journals.sfu.ca/cjsdw 252 3. blue to highlight HOCs and LOCs attended to in the revision

orange to highlight HOCs and LOCs not attended to in the revision
In the margins of the drafts, we listed the topics represented in the tutorials and compared the post-tutorial draft to the pre-tutorial draft looking to see what, if any, impact the tutorial discussion had had on the students' immediate revisions. Finally, following the same highlighting procedure, we examined final drafts for any additional revisions tied to the tutorial discussions.
In addition to the colour-coding, we made paragraph-by-paragraph outlines of each draft.
These outlines gave us a quick overview of the organization of each draft when we were referring back to the tutorial session.
In this way, the analysis had two parts. The first part was a comparison of tutor feedback on immediate revisions. These data were drawn directly from the transcript and two drafts: pre-and post-tutorial. The second part involved looking at the final draft to see if tutor feedback that was not attended to in the post-tutorial draft was later attended to in making final revisions to the draft. In relation to data analysis, data triangulation included assignment drafts (pre-and posttutorial draft and final draft), observation notes (professor and tutor), and tutorial transcripts.

David
In the notes, the tutor wrote that David "needed more material, guidance, needed to talk it out." David brought a four-page draft to the tutorial. The topics of the tutorial generally followed the order of the writing centre feedback sheet the tutor used for noting comments. 2 In David's case, the 30-minute tutorial consisted exclusively of HOCs connected to content: thesis, organization, and development including clarification of thesis, support, and counterarguments. As illustrated in Table 1, David attended to some of the topics raised during the tutorial. The final draft reflected attention to the tutorial feedback especially in reference to making the organization clearer.
The pre-tutorial draft lacked development; it had four pages compared to the eight-to ten-page requirement. The pre-tutorial draft included two introductory paragraphs, three body paragraphs covering places where holiday consumerism occurs, one paragraph addressing waste, and one incomplete conclusion paragraph. In addition to a lack of development, the structure of David's paper was disorganized and disconnected. Even though the paper was fraught with sentence-level issues, the tutorial covered HOCs exclusively. A large part was directed toward brainstorming to help flesh out ideas and work on organization.
Canadian Journal for Studies in Discourse and Writing/Rédactologie Volume 28, 2018 http://journals.sfu.ca/cjsdw In David's post-tutorial revision, the thesis remained at the end of the second paragraph, but the wording changed to include the key terms "consumerism" and "waste," which were identified as David began this paragraph in his final draft as follows: "Food is not the only thing being overly consumed during the holiday season, wild moss is slowly starting to disappear from the wild." He then went on to discuss how wild moss is connected to Christmas decorations and how its disappearance affects the wildlife that depend on it for food and shelter. Later in the paragraph he tied in the idea of Christmas trees: "Another threat to natural habitats is the consumption of trees. During the Christmas period a vast amount of trees are cut down for consumers to use during Christmas." He closed the paragraph by tying the two points together: "The consumption of trees and moss is proof that Christmas is a consuming monster that is being sold commercially to the public." To his original sentence: "Christmas is an environmental health issue that consumers are partaking in" he added "whether it be buying gifts or a Christmas tree." It appears that, to some extent, David attempted to apply the tutor feedback to his revision by reorganizing his points and adding some details. However, directive comments alone did not always lead to revision. The feedback that he completely ignored was the tutor's comments regarding counterarguments. He failed to address those at any point in the paper even though the tutor had discussed it with him. Roughly one-third of the tutorial included brainstorming as the tutor tried to help David add content to his paper. None of this feedback was attended to in the post-tutorial draft though he did attempt revisions in the final draft.
On the surface, the session would appear successful from a Vygotskian perspective in that David played an active part in the tutorial, sometimes explaining his strategy other times explaining his ideas. However, even though David added more content to the final draft, the length was still two pages shorter than required, and he neglected to attend to the tutor's encouragement to add counterclaims to his paper. His final draft was also riddled with sentence level and citation errors. The final draft received a D. The instructor wrote a summative comment: "Good idea -just not finished."

Alan
The tutorial with Alan focused almost exclusively on the HOC topic of content especially as it related to organization. His focus on the topic of diabetes reflected a personal interest as well as a professional interest. In her notes, the tutor observed that Alan needed work on organization. Alan brought an eight-page rough draft to the tutorial. It was a full draft with works cited, in-text citations, and even a figure he wanted to work into the text. It reflected thought and effort. Alan attended to the main points of the tutorial immediately following the tutorial (See Table 2). The tutor opened the session by asking Alan to show her where the thesis was stated. Alan had put his thesis at the end of his introduction, but his introduction was three pages long, which was over one-third of his whole draft. The tutor then asked him to explain the organization of the rest of his paper, which he did. This explanation lasted over sixteen minutes of the thirty-minute tutorial. During this part of the tutorial, the tutor often sought clarification with statements or questions, which Alan ably provided. As they reached the conclusion of his draft, she said directly, M: Your organization seems kind of like jumping all over the place.
A: I know.
She then used directive language with reference to his introduction, suggesting that he should "keep the introduction short" and use it to give a "glimpse" of what his paper will argue:

M: I would suggest in your introduction stating…just briefly, what diabetes is.
Alan made this change in the post-tutorial draft, moving a paragraph from pages five and six of his pre-tutorial draft to the second paragraph on page one. Another example of directive language occurred in reference to the content of his introduction. She advised that he save much of the research in his introduction until after he stated his thesis: Alan followed this advice; he reworded the opening paragraphs to include information about relevant studies, but he moved the details of the studies to after his thesis. The tutor continued to advise him, using directive language, regarding his organization. In his post-tutorial draft, Alan moved a paragraph about diet from the end of his pre-tutorial draft (paragraph nineteen, page seven) to page two paragraph three of his post-tutorial draft right before the paragraph about surgery.
Alan's engagement in the tutorial is illustrated by the following excerpt in which he asked whether his summary of a key study was "an acceptable length." The tutor pointed out that the length of the summary was not as problematic as the location. She explained that he was trying to do too much in his introduction before he stated his thesis and began supporting his argument.
Alan continued asking advice about stating his points, to which the tutor reminded him:

M: Remember it's your introduction. You don't have to throw in all the details of what you
found out.
The tutor instructed Alan to limit his introduction to one page and to save the specifics of the research study for after the thesis. The tutor suggested merging the information about dieting and surgery because they were both solutions to the problem "and then talk about counterarguments." The following excerpt reveals a mixture of directive and non-directive language in which the tutor concedes that ultimately Alan would decide what to do with the organization. The transcript shows the tutor mixed directive and non-directive statements and that neither seemed to curtail Alan's engagement in the tutorial. The post-tutorial draft shows that Alan did not slavishly or mindlessly follow the suggestions of the tutor.
All in all, Alan's post-tutorial draft showed clear changes in organization. He rewrote sentences as necessary to make them work with the changes in paragraph organization. Although he did not Canadian Journal for Studies in Discourse and Writing/Rédactologie Volume 28, 2018 http://journals.sfu.ca/cjsdw 258 keep his introduction to one page/two paragraphs, he did shorten it from eight paragraphs to five paragraphs in the post-tutorial draft. He followed the tutor's advice to explain why diabetes is an issue early on by moving a paragraph from pages five and six to page one. He also moved a paragraph from page seven to the third paragraph of his essay. He followed the tutor's suggestion to keep information about surgery and diet together in discussing the experiment, moving and adding information from a paragraph on page three to page two, just before his thesis. He followed her advice to give the details of the study in the body of his paper rather than the introduction by moving that paragraph from page one to page three and reorganized his counterclaims, moving paragraphs from pages three and four to page five. He absolutely followed the advice about reorganizing his points about diet and surgery from page five and six in his pre-tutorial draft to page seven of his post-tutorial draft. Alan was the only student in the class to earn an A on the final paper. On his final draft, the instructor wrote: "Well done! Very well organized!"

Tom
In her notes, the tutor observed that Tom had a "strong outline" but identified "issues with citing [and a] really broad topic." She felt that he was very engaged in his paper and in the tutorial.
During the revision period following the tutorial Tom worked for sixty minutes, which was twice the amount of time required. Tom also signed up for a voluntary extra tutorial. See Table 3 for Tom's feedback record. Tom's final draft received a B-due to a lack of attention to details most of which were never addressed in the tutorial, such as when to use quotation marks, when to italicize or underline, the need to refer to authors by last name, and the need to distinguish between two articles by the same author. In addition, Tom turned in his final draft with a template cover page, which was unnecessary, and with the first page of the paper missing. The teacher had to refer to his most recent draft to read for his introduction and thesis before returning to the final draft. The tutorial largely addressed larger concerns, but these finer points detracted from the final grade. These are largely editing concerns and there was some carelessness in omitting page one. Tom had a very strong, well-researched argument, but he did not submit an A paper because of these lapses. On the graded draft, the instructor wrote, "What form is this?" on page one of a template style cover page. On the last page, she noted "good research."

Discussion
The findings from this study suggest the following: (1) Students generally attended to feedback that was directive and straightforward. Students did not attend to sentence-level issues that were not addressed in the tutorial or citation feedback unless it was direct and explicit. With topics concerning thesis, organization, and style, students responded to directive feedback. Their revisions improved the text showing they understood and applied the feedback. On the other hand, indirect feedback sometimes led to ineffective revisions.
For example, students often misinterpreted documentation feedback that "you're doing this right" and overgeneralized what "this" referred to. Consequently, this study suggests that writing centre tutors and composition teachers should consider giving direct feedback.
(2) Tutorial engagement does not necessarily indicate attentive revisions. From the transcript alone, these tutorials reflect engagement. David's session reflects the experience of a first-year student who comes to the writing centre with an unfinished idea that the tutor helps him "flesh out" and organize. Tom's session reflects the experience of a highly-motivated non-native English speaking student who comes to the writing centre ready to discuss a draft reflecting a clear sense of thesis and organization, but who seeks help with LOCs such as citations and punctuation. Alan's session reflects the experience of a highly motivated student who comes to the writing centre with a completed rough draft and who needs feedback concerning organization and coherence.
(3) Critical reflection on the transcripts and drafts present a kind of good news/bad news scenario. The good news is that students generally attended to tutor feedback. The bad news is that they did not always attend to it effectively due to overgeneralizing or misunderstanding the feedback. In any case, engagement in a writing centre tutorial may not be evident simply from the length and number of turns occurring between the tutor and student. In other words, the transcript of the tutorial might (and did) reveal single, one-syllable responses from some students, but this did not correlate to lack of revision or attention to the tutor feedback. In fact, engagement with the writing process may be a more accurate measure of engagement overall. Although all three students seemed invested in their topics, Alan and Tom demonstrated more commitment to the writing process by bringing a complete, fully formed rough draft (Alan) or scheduling additional, voluntary conferences (Tom). David did neither. He was committed to his argument and eagerly discussed it with the tutor, but he did not take extra steps to adequately revise and edit his final paper.

Conclusion
Tutors may feel like a session is successful when the student is engaged in the discussion of the paper and that this engagement has led the student on the path toward being a "better writer." Students, however, may feel like a session is successful when their final paper receives the grade they feel it deserves, specifically when they believe that the writing centre visit positively affected the grade. In this study, grades were often negatively affected by changes the student made independent of the writing centre feedback or by simply failing to meet the assignment guidelines.
For example, Tom added a cover page from a template that did not follow any style guideline rather than just following MLA or APA as instructed. He was also missing the first page of his final draft. David copied and pasted the introduction from his annotated bibliography to use as the required abstract, which did not fit well. As already mentioned, David's final draft was two pages shorter than required, and he was missing the required number of sources. Then there were the multiple sentence-level and citation errors. These examples in the final drafts were influenced by a lack of attention to LOCs and HOCs not fully addressed in the tutorials. David and Tom's papers would have benefitted from careful editing, which could be classified as a "higher order skill" and one which is not fully addressed in "typical" composition classrooms and certainly not in "standard" writing centre tutorials.
This study was restricted by certain limitations. First, it represents a small sample size as is often the case with qualitative studies. Furthermore, as a summer course, the students and researchers left campus as soon as the course ended. This prevented the researchers from scheduling follow-up interviews which might have provided details about choices students made (or neglected to make) in their revisions. Also, the intensive nature of the course-one class day equaled one week of a normal 15-week semester-meant there was little time for revision between drafts. Finally, because the writing centre was closed in the summer, the tutor Canadian Journal for Studies in Discourse and Writing/Rédactologie Volume 28, 2018 http://journals.sfu.ca/cjsdw volunteered her time and had to fit in the appointments around her work schedule which necessitated limiting the tutorials to thirty minutes per student, which was not a large amount of time given the depth of the assignment. Nevertheless, as a collaborative study, it offers insights into tutorial effects and assignment grades.
A fairly recent discussion in WCENTRE Digest illustrates the interest and complexity of investigating the relationship between writing centre visits and grades. Our study underscores the need for caution in making this connection. As mentioned earlier, had the tutorials been judged by the transcripts alone, they would have reflected "engaging" tutorials focused on HOCs over LOCs. By that standard, some might have deemed them "successful" tutorials. Nevertheless, the engaging tutorial did not present a formula for an A paper, or even in David's case, a passing paper.
In conclusion, there remains relatively little research on the role that writing centre tutorials play in student revisions with emphasis on engagement, directive feedback, and consequential revision after writing centre tutorials. Despite the challenges of examining tutorial effectiveness, writing centre researchers should continue to find ways to investigate what happens following the tutorial. In a 2015 WLN article, Bromley et al. (2015) closed with a comment regarding "the importance of robust writing centre-based research to elucidate the complexities of student learning" (p. 6). We offer this study as a step in that direction.

Notes
1."Attend to" is operationalized here as to show understanding and an effort to revise with the information in mind.
2. Tutors in this Writing Centre had been trained not to write on student papers but to use the feedback sheet instead. The student writer received this written feedback at the end of the session.