9.4 Using Functional Grammar Analysis to Evaluate ...
Not only can functional grammar analysis be used in the evaluation of factual genres such as report, it can also be applied in the evaluation of other genres. This section illustrates how functional grammar analysis can be used to evaluate the quality of reading responses, a type of personal writing highly prized in schooling. In English literature classes, for example, one of the key assignments is to have students read and respond to literary works of interest and significance. In high-stakes examinations, students are likewise routinely asked to read and respond to an unseen narrative text. In other words, the capacity to respond to and express opinions about other texts has "a privileged status" in literary studies (Christie & Derewianka, 2008, p. 58). Not only is response writing central to the English literature curriculum, it is also critical to academic success in general. As Graham and Hebert (2010) reported in their review of writing research, instructional practices that involve writing responses to a text (literary or otherwise) "yielded a relatively large average weighted effect size" in improving students' overall literacy and content learning (p. 28).
Despite the importance of reading response in school learning, there are at present few effective methods for evaluating students' written responses. Most teachers rely on generic rubrics, such as the one presented in Table 9-4, to evaluate students' reading responses. One issue with the rubric is that it lacks specificity. It does not describe the linguistic resources that students can use to make their responses "informative", "original", or "superior". Nor does it elaborate on what "high level of understanding" or "carefully supported" means. More importantly, the rubric does not seem to be grounded in any sort of theory, failing to enable a systematic view on students' response strategies with respect to such important questions as those raised by Macken-Horarik (2006a, p. 10): (a) how does the student writer construe experience in the stimulus text (e.g., story)? (b) by what logic does the student writer generate a response? (c) how does the student writer evaluate the stimulus text? and (d) how does the student writer engage with the stimulus text? To answer these questions, attention to the language choices used in constructing reading responses is crucial. That is, there needs to be a clear articulation of the specific linguistic requirements for constructing successful or valued responses.
Table 9-4 Sample Rubric for Evaluating Reading Responses
Reading response is, in essence, an interpretive task involving analysis and evaluation of a stimulus text, most frequently, a literary text such as a short story or a novel. It requires students to read a text and then construct a response appraising and interpreting the text. Christie and Derewianka (2008) developed a topology of reading responses, presented in Figure 9-3, that captures the developmental progression of students' capacities to respond to and evaluate texts. At the lower end of the developmental continuum is personal response, which expresses immediate feelings about or gives simple attitudinal response to a story. Further along in the continuum are review and character analysis. Review involves retelling of the events in the text and brief evaluation of its qualities, whereas character analysis involves some discussion and evaluation of story characters. At the high end of the continuum is thematic interpretation, which involves generalizing from the specific characters and concrete events of the story and constructing reflections and/or abstract statements about these characters and events and the motifs of the story.
Least Successful ———————————————> Most Successful personal response —> review —> character analysis —> thematic interpretation immediate response ———> generalization ———> abstraction
Figure 9-3 Continuum of Reading Responses from Least to Most Successful
This response continuum suggests that more successful responses demand progressively more sophisticated literacy skills, moving beyond immediate personal reaction to generalizations and abstraction. Recent studies of student responses have shown that successful reading responses, such as thematic interpretations, make "connections between concrete (often recurring) motifs in a text and the abstract thematic preoccupations of its author" (MackenHorarik, 2006b, p. 103) and "organize and marshal information about the texts to be responded to, in such a way that a clear focus is offered and pursued in an orderly manner" (Christie & Dreyfus, 2007, p. 246). The grammatical resources for realizing these meanings include clauses with relational processes (i.e., linking verbs), connectives for elaboration, evaluative statements, and semiotic abstractions (Macken-Horarik, 2006a-c). Without a clear understanding of these linguistic resources and their functions in response writing, many students are denied access to the secrets for writing successful reading responses, often viewing reading response as an invitation for open responses and generating personal responses that are less likely to be valued by teachers and examiners.
A special kind of academic genre, reading response should be evaluated using criteria different from those for evaluating other kinds of writing. A key focus of the evaluation should be on seeing how students interpret the stimulus text and then organize their interpretations in ways that facilitate the presentation of information and development of argument. The rubric that is currently used in many English literature classes gives teachers few insights into what it is that makes a response more or less successful/valued and is tangentially useful in informing instruction. In recognition of the situation, a new tool for evaluating students' reading responses is proposed in Table 9-5. The tool draws on the functional grammar analysis framework presented earlier (Chapter 3), but also considers Macken-Horarik's (2006a-c) work exploring the linguistic features of successful and unsuccessful reading responses in high-stakes English examinations in Australia and the topology of reading responses charted by Christie and Derewianka (2008). It enables teachers to render judgment on the quality of reading response based on an analysis of the language patterns in student work. Specifically, it allows teachers to evaluate what students have to say about the content of the story, the logic they use in constructing the response, the stance or perspective they adopt in evaluating the story, and the ways they engage with the story structure.
Table 9-5 A Framework for Evaluating Reading Response
Using the tools presented in Table 9-5, let's evaluate the quality of one sample reading response (Text 9-3), written by a college student in response to The Things They Carried, a collection of linked short stories by a noted American novelist Tim O'Brien (1990). We can focus on the four broad questions (i.e., how does the student writer construe experience in the story? by what logic does the student writer generate a response? how does the student writer evaluate the story? and how does the student writer engage with the story?) and the associated linguistic resources that are central to constructing the kind of reading responses valued by school. According to Macken-Horarik (2006b), successful reading responses are characterized by symbolic abstraction, which is a kind of reading that connects the story's concrete motifs and its abstract themes. In crafting successful responses, students need to draw on a constellation of grammatical resources to actualize symbolic abstraction. For example, to construe experience in the story (i.e., encode experiential meaning), students need to use relational clauses to ascribe abstract meaning (or significance) to events, identify one thing with another, relate one event (or character) to another, or describe and classify the participants and processes of narrative experience. To link messages in the response (i.e., encode logical meaning), students need to use logical connectives such as in other words, for example, namely, and such as to expand on clause message by further specifying it, describing it, restating it, clarifying it, and refining it. To appraise the story (i.e., encode interpersonal meaning), students need to provide evaluation of the overall significance of the story as a whole, instead of merely offering subjective evaluation of or simply projecting thoughts and feelings about the story characters. Finally, to engage with the story (i.e., encode textual meaning), students need to demonstrate a global orientation to the text, moving from the material phenomena (i.e., story or its characters) to abstraction (i.e., psycho-symbolic significance). Analysis of these lexicogrammatical patterns can reveal patterns of meaning configured in the response.
Text 9-3
"The Things They Carried", by Tim O'Brien at first seemed to be just another war story. As I started reading I thought I was not going to have any interest at all in the story; however after I got into the story I found myself more interested than I thought I was going to be. This story is an excellent depiction of war itself. It is very realistic and easy to relate to even without any personal experience with war. The title itself paves the way for the entire story, it could not be any better for the story then it is. The entire story is about "the things they carried", and the author does choose to go into great detail about the things that they did carry.
While I was reading the story one question that kept running through my mind was in regard to the depth of information about the physical weights of the soldiers' items. Why would O'Brien go into this much depth about how much every individual item weighs? One thought on this that came to mind was that perhaps it is not as much about the physical weight as much as it is about the overall psychological experience. War is very stressful and can very easily become more then one can handle. I think that Tim O'Brien does a very good job at throwing in all of that information about how much this weighs and who has to carry that to almost overwhelm the reader. When he "overwhelms" the reader he makes it much easier for the reader to see the story from the frame of mind that a soldier would see it. The author also breaks in to the detailed list disrupting the flow of the items by informing the reader of the deaths of soldiers. He talks about them in giving us personal information about the soldiers and then all of the sudden another one is dead. This also helps the reader get a clear representation of war because in war it is very dangerous and you never know what is going to come your way.
Another question did catch my attention. Are these items really all that the soldiers carry or is there more? While I was reading I noticed that the author did leave hints in the story about the more deep variety of the things that they carried. He used phrases such as "They shared the weight of memory. They took up what others could no longer bear", "They were afraid of dying but they were even more afraid to show it", and "They carried all the emotional baggage of men who might die". This suggests that the actual items that the soldiers carry are essentially unimportant, even though they take up a large portion of the story. What is important however is what these soldiers are actually carrying with them. What the soldiers are carrying around with them is the devastating psychological affects of war. I think one good question for the class is do you think that Tim O'Brien effectively portrays the psychological affects of war? Also I want to add in one quote that I think is very important to think about to understand the story. "Men killed, and died, because they were embarrassed not to. It was what had brought them to the war in the first place, nothing positive, no dreams of glory or honor, just to avoid the blush of dishonor."
(Source: http://pirate.shu.edu/~jonesedm/ReadingResponseExample.htm,accessed on June 27, 2019 and reproduced without any correction to grammar, punctuation or spelling)
9.4.1 What Is the Story About for the Student Writer?
To understand what experiential reading the student writer made of the novel, we can examine the process types used in the response. It is clear that the relational process (is, was, have, become, suggests) figures prominently in the sample response; it is used in over one-third of the clauses, mainly to evaluate the story (e.g., This story is an excellent depiction of war itself. The entire story is about "the things they carried" …), characterize its central topic or motif (e.g., War is very stressful and can very easily become more then [than] one can handle. … because in war it is very dangerous. This suggests that the actual items that the soldiers carry are essentially unimportant, even though they t ake up a large portion of the story.), and identify issues for reflection or discussion (e.g., …one question that kept running through my mind was in regard to the depth of information about the physical weights of the soldiers' items. One thought on this that came to mind was that perhaps it is not as much as the physical weight as much as it is about the overall psychological experience. What is important however is what these soldiers are actually carrying with them. What the soldiers are carrying around with them is the devastating psychological affects [effects] of war.).
Other processes are used much less frequently. The material process, for example, is used mainly to describe what the novelist did in crafting the story (e.g., Why would O'Brien go into this much depth about how much every individual item weighs?; When he "overwhelms" the reader he makes it much easier for the reader to see …; The author also breaks in to the detailed list …). The mental process is used to describe the student writer's cognitive act of reading (As I started reading …; While I was reading the story …), as well as her thinking and feeling about the story (e.g., I thought …, I found …, I noticed …, I think …). Taken together, this student avoids providing too much plot details, focusing instead on expressing personal feelings but at the same time making generalizations about the story and reflecting on the themes and values found in the story.
The sample response contrasts with low quality responses, where students tend to focus heavily on what happened in the story; that is, they retell selected parts of the story by describing main characters' actions, thoughts, feelings, sayings, identities, and attributes, as well as their own emotional reactions to the characters and events. Text 9-4 and Text 9-5 (reproduced below), written by two U.S. middle school students (see Fang et al., 2014, pp. 539-540), are such examples. Text 9-4 is a reading response to William Pene du Bois' (1947) The Twenty-One Balloons, a book about the hot-air balloon trip around the world taken by a 66-year-old retired arithmetic teacher, William Waterman Sherman, who wanted to get away from everyone. The reading response is essentially a retelling of concrete events in the story (material processes underlined), with little evaluation or interpretation of the story's symbolic significance. Text 9-5 is a reading response to Adrian Fogelin's (2002) Crossing Jordan, a tale about a young white girl (Cass) who overcomes family prejudice and cultural differences to befriend a black girl (Jemmi) in a small working-class town. It focuses on what the story characters thought and felt. The mental and verbal processes (underlined) are juxtaposed with the material process in retelling parts of the story. The response also uses a few relational processes, primarily to identify story characters or events and to attribute qualities to them (e.g., One of Cass' positive actions is showing kindness towards the new neighbors, especially Jemmi.) but only occasionally to interpret the abstract significance of the story symbolically (e.g., This was a heart-warming novel about two friends of different races coming together to be friends …).
Text 9-4(https://www.daowen.com)
In The Twenty-One Balloons by William Pene du Bois, the main character, Professor Sherman, ends up shipwrecked with 21 balloons three weeks after departure. When he said he would be spending a year on the Pacific Ocean, he was found by a ship and traveled to San Francisco by train. He lived there and he was part of an explorer's club. At a parade in his honor, he accepts a little girl's bouquet of balloons, showing a positive behavior. However, when he was resting up at a hotel, the mayor gave him the key to the city in exchange for the details of his story.
Text 9-5
One of Cass' positive actions is showing kindness towards the new neighbors, especially Jemmi. She shows kindness by not automatically avoiding them because of their skin color. One of her negative actions is yelling back at her father. When Jemmi's mom bought Cass new shoes, her father didn't like this because he thinks that Jemmi's mom is trying to top them off and be better, but Cass sees it as no big deal and neither does Jemmi's mom. On a scale 1-10, I would give Cass an 8 because she kept believing that her parents could possibly be friends. And she stayed friends with Jemmi anyway. But sometimes, not often, she would think maybe her parents would never be able to get along. This was a heart-warming novel about two friends of different races coming together to be friends and they believe that one day they won't have to sneak behind their parents' back just to hang out. This story reminds me of the time when my sister had to stand up for my handicapped brother because people were prejudiced against him and calling him names and making fun of him. I remember the part when they entered that race but didn't win because Jemmi fell down and Cass went back for her. They got in the newspaper as their team name "Chocolate Milk". They even forgot that they didn't win first place. But Nana Grace says that they won for crossing the Lewis-Bodine line first and that's what really mattered.
9.4.2 By What Logic Does the Student Writer Generate a Response?
To examine the logic with which the college student generated her response, we can consider how the student combines clauses and makes connections between sentences. Specifically, we can look at the logical-semantic links (e.g., extending, enhancing, elaborating) within and across sentences. A perusal of the response suggests that the content of the response is expanded primarily through the logic of elaboration, much less frequently through the logic of enhancement, and rarely through the logic of extension.
As discussed in Chapter 2, in an elaborative relation, one clause or sentence elaborates on the meaning of another by further specifying it, describing it, restating it, clarifying it, explaining it, and refining it. In the first paragraph of Text 9-3, the statement, This story is an excellent depiction of war itself, is explained in the next three sentences (It is very realistic and easy to relate to … about the things that they did carry.). The second paragraph starts with a question (Why would O'Brien go into this much depth about how much every individual item weighs?), which is then answered in the rest of the paragraph. Similarly, the third paragraph begins with a question (Are these items really all that the soldiers carry or is there more?) that is subsequently addressed (While I was reading I noticed … the devastating psychological affects [effects] of war.). Later in the paragraph, the author wrote Also I want to add in one quote…, and the next two sentences then supply the quote, although there is no elaboration about how the quote connects with the point made in the previous sentence (i.e., how the quote is important to the understanding of the story).
In many sentences, embedded and comparative clauses are used to elaborate on what is being discussed (e.g., I found myself more interested than I thought I was going to be; one question that kept running through my mind; it could not be any better for the story then [than] it is; One thought on this that came to mind was that perhaps it is not as much about the physical weight as much as it is about the overall psychological experience; the things that they carried; the actual items that the soldiers carry). Within some clause complexes, one clause often elaborates on the other, as in The title itself paves the way for the entire story, it could not be any better for the story then [than] it is and The entire story is about "the things they carried", and the author does choose to go into great detail about the things that they did carry, where the second clause refines or specifies the message in the first (or main) clause.
A handful of clause complexes use the extending logic. This is mostly done through projection, i.e., verbs of mental process + relative pronoun that, as in I think that Tim O'Brien does a very good job at … and I noticed that …. Other extending logic relationships are realized explicitly through additive conjunctions (e.g., and, and then), as in War is very stressful and can very easily become more then [than] one can handle, … in war it is very dangerous and you never know what is going to come your way, and He talks about them in giving us personal information about the soldiers and then all of the sudden another one is dead.
In a few clause complexes, the enhancing logic is used. In this logicalsemantic relationship, one clause expands another by qualifying it with some circumstantial features. In Text 9-3, the qualification comes mostly with the circumstantial feature of time (as, after, when, while) and much less frequently with cause (because) or contrast/counter-expectancy (however) (e.g., As I started reading, I thought I was not going to have any interest at all in the story; however after I got into the story I found myself more interested than I thought I was going to be; When he "overwhelms" the reader he makes it much easier for the reader to see the story …; This also helps the reader get a clear representation of war because in war it is very dangerous …).
To summarize, Text 9-3 relies heavily on the elaborating logic by stating a point, a message, or a question and then further explaining, characterizing, or addressing it. However, this logic is often not transparent because no explicit signal phrases such as "for example" and "in other words" are used. Nor are all statements well supported through elaboration or clarification. To a less extent, the sample response also uses extending and enhancing logic to connect its massages, which helps aggregate details about story events and accumulate reactions to the story characters or events. This is unlike low quality responses, such as Text 9-6 (Fang et al., 2014, p. 542), which often rely heavily on extending and enhancing logics to retell parts of the events in the original story (see underlined items) rather than on elaborating logic to explain the reader's interpretations of the story. Text 9-6 was written by a middle school student in response to Andrew Clements' (1996) Frindle, which is a story about a boy named Nick who likes to make class time exciting. It relies on the extending and enhancing logic as the dominant organizing strategy.
Text 9-6
In the book Frindle, a kid named Nick is a troublemaker. When Nick was in third grade, he started making a peep sound in school. The teacher thought [that] it was a girl named Janet Fisk. And she got yelled at by the teacher. Then she said it wasn't her. After school, Nick apologized to the girl because it was his fault. This is a positive behavior because he was asking for forgiveness because she got yelled at in class. A negative behavior is when Nick made the entire classroom into a tropical paradise. He put construction paper palm trees in the corners of each desk. He wore sun glasses and turned the thermostat up to ninety degrees and put sand all over the classroom. The janitor was not happy when he had to pick it all up. The book makes me feel good because it is funny. When I lived in Connecticut in the winter, I would have like to make the school room a tropical paradise to make it feel like it was warmer. One of the things I will remember most about this book is that Nick was a troublemaker. Mrs. Granger thought[that] the dictionary was the law. So Nick had to do a presentation about the dictionary. When Nick did the presentation, he made it so long that there was only eight minutes left of class. Janet found a golden pen and Nick by mistake knocked her off the sidewalk. He picked up the golden pen and decided to make a new word for the pen called Frindle.
9.4.3 How Does the Student Writer Evaluate the Story?
To determine how the sample response evaluates the story, we can first identify evaluative clauses in the response and then sort these clauses into three groups: those representing the student writer's emotional reactions to parts of the story (events, characters), those summarizing or projecting story characters' feelings, and those making judgment about the overall significance of the story. Text 9-3 is populated with evaluative comments. These comments have to do mostly with appreciation (i.e., the worth and quality of the story) and much less with affect (i.e., emotional reactions to the story) and judgment (moral evaluation of story characters). These evaluations are occasionally amplified (e.g., very, essentially) and often presented in no uncertain terms due to the absence of modality. Except for one comment that expresses the student writer's personal feelings about the story (i.e., I found myself more interested than I thought I was going to be.), the vast majority of the comments provide an overall evaluation of the story, including its motifs and themes. They include the following:
· This story is an excellent depiction of war itself. It is very realistic and easy to relate to even without any personal experience with war.
· … it is not as much about the physical weight as much as it is about the overall psychological experience.
· When he "overwhelms" the reader he makes it much easier for the reader to see the story …
· This also helps the reader get a clear representation of war because in war it is very dangerous and …
· This suggests that the actual items that the soldiers carry are essentially unimportant, even though they take up a large portion of the story. What is important however is what these soldiers are actually carrying with them. What the soldiers are carrying around with them is the devastating psychological affects [effects] of war.
· I think one good question for the class is do you think that Tim O'Brien effectively portrays the psychological affects [effects] of war?
· Also I want to add in one quote that I think is very important to think about to understand the story.
When comments are made about a particular part of the story or a particular craft the author employs to fashion the novel, they are generally presented as evidence to support the overall evaluation of the story and its themes (e.g., The title itself paves the way for the entire story, it could not be any better for the story then [than] it is. Why would O'Brien go into this much depth about how every individual item weighs? I think that Tim O'Brien does a very good job at throwing …).
Text 9-3 includes few personal reactions to story characters or specific events in the story. Nor does it attempt to summarize or project the story characters' thoughts and feelings. This is a noteworthy accomplishment. According to Christie and Derewianka (2008), the capacity to express personal feelings about the story and make generalizations about its events and characters is an essential aspect of the interpretive tasks involved in writing reading responses; however, to compose high quality responses, students need to move beyond this to offer an overall evaluation of the story and its themes. Text 9-3 goes beyond simple commentary on the story characters or parts of the story to present an appraisal of the story as whole, including its thematic significance and the values and attitudes embodied in the entire story.
9.4.4 How Does the Student Writer Engage with the Story?
Finally, we examine how Text 9-3 engages with the story by analyzing the Theme and Rheme patterns in each clause of the response. Specifically, we categorize Themes into two types: (a) experiential Themes that refer to either the story characters or the student writer, and (b) semiotic Themes that refer to the story itself, parts of the story, or the story writer (which can be interpreted as referencing the stimulus text). While both types of Themes are present in the sample response, semiotic Themes make up roughly two-thirds of the Themes used, including "The Things They Carried" by Tim O'Brien, this story, it, the title, the entire story, the author, one question that kept running through my mind, Tim O'Brien, one thought on this, he, the author, this, another question, war, and what the soldiers are carrying around with them. Experiential Themes, which constitute about one-third of the total Themes used in the sample response, are mostly about the student writer (I), with one exception (you), which refers to the reader in general. This is unlike in Text 9-6, where most clauses thematize the story characters or the student writer. This disparity in Theme choices reflects a difference in the orientation, or angle, of literary interpretation and results in responses that feature different levels of engagement with the story, with Text 9-3 oriented toward thematic interpretation, whereas Text 9-6 oriented toward a summary of and personal reactions to story plots.
It is clear that Text 9-3 engages deeply with the symbolic features of the story (e.g., war and its effects on soldiers). However, the juxtaposition of different experiential Themes and semiotic Themes — the student writer (I), the generic (you), the author of the novel (Tim O'Brien), the story itself, parts of the story, and the themes of the story — makes the response sound somewhat rhetorically problematic. In other words, this constant shifting of Themes can distract from the focus of the response and contribute to a lack of discursive flow. This lack of a stable focus in the interpretive task may be indicative of insufficient maturity in managing engagement with the story at both global and local levels.
The Rhemes in the clauses of Text 9-3 contain abstractions that suggest the student writer engages with the psycho-symbolic significance of the story. These abstractions include an excellent depiction of war itself, the depth of information about the physical weights of the soldiers' items, … the psychological weight … the overall psychological experience, the deaths of soldiers, a clear representation of war, the more deep variety of the things that they carried, and the devastating psychological affects [effects] of war.
Taken together, findings from the Theme/Rheme analysis of the responses indicate that the sample response (Text 9-3) engages with the story both globally and locally, but struggles to maintain a consistent interpretive focus. There is also ample evidence that the response engages deeply and for the most part successfully with the symbolic features of the story.
9.4.5 Summary
The English education curriculum attaches great importance to developing students' capacity to respond to and evaluate texts; it has as one of its primary goals the development of students' ability to talk, write, and make judgment about literary texts, as well as to reflect upon the values, themes, and motifs of these texts (Christie & Derewianka, 2008). This means that reading response is a highly prized academic genre for not only subject English but other disciplines as well. The reality of assessment is that not all responses are equally valued in school. Given that language is the major resource for making meaning, teachers have a responsibility to make their expectations for language use explicit in their teaching and assessment practices so that students from all backgrounds are equipped to fashion their reading responses in ways that meet teachers' expectations.
The SFL-informed assessment framework used in evaluating Text 9-3 goes beyond the traditional rubrics for writing assessment to include a four-part focus that is relevant and specific to the reading response genre. By analyzing how the student writer construes experience in the story, what logic she uses to construct her responses, how she evaluates the story, and how she engages with the story, we gain insights into the quality of her reading response. Overall, the sample response (Text 9-3) goes beyond presenting personal response (i.e., immediate feelings about the story), conducting review (i.e., retelling of events in the story and statements about its qualities), or doing character analysis (i.e., discussion and evaluation of story characters) to engage in thematic interpretation, which involves generalizing from the concrete events of the story and constructing reflections and/or abstract statements about the story's experience and motifs. While there are issues with focus, word choice, punctuation, style, and management of discursive flow, the sample response as a whole includes many linguistic devices that characterize quality responses, including use of relational processes of a synoptic kind, elaborating logic, evaluative comments on the motif of the story, and semiotic Themes referring to the story or its abstract significance.