170 Beyond “Good Job!”: How Online Peer Review Platforms Improved My Students’ Writing and Made My Life Easier

by Liza Long

“Good job!” “Nice work!” “I think you need a comma here.”

Most instructors are familiar with these types of student comments during peer review. Nearly every composition instructor uses peer review in the classroom because we know that this activity is important. Brammer and Rees note that all experienced instructors recommend peer review to new instructors, yet both students and instructors are often unhappy with both the peer review process and the results. However, composition scholars have reached consensus that “the composing process is social, and peer review is an integral part of that process” (Brammer and Reese, 72). In an online environment, promoting student engagement can be especially challenging.

What does peer review look like in your classroom? In my face to face classes, I used to put students in groups and ask them to print out multiple copies of their drafts, then use a rubric that I created to have them check their peers’ work, with a vague student-directed “workshop” style discussion after the rubrics were complete. In an online class, I used a peer review discussion board, asking students to post their drafts and complete those same instructor-created peer review templates on each other’s work. But what were my students really learning through these peer review activities? Did peer review improve their writing? How could I make peer review a transferable skill?

One challenge I face with constructing meaningful peer review is that my students come from diverse backgrounds and reach my course with dramatically different previous academic experiences. I teach at the College of Western Idaho, the largest college in our state, with more than 24,000 students. Our two-year commuter college provides education to a wide variety of learners, from traditional students seeking a lower-cost option for their general education coursework to career and technical education students who are required to complete a written communication requirement. My English 102 Rhetoric and Composition classes, capped at 26 students, typically include a diverse mix from a variety of majors: traditional learners, adult learners, English language learners, and even dual credit high school students.

In  2017, on the recommendation of my colleague Meagan Newberry, I took a risk and tried out my first online peer review platform, Eli Review. I quickly learned that online platforms play an important role in facilitating more meaningful peer review experiences while also saving time for instructors. We are now feeling a new sense of urgency about online tools because the sudden shift to online learning precipitated by the COVID-19 pandemic during the Spring 2020 term made many of us rethink our traditional approaches to instruction. Online peer review is one tool that all instructors should consider. In a study of online peer review, Hicks et al. noted online platforms can provide engaging and enriching peer review experiences, but that the way instructors frame online peer review assignments is critical to the quality of feedback that student peers provide, and that “chunking” writing assignments leads to higher-quality feedback. In this essay, I’ll share my experience with two online peer review tools and explain how the iterative and constructive process of online peer review has transformed my students’ writing while also promoting critical thinking skills that prepare them for the workplace—and saving me time in the process.

Two Peer Review Platforms: Eli Review and Peergrade

After a colleague’s enthusiastic recommendation as noted above, I began using Eli Review, an online peer review platform, in 2017 for both my online and hybrid classes. I later adopted it for face-to face classes as well, dedicating one classroom day each week to the peer review drafting and revising process. In 2019, I piloted a peer review platform called Peergrade, which is the platform I currently use. I currently use Peergrade in my English 101 Writing and Rhetoric I, English 102 Writing and Rhetoric II, and English 211 Literary Analysis (advanced writing requirement) course s. I use the peer review tool differently in each course. For English 101, where we are focusing on using sources to support arguments in academic essays, I use Peergrade as an iterative writing and feedback tool to help students learn to craft paragraphs and draft thesis statements. In English 211, I use it for a more traditional peer review, asking students to assess each other’s literary analysis essay drafts using the grading rubric. Peergrade is a major component of my English 102 course design, so I will focus on that course as I share what has worked for me.

Both Eli Review and Peergrade function in similar ways. Eli Review is available as a low-cost subscription to students and offers a stand-alone website. Students purchase the subscription at the campus bookstore or online. According to Eli Review’s website, a three month subscription to the service is $12.50 per student. I switched to Peergrade because we have an institutional license and it integrates with our LMS (Blackboard), which means that students can access the platform directly from our Blackboard class. At the College of Western Idaho, we do not currently pass the Peergrade license cost on to our students. For Peergrade, students are automatically enrolled. With Eli Review, they have to enroll in a course I create using their school email accounts. Both tools allow instructors to collect data on student engagement and performance. While I personally think that Eli Review is a slightly superior tool with a better graphical interface, data analytics, and user experience, the LMS integration and the lower cost for students made it easy to switch to Peergrade. Both platforms offer excellent client support and are responsive to instructor feedback.

Introducing the Online Peer Review Platform and Process

I introduce the online peer review platform on the first day of class. My students’ first writing task for the platform is also my first week writing assessment:

In 2-3 paragraphs, describe how much experience you have with peer review and academic writing, how you worked with peers (in pairs, groups, etc.), and how you interacted (reading essays out loud, group conversations, etc.). Describe in detail the kinds of feedback you received from your reviewers, but also the kind of feedback you gave to the people you reviewed. Did you find that the feedback you received was useful? If not, what would make feedback more useful to you as a writer?

When I am teaching hybrid or face to face classes, we respond to this prompt in class together using our college’s laptop cart, giving students the opportunity to try the online peer review platform with my support. For online classes, I provide and record a brief tutorial throughout my LMS to walk students through the submission process. This prompt gives me the opportunity to create a baseline of peer review experience and attitudes for the class. Hicks et al. note that this kind of background research into students’ experiences with and cultural assumptions about peer review is an important first step in framing high-quality peer review tasks.

In our next class session (or the next week online), I introduce the peer review process and explain how it will work in our course. Teaching students what peer review is and how they are expected to do it is an evidence based practice. For example, Brammer and Reese note that students who receive instruction in how to peer review feel more confident in their abilities and have more positive experiences with the peer review process. After discussing my students’ writing submissions and addressing any concerns they have about peer review, I ask them to share examples of where they encounter reviews in their daily lives. In fact, reviews and surveys are nearly ubiquitous. We can’t even purchase a new pen on Amazon without being asked to review our shopping experience. Students quickly generate workplace examples of peer review: for example, their work is reviewed by supervisors, or if they themselves are supervisors, they may have to review others. Through this initial discussion, students establish the “why” of peer review. They also start thinking about the “what.” Are we just looking for misplaced commas? Or are we ensuring that the meaning of the essay is clear to the reader? As instructors who have created peer review rubrics know, establishing criteria for a review is not an easy task.

Using the Peer Review Platform for Writing and Revising

Each week, my students have an assigned writing task. They complete the writing task by Sunday for face to face and hybrid courses, then complete their assigned peer reviews before our next on-ground class. In online classes, I set a midweek deadline for writing tasks and a Sunday deadline for review tasks. Both Eli Review and Peergrade allow the instructor to set parameters for reviews, including the number of reviews required, whether or not the reviews will be anonymous, and whether late work is allowed. In my classes, we decide together whether to use anonymous feedback. In peer review, students are asked to identify required traits in their peers’ essays, then evaluate the essays on defined criteria. I also ask students to provide a final comment using the “Describe-Evaluate-Suggest” model of peer feedback as explained by Eli Review here. While I do not comment on every assignment, I use some of these formative assignments to ensure that the students are on task and understand the goal of the essay we are working on. For an example of my exploratory research weekly tasks and rubrics, including a student-created essay draft rubric, see Appendix I.

Learning to rate each other’s feedback has required norming exercises. Baker has noted that one area of concern for faculty is whether students are capable of giving high quality feedback. I ask students to rate each peer’s final comment on the five-star helpfulness scale, based on how well it meets Eli Review’s “Describe/Evaluate/Suggest” model for feedback.

• Five stars = Will transform the writer’s draft: all three parts are included, are specific, and the reviewer went above and beyond.
• Three stars = Solid, helpful, specific, and includes all three pieces.
• One star = May not include any aspects of the model or be too vague to be helpful.

Initially, peer feedback scores tend to skew high. We look at anonymized examples to discuss the kinds of comments that are truly helpful. I also spotlight “feedback stars” with examples of high-quality feedback at the beginning of class when we debrief about the week’s writing and review assignments.

The bulk of my students’ course grade now comes from peer review. Rather than grading individual essay drafts, I assign 100 points to each essay’s peer review tasks. Part of the grade depends on how students rate their peers’ submissions, and another part of the grade depends on how their peers rate the quality of their feedback. Both Eli Review and Peergrade allow students to numerically assess each other’s work and each other’s feedback. As noted above, I expect that students will provide feedback that is helpful at a “three stars” level (out of five) to earn full credit for their reviews. Baker’s study of a four-week peer review process supports this approach for high-quality feedback. See Appendix 2 for a rubric and examples of student feedback.

I do not provide an evaluative grade (other than complete/incomplete) for any of my students’ essays until they submit their final portfolios at the end of the semester. One of my colleagues, Meagan Newberry, uses a similar peer review approach with a grading contract proposed by Asao Inoue in his book Antiracist Writing Assessment Ecologies. In Newberry’s class, students who complete their writing and review tasks at the expected “three star” level earn a B in the course; for an A, students must complete an additional assignment.

How Students Responded to Online Platforms

I have seen improvement in student engagement and satisfaction as I have become more familiar with these online tools. In Fall 2016, the semester before I started using online peer review platforms, only 50% (12/24) students completed the end of course survey, with 100% expressing that they were “satisfied” (17%) or “very satisfied” (83%) with the course. Their comments tended to focus more on me as an instructor and less on their own writing process. In the first semester in which I taught the course using an online peer review platform, my student evaluations were less positive: I had a 44% response rate for my end of course evaluations, and 90% of the students who completed the end-of-course survey expressed that they were “satisfied”(10%) or “very satisfied ”(81%) with the course, and 9% rating their experience as “neutral.”

In Spring 2018, after I had used online platforms for one year and refined the peer review feedback process, 60% (15/25) of my students completed the end of course survey, with 100% reporting that they were either “satisfied” (27%) or “very satisfied” (73%). Seven commenters specifically mentioned Eli Review as a reason that they enjoyed the course. In 2019, after I switched to the LMS-integrated PeerGrade platform, 67% (14/21) students completed their end of course survey, again with 100% reporting that they were “satisfied” (35%) or “very satisfied” (65%) with the course. While none of the students mentioned Peergrade in their comments, many of them commented on how their writing had improved. I think there’s a possibility that students prefer the user experience of Eli Review to Peergrade, which may account for the lower numbers of “very satisfied” students as well as the fact that the online platform was not mentioned in student comments.

Spring 2020 was a special case for all of us, and I did not have high expectations for my course evaluations. While fewer students completed their end-of-course surveys, all three of my sections finished the year strong despite the challenges of COVID-19 and moving online. In one section, 13/22 students (59%) completed the end of course survey, with 100% reporting that they were very satisfied (53%) or satisfied (46%) with the course. Honestly, given the fact that none of these students had signed up for what became an online course, I am proud of this assessment. In their final comments, students noted that they had learned the revision process and that they were proud of their final portfolios and the work they had accomplished. Not a single student complained about the move from hybrid to online in their comments.

What I Have Learned from My Students’ Online Peer Review Experiences

While I was initially hesitant to try this new tool because I was concerned about the time involved in learning a new system, using online peer review platforms has transformed the way I teach my English 102 courses. These platforms provide structure and organization to support student writing, and they foster a collaborative classroom experience in hybrid and online courses. Here are some of my top takeaways after three years of using this type of peer review:

1. Using an online platform for peer review can provide engagement and accountability, allowing for more constructive feedback from classmates. When students write for an audience of their peers, they try harder. My students consistently report high levels of engagement with the course and each other. As Baker notes, “Research consistently demonstrates that engaging students in the feedback process improves the quality of students’ final submissions” (180).
2. For hybrid or online classes, these platforms are especially valuable. For example, during the quick switch from hybrid to fully online courses precipitated by COVID-19 in the Spring 2020 term, our peer review platform provided continuity and a simple way for me to check that students were completing their regular assignments. Students told me that my courses felt “normal” to them, which was reassuring in a time when so many other areas of their lives were changing.
3. The peer review process supports student writing because it reinforces the iterative nature of the writing process. Students break essay tasks into chunks, then incorporate these chunks into the larger essay project (Baker, Hicks et al.).
4. The online peer feedback platform provides an easy way for me to give early feedback on formative assessments so that I can intervene and make writing center referrals for students who were struggling with course concepts like thesis statements, source use, or paragraph unity.
5. Students learn to create their own assessment rubrics, enabling them to think more critically about their rhetorical process. This metacognition helps to reinforce the rhetorical situation for each essay that I teach (see Appendix I: Week Four Peer Review, Student Created Rubric).

Ultimately  , I have concluded based on my experiences that using the online platform’s writing and review process has led to stronger student essays with less time spent on my part Because students know that a large portion of their grade depends on their revisions, they feel free to take more creative risks with their work. They choose more difficult topics for their exploratory and persuasive essays, and they learn to embrace the idea that writing an academic essay is a process. Students have reported high levels of satisfaction with both Eli Review and Peergrade in every course where I have used these online platforms. Most importantly, these tools have saved me time as an instructor while also increasing students’ perception of my engagement. Overall, I give online peer review platforms five out of five stars.

Works Cited

Baker, Kimberly M. “Peer Review as a Strategy for Improving Students’ Writing Process.” Active Learning in Higher Education 17.3 (2016): 179-192. https://www.hendrix.edu/uploadedFiles/Academics/Faculty_Resources/Faculty_Development_Newsletters/Peer-review-student-writing.pdf

Brammer, Charlotte, and Mary Rees. “Peer Review from the Students’ Perspective: Invaluable or Invalid?.” Composition Studies 35.2 (2007): 71-85. https://www.uc.edu/content/dam/uc/journals/composition-studies/docs/backissues/35-2/Brammer%20and%20Rees%2035.2.pdf

Eli Review. “The Describe-Evaluate-Suggest Model of Peer Feedback.” (2016) https://elireview.com/2016/08/03/describe-evaluate-suggest/

Hicks, Catherine M., et al. “Framing Feedback: Choosing Review Environment Features that Support High Quality Peer Assessment.” Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2016. https://dl.acm.org/doi/pdf/10.1145/2858036.2858195

Inoue, Asao B. Antiracist Writing Assessment Ecologies: Teaching and Assessing Writing for a Socially Just Future. Parlor Press (2015). https://wac.colostate.edu/books/perspectives/inoue/

Peergrade. “Getting Started.” https://www.peergrade.io/getting-started/

Appendix I: Exploratory Research Essay Writing Tasks and Peer Review Rubrics

Week One Writing Task

Post your narrowed and focused research question for Essay One (Exploratory Research) here. Then write 2-3 paragraphs explaining why you chose this topic and what you expect to find in your research. Your instructor will provide you with feedback on this assignment.

Week One Peer Review Rubric

Trait Identification

• The research question is phrased as a single question. (Y/N)
• The research question is focused (not too broad, not too narrow) *Y/N)
• The research question is objective (does not take sides). Y/N
• The research question is appropriate for a 6-8 page college essay. Y/N Please comment on your response.

Evaluation

I would like to read a paper that answers this research question.

• Meh
• Sure
• Wow, this paper will change my life!

Please comment on any part of the research question that you think could be improved using the Describe-Evaluate-Suggest model. For example, is the question objective, or does it contain value statements (words that have positive or negative connotations)? Is the topic too broad or too narrow? If so, suggest a way to narrow or expand the topic. If you know of a good source for this question, feel free to share it! Your response must be at least 75 words in length.

Week Two Writing Task: Academic Article Summary and APA Style Reference

1. Choose one academic source you plan to use for your exploratory research essay.
2. Summarize the source in 5-6 sentences.
3. Use at least one template from Chapters 1-3 of They Say/I Say in your summary.
4. Include one direct quote from your source and cite the quote correctly in APA style.
5. Create the APA-style full reference for your source and place it at the bottom of your paragraph.
6. Check your paragraph for correct grammar, spelling, punctuation, and academic style. 

Week Two Peer Review Rubric

Trait Identification

• The summary used an appropriate academic source (not Wikipedia or a fake news site) Y/N
• The assignment summarized the article in 5-6 sentences. Y/N
• The summary used a They Say/I Say template from Chapters 1-3. If no, please suggest one. Y/N
• The summary included a direct quote with an APA style in-text source citation. Y/N
• The summary included an APA style full reference for the source at the end of the paragraph. Y/N
• After reading the summary paragraph, I understand the main ideas of the source. Y/N
• The summary paragraph displays the conventions of academic writing, including style, grammar, and correct punctuation/spelling. Y/N

Please make sure that you include a final comment that your peer would rate as at least three stars for helpfulness here. Remember to like your peers’ feedback if you would rate it as three stars or higher.

Week Three Writing Task: Introduction (aka “Hook”)

How do you get your audience’s attention? The introductory paragraph is one of the most important parts of your essay. An effective hook catches the reader’s interest in a specific, concrete way. Look at the introductory paragraphs in the essay section of They Say/I Say for some examples of effective hooks.

• Outrageous statement or exaggeration: In “Don’t Blame the Eater” (p. 241), David Zinczenko invites the audience to consider a headline suitable for a Jay Leno monologue.
• Question: Zinczenko also uses a series of questions to get the reader’s attention. “Whatever happened to personal responsibility?” Richard Muller similarly uses the questioning technique in combination with a strong statement in “Nuclear Waste” (p. 252).
• Strong statement: In “Hidden Intellectualism” (p. 244), Gerald Graff states, “Everyone knows some young person who is impressively ‘street smart’ but does poorly in school.”
• Anecdote: In “The (Futile) Pursuit of the American Dream” (p. 260), Barbara Ehrenreich uses anecdote and vivid imagery to set the tone of her essay.
•  Vivid Imagery or unusual detail. Flannery O’Connor starts off her short story “Everything that Rises Must Converge” (p. 272) with vivid imagery.

Other ways to start your essay include (with my examples):

• A quotation: In a 1965 sermon the day after nonviolent resisters faced police brutality in Selma, Alabama, Dr. Martin Luther King, Jr. said, “Deep down in our non-violent creed is the conviction there are some things so dear, some things so precious, some things so eternally true, that they’re worth dying for.”
• A statistic or fact: According to the National Institute of Mental Health, one in five children under the age of 18 lives with a severe and debilitating mental disorder.

Choose at least one of these techniques to write a draft of your introductory paragraph(s) for Essay One.

• Remember that the purpose of your essay is exploratory and the tone should be objective, so some of these techniques may be more appropriate than others.
• Avoid the second person pronoun you in your draft (you may use first and third person pronouns).
• Your hook should end with a thesis statement that answers your research question. For an example, see Michaela Cullington’s essay, “Does Texting Affect Writing?” Her thesis statement comes at the end of paragraph three: “In fact, it seems likely that texting has no significant effect on student writing.”

Week Three Peer Review Rubric

Trait Identification

• The introduction starts with a “hook” to catch the reader’s interest. Y/N
• The introduction avoids the second person pronoun “you.” Y/N
• The introduction is at least one paragraph in length. Y/N
• The introduction is specific and concrete. Y/N
• The introduction ends with a thesis statement that answers the research question. Y/N

Evaluation

This introduction makes me want to learn more about the topic.

• Meh
• Sure!
• I cannot wait to read this essay!

This introduction uses the conventions of academic style, including good grammar, punctuation, spelling, and academic tone. Please provide a brief comment to explain your answer.

• Needs some work
• Solid style
• Wow! You’re an academic style pro!

Please make sure that you include a final comment that your peer would rate as at least three stars for helpfulness here. Remember to like your peers’ feedback if you would rate it as three stars or higher.

Week Four Writing Task: Exploratory Research Paper Rough Draft

Writing task is due Sunday at 11:59 p.m. Reviews are due by Thursday before class. The final revised essay is due to Blackboard on the following Sunday at 11:59 p.m.

Note: The reviews for this assignment are not anonymous. You should have only two papers to review. I have also assigned a self-assessment for your essay, which means you will have the chance to check your own work against our class-created rubric.

Week Four Peer Review: Student-Created Rubric (Note: These differ by class)

Trait Identification

• The essay contains an effective hook and introduction.
• The essay contains five or more sources.
• The sources used are credible.
• The paper uses APA citation style.
• In text citations are mostly correct (author, date).
• The entire essay supported the research question and answer (thesis statement).
• The essay meets the length requirements.
• The essay includes an introduction, thesis statement, and conclusion.
• The essay is objective in tone (free from bias). If no, please comment.
• The essay does not use the second person pronoun “you” (except in direct quotes).
• The paper uses They Say/I Say templates.

Evaluations

The essay taught me something new.

• I didn’t learn anything I didn’t already know
• I already knew something about this topic, but I learned something new
• I hadn’t heard anything about this topic, and I learned something new

Evaluate the essay’s grammar and punctuation

• Needs a lot of work
• Has some issues
• Mostly effective

Evaluate how informative the essay is.

• Meh
• Gets persuasive or is slightly off topic
• I feel way more informed about this topic! Nice work!

The essay contains a good balance of sources and original ideas (They Say/I Say)

• Okay
• Good
• Great!

The paper is engaging and interesting

• Meh
• I was interested but it could be better
• I was so intrigued by this paper, I reread it!

How likely are you to recommend this essay to a friend? Please explain your response.

• I could not recommend this essay to anyone
• I might recommend this essay to someone
• I would definitely recommend this essay to someone.

Appendix II: Peer Review Grading Rubric and Examples of Student Feedback

Possible Deductions for Each Week

• -5 submission score below 70%
• -5 reviewer score below 70%
• -5 for each missing review

Examples of Student Feedback

• I completely agree that many problems lurk on the internet. I also agree that parents should take the responsibility to protect their kids from harm of all kinds including those found on the internet. I like the idea of educating kids and holding them accountable with the contract.  Your Rating: One Star
• Have you thought about having the parents take turns volunteering between their classes? If enough parents participated and volunteered in the program you are suggesting, maybe they could cover most of the time slots and get a break on their fees. Also, CWI and other schools offer online and/or hybrid classes (like this one that we are in), which allow students more freedom in scheduling their school work. Maybe online classes could be part of the “plan of action” that you were talking about.  Your Rating: Three Stars
  
• I think everybody wants to fix this “very expensive health insurance” problem. All of us wish we could live a healthy life. I believe that without our health we can’t live happily. It would be so sad if we couldn’t see a doctor because we don’t have health insurance. What I hear you saying is that we need a universal healthcare system. Since the assignment asks us to first prove there’s a significant problem, I’m wondering which you plan to focus on? Will it be an insurance company or medical costs? It might be that both are connected, but if you focus on either one the problem of your argument will be more clear and strong. I would suggest finding evidence that convinces the reader how much the cost differences are between employer insurance and private insurance in Idaho. If you don’t have a job you don’t have insurance? You can’t get any healthcare at all? You might be able to get useful information on the website for The Idaho Department of Health and Welfare. http://healthandwelfare.idaho.gov/Medical/tabid/61/Default.aspx Your Rating: Five Stars