Improving Peer Feedback with Peergrade

Lisa Spitz is an assistant professor at Lesley’s College of Art and Design and the program director for the User Experience online BS degree program. Lisa worked with eLIS this fall to pilot a peer feedback tool, called Peergrade, in her Sketching for Interactive Design course. Below she shares her and her students’ experiences using Peergrade.

In this course, students use sketching to document research insights, tell a story, and visualize mobile interface concepts and interactions. Each individual assignment includes a period of sketching and revising, where students provide peer feedback and then revise their own sketches for overall clarity. My initial experience teaching this course in Blackboard raised a number of challenges with the peer review process: not all peer feedback was of the same caliber and not all students received the same amount of feedback. This meant I was compensating for poor and/or incomplete feedback. I was also manually tracking the quality and quantity of feedback each student provided to their peers, for grading purposes. Further, due to inconsistencies in the types of feedback received, students reported finding it difficult to revise their work. 

Over the summer, John McCormick in eLIS introduced me to an online peer review platform called Peergrade. The overall format and structure of Peergrade was a good match for my particular assignment structure and I was interested in seeing how it might better support our students in the peer review process.

The tool itself was fairly easy to learn. As an instructor, I was able to set up my “classroom” in Peergrade and create each of my assignments. For students, their experience entailed posting their sketches in Peergrade and then evaluating their peers’ sketches based on a custom rubric (which I set up in advance of the course running). The biggest challenge I faced was in tailoring the rubrics to each individual assignment. Students evaluated their peers work based on quantity and diversity of sketches as well as unique requirements for each assignment. The rubrics I created provided students with both quantitative and qualitative feedback on their sketches; and the system guaranteed that each student received feedback from three other students.

Students responded favorably to the use of Peergrade. They were fairly self-sufficient in using the Peergrade platform. It required very little technical support from my end; for instance, allowing late assignment submissions and permitting students to re-upload their work. Some adhoc quotes found in journal entries and the course evaluation include:

  • “I really like Peergrade, I only wish the rest of my courses used this site. It is so much easier to give the feedback and receive the feedback that you want without upsetting another peer about your opinion. Since it is anonymous it is easier to be truthful if you have suggestions on changes.”
  • “That program allowed me to finally get honest feedback from my peers on how they truly felt about my work.”
  • “By reviewing other students, I often could improve my own work through just that process alone.”
  • “The peer commenting system was a great way to discuss among other students each other’s work before turning in the final assignment each week. Critiquing helped me understand my own skills better.”
  • “Peergrade was life changing, love it.”

In an ideal world, I’d spend more time user-testing assignment rubrics before launching these assignments with a live class. However, design is an iterative process and course design is no different and I anticipate refining each assignment rubric with each course instantiation.

If you’re just not sure how this would work within your classroom context, I’d say start small. Choose one project in which you’d like students to give and receive quality peer feedback. Decide what a “good” assignment submission looks like and set up your rubric to probe specifically on those areas. Then, see what the experience looks like from both the Instructor and Student views. Having an initial experience with Peergrade will help you to determine just how and when it might be an asset in your courses. 

Peergrade is currently be used in a small number of online and on campus courses at Lesley. If you would like more information on using Peergrade or peer feedback in your courses, contact elis@lesley.edu.

Provide Grading Feedback With Audio and Video

Blackboard now allows you to embed an audio/video recording of your feedback as you grade attempts. This feature is available for all graded content, including assignments, graded discussions, or in the Grade Details area of any Grade Center column.

Why Provide Audio or Video Feedback?

Providing feedback using audio or video allows you to connect with your students, especially those at a distance. You can give them a glimpse of your personality and show them the real person behind the text. Creating this connection can make it easier to interact, share information, and ensure that your students don’t feel isolated.

Text-based feedback can lend itself to misinterpretation – students may weight all of the feedback equally or be overwhelmed by a lot of text on the page. Audio allows you to add tone and emphasis, perhaps even humor and support, while video allows you to add facial expressions and gestures.

How Do I Add Audio or Video Feedback in my Course?

You can add audio or video feedback on any gradable item in your course. To do so, access the Feedback to Learner area. You may do this directly from an Assignment or from within the Full Grade Center.

From within an Assignment:

From your student’s submission, navigate to the Assignment Details area. In the Feedback to Learner area, In the Feedback to Learner area, click on the Full Content Editor button. This will open the full text/content editor.
Screenshot of Feedback to Learner area of Assignment

For more information on grading assignments or accessing the students’ submissions, see Grading Assignments in myLesley.

From a Grade Center column:

Navigate to the Full Grade Center and hover your mouse over the item you wish to offer feedback. Click the chevron to open a menu and select Quick Comment.

screenshot of accessing Recording Feedback from within the Full Grade Center

Launch the Recorder:

From the Full Text/Content editor, click the Record Feedback Button (first button on the third row of the text/content editor).screenshot of record feedback button

This will launch the recorder. Click the video button to launch your webcam or click the record button to record audio only. Each recording may be a maximum length of five (5) minutes.screenshot of record feedback

Once you have finished recording, your audio or video recording will be added to the student’s feedback area. Detailed instructions for using the Record Feedback option may be found here: Record Audio and Video.

What if I want to upload a video I recorded elsewhere or provide my students with a screencast?

If you recorded a video in a different tool, you may upload it using the Kaltura Mashup Tool. You may also upload a video to Kaltura directly from your mobile device.

If you would like to record a screencast, you may use Kaltura CaptureSpace Lite.

Where can I find more information on providing feedback to my students?

To find out more about providing feedback to your students, check out the resources below:

 

A Student’s Take on Peer Review

In this summative assignment in a freshmen Honors English Composition class, students were asked to review their papers and assignments from the course, and determine 1-3 specific areas of growth or improvement, as well as specific classroom activities, assignments, etc. that contributed to the improvement. Students were then asked to demonstrate this in a creative work in any format/mode, and present the project to the class. A goal of the project was to reflect on one’s learning in a creative style which reflects both the learning itself and the personality or talents of the student.

Emily Tran created this awesome reflective video on her experiences with peer review.

Grading Offline This Winter Break

During the upcoming winter break myLesley will transition to a SaaS environment and upgrade to the latest version of Blackboard (Q4 2017). The upgrade and migration to the new servers will take place from Saturday, December 30, 2017 to Wednesday, January 3, 2018. You will not have access to myLesley during this time. All other Lesley services will be available.

We highly recommend completing your grading by Friday, December 29th. However, we understand that many faculty use the winter break to do their grading. With that in mind, we have provided some resources for downloading your students’ gradable items so that you can grade offline when the server is down. The instructions include downloading student submitted assignments, collecting and printing discussions, saving blog, journal and wiki content to PDF, and downloading a copy of your Grade Center.

Please keep in mind that you must complete these steps by December 29th. You will not have access to myLesley from December 30 – January 3.

If you have any questions or need assistance, please email elis@lesley.edu.

Lessons learned from running our first online Design for User Experience course

Today’s post is by Lisa Spitz, Lesley Assistant Professor and consultant for the College of Art and Design’s bachelor’s program in design for user experience.


In Fall 2, 2016 we ran our first course in the Design for User Experience program, Typography 1. 10 students signed up for the course. Excitement ensued. And then I started looking into the class roster. Of the 10 students, just 1 was a Design for UX student. The remaining students represented a mix of Business, Counseling, and Psychology programs. As a new program in an entirely new category for Lesley, I realize that it takes time to market and enroll new students. Nonetheless, I was a bit disappointed by the turn out. I didn’t question the applicability of the content to individuals “outside the field”. Principles of good typography is something anyone can benefit from. But I was worried about the complexity of the learning activities I’d planned and the Adobe software that was required to complete them.

What I learned over the subsequent 8 weeks is the importance of being flexible and the benefit of testing a course with individuals outside your domain. Let’s start with the latter point. For those familiar with Universal Design for Learning or Inclusive Design, it’s a bit like that. If you can make your course “work” for individuals outside your program, chances are it will work better for those inside your program as well. I’m not talking about “dumbing down” content or removing requirements. I’m talking about adding instructional supports to make the course content and expectations clearer. Here are a few ways I made that happen while the course was still in flight:

Providing better prompts
As a typography course, students were expected to create several designs and critique the work of their peers. However, journal entries revealed that students lacked the confidence to do so and some even felt hypocritical critiquing their peers’ work. The original critique questions I’d provided assumed they could judge which design was best (or worst) and give concrete recommendations on what to do next. But students were not sure how to assess the work of their peers. How would they know which was best? They certainly could tell which one they liked, but could not articulate why it was better. So, I went back to the drawing board and made the questions more personal. “What words would you use to describe this?”; “What is being emphasized?”; “What interests you about the design?” Etc. These questions were easier to answer. They required students to respond based on what they saw and how they felt, not what they deemed to be “good” or “bad”.

Original critique language:Critique_Before

Revised critique language: 
Critique_After


Creating more explicit directions

As a visual learner, one of the biggest challenges I faced when creating my own online course is finding ways around the “wall of text”. To explain an activity requires quite a bit of documentation. Aside from using all video or images, there’s almost no way around it. And when confusion arises, the tendency is to double down with more explanation. Instead, I took a step back, added images, cut text, and used more headings and bulleted lists – detailing process, specifications and steps for completion.

Original assignment description: (click for full size image)
direction_before_crop

Revised assignment description: (click for full size image)
directions_after_crop

Personalizing the feedback process
As students submitted their design work each week, I used the Assignment Tool to provide feedback. Originally, I defaulted to the WYSIWYG editor and took to writing what I thought worked/didn’t work and needed improvement. However, it felt as if some of my feedback was getting lost in translation. Again, the wall of text. Midway through the course I switched to video. Instead of writing a single piece of feedback, I recorded my screen as I looked at each of their design options and spoke about their use of typography in great details. If I’d have typed that feedback out, it would have been a novel. But to record it took just a few minutes. Students appreciated the new format and commented on how incredibly helpful it was.

All of these changes required a great deal of flexibility on my part. I ended up re-writing each week’s content before it went live; I added images to show, not tell; I created videos that demonstrated how to do the assignments; I offered up 30 minute 1:1 time slots to address individual challenges; and I gave feedback that was personal and specific. In the end, I had students comment on their appreciation for typography and design. But more importantly, I witnessed their transformation. When week 1 started, students proclaimed themselves unable to be creative. When week 8 finished, they professed the ways in which they were using their new knowledge of good typography to impact their professional and academic lives. As for myself, I still have some work to do within the course curriculum – but am confident that the results will be even better the next time around.