According to specs…

During the fall 2016 semester, I participated in a multi-session faculty learning community that explored what specifications grading is, how it might be implemented, and what it might do for our students and us. In her book Specifications grading: Restoring rigor, motivating students, and saving faculty time, Linda Nilson claims that specifications grading can make grading faster, clearer, and more precise, allowing the instructor to focus more on promoting improvement in students, rather than worrying about justifying a grade.

The basic tenets of specifications grading are to provide a very clear, detailed description of what is expected of the student for a given assessment (the specification), and then evaluate their submitted work as complete or incomplete against that specification—they either met the specification, or they did not. Simply providing clear expectations can make a significant difference in a student’s ability to submit what you are wanting. It almost seems too good to be true.

Historically, I have assigned points to everything students do for a course, and then determined their course grade based on the total of their earned points compared to what was possible to earn. This means that a student can do poorly on some assignments (or not do it at all), and great on other assignments, and then depend on the “extra” points from the great assignment to offset the lack of points on the poor assignment. This effectively generates an “average grade”, but does not necessarily reflect what the student accomplished. Students were also inclined to beg for partial credit.

I decided if even one of the claims Nilson made in her book title was true, it would be worthwhile implementing specs grading. My first opportunity was for a discussion-based course (CS 200: Computers and Society) during the spring 2017 semester. I converted my evaluation of all assessments to complete/incomplete—they met the detailed specifications I provided, or they did not. One exception I made to grading complete/incomplete was for the exams, although, even here, I still altered my approach. Previously I often awarded partial points for a response that contained some of the answer for which I was looking, but not all of it. For this course, I evaluated each question on a complete/incomplete basis—full points or zero. Since most of the assessments were an all or nothing situation, I did offer the students limited opportunities to resubmit incomplete assignments. In the absence of points to add up, I determined the final course grade based on how many of each assessment category the student completed. For example, if a student successfully completed at least five of six quizzes, that gave them an A, provided they had an A in all other assessment categories. If they only passed four, they dropped to a B, etcetera.

I found it critical to clearly (and repeatedly) explain to the students why I was grading this way, and what the benefits were for them and their learning. I made minor adjustments, based on my experiences and student feedback, and implemented specs grading for the same course during the summer 2017 semester as well. I have just completed the fall 2017 semester during which I implemented specs grading on two more courses. One of these courses (CS 239: Social and Professional Issues) I implemented the same way I did the CS 200 course during the spring 2017 semester. For the second course (CS 222: Advanced Programming), I used specs grading on all assessments, excepting the programming projects. For the projects, I continued to assign points based on how close to a correct solution they were. The projects were a category of assessment items, and had a different minimum points the student had to accumulate for each letter grade.

Reflecting on my experiences teaching courses utilizing specifications grading, I observed a variety of positive outcomes, including the following.

• Students did not beg for missed points. Although there were a few (very few!) times when a student asked if I would consider passing a given assignment, rather than the failing evaluation I had provided.
• As a whole, final course grades stayed substantially the same in the specifications-graded courses, compared to before I implemented specifications grading. I believe however, that the grades more accurately reflect the level of course content mastery. That is, if a student received a course grade of “B”, they had mastered most all of the course material at a “B” level or better, as opposed to some material at an “A” level, and other material at a “C” level.

Not all was perfect, however. My experiences and student feedback did highlight some areas that needed adjustment, including the following.

• I needed to provide a better explanation and justification for using specifications grading at the start of the semester.
• I did not originally provide enough detail in a few of the specifications. Others contained minor ambiguities.
• Specification grading has not significantly reduced my grading time. In large part, this is because I want to provide the student with as much feedback as possible as soon as possible, rather than stopping my evaluation as soon as I found a specification they did not meet.
• Some students became discouraged and gave up when they failed too many assignments.

Thus far, students tend to either like, or dislike the use of specifications grading; there is not much middle ground. Some student quotes from the end-of-semester course evaluation may serve to illustrate this. Students in the CS 222 course offered these comments about specification grading (I list the positives, and then the negatives):

“The grading system was very clear. Everyone knew what needed to be done for a specific grade.”

“Unique grading system, which might seem daunting at first, but is overall a worthwhile system.”

“Grading system is a complete mess, I know it's newish, but it really doesn't have a place in an advanced programming class.”

“Specification grading encouraged me to not complete more achievements because I knew that I could not get a grader higher than a C because of my project scores. In that regard, I think standard weighted grading systems are better as every assignment counts. In this system, students will always have incentives to work hard on every remaining assignment because they all impact the student's grade.”

The student comments in the CS 239 course were all on the negative end of the scale.

“The fact that you could have one hundred percent on everything throughout the semester, but then get a B on the final and it would bring your final grade down to a B was kind of ridiculous, especially for a one credit hour class”

“This course has a horrible grading system. Pass/Fail for sections would have been alright, if they were then averaged.”

“The grading system needs revision.”

“I hate this grading scale. People are naturally going to be better in certain areas and it seems unfair for their skill in those areas not matter if they don't do well in another area. Because of the specification grading I did only the bare minimum on all of the assignments because I knew there was less weight placed on my actual skill on that assignment.”

This course has a clear grading system.

Interestingly, if we look at the student responses to Likert scale questions that are part of the course evaluation, we get a much more positive response for both courses. The response rate for CS 222 was 20 of 39 students or 64.5%, and for CS 239 was 22 of 30 or 73.3%. The questions related to specifications grading were the following:

The use of specification grading in this course encouraged me to complete more of my work well, rather than depending on a good grade on one item to cover lesser performance on another item.

The use of specification grading in this course encouraged me to attend class regularly, since I knew it would have a direct impact on my final course grade.

I felt like I could determine my final course grade (or my path towards it) more accurately at any time during the semester because of the use of specification grading.

Once I got used to it, I wish more instructors used specification grading for their courses.

All five questions had a combined strongly agree and agree that was beyond 65%, with many of them being 80% or better. Apparently those that disliked specifications grading must have been more inclined to provide comments.