But why specifications grading? What's wrong with points?

I use Specifications Grading for the courses I teach. This grading approach does not assign any points to assessable items. That is, there are no points to add up at the end of the semester to determine the final course grade. Each assessable item is either determined to have met the specifications I provided for it (in which case the item is marked “complete”), or that it did not meet the specifications (in which case the item is marked “incomplete”). The assessable items are grouped into a variety of categories (depending on the course). To earn a particular final course grade, a learner must “complete” a specified number of items in each assessable category, with lower grades typically requiring fewer “complete” items in each category.

But why!?

I sometimes get asked why I use specifications grading. This is usually followed with the question, “What’s wrong with points? That’s what we’ve been using since I was in school, and what today’s students are familiar with.” Let's explore these questions a bit.

Bucket of points

With a traditional point-based grading system, there is no way to _guarantee_ that a course’s student learning outcomes (SLO) are met, unless the _only_ thing that is _ever_ assessed are the SLOs. And even in that scenario, there is no (practical) way to know if they met _all_ of the SLOs (and if not, which ones did they meet?), unless they earn 100% of the possible points for the semester. Because all points go into the same “point bucket” to determine the final course grade, they get mixed together and “averaged,” and the ability to identify what the learner has shown evidence of learning is lost. Areas the learner does well in (earning lots of points) help mask area they may not have learned at all, due to the “extra” points from the good areas filling in the voids. As such, it is possible that we have learners completing a course with a grade of C having fully met very few (maybe none?) of the SLOs. Further, if they earned a B or C (or maybe even an A), we have no way of knowing which/how many of the SLOs were met.

Just throw them in the bucket...

Consider the following table showing scores for three learners. Do you consider all three of these learners B students? They all ended the semester with 82.5% of the possible points.

As I look at this data, I see three very different learners. On the one hand, Benny has pretty consistent scores on everything, in each case earning 80-83% of the possible points. Hoton, on the other hand, starts out very well, but then misses a couple of quizzes, and earns marginal scores on Exam 1 and the Final Exam. On the third hand, Dee doesn't do very well on the first few quizzes and Exam 1, but then does very well towards the end of the semester. 

I suspect we can agree that Benny is a B student. We may not be able to agree if Hoton and Dee are, however. If we just look at the Final Exam score, it's not clear that Hoton has learned course material at a B level, since they only earned 71% of the possible points. Did they not learn the material, or did they juust have a bad test day? However, again looking at the Final Exam score, Dee did better than Benny, earning 90% compared to Benny's 82%. Could that mean Dee simply needed more time to master the content, and maybe her Final Exam score indicates she should be considered an A student? Following similar logic, Is Hoton a C student? Or, do we declare all of them B students because they all earned 825 points? Because all of the points are dumped into the same bucket, we have an "average" of  the semester, and loose the details that might be beneficial in making decisions.

Let's fly across the country...

Before I describe this scenario to you, let me assure you that I do know this is not how airplanes are designed. But please hear me out, and just consider it an example, however contrived it may be.

Let's imagine there is a school you can attend where you take four courses and are considered to be fully qualified to design an entire airplane, with no minimum grade needed to pass a course. Further, let's imagine two particular engineers have taken these four courses, and both graduated from the program with identical 3.0 GPAs. Upon graduation, each engineer designed an airplane, and airplanes have been built to the engineer's specs. Does it matter to you which airplane you get on to fly across the county? Based on each engineer's GPA--the only thing you can base our decision on--your answer is likely that it does not matter.

But, what if I told you that Engineer 1 earned course grades of A, A, A, and an F, and Engineer 2 earned all Bs? Does that added information sway you one way or the other? Many people choose the airplane that Engineer 2 designed, even though they did not earn any As, because Engineer 1 earned an F. But, now what if you knew what the four courses were? Could that make a difference in our decision? If Engineer 1's F was in wing design, engine design, or cockpit design, you'd likely for sure choose choose the Engineer 2's plane. But if Engineer 1's F was a class about seat and interior design, you might opt for their plane, since you'd know that the engine, wings, and cockpit should be based on A-level work. You just might have an uncomfortable seat. Being able to know those details is important. Just having the average grade (the GPA) of the four courses hides the needed information you need to make the best decision.

Binned countable items

Specifications grading can provide a solution to the “problems” I just described. If the final course grade specifications are structured to do so, a particular final course grade can tell you exactly what/how many SLOs were met by a learner. For example, let’s assume a course has 5 SLOs. The specs could be configured such that a final course grade of C means they met at least 2 of the 5 SLOs. A grade of B means at least 4 of the 5, and an A means they met all 5. If structured even more carefully, the specs could even be configured such that a Grade of C would indicate exactly which 2 SLOs were met. I see this as a benefit! It allows a final course grade to mean something specific. Using the example I presented above, if a learner earns a B we can know they showed proficiency in at least 4 of the 5 SLOs, and maybe even know what they did not show proficiency in. For a points-based-graded course, all we can say is they earned at least 800 out of a 1000 points, but have little idea of what they actually learned. 

For me, points just don't add up!

Ever since I realized that a point-based grading system hides deficiencies as well as strengths in an average, I've been very uncomfortable using it. Fortunately, I discovered specifications grading's benefits many years ago, and have been using it ever since. There is no going back for me, because points just don't meet my specs.

My century in the last decade

As I was compiling data to update my curriculum vitae last week, I realized I’d hit a milestone of sorts—a century. As of the 2019 fall semester, I have been responsible for one hundred sections of courses while serving as a full-time faculty member in the Computer Science department of Ball State University. Since I started teaching full-time in the fall of 2010—the start of the just-ended decade, I decided now was as good a time as any to review what I’ve done in my academic world for the last half-score years. As I reflected, I was a bit surprised at all I've done. Here’s some of what I discovered.

Courses taught

During these 9.5 years, I’ve taught sixteen different courses. A dozen of them have been computer science courses, and I’ve also taught four different courses for the Honors College. Some semesters I taught multiple sections of some of these courses. The numbers provided here are counting sections. Details are provided below in a table, along with a chart showing the same data, for those people who just want to look at the pictures.

Course	Sections taught
CS 104: Intro to Computers	8
CS 110: Intro to CS & Web	14
CS 120: CS 1: Prog Fund	22
CS 200: Comp & Society	11
CS 204: Personal Comp	1
CS 205: Multimedia Prog	1
CS 206: Digital Image for Web	1
CS 222: Advanced Prog	15
CS 239: Soc & Prof Issues	16
CS 339: CS Ed & Hist	1
CS 499: Independent Study	3
CS 499: CS4MS+	3
HONR 296:CS & Algo	1
HONR 390: SED	1
HONR 390: Orwell 1984	1
HONR 499: Honors Thesis	1
Total sections	100

I was the first to develop and teach over half of these courses. This includes six of the CS courses (CS 204, CS 205, CS 206, CS 239, CS 339, and CS 499: CS4MS+), and three Honors College courses (HONR 296, and both HONR 390s). Four of the CS courses were developed from a master syllabus, and two from scratch (with one of them being an international field study, and one being an immersive learning course). All three Honors College courses were developed from scratch. (HONR 390: SED was an immersive learning course I co-developed and co-taught with Lynne Stallings and Carolyn Dowling). I also developed an online version of the established CS 110 course.

Students and grades

During this time, I’ve taught/mentored 2,693 students (ignoring the fact that some students took multiple courses with me). Having taught one hundred sections of courses, that results in an average of 27 students per section. Student enrollment, average enrollment, and number of courses (by semester) is shown below.These students earned 924 As, 981 Bs, 468 Cs, 147 Ds, and 173 Fs. The relative percentages are shown in the chart to the right. I did award plus and minus grades, but have consolidated them into the base grade.

But wait, there's more...

Besides teaching, I've been able to present at a variety of conferences, publish my research, try out different pedagogy, and even lead an international field study course during the last decade.

Conferences and publishing

My very first academic conference actually occurred a couple weeks before I officially started my first semester of full-time teaching (fall 2010). I submitted a proposal to International Workshop on Computing Education Research (ICER) earlier in the year, and much to my surprise, it was accepted. Due to the conference being held at Aarhus University in Denmark, I had to obtain a passport for the first time. And thus starting my international travels.

Since then, I've presented 42 times at a variety of conferences, seminars, and workshops (some of them multiple times), including the conferences in the following list. I've also served on the CCSC:MW conference committee since 2010, serving as chair twice.

• Technical Symposium on Computer Science Education (SIGCSE)
• Consortium for Computing Sciences in Colleges: Midwest (CCSC:MW)
• Lilly International Conference on College Teaching
• Edward C. Moore Symposium on Excellence in Teaching
• Diversity Research Symposium

During the last decade, I've had 17 peer-reviewed publications in book chapters, journals, and conference proceedings. I've also had 4 magazine articles and white papers published.

Pedagogy

I've experimented with a variety of pedagogical approaches during the last decade, including the following. 

• Specifications grading (see my posts here and here)
• Learner-centered teaching
• Flipped learning
• iClicker response system
• Immersive learning (CS4MS+)
• CS 120 All-section Art Show
• International field study (Computer Science, Education, and History)

To support flipped learning in CS 104, CS 110, and CS 120, I've created over one hundred instructional videos totaling approximately 21 hours in length.

Faculty Advisor

During the decade I've served as the faculty advisor for two student organizations.

• Golden Key International Honour Society
• Colleges Against Cancer

Recognition

Some people thought I did good work, and chose to recognize my efforts.

• Recognized as a Senior Member by the Association for Computing Machinery (ACM).
• My Journal article Measuring and understanding team development by capturing self-assessed enthusiasm and skill levels, ACM Transactions on Computing Education 16, 2 (Mar. 2016), Article No. 6 was included on the ACM Computing Reviews 21st Annual Best of Computing: Notable Books and Articles list.
• National Residence Hall Honorary, Ruth Peters (Ball State) Chapter: I was independently nominated by two students for my “dedication to academic excellence and student success.”

OK, that's enough...

I could have listed a few more things, but this is already too long, so I'll just stop here. Leave me a comment about what your last decade has been like.

Image credits:
Dave talking: Lilly International Conference on College Teaching