But why specifications grading? What's wrong with points?

I use Specifications Grading for the courses I teach. This grading approach does not assign any points to assessable items. That is, there are no points to add up at the end of the semester to determine the final course grade. Each assessable item is either determined to have met the specifications I provided for it (in which case the item is marked “complete”), or that it did not meet the specifications (in which case the item is marked “incomplete”). The assessable items are grouped into a variety of categories (depending on the course). To earn a particular final course grade, a learner must “complete” a specified number of items in each assessable category, with lower grades typically requiring fewer “complete” items in each category.

But why!?

I sometimes get asked why I use specifications grading. This is usually followed with the question, “What’s wrong with points? That’s what we’ve been using since I was in school, and what today’s students are familiar with.” Let's explore these questions a bit.

Bucket of points

With a traditional point-based grading system, there is no way to _guarantee_ that a course’s student learning outcomes (SLO) are met, unless the _only_ thing that is _ever_ assessed are the SLOs. And even in that scenario, there is no (practical) way to know if they met _all_ of the SLOs (and if not, which ones did they meet?), unless they earn 100% of the possible points for the semester. Because all points go into the same “point bucket” to determine the final course grade, they get mixed together and “averaged,” and the ability to identify what the learner has shown evidence of learning is lost. Areas the learner does well in (earning lots of points) help mask area they may not have learned at all, due to the “extra” points from the good areas filling in the voids. As such, it is possible that we have learners completing a course with a grade of C having fully met very few (maybe none?) of the SLOs. Further, if they earned a B or C (or maybe even an A), we have no way of knowing which/how many of the SLOs were met.

Just throw them in the bucket...

Consider the following table showing scores for three learners. Do you consider all three of these learners B students? They all ended the semester with 82.5% of the possible points.

As I look at this data, I see three very different learners. On the one hand, Benny has pretty consistent scores on everything, in each case earning 80-83% of the possible points. Hoton, on the other hand, starts out very well, but then misses a couple of quizzes, and earns marginal scores on Exam 1 and the Final Exam. On the third hand, Dee doesn't do very well on the first few quizzes and Exam 1, but then does very well towards the end of the semester. 

I suspect we can agree that Benny is a B student. We may not be able to agree if Hoton and Dee are, however. If we just look at the Final Exam score, it's not clear that Hoton has learned course material at a B level, since they only earned 71% of the possible points. Did they not learn the material, or did they juust have a bad test day? However, again looking at the Final Exam score, Dee did better than Benny, earning 90% compared to Benny's 82%. Could that mean Dee simply needed more time to master the content, and maybe her Final Exam score indicates she should be considered an A student? Following similar logic, Is Hoton a C student? Or, do we declare all of them B students because they all earned 825 points? Because all of the points are dumped into the same bucket, we have an "average" of  the semester, and loose the details that might be beneficial in making decisions.

Let's fly across the country...

Before I describe this scenario to you, let me assure you that I do know this is not how airplanes are designed. But please hear me out, and just consider it an example, however contrived it may be.

Let's imagine there is a school you can attend where you take four courses and are considered to be fully qualified to design an entire airplane, with no minimum grade needed to pass a course. Further, let's imagine two particular engineers have taken these four courses, and both graduated from the program with identical 3.0 GPAs. Upon graduation, each engineer designed an airplane, and airplanes have been built to the engineer's specs. Does it matter to you which airplane you get on to fly across the county? Based on each engineer's GPA--the only thing you can base our decision on--your answer is likely that it does not matter.

But, what if I told you that Engineer 1 earned course grades of A, A, A, and an F, and Engineer 2 earned all Bs? Does that added information sway you one way or the other? Many people choose the airplane that Engineer 2 designed, even though they did not earn any As, because Engineer 1 earned an F. But, now what if you knew what the four courses were? Could that make a difference in our decision? If Engineer 1's F was in wing design, engine design, or cockpit design, you'd likely for sure choose choose the Engineer 2's plane. But if Engineer 1's F was a class about seat and interior design, you might opt for their plane, since you'd know that the engine, wings, and cockpit should be based on A-level work. You just might have an uncomfortable seat. Being able to know those details is important. Just having the average grade (the GPA) of the four courses hides the needed information you need to make the best decision.

Binned countable items

Specifications grading can provide a solution to the “problems” I just described. If the final course grade specifications are structured to do so, a particular final course grade can tell you exactly what/how many SLOs were met by a learner. For example, let’s assume a course has 5 SLOs. The specs could be configured such that a final course grade of C means they met at least 2 of the 5 SLOs. A grade of B means at least 4 of the 5, and an A means they met all 5. If structured even more carefully, the specs could even be configured such that a Grade of C would indicate exactly which 2 SLOs were met. I see this as a benefit! It allows a final course grade to mean something specific. Using the example I presented above, if a learner earns a B we can know they showed proficiency in at least 4 of the 5 SLOs, and maybe even know what they did not show proficiency in. For a points-based-graded course, all we can say is they earned at least 800 out of a 1000 points, but have little idea of what they actually learned. 

For me, points just don't add up!

Ever since I realized that a point-based grading system hides deficiencies as well as strengths in an average, I've been very uncomfortable using it. Fortunately, I discovered specifications grading's benefits many years ago, and have been using it ever since. There is no going back for me, because points just don't meet my specs.

Assessing Assessment

For the last two years, I’ve served my college as one of three “Pedagogy Fellows.” Our role has been to be a pedagogy resource to the college’s faculty in a variety of different ways. One of those ways has been to write short messages that we emailed to them on an (almost) weekly basis. The following is a slightly edited version of the last post I was responsible for writing.

It’s the second week of April, with just a couple more weeks of instruction left this semester. The spring semester is nearly over, and course grades will soon be posted. Once they’re submitted our thoughts will turn to preparing for fall courses. OK, we’ll likely take a few days or weeks off to catch our breath, but sometime in the next few months we’ll start that preparation. Some of us like to address it early in the summer and then devote the remainder of the summer to other activities, and some of us flip that around and prepare at the end of the summer.

What are you assessing?

As you start your fall course planning, I encourage you to consider what you’re assessing. Are you assessing (all) the course learning objectives? Are you (sometimes) assessing learner behaviors (such as meeting due dates) rather than course learning objectives? Is it important that those behaviors are assessed, or is that occurring because that’s the way we’ve always done it? Take time to identify what knowledge and skills you want your learners to leave your course with. How well does your list match up with the course learning objectives? If there is a significant difference between the two lists, is that OK, or should they be reconciled, perhaps removing some from one list, and adding some to the other?

How are you assessing?

Besides considering what you are assessing, reflect on how you are assessing? Do your assessments measure learning and understanding, or just the learner’s ability to memorize content for a few hours or days? Is the course grade calculation structured in such a way that a learner can do poorly in one part of the course, but make up those points in another part of the course, thus producing an average grade? Is it OK that the course grade may “hide” the fact that they did poorly in part of the course? Is it possible for a learner to do so poorly on an exam during the start of the semester that there is no way their course grade can recover, regardless of how well they’ve learned the course content by the end of the semester? Does it matter if a particular learner has not gained understanding by a particular date early in the semester, or just that they have the understanding by the end of the semester?

As you are pondering all of this, I encourage you to check out low-stakes and authentic assessment. Also, review material on the mastery learning model and backwards design. And lastly, consider moving away from a points-based system to other alternative grading systems, such as collaborative grading, and specifications grading. Both approaches help center and encourage the learning that you want to occur, rather than having the learners focus on “what do I have to do to earn points to get a B.” Lastly, review this list of blogs and podcasts to see what you can glean from them.

Blogs

• Faculty Focus: Higher Ed teaching strategies from Magna Publications.
• Grading for Growth: Innovative grading methods that work with rather than against student learning: specifications grading, standards-based grading, ungrading, and more.
• BSU Teaching Innovation Blog: A virtual hub for dialogue and community: learn about and share research-informed, learner-centered pedagogy, from teaching with technology to experimental and active learning strategies.
• Paul Gestwicki's Blog: A blog for reflective practice focused on Computer Science, but applicable to all.

Podcasts

• Tea for Teaching: Informal discussions of innovative and effective practices in teaching and learning.
• Teaching in Higher Ed: Faculty development for professors.
• Think UDL Podcast: Discussions with people who are designing and implementing strategies in post-secondary settings with learner variability in mind.

Time to act!

Since you’ve got to this point in my ramblings, I must have held your interest. What caught your eye? What has caused you to consider doing something different? Do you have other suggestions of assessment-related things we should consider as we prepare our courses? Now’s the time to act. It’s time to assess your assessment.

Image credit: https://stock.adobe.com/

The teacher becomes the learner: Assesment

Continuing my intentional path to explore what I can learn to become a better teacher in a remote synchronous online learning environment, here's part 3. I am participating in a Faculty Learning Community (FLC) sponsored by the Division of Online and Strategic Learning of Ball State University. I will periodically post my reflections in this blog to document my progress, and to serve as a resource for a future me, as I come back to review these posts. Perhaps there will be something of use for other readers that aren't me.

Module 3: Assessment

This module focused on formative and summative assessment. The following notes summarize what I gleaned from the module.

Formative assessment...

From Remote Learning and the Formative Assessment Process 

• Remote learning means there is more student autonomy
• Distance learning relies on the formative assessment process for success
• The formative assessment process has four attributes:
• Clarify: determine what students will learn and how they will know they have learned it;
• Elicit: generate evidence of student learning, such as asking questions;
• Interpret: review evidence to determine students’ progress towards the learning goal(s);
• Act: take instructional next steps to move students from where they are to where they need to be, such as re-teaching using a different mode.

From Formative Assessment in Distance Learning

• Know your purpose
• Focus on feedback
• Peer feedback: Be kind; be specific; be helpful
• Leverage personal conversations
• Check in on SEL (social emotional learning)

Summative assessment...

From Alternative to Exams for Remote Teaching

• Online discussions
• Student created videos
• Group projects
• Peer review
• Blogs
• Creative projects

From Summative Assessment in Distance Learning

• What is “need to know” versus “nice to know”?
• Which standards are priorities?
• Move from one big assessment to a series of smaller ones

Summary

Overall, this was another reassuring module for me, in that it focused on a lot of things I am already doing, although I may not have known the names by which I should call them. I have not felt the need to make any significant adjustments to how I am assessing the learners in courses I am teaching in a remote synchronous modality, since I appear to already be doing the right things.