But why specifications grading? What's wrong with points?

I use Specifications Grading for the courses I teach. This grading approach does not assign any points to assessable items. That is, there are no points to add up at the end of the semester to determine the final course grade. Each assessable item is either determined to have met the specifications I provided for it (in which case the item is marked “complete”), or that it did not meet the specifications (in which case the item is marked “incomplete”). The assessable items are grouped into a variety of categories (depending on the course). To earn a particular final course grade, a learner must “complete” a specified number of items in each assessable category, with lower grades typically requiring fewer “complete” items in each category.

But why!?

I sometimes get asked why I use specifications grading. This is usually followed with the question, “What’s wrong with points? That’s what we’ve been using since I was in school, and what today’s students are familiar with.” Let's explore these questions a bit.

Bucket of points

With a traditional point-based grading system, there is no way to _guarantee_ that a course’s student learning outcomes (SLO) are met, unless the _only_ thing that is _ever_ assessed are the SLOs. And even in that scenario, there is no (practical) way to know if they met _all_ of the SLOs (and if not, which ones did they meet?), unless they earn 100% of the possible points for the semester. Because all points go into the same “point bucket” to determine the final course grade, they get mixed together and “averaged,” and the ability to identify what the learner has shown evidence of learning is lost. Areas the learner does well in (earning lots of points) help mask area they may not have learned at all, due to the “extra” points from the good areas filling in the voids. As such, it is possible that we have learners completing a course with a grade of C having fully met very few (maybe none?) of the SLOs. Further, if they earned a B or C (or maybe even an A), we have no way of knowing which/how many of the SLOs were met.

Just throw them in the bucket...

Consider the following table showing scores for three learners. Do you consider all three of these learners B students? They all ended the semester with 82.5% of the possible points.

As I look at this data, I see three very different learners. On the one hand, Benny has pretty consistent scores on everything, in each case earning 80-83% of the possible points. Hoton, on the other hand, starts out very well, but then misses a couple of quizzes, and earns marginal scores on Exam 1 and the Final Exam. On the third hand, Dee doesn't do very well on the first few quizzes and Exam 1, but then does very well towards the end of the semester. 

I suspect we can agree that Benny is a B student. We may not be able to agree if Hoton and Dee are, however. If we just look at the Final Exam score, it's not clear that Hoton has learned course material at a B level, since they only earned 71% of the possible points. Did they not learn the material, or did they juust have a bad test day? However, again looking at the Final Exam score, Dee did better than Benny, earning 90% compared to Benny's 82%. Could that mean Dee simply needed more time to master the content, and maybe her Final Exam score indicates she should be considered an A student? Following similar logic, Is Hoton a C student? Or, do we declare all of them B students because they all earned 825 points? Because all of the points are dumped into the same bucket, we have an "average" of  the semester, and loose the details that might be beneficial in making decisions.

Let's fly across the country...

Before I describe this scenario to you, let me assure you that I do know this is not how airplanes are designed. But please hear me out, and just consider it an example, however contrived it may be.

Let's imagine there is a school you can attend where you take four courses and are considered to be fully qualified to design an entire airplane, with no minimum grade needed to pass a course. Further, let's imagine two particular engineers have taken these four courses, and both graduated from the program with identical 3.0 GPAs. Upon graduation, each engineer designed an airplane, and airplanes have been built to the engineer's specs. Does it matter to you which airplane you get on to fly across the county? Based on each engineer's GPA--the only thing you can base our decision on--your answer is likely that it does not matter.

But, what if I told you that Engineer 1 earned course grades of A, A, A, and an F, and Engineer 2 earned all Bs? Does that added information sway you one way or the other? Many people choose the airplane that Engineer 2 designed, even though they did not earn any As, because Engineer 1 earned an F. But, now what if you knew what the four courses were? Could that make a difference in our decision? If Engineer 1's F was in wing design, engine design, or cockpit design, you'd likely for sure choose choose the Engineer 2's plane. But if Engineer 1's F was a class about seat and interior design, you might opt for their plane, since you'd know that the engine, wings, and cockpit should be based on A-level work. You just might have an uncomfortable seat. Being able to know those details is important. Just having the average grade (the GPA) of the four courses hides the needed information you need to make the best decision.

Binned countable items

Specifications grading can provide a solution to the “problems” I just described. If the final course grade specifications are structured to do so, a particular final course grade can tell you exactly what/how many SLOs were met by a learner. For example, let’s assume a course has 5 SLOs. The specs could be configured such that a final course grade of C means they met at least 2 of the 5 SLOs. A grade of B means at least 4 of the 5, and an A means they met all 5. If structured even more carefully, the specs could even be configured such that a Grade of C would indicate exactly which 2 SLOs were met. I see this as a benefit! It allows a final course grade to mean something specific. Using the example I presented above, if a learner earns a B we can know they showed proficiency in at least 4 of the 5 SLOs, and maybe even know what they did not show proficiency in. For a points-based-graded course, all we can say is they earned at least 800 out of a 1000 points, but have little idea of what they actually learned. 

For me, points just don't add up!

Ever since I realized that a point-based grading system hides deficiencies as well as strengths in an average, I've been very uncomfortable using it. Fortunately, I discovered specifications grading's benefits many years ago, and have been using it ever since. There is no going back for me, because points just don't meet my specs.

Assessing Assessment

For the last two years, I’ve served my college as one of three “Pedagogy Fellows.” Our role has been to be a pedagogy resource to the college’s faculty in a variety of different ways. One of those ways has been to write short messages that we emailed to them on an (almost) weekly basis. The following is a slightly edited version of the last post I was responsible for writing.

It’s the second week of April, with just a couple more weeks of instruction left this semester. The spring semester is nearly over, and course grades will soon be posted. Once they’re submitted our thoughts will turn to preparing for fall courses. OK, we’ll likely take a few days or weeks off to catch our breath, but sometime in the next few months we’ll start that preparation. Some of us like to address it early in the summer and then devote the remainder of the summer to other activities, and some of us flip that around and prepare at the end of the summer.

What are you assessing?

As you start your fall course planning, I encourage you to consider what you’re assessing. Are you assessing (all) the course learning objectives? Are you (sometimes) assessing learner behaviors (such as meeting due dates) rather than course learning objectives? Is it important that those behaviors are assessed, or is that occurring because that’s the way we’ve always done it? Take time to identify what knowledge and skills you want your learners to leave your course with. How well does your list match up with the course learning objectives? If there is a significant difference between the two lists, is that OK, or should they be reconciled, perhaps removing some from one list, and adding some to the other?

How are you assessing?

Besides considering what you are assessing, reflect on how you are assessing? Do your assessments measure learning and understanding, or just the learner’s ability to memorize content for a few hours or days? Is the course grade calculation structured in such a way that a learner can do poorly in one part of the course, but make up those points in another part of the course, thus producing an average grade? Is it OK that the course grade may “hide” the fact that they did poorly in part of the course? Is it possible for a learner to do so poorly on an exam during the start of the semester that there is no way their course grade can recover, regardless of how well they’ve learned the course content by the end of the semester? Does it matter if a particular learner has not gained understanding by a particular date early in the semester, or just that they have the understanding by the end of the semester?

As you are pondering all of this, I encourage you to check out low-stakes and authentic assessment. Also, review material on the mastery learning model and backwards design. And lastly, consider moving away from a points-based system to other alternative grading systems, such as collaborative grading, and specifications grading. Both approaches help center and encourage the learning that you want to occur, rather than having the learners focus on “what do I have to do to earn points to get a B.” Lastly, review this list of blogs and podcasts to see what you can glean from them.

Blogs

• Faculty Focus: Higher Ed teaching strategies from Magna Publications.
• Grading for Growth: Innovative grading methods that work with rather than against student learning: specifications grading, standards-based grading, ungrading, and more.
• BSU Teaching Innovation Blog: A virtual hub for dialogue and community: learn about and share research-informed, learner-centered pedagogy, from teaching with technology to experimental and active learning strategies.
• Paul Gestwicki's Blog: A blog for reflective practice focused on Computer Science, but applicable to all.

Podcasts

• Tea for Teaching: Informal discussions of innovative and effective practices in teaching and learning.
• Teaching in Higher Ed: Faculty development for professors.
• Think UDL Podcast: Discussions with people who are designing and implementing strategies in post-secondary settings with learner variability in mind.

Time to act!

Since you’ve got to this point in my ramblings, I must have held your interest. What caught your eye? What has caused you to consider doing something different? Do you have other suggestions of assessment-related things we should consider as we prepare our courses? Now’s the time to act. It’s time to assess your assessment.

Image credit: https://stock.adobe.com/

Providing ingredients for learner success

In preparation for a Community of Practice meeting about active learning, our facilitator asked us to read Dr. Dunja Trunk's Faculty Focus post entitled "A Recipe for Student Success." In it, they describe five ingredients needed for learner success, and provide details and examples for each of the five. (I encourage you to take a few brief minutes to read their worthwhile post, if you've not already done so.) The five ingredients are the following.

1. Clear goals and objectives
2. Develop effective study habits
3. Establish effective communication channels
4. Take care of physical and mental well-being
5. Embrace continuous learning and growth mindset

This got me thinking

I fully support this list and have nothing to add to it, other than an an obvious addition—acquire domain knowledge and understanding. The post did get me thinking, however. To what extent are we teachers/mentors responsible for enabling a learner to utilize these ingredients? Our response to this question likely determines how we teach. In particular, should we...

1. Assume the learners know about the ingredients and how to use them?
2. Tell the learners about the ingredients and how to use them?
3. Give the ingredients to the learners?
4. Show the learners how to use the ingredients?
5. Have the learners use the ingredients?

I'll be developing my response to my questions for the rest of this post. I invite you along for the journey, and will be curious to see if you follow me, or take a different path.

Preparing a meal

If we use an analogy of preparing a meal, it seems to me that an individual is not likely to be (consistently) successful simply by reading and following a recipe, unless they have a lot of past experience doing so. So if our learners have little experience or knowledge, we best avoid assuming they know what's needed for the meal. Further, we can't simply tell them the ingredients they need, as they may not know where to acquire the needed ingredients. Also, telling someone how to put icing on a cake is not likely to lead to success for them  the first time. It takes practice; sometimes lots of practice.

So, perhaps we need to be giving our learners the ingredients, and showing how to use them. How much garlic is enough, but not too much? Does the water need to be cold or warm when we add it to the other ingredients? What part of the meal should we start preparing first, if we want everything ready at (roughly) the same time? Preparing a meal is not easy. Getting it right the first time is quite a challenge. Learning is not easy either, nor is getting it right the first time.

But, simply giving our learners the ingredients and showing them what to do with them still doesn't seem like enough. We need them to actually try (and likely fail) to make the meal themselves. It is in the doing (and failing) that learning happens. The more we do, the more practice we have, the better we get.

But, what about chefs?

But, what if we have a learner who is very experienced, and has a broad understanding of the material? I think I still arrive at the same point as I did before. However, I won't be explaining and demonstrating near as much, and certainly would not expect to be discussing or showing foundational ideas and skills. If I'm working with a "chef" to prepare a meal, it is going to be more of a collaboration—the two of us will be sharing ideas and questioning each other. But there will still likely be some advanced knowledge or skills that they do not yet have, and need to observe and practice. And yes, I'll likely learn something from them as well during that collaboration.

My teaching

How does all this impact how I teach and mentor? I've posted about how I teach a few times, here and here, for example. I strive to develop and repeatedly present clear course goals and objectives to the learners. I provide opportunities for them to develop effective study habits. I do my best to have clear and consistent communication with the learners, and encourage them to do so, as well. I consider the individual learner when making decisions, and I am constantly promoting a growth mindset and reminding the learners that making mistakes is fine, as long as we learn from them.

My teaching heavily utilizes the following.

• Active learning
• Flipped learning
• In-class demonstrations and activities
• Small group discussions
• Class discussions
• In- and out-of-class group work
• Learner-centered focus
• Specifications grading, and sometimes ungrading
• Flexible due dates, based on circumstances
• Very little lecturing

I strive to provide a learning environment focused on the learner, not me. I strive to be aware of what the learners do and do not know, and meet them where they are. Sometimes I'm able to accomplish this in class, and sometimes it is accomplished in one-to-one sessions in my office or via Zoom. When teaching higher-level courses, I do expect the learners have more experience, but I try not to assume it. That is, I strive to verify what experience they are bringing to the course. 

I strive to always provide examples of, and opportunities for the learners to practice, the five ingredients needed for learner success that Dr. Dunja Trunk described in their post.

Your teaching?

How do my thoughts and approach to teaching/mentoring match up with yours? What can we learn from each other? Please share your thoughts and ideas so that I (and other readers) can benefit from them. I look forward to it, as I continue to learn how to treach.

Image credit: https://stock.adobe.com/

How I teach

A photo of me doing what I rarely do--lecturing

I was recently asked about how I teach. Obviously, teaching is something I do daily, and I do give it a fair amount of thought, but I’d never been asked this all-encompassing question before—or at least not for a long time. It got me thinking about how I actually do teach, and how I create a learning environment in which learners can learn.

How do I teach?

The person asking the question offered a few prompts to help focus my thinking. The following are my bullet-point responses to his questions.

What strategies do you use to get and keep students engaged in class?

• Active learning: What better way is there to learn, other than by doing, and learning from your mistakes while doing?
• Flipped learning: Let’s apply new knowledge and skills during class, when questions can be asked, and save the lower order of Bloom’s taxonomy for individual learning outside the classroom.
• Learner-centered teaching: Teaching and learning is about enabling the learner to learn, not about enabling the teacher to teach.
• iClicker learner response system: This and similar tools provide a great way to provide (anonymous, if desired) responses and feedback in the classroom.
• Check in surveys throughout the semester: Giving the learners opportunities to reflect on how their learning is progressing during the semester, and what you might be able to do to create a better learning environment can be powerful for both learner and teacher.
• Reflection papers: Like the check in surveys, creating opportunities for learners to reflect on what and how they are learning is powerful.
• Specifications grading (and exploring ungrading): Points are not the only way to represent what an individual has learned. I’ll even go further and say they may be a bad way to do so. I’ve found specifications grading to be a much better option and have recently been dabbling with ungrading. I've previously posted about specifications grading.
  

What strategies do you use to get and keep students engaged in community and to develop an identity within your class? (That is, to encourage engagement in campus life or contact outside class, i.e., studying together.)

• Pair-programming and group work: Have learners work together on a regular basis in class, and the relationships will inevitably flow to outside of class.
• Small group discussions: Taking time to share opinions and understanding with others—and listening to their point of view—opens a communication channel.
• Immersive learning: Learning while doing, and satisfying others needs in the process is a tremendous motivator.
• Expert of the Day: Asking learners to complete work on their own before class, and then serve as a teaching assistant during class helps reinforce their learning. It also provides multiple opportunities to interact with others.
  

What strategies do you use to lecture or otherwise deliver course content effectively?  

• I try to avoid lecturing! I’ve found that talking about a topic for an extended period of time seldom is productive. Learning by listening is not near as productive as learning by doing.
• Flipped learning: See above.
• Active learning: See above, again.
• Discussion (mostly Socratic): For teaching things other than skills, discussion is great—especially if I can avoid stating my personal opinions, but rather provide an environment in which the learners critically think about their own ideas and understanding, share that with others, and learn from others.
• Disability awareness and accessible Canvas course site: If you have challenges accessing the course site—for any reason—your learning opportunities are going to be diminished. Every learner deserves full access to every learning opportunity.
• Diversity, equity, and inclusivity: Understanding what others bring to the classroom provides opportunities to reduce or remove barriers to learning, as well as providing rich opportunities for all to learn.

How do you think about course design (syllabus, sequencing, assessment, assignments, activities, etc.)?

• I think about Understanding By Design (Wiggins & McTighe), but admittedly seldom am successful at fully implementing it due to the time I provide myself. I need to do better.
• I keep Universal Design for Learning (UDL) in mind: If I do something to help one learner, I likely will help many others.
• I tie course activities/learning outcomes to workforce competencies (NACE): Beyond content knowledge (which may become obsolete, given time), learners also need to learn and recognize non-domain transferable skills. Anything I can do to establish the relationships between class activities and workforce competencies will be a good thing.
• I have a very long, but “warm,” supportive, learner-focused syllabus: My approach is to incorporate useful (non course) content into the course syllabus in the hope that the learner will discover some other content while they’re looking for the first thing.
• Diversity, equity, and inclusivity, and having an accessible Canvas course site: See above.

How do you balance rigor with flexibility?

• Rigor is an interesting word! (See www.merriam-webster.com/dictionary/rigor.) Brief definitions follow:

1. harsh inflexibility in opinion, temper, or judgment: severity
2. the quality of being unyielding or inflexible: strictness
3. severity of life: austerity
4. an act or instance of strictness, severity, or cruelty
5. a tremor caused by a chill
6. a condition that makes life difficult, challenging, or uncomfortable
7. strict precision: exactness
8. obsolete: rigidity, stiffness

Based on those definitions, I’m not sure I want rigor. (I’ll admit I don’t have a better word at the moment, however.) Obviously, I do want learners to learn, and want a reliable way to measure that learning.

• Specifications grading: Beyond what I’ve said above, and elsewhere, Specs grading encourages learners to work on assignments and projects until learning occurs, rather than measuring the learning at one predetermined moment in time. I teach so others can learn, not so that I can assign a grade.
• Open/flexible assignments: Having flexibility in assignments can provide options to the learner, and greatly reduce the temptation to copy someone else’s work. If your project is different than someone else’s, you can’t do exactly the same thing. You can share ideas and learn from each other, however.

To what extent, and how, do you keep up with discussions in your field about pedagogy?

• Conferences: I attend and present at as many as I can. Some of my regulars include:

• SIGCSE (ACM Special Interest Group on CS Education)
• CCSC:MW (Consortium for Computing Sciences in Colleges: Midwest)
• Lilly Conference on College Teaching
• The Grading Conference

• Blog and Twitter: I tend to use my blog and social media to be reflective about my academic life and experiences. I also learn from others who do the same thing.
• Professional development:  I’m very fortunate to have a university that is focused on ensuring every faculty member has an opportunity to be the best they can be. I’ve taken advantage of many of these opportunities.

What are the most common barriers to learning for your students? How do you approach these barriers?

• Fixed mindset: If you don’t think you can do something, you’re probably right. I fight this by being supportive, encouraging, and affirming. I also strive to dd “yet” to their vocabulary, as in, “I can’t do it yet.”
• Life gets in the way: Learners often struggle because of “life.” I provide Oops Bits to the learners for them to “spend” when they need to submit something late or miss a class. I’m also flexible with due dates when learners have extreme circumstances occur in their lives.
• Lack of preparation prior to the course: Sometimes learners come into my class without a specific skill or knowledge. To help these learners, I often offer recurring/scheduled office visits. Depending on their need, I may also refer them to our Learning Center or online resources.
• Lack of preparation during the course: To encourage learners to continue to prepare for the course all semester, I administer reading quizzes, offer recurring/scheduled office visits, and refer them to our Learning Center or online resources.
• I tell struggling learners: I will work at least as hard as they do to help them learn the material, thus communicating that they must try, but I will be right there with them.

Let's share and learn

If I were to ask you how you teach, what would be your response? Would it be similar to mine, or completely different? What do you do that works for you and your learners? Please share; let’s learn from each other.

Lilly Conference reflections: 2019 edition

I recently spent three wonderful days at the 39th Original Lilly Conference on College Teaching at Miami University in Oxford, Ohio. This is the fifth time I’ve been able to attend this conference and have been privileged to present at the conference all five times as well. As I said in my post last year, if you can afford the time and money, I highly recommend attending this conference which is held the weekend before Thanksgiving every year.

The last two years, I’ve talked about my experiences combining specifications grading with learner-centered teaching. This year I focused on just specifications grading, leading a three-hour pre-conference workshop and delivering a conference presentation. I've blogged about specs grading here and here. I also volunteered to host a discussion table during lunch on Saturday for the first seven people who signed up to eat with me and discuss specs grading. All three events seemed to go well with lots of participant interaction and positive responses from the participants.

I’ve said it before, but every time I’ve attended Lilly, I’ve come away from the experience rejuvenated. When you spend time in conversations with others excited and interested in finding the best way to teach, how can you not become excited yourself? The conference attendees are a very friendly group of people. One has a sense of attending a homecoming and getting reacquainted with old friends when one attends Lilly.

I always come away from Lilly feeling like I was trying to drink from a fire hose; there is so much good information received in such a short time. I’m still mentally processing this year’s insights, so I’m simply going to provide a list of very briefly annotated quotes I wrote down while listening to presenters. They are presented here in the order in which I experienced them.

• “If you believe the words in your video are important, why wouldn’t you want all people to have access to them?” (Brenda Jo Brueggemann) Seems like a pretty obvious justification for captioning to me.
• “On quizzes, include questions about the new material, as well as previous material.” (Sheri Stover) Retrieval learning reinforces previous learning.
• “Cramming works for a day. A week later, you’ve lost most of it.” (Sheri Stover) Spread out the studying if you really want to learn the material.
• “Education is the most powerful weapon which you can use to change the world.” (Nelson Mandela) What more is there to say?
• “Education is not the learning of facts, but the training of the mind to think.” (Albert Einstein) If you can think, you can continue to learn.
• “Teaching is leading students into a situation in which they can only escape by thinking.” (Unknown) Sounds like a great situation to be in!
• “If we teach today’s students as we taught yesterday’s, we rob them of their tomorrow.” (John Dewey) As we learn of better ways to teach, we must adopt them.
• “In times of change, learners inherit the earth, while the learned find themselves beautifully equipped to deal with a world that no longer exists.” (Eric Hoffer) We must be life-long learners, lest we get left behind.
• “Engagement requires both motivation and active learning.” (Elizabeth Barkley) One of them by themselves is not sufficient.
• “Rewards and punishment is the lowest form of education.” (Chuang Tzu) Carrots and sticks don’t work as well as other approaches.
• “Forget the mistake. Remember the lesson.” (Unknown) Move towards a growth mindset.
• “Unpredictability can generate emotional learning.” (Mike Pinter and Peter Giordano) Shake things up a bit and get people’s attention.

Share your thoughts below. What quotes resonate with you? What ones raise questions? Do you find fault with any of them?

Image credits: All images by Dave Largent

SIGCSE 2019: Where the inspired inspires the inspirer

I attended my first SIGCSE Technical Symposium as a faculty member last week. I attended one other time, way back in 2010, when I was a graduate student. At that time, I delivered a Student Research Competition talk and poster about my thesis work related to small group development as it relates to software development teams. I also served as a student volunteer for the symposium. It was a great experience, but I only experienced part of the full event. This year, attending as a faculty member, I was able to experience the full gamut that SIGCSE has to offer, including being both giver and receiver.

One of the symposium co-chairs is a two-time graduate of Ball State University's Department of Computer Science. Manuel Pérez-Quiñones now serves as the Associate Dean of the College of Computing and Informatics at the University of North Carolina at Charlotte. I first met him when he returned to the Ball State campus a few years ago to deliver a keynote at a diversity research symposium. It was good to greet him again and deliver a “hello” from BSU.

Dave, Christian, and James

I was fortunate to have a workshop proposal about specifications grading accepted for presentation Wednesday evening, at the start of the symposium. (I've previously posted about specs grading.) James McGuffee (Christian Brothers University), Christian Roberson (Florida Southern College), and I jointly authored this proposal, and I am indebted to their help making it a reality. The material we presented resonated well for the thirty-plus workshop attendees, and I believe we made a few converts that evening. We had lots of great questions and discussion, with many attendees staying well past the 10:00 PM ending time discussing possible ways to implement specs grading in their courses. All in all, it was a great way to start the symposium off on a high note.

I enjoyed the keynote presentations. Marie desJardins (Simmons University in Boston) focused on inclusion in computing, ultimately discussing these “Five Pernicious Myths.”

“He was born to be a computer scientist.”
“Computer scientists are… {insert stereotype here}.”
“Anybody can be a computer scientist—girls just don’t want to.”
“It’s a joke—don’t you have a sense of humor?”
“’Diversity programs’ are just political correctness.”

Check out her presentation slides for a wealth of information and resources.

Gloria Townsend (DePauw University) provided a top-ten list that identified the most effective strategies used at DePauw to recently award 47% of their computer science undergraduate degrees to women. Her list included these items, many focusing on women:

10.  Celebrations
9.    Networking
8.    ACW-W chapter
7.    Mentoring
6.    Role modeling
5.    Outreach
4.    Content preview
3.    Social activities
2.    Female CS1 teachers
1.    Encouragement

Mark playing a ukulele

Mark Guzdial (University of Michigan) delivered a keynote in which he posited that “teaching programming as a way to express ideas, communicate with others, and understand our world is one of the oldest goals for computing education,” and that “Computer Science was invented to be a tool for learning everything else.” His justification can be found in his presentation slides. At the end, he gave us a call to action to find our allies and grow the community, and to invent, mutate, and evolve. His blog is well worth your time, so take a look.

I first met Mark when he chaired my paper session at ICER 2010 in Aarhus, Denmark. This was my first international academic conference. To put things into perspective, I started my first semester of full-time teaching at a university a few days after the conference. I presented my thesis work related to small group development as it relates to software development teams. After collecting five years worth of data, I turned it into an ACM TOCE journal article. If I recall correctly, my table-mates for dinner that evening were Mark, Beth Simon, and someone from NSF, whose name I have long since forgotten. Talk about being out of my element! But they were all very gracious, and it was a great dinner. Beth and I talked about peer instruction and the use of clickers. I started using clickers that fall, but it was three more years before peer instruction made its way into my classrooms.

I attended a variety of paper presentations and a workshop. Since this post is already lengthy, and I have one more big thing I want to share, I’m going to skip the details of the papers and the workshop. If you really want to know, ask me about them.

But wait; there's more...

My two presentations served as bookends for my symposium experience; I started with the specs grading workshop and ended Saturday morning by presenting a poster about the CS1 Art Show I’ve been coordinating for six years. For much of that time I anecdotally knew the show had a generally positive impact on our students, the department, and how others view us. This past summer I started collecting data, and now can more formally make that claim, with data to back it up. My poster presented some of that data and described the details and logistics of producing the art show. Included were student and judge comments.

I talked with a dozen or more people during my two-hour poster presentation who seemed genuinely interested in doing something similar with their own course. I remember one person exclaiming “This is exactly what I’ve been looking for!” All these interactions provided a wonderful affirmation that I did something good. But there were three visitors, in particular, who but me over the top, providing a great way to end my symposium experience.

In the fall of 2013 we started using the media computation textbook authored by Mark Guzdial and Barb Ericson. That same semester, I was inspired to hold an art show based on collages our CS1 students were creating for one of their course projects. The idea originated from an article by Beth Simon and Leo Porter about a similar event they produced. I took their idea and modified it to work in our environment.

Fast forward to a few days ago. I wondered if Mark, Barb, Beth, or Leo (all of whom I knew were present at the symposium) might be interested in my research about the art show, so I sent them all a message letting them know I was presenting my poster Saturday morning. They all expressed interest, indicating that they’d try to drop by my poster session, but some of them indicated they had busy schedules. As such, I wasn’t holding my breath.

To my delight Mark arrived and talked with me for 10-15 minutes. While reviewing the presented data, he seemed excited about how learner-centered and engaging the whole art show process is. After he left, Barb arrived a few minutes later. I was talking with another individual at the time, but she very patiently waited until they were done, spending time reviewing my poster. She also provided lots of encouragement about the art show project.

Image by rawpixel.com

It was getting close to the end of the poster session when Leo arrived. We talked at length about the logistics of how I run the show. The more we talked, the more excited he became, and the more questions he asked. I could see a light bulb starting to glow. Now remember, Leo originated the art show idea with Beth when he was serving as a Teaching Assistant at the University of California, San Diego (UCSD), and operationalized it while at Skidmore College. But, after moving back to UCSD, he was faced with very large class sizes, and dismissed the possibility of continuing the art shows. Now, after years away from the art show, he was seeing a way to make the process scale to larger class sizes. Maybe he could produce an art show with large class sizes at UCSD. The light bulb was definitely on at this point, and that made me very happy. What a way to end my SIGCSE Technical Symposium experience.

The inspired (me), had just inspired the inspirer!

Lilly Conference reflections: 2018 edition

I spent the last three days at the 38th Original Lilly Conference on College Teaching at Miami University in Oxford, Ohio. This is the fourth time I’ve been able to attend this conference and have been privileged to present at the conference all four times as well. If you can afford the time and money, I highly recommend attending this conference held the weekend before Thanksgiving every year.

At my first Lilly (2013), I presented a poster with three of my colleagues (Rebecca Pierce, Lynne Stallings, and Petra Zimmermann) titled “Classroom Interaction Redefined: A Multidisciplinary Perspective on Moving Beyond Traditional Classroom Spaces to Promote Student Engagement.” We translated this into a journal article, which was recently published online.

In 2016, I talked about helping computer science students explore the issue of lack of diversity and inclusivity in our educational programs and profession. The last two years, I’ve talked about my experiences combining specifications grading with learner-centered teaching. I've blogged about specs grading here and here.

Every time I’ve attended Lilly, I’ve come away from the experience rejuvenated. When you spend time in conversations with others excited and interested in finding the best way to teach, how can you not become excited yourself? The conference attendees are a very friendly group of people. One has a sense of attending a homecoming and getting reacquainted with old friends when one attends Lilly.

Rather than try to detail everything I absorbed at the conference this year, I’m simply going to provide a list of very briefly annotated quotes I wrote down while listening to presenters. I’m not going into depth partly because I don’t have the time to do so right now, and partly because I’m yet to fully process most of what I experienced. They are presented here in the order in which I experienced them.

• “Get more sleep.” We have to have adequate sleep for our brains to make memories.
• “We can't solve our problems with the same thinking we used to create them.”  Time to think outside of the box.
• “Give authority to students and be prepared to be amazed.” I’ve had this experience many times, as recently as a week ago with honors colloquium students.
• “Memorizing is what you do when something doesn't make sense.” To truly learn something, it has to make sense.
• “What would students do if we gave them wrong or missing instructions?” Should we do this to make them think and question?
• “Students must have past knowledge to which they can connect the new information we give them.” Our brains store new information by attaching it to previous knowledge and experiences.
• “Test for what you want your students to know a year from now.” If it’s not important enough that you want the student to know it a year from now, why test them on it? What does it matter if they know it now?
• “Peer feedback should be thought of as reciprocal teaching, not an evaluation.” This changes the nature of the conversation.
• “Flipped learning may not be evidenced by short term assessments gains but will be reflected in long term knowledge.” Learners can cram for a test and do well but are much less likely to remember it later. True learning is for the long term.
• “We can’t change what we cannot see.” If we don’t know there is an issue, how could we know it needs changed? We have to look and observe.
• “What are the things going on in our students lives? How might those things affect their learning? How could it change our teaching if we knew?” We must do more than simply teach content.
• “Everything we do is a rehearsal for the future.” Extremely few things are ever done only once. Always strive to do better.
• “Impatience is the enemy of empathy.” Take time to understand.
• “Grades should be indicative of the quality and quantity of learning.” The key here is measuring learning, not testing.

Share your thoughts below. What quotes resonate with you? What ones raise questions? Do you find fault with any of them?

What we learned in CS 222: Spring 2018 edition

I've had the pleasure of teaching the CS 222 course since fall 2015. Having just started to blog last fall, last December was my first opportunity to blog about the end-of-semester activity we conduct in this course. This blog entry will report on this spring's course.

The final exam process for CS 222 consists of three parts. First, the students are asked to list everything they can think of that they learned--directly or indirectly--because of their participation in CS 222 this semester. Once the consolidated list is compiled (and cleaned of duplicates), each student has a limited number of votes to cast to identify the items they feel are the most significant. And lastly, each student then selects one of the items which received the most votes and writes a reflective essay about how they learned that item.

As we cleaned the list, we decided to place some individual items under a heading (e.g. Clean Code); it was the heading that could then be voted for. The consolidated list contained 97 items (after creating these consolidations and removing duplicates). Each student had seven votes to cast for the items they felt were the most significant for them. Because of a tie for seventh place, we included eight items in our "Top N List," which ended up including these items (votes shown in parentheses):

• Clean Code: Best practices of code structure (23)
• Test-Driven Development: Using test classes to guide production code (22)
• Git (12)
• Object Oriented Programming Model (9)
• Behavior Driven Development and User Stories (9)
• Stages of group development (Forming, Storming, Norming, Performing, Adjourning) (8)
• Empathy cycle (acceptance testing) (8)
• Plan and track your work (8)

The students in the fall 2017 section of the course developed this list:

• Clean code techniques (15)
• TDD (15)
• Stack overflow (14)
• It’s easier to take the time to write cleanly than to write dirty code and fix it later. (8)
• Code is like art. Anyone can look at code and call it beautiful, but it’s rare to find someone who can tell you why. (7)
• Time-management is a key factor in a lot of things (7)
• Consider what the needs of the end user are when developing functions, rather than developing functions based on satisfying your own goals for your application. (5)
• Navigation of GitHub (5)

Comparing the two lists, I see many similarities. Both lists include references to clean code, TDD, Git, and empathy/considering the users' needs. Clean code and TDD are major themes of the course, both of which are completely new concepts to the students. As such, they struggle with them. Similarly the use of Git and GitHub is new to virtually all students, and causes its own challenges, until they take the time to figure it out. The empathy theme comes from a discussion about a design cycle model a few weeks before the end of the semester. Planning and tracking work, and time-management are related items on each list, but have a bit different focus. Few students have had to manage a project that lasts more than a few days, so it is no surprise this would make it onto the list.

New to this year's "Top N List" is object oriented programming (OOP), user stories, and stages of group development. OOP terminology was on last fall's master list, but only received four votes. User stories received one vote last fall. Stages of group development was mentioned last fall, but did not receive any votes. It is rewarding to have it make it onto this year's list, because I added this topic to the course when I started teaching it. Noticeably missing from this year's "Top N List" is Stack Overflow, although is did receive six votes, so it was close.

There were a few items that I found interesting, but didn't receive enough votes to make the "Top N List."

• How much fun and how rewarding teamwork can be
• What I enjoy working on versus what I don’t enjoy working on
• Implementations are almost always more complicated than they appear at the surface

I like the that a few students now believe teamwork can be a positive experience. The fact the it didn't make the list could be that most students had already figured it out, or maybe they didn't experience the great teamwork these students did. It is always good to know what brings you joy, and what does not. The few students that voted for that item may well save themselves some grief in the future. The last item relates to the "plan and track your work" item which did make the list. Again, few students have had to deal with a "large" project, so there were lots of surprises awaiting them.

Comparing the course grades of this semester's students to those of the fall semester, the grades dropped slightly, though not what I'd consider significantly. Percentage wise, there were fewer Bs, more Cs.

	Fall 2017	Spring 2018
A	42%	42%
B	48%	42%
C	10%	16%
D	0%	0.0%
F	0%	0.0%

Last fall, I started using specifications grading for this course, but the assessable items did not change--only how they were evaluated. I reflected on my experiences with this transition in my previous blog entry. I continued with this method of grading this semester as well. The drop in grade for a couple students can be attributed to one student not "carrying their weight" within their team, and thus was rated low on the final project (which had a significant impact on their course grade), and another student simply not submitting much work, other than the projects.

Of programming projects, specifications grading, students dropping my course, and Charles Dickens

With apologies to Charles Dickens… It was the worst of days. It was the best of days.

My day started by learning that a student had dropped my CS 120 course. Yes, I’m usually a bit saddened when this happens, but I understand there are lots of reasons students drop a course for which they’ve enrolled, and that most of those reasons have nothing or little to do with me. But this time it was different; the initial reason she gave to me for her dropping the course was how I was grading her work. I was particularly saddened and discouraged that I appeared to be the major reason for her dropping the course. Further, since there are few females who enroll in computer science courses, I really like to keep the few who do enroll. To think I might have chased one away was unacceptable to me.

I am utilizing specifications grading for the first time in our CS 120 course this semester. (Read my previous discussion of specifications grading in a different course.) This course serves as our first introduction to programming for our students. I am evaluating most assessable items in the course as complete/incomplete—either they fully met the detailed specifications I provided to the students, or they did not. I am evaluating the outside-of-class, work-by-yourself projects based on an A-F scale, where students must complete certain specifications to earn a given grade. The better the grade, the more items that have to be successfully completed. However, the highest incomplete specification determines your grade.

In the instance of this student, she completed nearly all specifications correctly, and would have received an A, except for two of the lower-level specifications which were required for a grade of C. As a result, I recorded a grade of D for her, as I described would happen in the project specifications document. The items she did not complete should have been relatively easy to complete; that’s why they were at the C grade level. She took exception to this, saying her program generated the correct answer, so therefore she deserved an A, and felt it was totally unfair to receive a D. Further, she stated that she was thinking about dropping the course. I do have some empathy for her stance; it does seem a bit unfair. On the other hand, had she paid a bit more attention to details, she could have easily earned an A. Further, I’ve provided for this sort of situation by providing every student four Oops Bits at the start of the semester. These Oops Bit can be used to resubmit a variety of assessable items which are not completed successfully on the first submission, or can be used to gain an extra day to complete an assignment.

In an attempt to try to help her understand my rationale for establishing the grading for the project in the way I did, I asked her to consider this analogy.

What if you were an excellent employee, and there were few people that could do your job better, but you had a hard time showing up to work on time, maybe even missing some days altogether. Most employers would not be willing to keep you as an employee—even though your work was excellent when you showed up—because they couldn’t depend on you showing up, a fundamental quality of a good employee. They would be more likely to retain the employee they could depend on, even if their work quality was a bit lower.

Up to this point, our conversation about her grade was all via e-mail. I encouraged her to meet with me in person to discuss the situation further, and she agreed. We met yesterday and had a relatively pleasant, healthy conversation for nearly an hour about specifications grading, and how I had structured the specifications for this particular project. One thing I asked her to do was to arrange the items in the project specifications into what she considered to be a better set of requirements for each grade. My intent in asking her to do so was twofold: I was truly curious how a student would arrange the list, and I wanted her to critically think about what qualities of the project justified an A versus a B, etc. She obliged and came up with the following list. The first two columns in this table indicate what is required for a given grade. All items below a particular grade are required as well. If any of the items marked as a grade of D are missing, a grade of F would be recorded. The first column is how I organized the list, and the second column is her ordering. (I’ve arranged the list based on my specifications for the project.)

My grade category	Student grade category	Assessable item description
A	A	Program accurately creates all 31 possible versions of the shifted alphabet
A	A	Program accurately lists all 31 possible translations, based on character shifts of 0 through 30 positions to the right
A	A	At the end of your reflection paper, tell me to what this encoded message translates.
B	B	Program lists all 31 possible translations, based on character shifts of 0 through 30 positions to the right (correct or not)
B	C	Program uses descriptive function and variable names
B	D	Program capitalizes the input parameter value
C	C	Program is formatted/indented similar to the text book authors’ examples
C	D	ALL files for the project are placed into a folder named project0. Compress this folder. Submit this compressed folder.
D	B	Reflection paper completed and submitted according to specifications provided in Canvas
D	C	Program will run in JES
D	C	Program uses the character set provided
D	D	Program will load in JES
C		Program accepts one input parameter

I’ve highlighted the items where there was a difference in our ordering. The items highlighted in yellow are one grade different, sometimes higher, and sometimes lower. The darker highlighting are items with a difference of two grades, again in both directions. I find it interesting that most of the ordering is relatively similar. I also find it interesting to see what this student valued compared to other items.

During our conversation, she shared some about other courses she is taking this semester, and how none of them are required for her major, including CS 120. She is transferring to another university next semester, and is simply taking core courses and electives which will transfer. By the end of our conversation, I felt I’d made some progress, and asked her to give the course three more weeks before making a decision about dropping the course. This would provide her more time to experience the course, and sit for the first exam. By then, she’d have a good idea how the course was going for her. As we parted ways, I believed I’d see her in class today. Unfortunately, she did not attend, and I received word after class that she had dropped. 

Initially, I was rather discouraged to have a student drop my course because of how I am conducting the course, and specifically because of how I am grading. Upon reflection, and remembering what she shared with me during our conversation yesterday, I have come to realize her reason for dropping the course was much more than how I graded her project, and perhaps that was not even the major reason. She was taking an eighteen-credit hour load, had multiple demanding courses which are requiring significant amounts of time, and the courses are not directly required for her major. Dropping CS 120 likely bought her a lot of time to devote to other courses, and is not likely to have a significant impact on her college career. I likely would have made the same decision myself.

So, by now, I’m feeling a bit better than I was earlier in the day.

And then the student for my 5:15 PM appointment arrived.

This student is one who had met with me a few days ago because he felt he was struggling to understand the course material. When I sense a student is capable, but maybe just needs a bit of a push and assurance that they are capable—a shot of self efficacy—I will offer to set up a standing appointment to meet at the same time every week when they can come in and ask what ever questions they might have. Students are always welcome to meet with me anytime our schedules permit, but for some students, having a pre-scheduled time seems to make a significant difference in their progress. He was one of those students, and was arriving for his first appointment.

I expected that he might have a question about one of the recent activities we’d worked on in class. This was not the case. Rather, he came very prepared with a mental list of items and concepts for which he had questions, or felt like he needed clarification. We very productively moved from one topic to another, and I could tell, based on the type of follow-up questions he was asking, that he was gaining a much better understanding of the material. We worked through course material for nearly an hour, that very quickly passed—at least for me. The more I worked with him, the more he impressed me with how truly committed he is to learn and understand the course material. He was a total joy to work with.

So, was it the worst of days? Or was it the best of days?

I’m going to go with the best.