Lilly Conference reflections: 2018 edition

I spent the last three days at the 38th Original Lilly Conference on College Teaching at Miami University in Oxford, Ohio. This is the fourth time I’ve been able to attend this conference and have been privileged to present at the conference all four times as well. If you can afford the time and money, I highly recommend attending this conference held the weekend before Thanksgiving every year.

At my first Lilly (2013), I presented a poster with three of my colleagues (Rebecca Pierce, Lynne Stallings, and Petra Zimmermann) titled “Classroom Interaction Redefined: A Multidisciplinary Perspective on Moving Beyond Traditional Classroom Spaces to Promote Student Engagement.” We translated this into a journal article, which was recently published online.

In 2016, I talked about helping computer science students explore the issue of lack of diversity and inclusivity in our educational programs and profession. The last two years, I’ve talked about my experiences combining specifications grading with learner-centered teaching. I've blogged about specs grading here and here.

Every time I’ve attended Lilly, I’ve come away from the experience rejuvenated. When you spend time in conversations with others excited and interested in finding the best way to teach, how can you not become excited yourself? The conference attendees are a very friendly group of people. One has a sense of attending a homecoming and getting reacquainted with old friends when one attends Lilly.

Rather than try to detail everything I absorbed at the conference this year, I’m simply going to provide a list of very briefly annotated quotes I wrote down while listening to presenters. I’m not going into depth partly because I don’t have the time to do so right now, and partly because I’m yet to fully process most of what I experienced. They are presented here in the order in which I experienced them.

• “Get more sleep.” We have to have adequate sleep for our brains to make memories.
• “We can't solve our problems with the same thinking we used to create them.”  Time to think outside of the box.
• “Give authority to students and be prepared to be amazed.” I’ve had this experience many times, as recently as a week ago with honors colloquium students.
• “Memorizing is what you do when something doesn't make sense.” To truly learn something, it has to make sense.
• “What would students do if we gave them wrong or missing instructions?” Should we do this to make them think and question?
• “Students must have past knowledge to which they can connect the new information we give them.” Our brains store new information by attaching it to previous knowledge and experiences.
• “Test for what you want your students to know a year from now.” If it’s not important enough that you want the student to know it a year from now, why test them on it? What does it matter if they know it now?
• “Peer feedback should be thought of as reciprocal teaching, not an evaluation.” This changes the nature of the conversation.
• “Flipped learning may not be evidenced by short term assessments gains but will be reflected in long term knowledge.” Learners can cram for a test and do well but are much less likely to remember it later. True learning is for the long term.
• “We can’t change what we cannot see.” If we don’t know there is an issue, how could we know it needs changed? We have to look and observe.
• “What are the things going on in our students lives? How might those things affect their learning? How could it change our teaching if we knew?” We must do more than simply teach content.
• “Everything we do is a rehearsal for the future.” Extremely few things are ever done only once. Always strive to do better.
• “Impatience is the enemy of empathy.” Take time to understand.
• “Grades should be indicative of the quality and quantity of learning.” The key here is measuring learning, not testing.

Share your thoughts below. What quotes resonate with you? What ones raise questions? Do you find fault with any of them?

Popular achievements in CS 222

One of the assessible items in CS 222 (Advanced Programming) is something we call achievements. Achievements are designed to encourage student’s independent exploration of relevant course topics they choose from a provided list. My colleague, Paul Gestwicki, introduced these into the course many years ago, and I’ve chosen to keep them. Most achievements are designed to be individual exercises, although some may be attempted by small groups—usually pairs, but potentially an entire final project team.

Each student may earn up to four achievements during the semester; they must complete four if they expect to receive an A for the course. To encourage planning, a student may only submit one achievement claim per week. Students make achievement claims by sharing their artifact in a document in the course’s shared Google drive, to which they have write access, but is closed to anyone outside the course.

Every semester or so, we review the list of available achievements, adding a few new ones, while dropping others. We substantially changed the list after the fall 15 semester, but there is a core set that has remained on the list since. The achievement options have included the following items.

• Campus Leader: For serving your campus community. Help lead or organize a campus club or event that is relevant to our course objectives. Write a reflective essay about the experience. (Added fall 2017)
• Capstone Connector: For investigating the senior capstone. Interview a Ball State University Computer Science capstone team about their project. Write an essay describing the project, its client, and the tools and techniques used by the team.
• Community Connector: For engaging with alumni. Interview a Ball State University Computer Science alumnus about the essential questions of our class. Write an essay about the experience.
• Crystallizer: For being principled about software development. Read 7 Properties of Highly Successful Projects from Crystal Clear with your team before an iteration. Document how you will abide by these principles during the upcoming iteration. At the end of the iteration, write an essay reflecting on the experience.
• Didact: For sharing discoveries. Give a five- to ten-minute presentation to the class to explain something you have learned this semester that is outside the required work. Share any slides or handouts you create in our shared folder.
• Diversity Seeker: For discovering we’re too much alike for our own good. Search for reasons (supported by data/research) why the CS profession, and/or our CS major students are not diverse (based on gender and ethnicity). Identify potential actions that can be taken to make the Computer Science student body more diverse. Write an essay summarizing what you have found. Describe practical steps you believe we can take so that our CS Department will have an environment where all students feel welcomed and included.
• Fair-Minded: For exploring future opportunities. Attend the Cardinal Job Fair, or an equivalent, pre-approved event. Speak to representatives from at least three companies. Write a reflective essay on the experience. (Added fall 2017)
• Filmmaker: For using video production skills to help our community. Create and share a video to help students learn tricky concepts from a 100- or 200-level Computer Science course. I must approve the topic ahead of time.
• Jammer: For making something delicious. Participate in a jam or hack-a-thon event. Share your experience in a short reflective essay, including your product or a link to it, as appropriate. (Added fall 2017)
• Judge: For evaluating art and code. Participate as a judge for the CS 120 All-section Art Show. Write an essay about your experience, including a reflection on how your coding has changed since you were a student in CS 120. (Added fall 2017)
• Open Source Contributor: For contributing to the commons. Contribute to an open source project. Contributions can take many forms, including bug reports, documentation, or source code, for example. Write a reflective essay about the experience, including links to your contributions. (Added fall 2017)
• Reflective Practitioner: For learning from failure. Consider one of your failures from this course and write a postmortem about it. Use the format described by Fitzpatrick and Collins-Sussman, explicitly labeling each step.
• Rereader: For discovering once is not enough. Reread a minimum of four chapters of Clean Code. Write an essay reflecting on what you (re-)learned by reading it again, and why you feel you missed it the first time you read the book. (Added spring 2017)
• Seeker: For learning from a community of practice. Attend a Computer Science Department Colloquium, Google Developer Group Muncie, or other approved presentation, seminar, or conference. Write an essay in which you share your experience and tie it into one or more of our course's essential questions.
• Studious: For learning how to learn. Read Bill Rapaport's How to Study: A Brief Guide and then write a study plan for this course, including a grade goal and assessment plan.
• Third-Party Librarian: For being pragmatic about reusing features. Meaningfully incorporate a third-party library into your final project. Describe the library you are using, why you chose it, and how it is used, using examples from your project. (Removed after summer 2016)

The following table presents statistics from the most recent six semesters I’ve taught the course. I find it interesting to look through the table to see what is popular (and what is not). For example, considering all semesters, the Reflective Practitioner, Diversity Seeker, and Crystallizer achievements have clearly been the most popular. Looking at individual semesters, the Rereader, Seeker, Studious, and Third-Party Librarian achievements have also been very popular.
 

As mentioned above, achievements are designed to encourage student’s independent exploration of relevant course topics. Even though the Third-Party-Librarian achievement was very popular, we chose to remove it because the projects being assigned in the course were requiring the use of libraries. As such the achievement was doing little to cause a student to explore a new topic.

A few of the recently-added achievements have proven to be quite popular. In particular, the Fair-minded, Judge, and Rereader achievements appear to be gaining a strong foothold. I suspect some of them will creep into the most popular list within a semester or two.

Five achievements (Campus Leader, Didact, Filmmaker, Jammer, and Open Source Contributor) have generally been the least popular. We have chosen to leave these as options for students to complete so students with a particular interest or skill in one of those areas will have something tailored for them.

Lastly, I’ve found the number of achievements students choose to complete interesting. While a great many students do complete four, the majority do not. As shown in the table, the average for all semesters is just over three completed achievements per student. Due to the one submission per week limitation, and their lack of planning, some students simply run out of time to get all four submitted.

I believe the achievement system is meeting its intended purpose within the course. I will find it interesting to revisit this a year from now to see how the popularity of individual achievements has changed.

What we learned in CS 222: Spring 2018 edition

I've had the pleasure of teaching the CS 222 course since fall 2015. Having just started to blog last fall, last December was my first opportunity to blog about the end-of-semester activity we conduct in this course. This blog entry will report on this spring's course.

The final exam process for CS 222 consists of three parts. First, the students are asked to list everything they can think of that they learned--directly or indirectly--because of their participation in CS 222 this semester. Once the consolidated list is compiled (and cleaned of duplicates), each student has a limited number of votes to cast to identify the items they feel are the most significant. And lastly, each student then selects one of the items which received the most votes and writes a reflective essay about how they learned that item.

As we cleaned the list, we decided to place some individual items under a heading (e.g. Clean Code); it was the heading that could then be voted for. The consolidated list contained 97 items (after creating these consolidations and removing duplicates). Each student had seven votes to cast for the items they felt were the most significant for them. Because of a tie for seventh place, we included eight items in our "Top N List," which ended up including these items (votes shown in parentheses):

• Clean Code: Best practices of code structure (23)
• Test-Driven Development: Using test classes to guide production code (22)
• Git (12)
• Object Oriented Programming Model (9)
• Behavior Driven Development and User Stories (9)
• Stages of group development (Forming, Storming, Norming, Performing, Adjourning) (8)
• Empathy cycle (acceptance testing) (8)
• Plan and track your work (8)

The students in the fall 2017 section of the course developed this list:

• Clean code techniques (15)
• TDD (15)
• Stack overflow (14)
• It’s easier to take the time to write cleanly than to write dirty code and fix it later. (8)
• Code is like art. Anyone can look at code and call it beautiful, but it’s rare to find someone who can tell you why. (7)
• Time-management is a key factor in a lot of things (7)
• Consider what the needs of the end user are when developing functions, rather than developing functions based on satisfying your own goals for your application. (5)
• Navigation of GitHub (5)

Comparing the two lists, I see many similarities. Both lists include references to clean code, TDD, Git, and empathy/considering the users' needs. Clean code and TDD are major themes of the course, both of which are completely new concepts to the students. As such, they struggle with them. Similarly the use of Git and GitHub is new to virtually all students, and causes its own challenges, until they take the time to figure it out. The empathy theme comes from a discussion about a design cycle model a few weeks before the end of the semester. Planning and tracking work, and time-management are related items on each list, but have a bit different focus. Few students have had to manage a project that lasts more than a few days, so it is no surprise this would make it onto the list.

New to this year's "Top N List" is object oriented programming (OOP), user stories, and stages of group development. OOP terminology was on last fall's master list, but only received four votes. User stories received one vote last fall. Stages of group development was mentioned last fall, but did not receive any votes. It is rewarding to have it make it onto this year's list, because I added this topic to the course when I started teaching it. Noticeably missing from this year's "Top N List" is Stack Overflow, although is did receive six votes, so it was close.

There were a few items that I found interesting, but didn't receive enough votes to make the "Top N List."

• How much fun and how rewarding teamwork can be
• What I enjoy working on versus what I don’t enjoy working on
• Implementations are almost always more complicated than they appear at the surface

I like the that a few students now believe teamwork can be a positive experience. The fact the it didn't make the list could be that most students had already figured it out, or maybe they didn't experience the great teamwork these students did. It is always good to know what brings you joy, and what does not. The few students that voted for that item may well save themselves some grief in the future. The last item relates to the "plan and track your work" item which did make the list. Again, few students have had to deal with a "large" project, so there were lots of surprises awaiting them.

Comparing the course grades of this semester's students to those of the fall semester, the grades dropped slightly, though not what I'd consider significantly. Percentage wise, there were fewer Bs, more Cs.

	Fall 2017	Spring 2018
A	42%	42%
B	48%	42%
C	10%	16%
D	0%	0.0%
F	0%	0.0%

Last fall, I started using specifications grading for this course, but the assessable items did not change--only how they were evaluated. I reflected on my experiences with this transition in my previous blog entry. I continued with this method of grading this semester as well. The drop in grade for a couple students can be attributed to one student not "carrying their weight" within their team, and thus was rated low on the final project (which had a significant impact on their course grade), and another student simply not submitting much work, other than the projects.

Of programming projects, specifications grading, students dropping my course, and Charles Dickens

With apologies to Charles Dickens… It was the worst of days. It was the best of days.

My day started by learning that a student had dropped my CS 120 course. Yes, I’m usually a bit saddened when this happens, but I understand there are lots of reasons students drop a course for which they’ve enrolled, and that most of those reasons have nothing or little to do with me. But this time it was different; the initial reason she gave to me for her dropping the course was how I was grading her work. I was particularly saddened and discouraged that I appeared to be the major reason for her dropping the course. Further, since there are few females who enroll in computer science courses, I really like to keep the few who do enroll. To think I might have chased one away was unacceptable to me.

I am utilizing specifications grading for the first time in our CS 120 course this semester. (Read my previous discussion of specifications grading in a different course.) This course serves as our first introduction to programming for our students. I am evaluating most assessable items in the course as complete/incomplete—either they fully met the detailed specifications I provided to the students, or they did not. I am evaluating the outside-of-class, work-by-yourself projects based on an A-F scale, where students must complete certain specifications to earn a given grade. The better the grade, the more items that have to be successfully completed. However, the highest incomplete specification determines your grade.

In the instance of this student, she completed nearly all specifications correctly, and would have received an A, except for two of the lower-level specifications which were required for a grade of C. As a result, I recorded a grade of D for her, as I described would happen in the project specifications document. The items she did not complete should have been relatively easy to complete; that’s why they were at the C grade level. She took exception to this, saying her program generated the correct answer, so therefore she deserved an A, and felt it was totally unfair to receive a D. Further, she stated that she was thinking about dropping the course. I do have some empathy for her stance; it does seem a bit unfair. On the other hand, had she paid a bit more attention to details, she could have easily earned an A. Further, I’ve provided for this sort of situation by providing every student four Oops Bits at the start of the semester. These Oops Bit can be used to resubmit a variety of assessable items which are not completed successfully on the first submission, or can be used to gain an extra day to complete an assignment.

In an attempt to try to help her understand my rationale for establishing the grading for the project in the way I did, I asked her to consider this analogy.

What if you were an excellent employee, and there were few people that could do your job better, but you had a hard time showing up to work on time, maybe even missing some days altogether. Most employers would not be willing to keep you as an employee—even though your work was excellent when you showed up—because they couldn’t depend on you showing up, a fundamental quality of a good employee. They would be more likely to retain the employee they could depend on, even if their work quality was a bit lower.

Up to this point, our conversation about her grade was all via e-mail. I encouraged her to meet with me in person to discuss the situation further, and she agreed. We met yesterday and had a relatively pleasant, healthy conversation for nearly an hour about specifications grading, and how I had structured the specifications for this particular project. One thing I asked her to do was to arrange the items in the project specifications into what she considered to be a better set of requirements for each grade. My intent in asking her to do so was twofold: I was truly curious how a student would arrange the list, and I wanted her to critically think about what qualities of the project justified an A versus a B, etc. She obliged and came up with the following list. The first two columns in this table indicate what is required for a given grade. All items below a particular grade are required as well. If any of the items marked as a grade of D are missing, a grade of F would be recorded. The first column is how I organized the list, and the second column is her ordering. (I’ve arranged the list based on my specifications for the project.)

My grade category	Student grade category	Assessable item description
A	A	Program accurately creates all 31 possible versions of the shifted alphabet
A	A	Program accurately lists all 31 possible translations, based on character shifts of 0 through 30 positions to the right
A	A	At the end of your reflection paper, tell me to what this encoded message translates.
B	B	Program lists all 31 possible translations, based on character shifts of 0 through 30 positions to the right (correct or not)
B	C	Program uses descriptive function and variable names
B	D	Program capitalizes the input parameter value
C	C	Program is formatted/indented similar to the text book authors’ examples
C	D	ALL files for the project are placed into a folder named project0. Compress this folder. Submit this compressed folder.
D	B	Reflection paper completed and submitted according to specifications provided in Canvas
D	C	Program will run in JES
D	C	Program uses the character set provided
D	D	Program will load in JES
C		Program accepts one input parameter

I’ve highlighted the items where there was a difference in our ordering. The items highlighted in yellow are one grade different, sometimes higher, and sometimes lower. The darker highlighting are items with a difference of two grades, again in both directions. I find it interesting that most of the ordering is relatively similar. I also find it interesting to see what this student valued compared to other items.

During our conversation, she shared some about other courses she is taking this semester, and how none of them are required for her major, including CS 120. She is transferring to another university next semester, and is simply taking core courses and electives which will transfer. By the end of our conversation, I felt I’d made some progress, and asked her to give the course three more weeks before making a decision about dropping the course. This would provide her more time to experience the course, and sit for the first exam. By then, she’d have a good idea how the course was going for her. As we parted ways, I believed I’d see her in class today. Unfortunately, she did not attend, and I received word after class that she had dropped. 

Initially, I was rather discouraged to have a student drop my course because of how I am conducting the course, and specifically because of how I am grading. Upon reflection, and remembering what she shared with me during our conversation yesterday, I have come to realize her reason for dropping the course was much more than how I graded her project, and perhaps that was not even the major reason. She was taking an eighteen-credit hour load, had multiple demanding courses which are requiring significant amounts of time, and the courses are not directly required for her major. Dropping CS 120 likely bought her a lot of time to devote to other courses, and is not likely to have a significant impact on her college career. I likely would have made the same decision myself.

So, by now, I’m feeling a bit better than I was earlier in the day.

And then the student for my 5:15 PM appointment arrived.

This student is one who had met with me a few days ago because he felt he was struggling to understand the course material. When I sense a student is capable, but maybe just needs a bit of a push and assurance that they are capable—a shot of self efficacy—I will offer to set up a standing appointment to meet at the same time every week when they can come in and ask what ever questions they might have. Students are always welcome to meet with me anytime our schedules permit, but for some students, having a pre-scheduled time seems to make a significant difference in their progress. He was one of those students, and was arriving for his first appointment.

I expected that he might have a question about one of the recent activities we’d worked on in class. This was not the case. Rather, he came very prepared with a mental list of items and concepts for which he had questions, or felt like he needed clarification. We very productively moved from one topic to another, and I could tell, based on the type of follow-up questions he was asking, that he was gaining a much better understanding of the material. We worked through course material for nearly an hour, that very quickly passed—at least for me. The more I worked with him, the more he impressed me with how truly committed he is to learn and understand the course material. He was a total joy to work with.

So, was it the worst of days? Or was it the best of days?

I’m going to go with the best.