S4xE7: Case Study: Peer Instruction
Episode Summary
In this episode, we got to continue talking to Dr. Beth Simon about peer instruction from the prior episode’s peer instruction Teaching Practice Byte. Our host, Kristin Stephens-Martinez, shares her experience with peer instruction and asks Beth for help to improve. The episode ends with three main takeaways that Kristin has since used in her course.
You can also download this episode directly.
Episode Notes
Craig Zilles, David P. Bunde, Jaime Spacco, Cynthia Lee, Leo Porter, and Cynthia Taylor. 2023. Spiffy Peer Instruction Questions. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 2 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 1226–1227. https://doi.org/10.1145/3545947.3569590
https://peerinstruction4cs.com/
Transcript
[00:02] Kristin: Hello and welcome to the CS-Ed podcast, a podcast where we talk about teaching computer science with computer science educators. I am your host, Kristin Stephens-Martinez, an Associate Professor of the Practice at Duke University. Rejoining me today is Dr. Beth Simon, teaching professor in Education Studies at UC San Diego, formerly teaching professor in Computer Science and Engineering. Beth, thank you so much for coming back on the podcast.
[00:25] Beth: Hi, Kristin. Thanks for the invitation. I’m super excited to be sharing my experiences with peer instruction with you and your audience.
[00:30] Kristin: So today’s episode is a continuation of the teaching practice byte on peer instruction. In the first episode, we talked about what peer instruction is, and Beth was generous enough to agree to record a second where you and I are going to discuss what I do and coach me through how to make my peer instruction even better. If you don’t know what peer instruction is, I strongly encourage you to listen to the episode right before this one in the podcast feed. But otherwise, could you provide a brief, like 1 to 2-sentence summary of what peer instruction is to refresh everyone’s memories?
[01:03] Beth: Sure, peer instruction, even though it sounds like a very general technique, is actually a very specific technique by which we implement a flipped classroom. And so just students um prepare outside of class, learn some of the easy things. As an instructor, we create multiple choice questions for students to engage with in the classroom, and each question is engaged with in a specific protocol. If you put the question up, you ask students to individually think about it in some way, record their answer, and then you have them discuss that uh discuss a question in groups, talking about both the correct answer and the incorrect answers. And then you ask for everybody to vote again, we sometimes call that the group vote. And then, at that point, you can have, lead a class-wide discussion and clear up any misconceptions that students might have.
[01:50] Kristin: Mm OK. So I think that we can start with me giving you a rundown of how I do it and a little bit of the context, and then you can, you can tell me what I’m doing wrong and help me fix it. So I teach intermediate data science, which is basically for it’s, it’s not just for majors, it’s for anyone who’s taken the CS2 or someone who’s learned Python in our CS1 and taken a stats class. So a lot of different levels of preparation are coming in, and it is a flipped class and also hybrid because I am that kind of crazy. So the students have to watch videos, there’s a couple of readings, most of its videos and then there are what we call prepare quizzes where like they have to take these quizzes, they get three tries. They need 80% to get to full credit. So it’s pretty low barrier. It’s just to like do this, this thing. And then I actually do just in time because, again, I am that level of extra where I look at how the students did in the quizzes. And I’m looking at across all of their attempts, like what did they do wrong on the first try, second versus third and I order my peer instructions basically by what they did worst on. So like, what they did worst is what we’re talking about first and then in that order, and there’s often a little bit of a mini-lecture after each peer instruction. But kind of like, all right, you all did worse in understanding about X, Y, and Z. So we’re going to do a peer instruction on that first, everybody. So we do that, and then based on how things go, I either quickly go through the mini-lecture afterwards or I like go through it in more detail depending on what wrong answers they chose.
[03:37] Kristin: My policy is that they need 80% of the possible points across the whole semester to get full credit. I think some students hate that because there are some weeks where there’s more peer instructions than other weeks. So I kind of like the idea that you mentioned in the prior episode about going like you need a certain number of the like submissions each day, and then you get one point for that day, I might adopt that to make it so that they’re less, like, upset about, like, what they were eight last week and there are only three this week. And I’m like, that’s just how it goes sometimes. And then during the peer instruction, I have the students vote individually, and I’m going to, we’ll set the hybrid part aside for now. We won’t talk about that part. I’ve recently changed things there that not many of them prefer to attend hybrid. I’ve clearly signaled hybrid is when you are sick, you’re an athlete, and you’re traveling, it is not because you don’t feel like schlepping to the lecture room.
[04:36] Kristin: So I have the students fill out Microsoft forms. So, I actually capture their, their email address. So I know who they are and I can get award points as appropriate and then if less than 75% get it right, then I have them discuss, but I don’t do that if they don’t, if they get over 75%, which rarely happens nowadays because it feels like I’ve, I’ve hit the points where the questions are hard enough that like they struggle with getting that marker. Um And then they discuss, and then I do a round two. Usually, I open it within like 30 seconds, one minute of having them start discussing because then I’m because I usually get up and I wander around the room during the discussion to try and get students who are not talking to anyone to either talk to me or talk to their neighbor. Um, and then what I usually do is I, since I know that 140 filled out round one, I say, like I will like, we’ll start, we like close out round two when 120 fill out round two.
[05:39] Kristin: And I open the form right when I use it, and I close the form right when I’m done using it because it’s, it’s to prevent the students from like filling it out later or something like that. So no, it’s like right when I’m using it, that’s the only chance to get. And then when students complain, I’ll be like, you only need to fill out one of the two, round one or round two to have earned the point for that peer instruction. Um, But that then leads to problems with round two. Sometimes they’re like, oh, I already did round one. I don’t, I can I like check out for the rest of this like whole thing. Uh, And is that it? Let me think if there is any other pieces to my peer instructions from there for the for the distractors for the wrong answers. I did get feedback this semester where students appreciated when I talked about the wrong answers. Like if you chose this wrong answer, it could be that you don’t understand X, Y, and Z um, and they appreciated that. So I started doing that more.
[06:35] Kristin: Sometimes my wrong answers are they, they have a reason there, but they don’t, they don’t have a misconception attached to it. So like, for example, I have the students apply how the neural net function is working, like apply this, remember this function here’s the inputs, what’s the output? And so all the numbers available to them are every possible number that could be involved in this calculation. And so like all the input numbers are there, all the weights are there, like all of those numbers are there. And it’s because I’m like, well, I have seen students just like just pick these random numbers that are like visible to them. And so like, let’s see how many of them do that in the end. So then there’s like 10 options, which I know is not the best. But it’s also one of those things of like if a student, I want a student to pick an answer and then signal to me like this is the one I’m like, I didn’t, I don’t know what’s going on. So, I’m just going to pick this number. Hm. Um, All right. I think that was all. So tell me what I’m doing wrong, Beth.
[07:39] Beth: All right. A few things I would comment on. Um, First off, your talk about like, you know, choosing to have a group discussion versus not um, based on the 75% mark. So great that you’re rarely hitting 75% the first time because that means you are making the questions hard enough. And so I think that shows expertise on your part. I will say that for anybody else who’s looking to pick up peer instruction for the first time, that can be the thing that’s very, very hard to do. One of the reasons that I always, two reasons I always say have the group discussion is just because people pick the right answer does not mean that they thought the right things about how to explain that. And we know that students going into industry are often, you know, they say that they’re lacking their presentation skills, their communication skills. And I tell students that, you know, we’re helping build your communication skills, and this is what employers say they want. So that’s why it’s really important that you all discuss.
[08:31] Beth: The other thing that I think is really valuable. And I’ve heard from students, and as I start looking into more of the psychological and neuroscience behind this is students like knowing what’s going to happen next, right? So if they’re like, if they just know there’s always going to be that discussion time and so they can just turn and ask others, it reduces their stress and, I think, allows them then to focus more on their learning. So, I think, it’s important for them psychologically to have that time. It’s also much easier as us on instructors, like, don’t have to do quite as much thinking in the classroom. Just do the discussion. You don’t have to be, like, ah, do I need to do this? And that’s especially important for, I think beginning faculty who are struggling to discuss. So, group discussion. You said 30 seconds to a minute for reasons that we don’t need to go into. I have spent an obscene amount of time going over my own podcasts, editing them up for purposes that, again, are not important. But I now I, I can tell you I spend more than a minute on discussion. It’s probably 2 to 3 minimum, minutes on discussion. But that also depends on like how engaged everyone is in discussion.
[09:48] Kristin: Let me be more precise. I turn on round two within like 30 seconds or a minute. The form, like I open the form because like the forms are only open when we’re actually using them and shut them down.
[09:58] Beth: Right so you want to give them that opportunity.
[10:00] Kristin: But the discussion can go for another like five minutes. I turn it on so that I’m like, I, I turn it on. Then I walk away from the podium. I’m walking around the whole room and then this and then I pop also it up a little bit so that they can I I I show just the top part so they can see how many people have submitted so far. They also know that the form is open.
[10:22] Beth: Ok. That makes complete sense. Sounds like a very good practice. Absolutely. And also the whole walking around the room while they’re discussing, even, like I’ve, um, the craziest thing I ever did is I lectured to 547 students. We have no lecture hall that serves 547 students. I had two 250-person lecture halls side by side, separated by a door and a room that was on the flat on the top that had like 20 people. And I, I was simulcast across and then I had TAs wandering the aisles with microphones. But anyway, I would go up and down the aisle. I would come in the other thing and you know, I did get a lot of comments like I was a little bit like Ellen, partially because I have short hair, but also it was like the Oprah walking the aisles, you know, depending on your age, you can pick your favorite telehost, whatever. But yeah, you do want to walk up and down the aisles and the students like actually kind of get to know you if you have a large classroom. I think it’s really great to bring undergraduate tutors in and to go up and down the aisles and encourage that discussion and be like, whatever.
[11:24] Kristin: Yeah, I feel like also every school has its own culture, right? And it feels like at Duke students. I struggle a little bit with the, like, walking around and talking to students. Um, and I think it’s one of those few things where it feels like the beginning of the semester is so important for setting that tone, and then I never get it right. And so, the rest of the semester, this little piece feels like a dumpster fire. And I’m like, no, we’re going to try and get this right this time.
[11:53] Beth: Good for you for continuing to try.
[11:55] Kristin: And so what it boils down to is I end up wandering around, and this is also just my own introversion, like hating having to walk around and talk to students.
[11:53] Beth: Same here. That’s why I hire tutors. They’re much better at it.
[12:06] Kristin: Yeah. My TAs, many of them are not available during class, and they also seem to have strong introverted tendencies at the same time. Um, so I wander around. I sit down next to someone who’s like, got nobody near them because they partially sometimes purposely are sitting so that no one is near them, and then I talk to them. Um, and it feels like I’m not sure how to, to move past this, to try and get them to talk to each other more. Do you have any advice on that one?
[12:41] Beth: I do. And by the way, I am with you. I’m totally introverted. People don’t believe it, but it’s very true. I only play an extrovert when I stand in front of the lecturer hall, but I don’t, I don’t really like that and I don’t actually, I never get to actually have interesting discussions with students. They’re usually like, it’s like, is there a midterm in this class? I’m like, you know, that might not be the most interesting thing to ask me right here. But I, that’s why I do. I hire undergraduate tutors to do that.
[13:07] Beth: But I really think the big difference, and I know that hardly any of my colleagues at UCSD do this, is that I assigned them into groups. So, if you come to the UC San Diego campus and you go into any lecture hall, you’re going to find that every seat has a letter-number combination on it. Just like at the theater, we have A1, A2, A3, A4.
[13:28] Kristin: Oh, we don’t have that.
[13:30] Beth:I know. Well, they didn’t either. And I asked for it in one classroom, and then they’re like, why do you want this? And I explained it and they thought it was really amazing. And then I think there must have been some extra money or something, and suddenly it was everywhere. And so, and then they also created diagrams of the seating layout and put them online where all the faculty could access them. And I literally draw little circle. I used to like draw a little circles around it and make little groups of three, and I assign students into there. Now, students will hate this, and I tell them this. You are going to think this is the worst thing ever. I ask you to give me two weeks and tell me if you can’t stand your group, if you can’t stand where you’re sitting, if you, whatever, if this isn’t working for you, you let me know and we’ll work something out. Nobody has ever asked to stop doing this. Oh, wait, actually, there was one, there was one person.
[14:23] Beth: Anyway, the reason I do this is, again, I was very much not um the kind of person who would raise my hand in class, even the small classroom. So I didn’t, I wanted to take away the stress of having to be like, find a group. Can I talk with you? You know, that’s just, that’s not good. The other thing is I started doing this in a CS1 class, and we all know the issues of lack of females in uh introductory programming and like feeling like you don’t fit in and I’ve had that my entire life. So, I would sort my class roster by gender. So I’ve got M’s and F’s, and then because I’m lazy, I would just start to send the seating that way. What that would mean is so any group, they’re all sitting in one spot. The first time I did it, I just did two men and a woman or two women. If there was going to be a woman in the group, there had to be two out of three. And they still had girls coming to me saying so and so won’t listen to me. So, you know, they got or whatever. So then I’m like, forget it. I’m just, it’s easier to sort anyway. So I sort, and then I do it, and then I end up all the women end up sitting in one spot in classroom, which I don’t know if they’ve ever noticed, I’ve only had one student though, like, halfway through the term. Like, did you notice all the girls are sitting up there? I’m like, hm, that’s interesting. And then I walked away, but I definitely, you know, I know the girls did notice that they were in groups with girls. Um, I don’t know if they noticed they were all sitting in the same area but I, I do wonder if that had any influence on their perception of being alone in the class because they all grouped together.
[15:54] Beth: But I absolutely do required seating would do it. Even if it wasn’t C1, I might not sort by gender if it was, you know, beyond CS1. I think when I did the CS2 I didn’t, um, often there’s some other things that go around with that. But what I think is really important is I take time, and I say you need to get to know the people in your group because you’re going to talk to them all term long. And that also takes away this, like putting yourself out there in front of other people and you, you, you’re forced to make some friends-ish. At least friends from being in the classroom. So I think it’s a, it’s a big, can be a big lift if you don’t have numbered seats and you do have to explain it to students about this is, you know, we’re doing this because I don’t want any of you to feel left out. I don’t want, you know, I’m doing this for very real reasons. But I think it’s a big thing that changes that dynamic of expectation of discussion. I used to, now, here’s a step that maybe is not needed too far. I used to when I was very worried at the beginning about students actually having equal voice in the group. I would say here’s three icons. We’re the Tritan. So I had a Tritan, I had a bug, and I had a Java coffee cup, and I was like, OK, these are the three icons, and each lecture, period, one icon is gonna be selected. And that’s the person who’s leading the discussion that day in your group. Everybody decides for the whole term. Are you the Java Cup? Are you the Tritan. Or are you the bug? And whenever, and I would have that person say and say, OK, who’s the bug? Raise your hand at the beginning of every class? OK, you guys are leading the discussion today I stopped doing that, and I don’t, you know, I don’t think that I think that was maybe a little bit of a step too far, but it was also, again trying to keep this open level playing field and I do think that, that just putting them in assigned groups where they’re with the same people solves a lot of the problem that would, you know, that you are experiencing, trying to feel like I really want them to engage and they’ll engage with each other then and then you don’t have to, yeah, to be honest.
[17:48] Kristin: I might go and investigate the, the room that I will likely be teaching in the fall to see if there’s numbers. I don’t think there are, but at the same time, it’s not that hard to, like, pull up a grid and you’re like, there’s this many seats across this is your spot. And yeah, I’ll have to think about how to, how to handle that because I, I suspect that I probably will get somebody who was like, I don’t want to be there. I don’t want to like, after two weeks, like, I don’t want to do this.
[18:15] Beth: Yeah. Well, so what I would do is keep the front row if you’ve got enough capacity, like, don’t assign the front row and save that for people. Like, you know, you’ve got a student who’s got hearing issues or whatever and their group gets to move to the front or if people are like, oh, if you don’t want to be in your group, that’s fine. You know, this is where there are open seats. I don’t know about you, but we’re pretty much like, we’re like, our seating is full up, man. So it’s like if they’ve got to sit in the front versus discussing their group and, you know, somebody comes up, you know, there’s this guy in my group, he just never discusses. And I’m like, you know, I appreciate that you’re trying and whatever, but don’t worry about it. It’s not, you know, your problem to deal with, and if they don’t want to, to engage in the learning, that’s OK. Are you good with your other partner? Yes. OK. Fine. And I make it groups of three to deal with absenteeism and things like that, that way you’ve always got something.
[19:04] Kristin: Okay, I’ll have to sit with whether or not I want to impose that level of structure. I agree with the idea that it very much reduces a bunch of anxiety by me being the awkward one and going like, I’m going to be the awkward one by forcing you all to sit in certain spaces and telling you you got to get to know your part like the triple that you’re in because of education research says so but I, I have to think about if I want to, like, go that far because I don’t know, I, I still have moments of wishing that I could win the popularity contest among the, the teaching faculty. And I know that I never will, but it’s still moments of like, I don’t like the fact that my, like, rate my professor, even though I don’t look, there very often is so polar, like, bifurcated and like how people perceive me.
[19:53] Beth: Yes, I understand that completely. And I think it was a point in my career, two things happened that made me really embrace that. So I started teaching in CS1, which is our majors, right? And then this thing came along um where I was asked to prototype the new AP computer science principles course for high school, which is essentially a CS0. And I was then teaching that to all of the psych majors on campus, which at the time is the largest major were required to take that class. They previously been doing like Excel and other boring, whatever. And um, and here I’m going to make them program. Yeah. And so I’m like, and and again, also doing this, this is the first quarter where I did this five, this three room 500 whatever. And I’m like, OK, yes, Beth is doing all the crazy things. So, you know, we’re peas in a pod there.
[20:43] Beth: So I hired a bunch of my undergraduates who were in my freshman course the previous year. And I said I need you all to help me with this because you understand why peer instruction is valuable. And I’m going to have to do this and I need your input on it. And they gave me a variety of important things. They said, first off, you can’t make it just be 5% of the grade. It needs to be 10% of the grade because that communicates to the students. This is important, and I’m like, yeah, but people and whatever, and they’re like, you can give exceptions. You know, the student is doing chemotherapy. No problem. They don’t, they can just get the five points. It doesn’t matter or 10% if you don’t make it so that you can’t get an A without coming to class, then you are not telling you are, you are failing them by not communicating how important this learning experience is. And you have the research to show it, and I was like, whoa, the students told me that. Ok, I will do that.
[21:36] Beth: The other thing is they said, yes, you have to have the numbered groups. It’s psych majors, so they’re half women. So we didn’t have to do the women and men thing. But they’re like, no, you need to, you need to do that, and they’re like, and we need to be there. It can’t just be you walking up and down. We need to be there. And I would assign one undergraduate tutor to a group of 30 students. I’m like, you own this row, these 2-3 rows. And I, at one point I even like left blank rows for them to wander up and down. And I had them introduce themselves to their group and be like, they’re the ones there.
[22:09] Beth: The other reason was because we were in the multiple rooms. But we have students share out. This is something I think more people should do. We had students speak into microphones because they’re in multiple rooms. And you know, lecture halls are set up so that we at the front can project, and it’s really hard for the voices in the audience to come through. What was better about the microphones for everyone is now nobody’s looking around to be like, who’s talking and looking. And I, you just, they put a, a handheld microphone in front of them. I call on that, and they’d be like, I’d be like group, microphone two you’re on, and they flip it on, they go hello and be like, hi. And the voice just comes out of everywhere, and nobody turns to look, and so you’re very, you’re like really anonymous. And so now you’re not having to scream, you’re not having, you know, you’re just speaking in your normal voice, and you’re completely anonymous, and these voices just come out. And I think it was honestly the best peer instruction experience I probably ever offered because again, I was very much worried about um, convincing these students it was valuable, et cetera. But it’s, that’s one of those things that’s gonna be a step really a step far for most people to be able to get wireless microphones. Uh uh you know, available in their classroom to be handing around.
[23:24] Kristin: What’s nice is I don’t actually have to worry about microphones. I do have microphones in most of my rooms that I get assigned. I think the and I think one thing that might convince me to do the assigned seats is that then I can say group five without having to call out a person’s name. Which would be great. So do you have any other thoughts about how I do peer instruction to make it better?
[23:52] Beth: No. And to be honest, I think most of the really critical components for really making it work. I would say I would encourage you to reduce those number of options. So, the way I think about my misconceptions and like how many other options I have, if it’s not interesting to discuss and have an aha moment around a particular misconception, then I don’t want it to be there. And that way, I can tell all students, it’s important for you to not just to discuss every answer option because I’m obviously not going to put this exact same question on the exam, but I might just tweak it. So that one of the other answers is right. So that’s another way that incentivizes them to discuss it. And then also, you know, really makes it so that I think three is actually the perfect number of answers because I’ve even been in a classroom and I’m like, I don’t want it like, I’m watching a friend of mine or something. I’m like, I don’t want to discuss all those other options. And then, if there’s five other options, you don’t even know which, which one should discussed. And so I feel it’s a way again with guiding the learning and the conversation to include just the ones that are really important to know. And I’ll, I’ll definitely attribute um Cynthia Bailey Lee really gave me a lot to think about in this regard because she started using peer instruction in theory of computation. And so maybe you’re starting the halting problem, you know, you have a lot of, you know, halts, does not halt or true or false, you know, and I’m like, how can that be good? It’s just 50/50. And it’s because it’s not 50/50. It’s about the discussion, the explanation of why it’s that and why it’s not that. And so I was, I really learned a lot, and uh her questions are available on our peer instruction. The number four cs.com website, look at those theory of computation questions, and you will be blown away.
[25:48] Kristin: Yeah, I think my voice is really going now to, to the audience. I have a cold, in case you can’t tell. Get to sound funny the whole time. How do you train your TAs to effectively wander around? Though, I guess since you are assigning them a set of students, it’s probably a lot easier. Do they also have discussion? Or back in the day did they have a discussion? They also were teaching those same students or anything like that?
[26:17] Beth: Right. So this is something that maybe is, well, maybe it’s unique to UCSD, I think it’s fairly common amongst UCSD. So you have a certain dollar amount of instructional assistant support that you can have. And if you choose to have a TA, they are expensive because you’re paying more, and then you’re also paying tuition remission. But if you choose to have undergraduate tutors and um they are much cheaper, and tutoring at the undergraduate level is exceedingly prestigious at our institution and always has been, so the best students want to do it. I have found I need, you know, with a big class, I need one TA to manage the undergraduates and to manage, you know, high-level things, but they may never even come to the classroom. I encourage them to if they’re interested in teaching, to learn about that.
[27:03] Beth: So the people I have going around and encouraging discussion are in fact, undergraduates themselves who have chosen to do this because they learned so much through the peer instruction process. So they are the best sellers of, you know, like students may even ask them like I “meh meh why are we doing” and they’ll be like, hey, you don’t understand this is great. This is so much better. I promise you’ll never fall asleep in this class. Um All that sort of thing. So I, I do believe that the undergraduate tutor model, especially at the lower division level is um really, really helpful for a wide range of things. But um, motivating the other students is, is important and, and going ahead and pushing them, you know, some of them are like, I’m kind of introverted too and I’m like, that’s fine. But you just got to pretend for the next 50 minutes that you aren’t. That’s, that’s what we all do. And like, just remember it’s, you got to convince them that this is going to be helpful to them if they, if they really engage in the classroom as students have sold me on their evaluations. I didn’t even have to study for the exam, cram for the exams because I already knew the material. Yes, I won. I won. I put the one so that it is, it is, it’s when students say things like that the other.
[28:16] Beth: Oh, here’s a bit of advice that might help you with your setting the tone thing. I do ask students in a required end of the term, like for 1% point of the overall course grade, you have to complete a learning reflection because I’m like, you need to reflect on what are the things that you did in this class that supported your learning and how which, you know, how did you know, how did homeworks either work or not work for you and, you know, help them think about the future. But I asked them to tell me, like, tell, give me a, tell me why peer instruction worked for you in the classroom. And then I get quotes from that, and I throw those up, the, I throw those up, not only the first day I put one up every week, it’s that slide that’s sitting up at the beginning is what a student said about peer instruction. And I think that is a bigger sell um, for them than anything I could say. Their, their fellow students said, you know, like I didn’t even have to cram for the exam. I already knew it by the time I got there, or I actually met someone in this class. And now, you know, we’re, we’re taking our classes together and things like that.
[29:20] Beth: So, um having, having students reflect on their own learning, I think, is something that we often don’t think about as computer science instructors, even those of us who care a whole, whole lot about learning. But taking that time at the last class period to have students do that, um to share some things out to come back and be like, hey, let’s take a second to think you did this weird thing with us all quarter one. Why did we do this? And you’d say that? And the funniest thing. You know, sometimes I just, you know, you don’t know what else to do in the last class. So I’m saying that, but our student affairs officer was a, had recently graduated student, and her younger sister was in my class, this psych majors class. And she told me that her sister was really influenced by that statement in that last class period because she’s like, by that point, you know, she’d really come to believe in you, and she, you know, yeah, this was all good. But then you told her again the research and why and whatever, and she’s like, right, I mean, students can really understand it better at that point um, about why it is. And I think, you know, just in general, in terms of helping students be able to go out and become better learners as they, you know, as they leave our class, that that’s a, a, an opportunity that we should be taking more advantage of. And it’s important, I think, getting to know we have a cognitive science department in our university and getting to know some of the faculty there and like looking at the courses they offer and really gave me more confidence in sort of my responsibility and opportunity to inform students about how they learn and uh share that with them so that they could take that on to other classes.
[31:00] Kristin: Is, is that so I know you’re on the quarter system, so that means it’s, it’s much shorter. But I’m on the semester system. So, is that something worth doing, like, two-thirds into the semester at all, you think?
[31:13] Beth: I don’t know. I think there’s a, a, I think maybe I’m gonna do it right before, like, a midterm. If I was gonna have that to, to be, or, or again, maybe it’s, I used to do, like, I throw a graph up from a research paper again with a quote every week just for something. I think that the, the power of the last lecture is it’s over, right? And it’s like this is what I’m leaving you with. Like I’m not just here to teach you data science. I am here to, you know, help you support you and your, your ongoing learning. And so I want you to think about how you can take what this you learned in this class and take it on to other classes that don’t even use, you know, peer instruction. Like, how do you keep yourself mentally engaged in the class? How do you know, can you, will you actually read before class so that you can’t be prepared? That’s a double-edged sword because it depends on how the faculty member is lecturing. But, you know, people said that they paid more attention. They changed how they were engaging in their other courses based on their experience. Um In my course.
[32:21] Kristin: Interesting, I think some of my teaching has changed over time. For partially for the like reaction to the pandemic and wanting to provide a bit more flexibility to the students and, and some of that, I think everything is a pendulum swing. You’re like swinging back and forth, trying to get to that like good spot and that spot keeps changing because the world keeps changing too. Um,
[32:47] Beth: I have to admit, I, you know, I haven’t taught with peer instructions since the pandemic, and I’ve heard it’s, it’s just even more difficult.
[32:56] Kristin: It does feel more difficult.
[33:00] Beth: It feels, yeah, you, you could talk to Leo Porter about this about again, knowing that this is the best practice and that it is good for students but needing to provide more flexibility for students and understanding what’s going on. There is a very interesting paper about students with ADHD, anxiety disorders, and other things and their experience specifically in peer instruction classrooms and you know that not everyone has the same experience, obviously, and different people have different experiences at different points in their quarter or their lives or whatever. But, you know, some of the students are saying yes, it makes me anxious to discuss with other people. But then some people are like, but I’m anxious to discuss people, and here’s this very narrow little thing, and I’m expected to do it and actually feel better afterward because I did whatever. And then I discussed with other people, and I realized that I wasn’t the only one that was having trouble that made me feel better. So, it’s a range of things that come out of it. And so it’s, it’s, you know, um the basic takeaway was, there are some good things that come out of, there are some bad things that come out of it, and we have faculty need to be cognizant of people’s situation and states and just, you know, you can’t run roughshod and say everyone, if you’re like not discussing, then the rest of your group, you should turn you in because they’re going to take away your points. It’s like, no, no, not like that. Uh, but I have to admit I I I don’t know what it really looks like in the classroom post-pandemic
[34:26] Kristin: For me, like one of the things is it, it’s getting students to discuss this, felt more like pulling teeth, like getting them to have some kind of discussion. Um And I part of me can’t tell is, is it, is it me, was it like me setting the semester tone wrong? Like what, what is going on? And I’m, I’m slowly like trying to get back to what it used to be like because probably the most shocking thing is I, I could probably go back and actually look at the data though like it’s biased in that before the pandemic, I taught mainly CS1 and after the pandemic, I’ve been mainly teaching data science, but I feel like the, the learning gains between round one and round two are much smaller.
[35:11] Beth: Ueah, it used to be pretty big.
[35:13] Kristin: It used to be pretty big.
[35:14] Beth: Like you said, less than 75% and then above 90.
[35:16] Kristin: Yeah, it’s always like, it’s below 75%. A lot of the time. But then you always break 75% of the round two. But, like, nowadays it feels like it’s still less than 75% every time. And I’m like, I don’t understand, I don’t understand why you all shifted only like by 5%. Like, what does this mean? Um And so I’m still part of me is kind of more willing to, maybe, maybe I need to be more drastic and just go like no assigned seats, assigned groups. We’re all going to like we’re going to try this experiment. And so I’m, like, slowly trying to figure out how do I get back to what peer instruction felt like it used to be like?
[35:51] Beth: I, I don’t know because this, you know, I thought we would have been past this already, but I wonder if it’s just a matter of time of, you know, getting the kids in our classrooms who weren’t as impacted or they had more time to recover their. Mhm. Group learning skills, their social skills, you know, because they’ve gotten now four years in high school, all in person and whatever. But it hasn’t happened yet.
[36:17] Kristin: Yeah, this year we’re graduating those who start first year was in the, during the pandemic. So, their first year of college.
[35:26] Beth: So I think there’s potentially, you know, three or four more years before we really come out. If there is any coming out the other side, it’s three or four years away. I would say that if anything peer instruction, you know, peer instruction implemented, the way that we know it should be is still one of those things that could be super beneficial for bringing kids back and for giving them the expectation about collaborating and communicating. And, you know, maybe it doesn’t happen when they’re in your semester. Maybe it’s the next semester with that next professor that implements peer instruction. But it gives them so much more feedback about what works and what doesn’t work. And like, you know, how to be a student, how to be a professional in doing data science or programming or whatever. And so I think this is a, you know, a potential hope. I think we just, I at least, used to be experienced in seeing like the 1st 2 to 3 weeks. It was kind of like, oh, no, no, no. But then they came around, they’re like, oh, this is great. Like, you know, I see what I’m getting from this, and I think it’s, it’s just, you know, there’s more for them to get past now and there’s less prior experience in, in the collaborating and whatever coming in that um that it, it, we might not individually be seeing it in the time span of our term or semester or whatever. But um, I do think it’s, it’s, you know, it’s almost like it would be, there was maybe there was a missed opportunity where we could have gone out post-pandemic and like had some university president uh or you know, dean of a school of engineering saying now is the time that all faculty need to be doing peer instruction because we got to get our kids back.
[38:14] Kristin: Your point about the developing the skill to explain things. I have not heard it framed that way, but I think I’ll definitely be using it because like in my data science class, one of the things I talked about is communication, like the reason why you have this laundry list of terms to learn is because we need to be precise in our communication when it comes to data science that people understand what you did and your argument for why it’s a good idea. Um I’m going to roll back real quick to your comment about undergraduate tutors. So when I said TAs, it’s a, you know, overloaded term. It means multiple things. When I said TAs, we have undergrad TAs, and we have grad student TAs and the undergrad TAs. I’m I’m figuring out like, how do I train the undergrad TAs to effectively support the discussion of what’s going on? Um Because it very much is like you got a channel and your most extroverted person that you possibly can. And then also come up with reasons to go talk to the students.
[39:18] Beth: can you recruit from your own student body? Like, can you recruit from the kids from this semester to be your tutor?
[39:25] Kristin: I Do. I do recruit them. Um, it’s a little bit hard because the, probably the most difficult thing is I’m kind of the time that I normally get for this course is very popular. And so I often have my undergrad TAs say, “I can make the Friday session but not the Wednesday session because I have a Monday-Wednesday class during that time.” Um So there’s that difficulty as well. And then I have another one who’s like always the hybrid one that’s in the corner to make sure that the mic is on and all of those kinds of things. Um, and manning the Zoom for me. So it’s, it’s a little difficult. I think I maybe, maybe I’ll try this experiment of just doing assigned groups and then see where that goes.
[40:10] Beth: I do think in the, in the checklist of things. Yeah, the assigned groups will be your biggest win.
[40:16] Kristin: Mhm OK. Maybe I’ve, I’ve convinced myself that I will try this, but I’m gonna sleep on it first.
[40:21] Beth: Well, and, um, to the point of you’ve got summer, you know, if you can find out what room you’re in and discover it doesn’t have seat numbers on there. It’s not that much money, like a little grant and be like, hey, peer instruction, best practice. Yada yada. This person told me her, her, her ed tech people went around and put, see there, it doesn’t cost that much to send a guy with a drill and these plates that you can buy on Amazon and drill them into every seat over the summer. So
[40:48] Kristin: I’ll try. OK, I’ll try and figure out what room I’m in and go check. Um This makes it more tempting. My main takeaways from this conversation is I should strongly consider assigning seats and assigning groups and then probably work on my peer instruction questions by removing the distractors, the wrong answers, that are not worth discussing. Um Always have students discuss regardless of the percent correct. And the question to have people share out is what did you discuss as opposed to like, which do you think is the wrong answer versus the wrong answer and all of that kind of thing? It’s just “what did you discuss?”.
[41:30] Beth: the last one is, is, is a good idea. It’s probably not as important as those other ideas. And, and what I like about, you know, your idea of removing the structures. That’s a super easy ask for you. Right. That’s, you can just sit down and do that. Right before you go into the classroom. Um, so, yeah, making groups is, is harder. But, uh, and then always having discussion that’s also easier on you too. So, those seem very likely to happen.
[41:54] Kristin: Yes, those seem very likely to happen. Um, assigned seats is going to think about that.
[42:02] Beth: Yeah, definitely a bigger, bigger lift.
[42:06] Kristin: Um, and then try and I, I think I, I like your point about the reflection, and I might, maybe I’ll put it in my, my, even my mid-semester survey. Like, I, I find that students are a little bit more open, willing and I, I need to figure out better how to layer back in reminders all over the place because to me, the reminders feel repetitive and so I don’t do them, but I know that I need to
[42:30] Beth: Yes. think of, think of this, you know, this my workers, student workers, younger sister, she’s like, you know, she talked about it all the time. I remember she said, but I, I like it really hit me then. Like, there’s, you know, beginning of the class, they’re all like ra ra ra doing whatever and like half of the time that you give a reminder at the beginning of the class, they’re seeing it, half of them aren’t because or whatever. So you’re getting the 50% or 25% hit rate per day that you have that reminder, and other people don’t mind, you know, as much. It’s like I know I’m telling you again, but again, this is so different, and like, I really want you all to understand why it works, and it’s, you have to hear about it, you have to experience it. I have to tell you again why it works. You have to experience it some more just to be, to be this sort of growing understanding. Um you know, you show them a spiral, like your understanding of like how to make this work for you and like how well why it works, et cetera is a, is a growing spiral.
[43:30] Kristin: Yes, growing spiral. That’s true. All right. So let’s close out because you’ve been very generous with your time. For the Too Long; Didn’t Listen, I feel like it’s my turn instead of your turn. Since you’re the one that’s coaching me. The peer instruction, make sure the question has a minimal number of distractors, only wrong answers that are worth talking about and then they vote individually. You always have them discuss regardless of how many people got it right on the first try. And then, strongly consider doing assigned seats in assigned groups, but I have to go check first if there are seat numbers in my room first. And thank you so much otherwise though, for hanging out with us, Beth and helping me out and thinking through how to make things better.
[44:17] Beth: I, I cannot tell you like, I’ve never gotten to do this with anybody. Even all the people I work have done peer instruction with over the years. When do you ever have time to make the time to have these sort of discussions to like, what do you do? What do I do? Like, why are you doing those things? I feel like we’ve missed out an incredible opportunity to help us all. Like my way is not just always the right way, it has ways in which it might not work in other specific settings, and Um Yeah, I, I, this has been such an amazing opportunity to have the time to reflect and, and remind even myself of, you know, why we did things we did.
[44:57] Kristin: Yeah. Well, this is one reason why I started my podcast because I was like, I want to have these conversations, and then like, I’ll just broadcast it to the world.
[45:01] Beth: Yes. Love it.
[45:04] Kristin: And thanks to everyone for listening. We’d also like to thank our patrons on Patreon! They help us keep this podcast ad, commercial, and sponsor-free. And a special heartfelt thank you to Clark Scholten, William Turkett, and Glenn Downing, three of our patrons that we randomly chose to shout out today. If you are currently not a patron but like what you hear, please consider supporting us on Patreon link on our website. And if not, at least share the podcast with your colleagues and give us a five-star rating in your app of choice so the algorithms know to recommend us to more people. And remember, teaching computer science is more than just knowing computer science. And I hope you found something useful for your teaching today.