S3xE9: Peer Teaching Summit at SIGCSE TS 2022
Episode Summary
In this episode, Sarah Heckman from North Carolina State University and our host discuss the Peer Teaching Summit at SIGCSE Technical Symposium 2022. We cover what a peer teacher is, more commonly known as an undergraduate or graduate teaching assistant, and how they support student help-seeking. The summit brought together many people with peer teachers at their schools where they discussed what they can and cannot do, and how every school is unique. Afterward, we focused on office hours and how there was a surprising variety of handling them, including what information students see in the office hour queuing app while they wait in the queue, what information peer teachers see, and the rules the peer teachers use to decide who is pulled off the queue next.
This episode was funded by NSF grant #1934965.
You can also download this episode directly.
Transcript
Kristin[00:08] Hello and welcome to the CS-ED Podcast, a podcast where we talk about teaching computer science with computer science educators. I am your host, Kristin Stephens-Martinez, an assistant professor of the practice at Duke University. Joining me today is Sarah Heckman, a teaching professor and director of undergraduate programs for the Computer Science Department at North Carolina State University.
Sarah[00:27] Thanks for having me on today.
Kristin[00:30] So, Sarah, first I’d like to ask, how did you get to where you are today? For full disclosure to everyone, Sarah and I have a grant together, but I don’t think I’ve ever asked you this question.
Sarah[00:40]So I was encouraged to study computing by my uncle. He was working on his Ph.D. in computer science when I was applying for undergrad college. So I started at NC State in 2000, and I really enjoyed my computer science classes and was able to start working as a peer teacher my sophomore year. And that led to an undergraduate research experience and later to the Ph.D. program. In 2009, I defended and started as the first teaching track faculty at NC State’s Department of Computer Science. I regularly teach our second-semester programming course called Software Development Fundamentals, as well as our junior-level software engineering course. And now, my research focuses on supporting student success through help-seeking and automated feedback.
Kristin[01:21] Awesome. And that is a perfect segway to our main topic, which is our peer teaching summit that we ran this past SIGCSE 2022. And I think probably the first thing we can cover is technically, it was the second one. And what is the peer teaching summit? Who went? What did we talk about there?
Sarah[01:42] Right. So the peer teaching summit, we had our first one in 2019 at the Technical Symposium as an affiliated event, and we were happy to run it again at the 2022 Technical Symposium. So the goal of the peer teaching summit was to bring together leaders who are working on having peer teachers in their classrooms, typically in their undergraduate introductory sequences. And we had 32 participants from 26 different institutions, and some of our participants included peer teachers themselves. We had some students there, which is fantastic. The institutions were a mix of public and private, had various sizes, but we tended to have more from slightly larger institutions, particularly where they had a significant number of peer teachers. So more than 15 or 20 in their organizations.
Kristin[02:28] And when you say peer teachers, what do you mean exactly?
Sarah[02:32] Right. So peer teachers that we talk about tend to describe undergraduate TAs, potentially tutors, or other near peers that support help-seeking. So depending on the institution, responsibilities of the peer teachers may include grading or running lab sessions or recitations, holding office hours, supporting asynchronous message board help-seeking. And those responsibilities do vary by institutional context.
Kristin[02:59] Yeah. So I think a question that some people might be wondering is why aren’t grad students kind of in this context considered a peer teacher?
Sarah[03:10] Well, I think that varies a little bit by institution too. I actually do call my grad student TAs peer teachers. But a lot of times we’re focusing on that near-peer relationship. And a student who has just recently taken the class, who’s helping someone who’s currently in the class, really provides that connection. A lot of grad students may not have done their undergrad at the institution where they’re now TAing. And so having that frame of reference through recently taking the class really helps with that near-peer relationship and especially around help-seeking.
Kristin[03:40] Got it. All right, so at the peer teaching summit, you know, it’s a summit, but what does that even mean in terms of what happened there?
Sarah[03:48] Right. So it was a half-day summit, and we ran a couple of breakout sessions, and as part of it, we like to mix people up. So the first breakout session, we had people sort of break into groups based on the size of their peer teaching program. So we had those that had, you know, more than 15 peer teachers in their program and then those who had fewer. And then, we had a group specifically for the students so they could discuss their way of doing peer teaching. Then in our next breakout, we really broke into groups around different topics that people might be interested in exploring further. So things like the logistics of running peer teaching programs, supporting help-seeking, and training.
Kristin[04:30] So, what came out of that that maybe surprised you the most?
Sarah[04:37] Right. So institutional context is huge when it comes to peer teaching. And I know that through the grant we’ve been working on, we’ve always talked about the differences between NC State and Duke and some of the other institutions we work with. But what the summit really brought forward was the whole variety of ways that institutions have come up with implementing peer teaching and also some of the constraints that institutions have. So, for example, an institution with unions may not allow peer teachers to do something like grading, where that’s required to be done by grad TAs. What was also really nice and kind of affirming is that we all have common challenges. We see challenges with scale, we see challenges with allocating resources to meet student demand, and we also see challenges with maintaining quality interactions and feedback. So it’s really nice to bring people together and see those common challenges and start discussing ways that we could potentially overcome them.
Kristin[05:29] Yeah, I remember being at that summit, and it was affirming to go like, oh, you have the same problems that I do, while also at the same time, I’m like, we can’t do that. I wish we could.
Sarah[05:39] Right. So I do hope that one of the things that we might be able to do more broadly is help provide that institutional justification for peer teaching so that, you know, budgets and other resources can grow appropriately.
Kristin[05:52] Yeah. So we are planning a white paper, but we definitely don’t want to do the equivalent of read you all the white paper. So what are the main things that will be in the white paper so people can look forward to it and then otherwise, we’ll probably just dive deep into one or two of the possible things in it?
Sarah[06:09] Right. So the goal of the white paper is to summarize the discussion around the summit and, in particular, some logistical side of things. So how do we recruit, hire, train, manage, and oversee quality control of peer teaching programs? We’ll also look at some of the challenges around help-seeking that we’re seeing, especially as our programs grow. And some of the strategies that different institutions use to overcome those challenges and how that might operate within particular institutional contexts. But ultimately, our goal is that the white paper can help instructors and departments start and refine their peer teaching programs and then hopefully again provide that justification for resources to support peer teaching long term.
Kristin[06:50] Yeah. So maybe how about we start with providing people with some mental models of generally how peer teaching programs are done currently, and then we can dive deep into help-seeking, which is kind of our current obsession, right now, I think. But peer teaching models first.
Sarah[07:12] Yes. Peer teaching models first. So we’ve noticed that a lot of the peer teaching models do tend to focus on the introductory sequences in undergraduate programs specifically. You know, some institutions, quite a number at our peer teaching summit, have dedicated staff, which is fantastic, to help support the logistical side of peer teaching, especially when you have, you know, 50, 60, 80, 100 peer teachers across a variety of courses. A lot of institutions also create hierarchical structures. So they have the instructors, maybe a lead peer teacher, and then the peer teachers themselves. There’s a lot of work to try to recruit and create pipelines. Maybe peer teachers start off in introductory courses and move up course sequences, and there’s also a specialization of peer teachers. So some might focus on grading, some might focus on answering message board posts. So there are lots of ways that peer teachers could be useful.
Kristin[08:08] So one of the biggest things that peer teachers do is support students when they are seeking help. And I definitely noticed the way that the different institutions did that both had commonalities and actually differences. And did you see that, too, when you were looking at what everyone reported and took notes on?
Sarah[08:31] Right. So at the summit, we really broke down help-seeking into two major areas. We had our asynchronous message boards like Piazza, and then we had our synchronous office hours. And there was a lot of commonality around message board usage. A lot of institutions use Piazza specifically, and a lot of the institutions tended to have unmoderated or student-driven message boards. So some institutions might require peer teachers as part of their responsibilities to regularly check in and answer questions. That’s something I like to have my peer teachers do at NC State. I also encourage batching, so it doesn’t become completely overwhelming.
Kristin[09:11] Oh yes. I feel like there’s always a TA that somehow, like, enters into the whack-a-mole, answer every post as soon as it goes up mode. And you kind of have to help that TA dial it back so that it doesn’t kind of overwhelm their lives.
Sarah[09:32] Because it can very easily overwhelm their lives.
Kristin[09:35] Oh, yes. Like the way that I tell my TAs to do, it’s like we do assign certain days like this is your day, but you only have to go once. You have to go once, at least some point in this day. But otherwise, you do not have to whack-a-mole every single post as soon as it goes up.
Sarah[09:50] Right. We have a similar strategy. I assign time periods throughout the day. So at some point in the morning, do a full pass- answer every question, ask for help, you know, behind the scenes if you need it, but full pass of the message board, clear all the questions. And then, on days of deadlines, I assign multiple TAs to each time period. And so that way, we can try to be very rapid with feedback on those days.
Kristin[10:14] Yeah, I think the other thing that I do to encourage the TAs to go to the forum is I remind them that like it’s your first practice at being able to help a student without it being synchronous. So you can practice. How do I write a good message? And then I’m more prepared in office hours if the student asks me a very similar question because you’ve already thought about it a bunch. And then you can also read everyone else’s responses, so you can get a sense of how to respond to these questions, both on the class forum but also in office hours, you can go, oh, that was answered in the class form. Let’s together, go there and look up the answer.
Sarah[10:53] That and identifying those common questions means you can take it to the rest of the teaching staff and let them know, hey, we’re seeing problems on this test case where students are struggling with this particular item and then the instructor can go and say something at the beginning of class.
Kristin[11:09] Yeah, I think for me, one of the things that also surprised me at the summit, though, is that not everyone was using a typical class forum. Like some people were using Discord or Slack. And I was like, no. Discord and Slack is a more synchronous text conversation thing, and that would horrify me because I don’t want to be that connected or have set an expectation of having that fast of a round trip time on messages.
Sarah[11:40] I personally don’t use Discord or Slack in my classrooms, but I have heard of some people who love it and think it’s a fantastic way to manage their classes. For some of the people that do use Discord or Slack, particularly Discord, I heard that they would create private rooms for each student with the teaching staff. So there is a channel specifically to have those particular interactions. But I also think they tried to manage expectations about turnaround time on those particular messages so that it wasn’t necessarily an immediate response expected, but that it did kind of tee it up, and in some ways, it might become a replacement for the message board.
Kristin[12:13] Yeah. I think the thing that I struggle with is generally when you think of Discord and Slack, especially, I feel like a little bit more of with Discord, given how it’s used outside of the classroom a lot more, that the expectation around round trip time is different than in a class forum. Does that make sense?
Sarah[12:33] It does. I unfortunately never tried it, so I’m not quite sure.
Kristin[12:38] I think the other reason why people really like Discord is that I believe Discord makes it really easy to write a Discord bot. And so that means you can add a lot of automation into your students’ conversations with each other and especially the student and TA conversations, compared to, say, a class forum. Like the barrier to entry to writing a Piazza or Ed Discussion bot is probably much higher compared to Discord. So I think that’s probably another thing that Discord and Slack give people that is harder to get in a typical class forum, especially if you potentially are like locked into your learning management systems class forum, which is not nearly as good as like Piazza or Ed Discussion.
Sarah[13:34] No. I heard of a really great Discord bot that was created that actually helped facilitate office hour queuing so they could use Discord to manage the queue, which I thought was a great way to utilize sort of all-in-one place for students to go for help seeking.
Kristin[13:54] So given office hours, definitely what did not surprise me at all was it felt like every institution has a homegrown queuing system for office hours, and often they built it themselves in some kind of app and, you know, Duke is no different. Duke has two actually. We use My Digital Hand for some of the courses, and we have our own homegrown system for the other courses. And the interesting thing that I heard from these queuing systems was that some of the solutions were different because the instructors had different goals in mind for their office hours. Did you see that at all when you were looking through everyone’s notes?
Sarah[14:42] So the queuing systems are interesting because it really comes down to the challenges I think the institutions are seeing during office hours. Are they seeing lots of, you know, long wait times due to high demands, are they seeing things around deadlines? Are they interested in focusing on throughput for students who are seeking help? And I think a lot of that probably informs the types of queuing systems or the ways that queuing systems are utilized by institutions. So, for example, at State, I like to have my PTF, especially on deadline days, ensure that everybody has been helped at least once. So in some ways, we might slightly prioritize a student who has not been to office hours that day over someone who was helped, you know, a couple of hours ago, earlier in the day. And so for us, it was very valuable to see when was the last completed help-seeking interaction so we can utilize that to, you know, ensure that everyone is seen at least once on a given day.
Sarah[15:37] But I see that other institutions would have different ways of looking at their queue. Maybe it really, truly is a first come, first serve. Even if people are repeatedly joining the queue.
Kristin[15:48] Yeah. I remember when I was a grad TA at UC Berkeley, one of the rules was very much look at what the student had a question about, like there was an actual box, I think it was a dropdown that they chose. And then you would potentially grab like five students across the entire queue, and you’d call all their names, and you pull them into a side room and like talk about that problem there. And then, like, that’s one way of getting more students off the queue regardless of where they were. So like, obviously, you, you look at the top student and the top students, like I’m struggling with problem three, and then you look through the entire queue and find every student struggling with problem three, and you pull them all out at once.
Kristin[16:28] And for us, though, my first reaction when I finally had the office hour queue data like more readily at hand was asking my office hours head TA how many, like are we going to need more TAs this coming week. And then she would often like she literally would go look at the historical data. And if she didn’t have access to it, like she’d get access to it, so that she could actually look at it. And usually, it was two semesters before because the spring and the fall are different in terms of certain things like Duke sports time. And yeah, she, what was great was the head office hours TA always had to also know like the context of when certain things happened because sometimes she’d go, I think like this blip in the spring semester, everyone came in a day earlier than normal because there was a sport thing happening. And so this time in the fall, though, I don’t think that’s going to happen. So we’d extrapolate to figure out how many more TAs need to be added to that particular week. That was for the CS1 though. It was nice that we had critical mass, and there was always enough TAs willing to put in extra hours during those times. While, like my data science class, we are stretched thin because the class isn’t really big enough to have a lot of extra TAs because the ratio I go for, for that, is like 1 to 20. While our lower division classes have a lot more TAs compared to the number of students.
Kristin[18:03] I think what surprised me then actually was when I shared that at the summit, it didn’t seem like anyone else was looking at their data in that way. That surprised me a little bit. It’s like you had it. It’s right there. Like often, it’s the easiest graph to plot, so it’s the first plot that someone makes with this data.
Sarah[18:22] So I was looking at my data in a slightly different way because I was concerned about wait times due to complaints that we’ve received in course evaluations and that wait times were too long. So I already knew that office hours tend to really peak right around deadlines. And sadly, in the spring, our sports don’t interfere quite so much. So we’re really focusing on trying to minimize those wait times. And I proactively schedule as many office hours as possible on deadlines as well as the day after the deadline because I do have a late period, and so I try to maximize having lots of office hours on those days. One of the things I have noticed, though, post-pandemic, is that my peer teachers are less likely to step in to help out when there’s really long queues on the days of deadlines. Before the pandemic, there was really a culture in the peer teaching group to like hold additional office hours and, you know, extend office hours on the days of deadlines. And I kind of haven’t seen that as much post-pandemic. So I hope to, you know, kind of build-up that, you know, extra hours are fine. You know, please come in and help out on those deadline days.
Kristin[19:29] I remember at Berkeley like there was also a culture like that, and here I don’t really see it very much at Duke, but I suspect that probably the reason why your PTFs don’t do that now is, my goodness, this pandemic kind of made everyone tired.
Kristin[19:54] One of the things that concerns me about the pandemic is a lot of our social ties, social skills, desire to socialize, have weakened to a certain extent. And I’m concerned that a lot of students and people are partially in this negative feedback loop where they’re like, I don’t really feel like socializing, so I’m not. But now I’m sad. And so now I really don’t feel like socializing. And so they kind of spiral downwards into becoming more isolated. And even I have this feeling. And so, for the students themselves, it’s probably harder for them because our brains are really good at tricking us into thinking how we know how to make ourselves happy. But we actually don’t necessarily do that. Like one podcast I listen to called The Happiness Lab, they talk about how socializing makes you happier, but your brain thinks that it’s not a good idea. So you don’t do it. And I feel like with the PTF’s, I’m coming back to this topic, I swear, with the PTF’s at least they have this little bit of built-in way of giving back and socializing and feeling like they are contributing and helping other people, which is a very genuine way to actually increase and boost your happiness.
Sarah[21:15] We want to, you know, better understand the impact of peer teaching on the peer teachers themselves. It’s something that we feel is understudied. And so if that’s one of the extra benefits that we can get for peer teachers and that they are able to make those connections and have that sense of belonging and being able to support people, I think that’s a huge benefit of peer teaching programs. And so measuring that and understanding that I think would be a really great contribution to computing education literature.
Sarah[21:44] It’ll be really interesting to measure if there is a change over the course of a semester or belongingness in computing, self-efficacy, maybe their own self-regulation by seeing some of the challenges that students run into that they may not have seen, especially if they’re good students. It’s always like, I always find it interesting when our new peer teachers are surprised the first time they see a student who’s really struggling or who is running into time challenges because they may not have. And for them to see that sort of breadth of experience that students have, I think, could be really beneficial, and then it might actually help them with their own study skills and their own performance in other classes.
Kristin[22:24] Let’s see, I’m trying to think back to the summit, and one of the things I think someone said about the office hours queue. They had a rule that the students knew about where the first time they’re there, they get popped to the top of the queue, and like everyone can see it like they can see that this person joined, and suddenly they’re like in front of everyone else. And I don’t remember what they said when I asked, like, how did the students feel about such a rule? I think generally the gist that I got back from them was the students understood it, they got it, and so they didn’t say anything. But since you actually have a similar rule, like how much are the students aware of this rule, and have any of them mentioned it or pointed out, or like told you how they felt about it?
Sarah[23:17] So students are generally concerned about the wait time. But the way I set up my version of My Digital Hand is that they can only see the number of students in the queue, not their position in the queue, specifically for this reason. So if they’re, for example, if they are listed as first in the queue, but I selected somebody who is fifth, that would you know, they saw the number go down. That would probably be a concern. But I just show the number of students who are currently in the queue, and the teaching staff will help behind the scenes. I haven’t had any major complaints about that, but I do think that there is a justification of, you know, you’ve been helped three times already today, and this student hasn’t been helped in two weeks. I think we need to help the student who hasn’t been helped in two weeks or is their first time in office hours because we want them to have that welcoming experience, so they feel like they’re comfortable coming back to office hours. Because sometimes, just that first attendance to office hours is a huge step for students to overcome because they might be concerned or intimidated about attending, you know, help seeking. And that actually kind of comes to some of the benefits of sort of the pandemic office hour experience. Some of the things I noticed is that office hour utilization increased significantly, which means that students, I think, felt slightly more comfortable coming to office hours potentially because they didn’t have the social concern of asking for help in front of other people. They were able to do that from the privacy of their dorm room or their living situation. And so we had a significant increase in attendance with an also increase in wait time.
Kristin[24:47] Yeah, I think for me what I kind of observed, I’m blanking on whether or not we really talked about this at the summit, was because everything went remote, office hours went remote. And I feel like students really liked having remote office hours. And I have not actually gone back to 100% all office hours are in person this past semester. I have, it’s split where the first shift is in person and the second shift is remote. And the room that we booked is actually booked for an extra like half hour or hour after the in-person so that people don’t feel like they have to leave the room immediately. But otherwise, the students can kind of decide if they want to go in person or if they want to go to remote. Oh, and to add to that, Duke has evening office hours. So I’m talking about office hours starting at 7:00. So from 7 to 9, it’s in person on campus because we are a mostly residential campus. And then from 9 to 11, it’s remote.
Sarah[26:02] Well, I mean, that is one of the interesting things about how Duke structures their office hours is that they’re all evening because mine are spread throughout the whole day. But interestingly, we didn’t really talk about, you know, pre versus post-pandemic help seeking at the peer teaching summit. I think a lot of it was really kind of focused on queuing and Zoom and some of the support there. But there wasn’t an in-depth discussion about, you know, are people planning on going back to in-person office hours or staying remote or having a hybrid mix? So at this point at State or in my class at least, I’m still doing online office hours. I’ve not yet switched to hybrid, though I do have both peer teachers and students who would really like to have in-person office hours. Right now, it’s because there’s a lot of flux with our space, so I don’t have a dedicated space for those office hours to be held. But once I am able to reclaim some of that dedicated space, I’m going to hopefully be moving to a hybrid model of office hours. However, we also serve a population of post-baccalaureate certificate students who are typically working professionals. So having those evening and online office hours are needed for that particular group of students in our distance education offering. And so I don’t want to go back to completely in-person. I do like having the virtual elements.
Kristin[27:18] I don’t think I’ll ever go back to 100% in-person. And it’s kind of like for many reasons, the biggest one is access so that students who live off campus, because even though we’re residential, there are students who live off campus and they don’t want to schlep all the way to campus to get help. The other one is like safety. So 9 to 11 p.m. is late. Campus is pretty much safe. But at the same time, like I don’t necessarily want to feel like I’m partially responsible for a student to be walking in the dark at midnight, potentially. And then the other piece that’s funny for me is that I made my office hours hybrid. Like my rule is I will have office hours for 2 hours each week, and one is in-person, and often it’s right after right before my class. Usually, it’s right after because that’s when students find me. And the other hour, I often am a little data-driven, I tell my students, all right, I’m going to have virtual office hours for one hour on Monday. You all are going to tell me what hour of the day is good, bad, or I’ll do it if you actually pick that hour, but I don’t like it. And I’ve had more students come to my virtual office hours compared to my in-person office hours because mine are during business hours, and due to Duke culture, they all prefer the evening hours. But I do have students come, and usually, it’s during the virtual ones because maybe maybe I’m less scary online, or it’s just more convenient.
Sarah[28:45] Right. I’m probably going to stick with virtual office hours for myself. Part of it’s my schedule is so slammed in the fall. And also my office can be difficult to find. So I’m kind of in a back hallway, and I have to give directions to my students. Once they find it, they usually have no problem getting here. But I also, when I was holding in-person office hours, I like to hold it in the same space as my peer teachers in that kind of more open, you know, working space where students could come in and work. And, you know, until we get that space again, I’ll probably continue with the virtual office hours.
Kristin[29:17] Yeah. Depending on where my lecture room was, if there was a lounge nearby and I had office hours right after class, I would just have it in the lounge and then like I’d leave when my office hours are over. If there was no real lounge area and my office nearby, then I would have it in my office, and students would just follow me like little ducks.
Kristin[29:36] But yeah, so I’m trying to remember if anyone talked about hybrid office hours because I already know that there are different ways to do hybrid. For example, I do it by time. So students know exactly when office hours are in-person versus hybrid. But I know that our CS1 it’s both at the same time. Like there’s a room during the whole time, and there’s people on Zoom the whole time, and they can kind of do it because there’s always at least 2 to 4 TAs for every single window. Which what’s funny for me is that when I have collected data from that class about how office hours are going, I don’t actually know if it was an in-person interaction or a Zoom interaction. And students in the in-person room might get helped by a Zoom TA so that they’re in the room, but they’re on Zoom, getting help that way. So it’s kind of a free for all right now, I believe, for our CS1.
Sarah[30:32] Well, I think we need to update data collection there so we can get a little bit more information about those hybrid office hours.
Kristin[30:39] Yeah, we can try. What are some open questions that you want to explore in this space?
Sarah[30:48] Yeah. And we have a lot of open questions. But we have a lot of data. We have data about who attends office hours, who interacts on message boards. We can look at how these interactions intersect. So do people who attend office hours also post on message boards and vice versa. But what we don’t know, at least one of the things I’m interested in, is that if these help interactions are actually successful. So while we do collect data in My Digital Hand about if the student was able to finish answering that question during office hours, I really want to have something a bit more concrete about what success looks like. Meaning can we see some type of measurable progress on a project? Are they now passing a test case they weren’t passing before? And can we potentially tie that to some type of help-seeking either message boards or office hours? But I think the other piece that we’re really interested in exploring is why students seek help. And getting into the qualitative piece of it. So and then, in some cases, why they don’t seek help.
Kristin[31:53] And there is actually quite a bit of literature from education psychology that has looked at this question. But I think the interesting thing is that most of that literature assumes that the help is from teachers or peers, but not peer teachers. So it’s like their roommate or someone like that helping them. So I think it’s definitely an interesting space. For me, what I am interested in, and I think this is more like the teacher side of me really trying to influence the research side of me. I want to better understand how teachers curate the student experience and behavior such as by these rules, like, if you haven’t been to office hours today, you get to the front of the queue. Like, for you, the students are not aware that this rule is kind of being applied, so you’re kind of only just told the TAs to do this. While I’m pretty sure there’s at least one other institution that uses this rule, but it’s visible in the app. And so everyone knows, the students know. And generally, it seems that the students are kind of like, yeah, that makes sense. It’s like a fairness thing. And there must be other rules out there because like when I heard this, I was like, oh, that makes sense that you can do that. But it just never occurred to me that I could do that. And I’m like, there must be more things like this out there. And I kind of want to sit down and just like find the map of all the ways that people do this. And then I’ll probably lift a bunch of rules and do them. I don’t know if it’s experiments because it’s not really experiments for me. It’s more like, oh, I like this where I’m going to do it. And then, oh yeah, I should measure that and see what happens.
Sarah[33:26] Right, but the measurement is so important. So the rule of helping a student who’s not been seen yet today or who’s never been seen before is one we’ve been doing for a while. But we have some preliminary results that show that about 50% of students will leave the office hour queue after they’ve been waiting about 49 minutes. And so if we want to, you know, be able to maximize throughput of students who are actually there, we do need to be mindful of the wait time. And so it might be that if someone has just joined and they are having help before and there’s a student who’s, you know, approaching minute 45 who maybe has been helped before, you have to kind of think about the balance of throughput in there, too. So it might be that the student who hasn’t been there before, they can wait a little bit while you help a student who’s been waiting for a while, so trying to find that right balance. So I’m not quite sure there’s, you know, one or two hard and fast rules that you always have to stick to, but understanding the data. So for us, knowing that there’s that threshold where, you know, 50% of the students leave, that’s worth knowing and using that data-driven information to make better decisions, hopefully.
Kristin[34:32] One of my first thoughts when you said that about like around this minute is when people start dropping off like, oh, in some ways you want a recommender system in the app to recommend which student to help next, but then you immediately followed it with, but humans need to know because humans seem to be in this loop rather than just take the recommendation at face value or as the gold standard. Because there is always a lot of context that you cannot code into the system. But I think my next follow-on thought immediately after that, though, is teaching is hard, and for peer teachers they’re just learning how to do this. And they need a lot of practice to be able to get to chunk what they’re doing into a few enough chunks, so they’re able to think about those meta things. So this is where, yeah, that balance, I think, is hard. And probably really depends on your context of both the institution, the course, the general culture, the TA itself, the teacher themselves. Like all of these. It’s complicated. Everything’s complicated.
Sarah[35:55] It is. It is. And it’s interesting because as TAs are getting started, we’ve noticed that they take longer in the interactions with students. So as TAs become more experienced with the course in, you know, future semesters, their interactions become more efficient. And so there is going to be that expectation that newer TAs might have lower throughput during office hours in particular because they are, you know, struggling to maybe find the right answer or find the right guidance to help that student move forward. And so trying to provide that, you know, safety net behind the scenes. So while I may not use Slack or Discord in my classes, I do use it with my teaching staff.
Kristin[36:38] Oh, yeah.
Sarah[36:39] And I always invite the TAs to, you know, say, if you don’t know the answer to something, please ask on Slack. We are happy to make you look as good as possible with helping the student by providing you with the resources you need to have a successful interaction. Because we don’t want, you know, people left hanging with these types of interactions. We want them to be successful and move forward.
Kristin[37:01] Yeah, I think my reaction to that is like that’s great for, say, a class with critical mass like our CS1. Like odds are good, there’s always multiple people on Slack. But for my data science class, where we’re a team of 15, and I’m counting myself and the grad TAs and everything, the odds are smaller that there will be someone available on Slack to help in the moment. So I think, the way that I handle that, so we don’t leave the audience hanging off, like, how do you solve that problem. I have a tradition in my TA meetings that I call TA win or TA I’d like feedback. And I literally generate a random number and count off, and I’m like, all right, there’s ten of us here. Random number eight. One, two, three, four, eight, you, or not you. It’s like, Scott, do you have a TA win you’d like to share, or would you like feedback? And most of the time, it’s a feedback request. And that is kind of my primary way right now that I both find out issues that might be happening in the class for my TAs or provide all of my TAs some tips, tricks, tools, ways of handling situation, also providing them some kind of like thought experiment for what they might also have to come across and have happened to them. So that they have some like key phrases or something that they could say to give themselves some time to think before they continue the sentence to try and help the student or potentially say things like, “My office hours are over. I am sorry, the next TA is going to help you.”
Sarah[38:44] I really like that because that’s one of the things I struggle with is that trying to find that common time for all of my teaching staff to meet means that I usually do a lot of the administration over Slack rather than through synchronous meetings. But and because of that, I feel like there’s a lot of knowledge lost both between the TAs, but also as the TAs letting me know of concerns because they don’t always make their way to me. So having something like that to facilitate, even if it is asynchronous or slightly more asynchronous through something like Slack, is a great way to make sure you’re informed as the instructor since, ultimately, it’s all on you.
Kristin[39:22] And to be clear, I don’t actually have a single weekly meeting with my TAs. I always end up having two weekly meetings with my TAs, and this is for my data science class with only like 15 of us. Like, it always ends up that I have to split it in two, and I meet half of them in one meeting, and I meet half of them on a different day at a completely different time. And when I was teaching the CS1, I’m only meeting with the head staff like I see everyone else, like at the kickoff meeting and the grading parties. And I think this semester I’m going to bring back as many in-person meetings as I can just because Zoom is not good for those; how are you doing? Like, what are you up to? While you’re waiting for everyone to show up. Or where are you going after this? And then, like when they walk together, like none of that, Zoom, does none of that. And then on top of that, like, a lot of my TAs don’t like to turn on the videos. I’m not, I’m never going to force them to do it, but it’s kind of a drag. Like talking to a bunch of black boxes with names.
Sarah[40:26] Yeah, but there you go. You’re making everyone happy by having group TA meetings without even telling them.
Kristin[40:32] Yeah, at least I’m trying to do something. All right, so let’s do something a little different, and then we’ll do TL;DL. So, first off, given that we have these questions that we want to answer and we definitely can’t answer them by ourselves. Do you have any asks for the listeners in the audience where they can help you out in some way? Or help us out in some way since it’s like our project together?
Sarah[40:57] Yes, because we are interested in characterizing this help-seeking. We want to understand how different institutions do that, how they use peer teaching, especially in their introductory computing sequences. And we are interested in looking at things across multiple classes, not just in a single class, too. So if you’re interested in similar types of data collection around help-seeking, you know, please reach out to Kristin and I. We would be very interested in collaborating with you so that we can start understanding more broadly how peer teaching and help-seeking all of that interacts and how students are impacted from that. And then ultimately, we want to bring together people with peer teaching programs to future peer teaching summits. Hopefully, you’ll see one at a future technical symposium in a few years and be able to learn from what everyone else is doing in the community.
Kristin[41:46] Awesome. All right. And with that, let’s do our last segment of TL; DL. Too long, didn’t listen. What would you say is the most important thing you want our listeners to get out of our conversation?
Sarah[41:58] All right. So peer teachers, they’re super important as near-peer resources for our students. And they’re one of the things that actually grows as our student population grows. We definitely want to consider the processes to recruit and train and hire and all the logistics behind it. But ultimately, help-seeking is where we really want to focus our effort and try to identify ways to ensure that our students are supported, and our peer teachers are also supported.
Kristin[42:22] And maybe look forward to our white paper whenever it gets out as a summary of the summit.
Sarah[42:26] Yes, very much so.
Kristin[42:29] And with that, thank you so much for joining us, Sarah.
Sarah[42:31] Happy to be here, thank you.
Kristin[42:34] And this was the CS-ED Podcast hosted by me, Kristin Stephens-Martinez, and produced by Amarachi Anakaraonye. And remember, teaching computer science is more than just knowing computer science. And I hope you find something useful for your teaching today.