S3xE7: Class Forums with Amogh Mannekote
Episode Summary
In this episode, we talk with Amogh Mannekote, a Ph.D. student at the University of Florida. He and others analyzed class forum data from three very different classes and discovered that a lot of factors influence how students use the class forum, including the kind of class assignments, the accessibility of other sources of help, and how the instructor or TAs answer questions on the class forum. After doing this work, he also strongly believes that instructors should download and analyze their data or we should create more “out of the box” open-source tools that do this for instructors. In addition, students need to be encouraged to use the class forum effectively and TAs need explicit instructions on how to interact with students on the class forum.
This episode was funded by NSF grant #1934965.
You can also download this episode directly.
Transcript
Kristin[00:10] Hello, and welcome to the CS-Ed Podcast, a podcast where we talk about teaching computer science with computer science educators. I am your host, Kristin Stephens-Martinez, an assistant professor of the practice at Duke University. Joining me today is Amogh Mannekote, a Ph.D. student in computer science at the University of Florida.
Amogh[00:28] Thank you so much for having me.
Kristin[00:30] So Amogh, first, I’d like to ask, how did you get here? We all have more winding paths than meets the eye, and I’d love to hear your story.
Amogh[00:38] Sure. So I did my undergrad in computer science and engineering in India, and during that time, I kind of got interested in machine learning and natural language processing, and I kind of worked a bit in the industry. I did some internships which revolved around doing NLP and even some machine learning engineering. I did an internship at a startup where I worked with a lot of text analytics and kind of dialogue systems. And, during my internship at IBM Watson, I actually was part of this team, which was building developer tools, and I really loved the feeling of empowering other creators and developers. And this kind of motivated me to kind of think about how people should be able to build whatever they want and not just be downstream consumers of products. And then I had like a tangent right after that, like a complete other tangent, where I worked on robotics for a year. Then after that, I got back to my initial interest, which is dialogue systems. When I joined Rasa, which is a startup that works on dialogue systems, it’s an open-source platform as well. And following this, I really wanted to dig in deeper into my interest in dialogue systems, and that is when I decided to go to grad school and I think it was around the time of late 2019, and unfortunately, COVID-19 came.
Kristin[02:16] Not good timing.
Amogh[02:20] Not at all.
Kristin[02:22] So, quick question. When you say dialogue systems and like kind of since we know where you ended up, you ended up working with Kristy Boyer for Computer Science Education. How did that interact with your interest in dialogue systems?
Amogh[02:38] So it’s funny because the reason I kind of saw Dr. Kristy Boyer’s web page was that, the lab is titled LearnDialogue, and the word dialogue got my attention. And after that, that was the first time I actually was, got introduced to this whole field of CS education and the research that our lab LearnDialogue works on and is basically trying to see how can we analyze dialogue in the context of learning? So this can be either in the context of something like pair programming. For example, two students, mostly middle school, high school children are working on some kind of a programming project and they constantly communicate with each other and this dialogue is very closely, you know, interwoven with what what kind of learning happens and also we can pick up on some interesting signals, like is there some breakdown of the dialogue or are they having productive conversation? Is it just engaging in what we call wheel spinning, which is just, you know, just unpredictably stuck at a problem? Yeah, there is also the other side of the dialogue system education interaction, which is the kind of the history from the intelligent tutoring community where you can build an automated system to tutor a student. So I see these as two different strands related, but you know, orthogonal.
Kristin[04:09] So it sounds like, though, you were more interested in the intelligent tutoring systems piece of dialogue, and then you just weren’t aware of the context that Kristy was applying it to.
Amogh[04:21] Yes, so most of the dialogue systems research today is focused on building chatbots or simple agents for doing things like setting alarms or booking some sort of a reservation of a flight or restaurant or these kind of simple things, they’re still challenging, but they are still much simpler, in my opinion, compared to something like teaching a student where there’s so much more chaos, I want to say.
Kristin[04:55] I don’t know if I’d call it chaos like the thought that I had when you listed out, like setting an alarm or booking a flight, is those are very short transactional conversations.
Amogh[05:08] Yeah. So unlike these systems, which are doing things like booking or booking hotels and restaurants or flights and setting alarms. The dialogue systems that need to teach a student some concept or act as a partner as it’s much more challenging because it’s you have to actually take initiative and think about what is good for the student and not just act as a passive robot that just does what it’s told.
Kristin[05:36] Yeah, there’s a bunch of other pieces that, like an expert teacher would be thinking about in that kind of dialogue with the student and be like, What does the student know already? What is the student currently working on? There is a difference between what the student wants in this interaction versus what the teacher might want in this interaction. Because like for example, my work in office hours kind of shows me that students often they’re like, I just want you to fix my bug. Like, that’s their goal in the dialogue interaction. While for the teacher, usually, the teacher’s goal is not to just fix the student’s bug. The teacher’s goal is to help the student find the bug themselves or help the student practice the skills of debugging. And so that’s a much more complex kind of interaction dialogue goal than book this flight for me. I feel like you’ve gone up an order of magnitude in terms of difficulty and challenge with a dialogue system for education. So that kind of brings us to our main conversation, which was the work that you did looking at class forums. And so, could you give me the main results from that, that would be useful takeaways for someone who is teaching computer science and using a class forum?
Amogh[06:54] Uh huh. Sure. So just to sort of set some little bit of background for this. So, the context was that we studied discussion forums because we were interested in how discussion forums played a role in helping instructors make sense of how extremely large class sizes are functioning during the courses. And you know, Piazza, Canvas, and places like these are very rich in data about how students are learning. And we wanted to kind of draw some useful inferences and insights about what can instructors do to make the conversations happening on these forums more productive. So essentially, we looked at three things I want to say. So the first is. What kind of questions are students asking on the forum? How are instructors and teaching assistants responding to those questions? And what are instructors doing to influence the nature of the conversations and discourse that happen on these forums?
Amogh[07:59] So yeah, we found a few interesting things from analyzing this data. And the first thing we found was that, you know, having more complex projects during the course of the course actually generated more discussion. So when students were given kind of long-term projects, they faced more roadblocks. They had more deep questions as opposed to something like how do I just get this small task done? So that kind of got them to really think about actually phrasing their questions and really articulating their problems in a way that, uh, a second person can actually understand it. So I’ll probably come back to this a bit later, but this is actually a very important thing from the perspective of learning. We also found that as much as the instructors and TAs want to encourage this kind of productive questioning, most of the students, well, a significant portion at least, resorted to something like just pasting their errors and asking for, “Hey, I am not sure what’s wrong. Can you please help?” And that kind of motivated the title of the paper, which is don’t just paste your stackflow. So we want students to do a little bit more work from the end and trying to figure out what’s wrong and explain what they tried already so that it helps not just the person helping them but also the very act of explaining it is good.
Kristin[09:38] Something we forgot to mention is that you did publish some of this stuff as a poster at SIGCSE 2022, right?
Amogh[09:44] Yes, that’s right.
Kristin[09:46] Okay. I was trying to remember where this work was published. And I should also mention that some of the data you analyzed was data from some of the CS1 data from Duke from a course I used to teach, but technically I did not teach that particular class. But I collaborate with that instructor all the time. So when I asked her for like the data, she’s like, sure, here you go.
Amogh[10:06] Yeah.
Kristin[10:07] So we are in this dataset.
Amogh[10:08] Yeah. And this was actually three courses. I mean, we had two CS1 courses, which was, the one was the one that you just mentioned from Duke University, which is taught in Python. The other one was from North Carolina State University, which was taught interestingly in MATLAB, I think mostly for engineering kind of audience. And the third one was a CS2 Java, which focused on object-oriented programing and software development principles.
Amogh[10:39] So the third kind of main finding they found was that the response rate of students answering their peers’ questions was actually quite low. And this is a very important indicator for a student performing really well in class. This was actually when we looked at the data, this was very, very highly correlated with the grade. The number of times a student answers a peers’ question, which they don’t have to, by the way, they don’t have any incentives to do that. But they still went out of their way to do it. That was seen as a huge thing. And one of the challenges is how do you encourage them to do more of that?
Amogh[11:21] And the final one was this is kind of exploratory, but we found that the number and type of help-seeking venues that a student has access to during the course, for example, office hours or Piazza or email. So all of these things need to be looked at holistically. And the nature of these venues, help-seeking venues, also influence the kind of conversations that happen on, say, Piazza. Because if most of the answers have already been answered in office hours, then it’s not so rich in the Piazza anymore. So which is not a bad thing, but it kind of does influence.
Kristin[11:58] What were the main differences that you saw across these three courses? Because I suspect that it wasn’t really the language that drove differences. It was more how the instructor approached the use of the class forum or, as you mentioned, the difference in other help-seeking venues.
Amogh[12:19] Yeah. So I think the first thing was that the CS2 course that we looked at, that was quite different from what the other CS1 courses, maybe unsurprisingly, but specifically in terms of the fact that they were dealing with a bit more complex setups in terms of the IDEs, the style checkers and other developer tools that they had to use, that was kind of different. They also, like I mentioned, were dealing with a more complex long-term project.
Kristin[12:53] Can you define long-term? What’s a long term project in this context?
Amogh[12:58] So a long-term project is like a semester end project that takes multiple weeks for them to complete, which has many pieces. This is in contrast to like, uh, a small assignment, which is just a one week or two weeks long.
Kristin[13:12] Okay. Because, like, I know that the CS1 here at Duke, we have assignments, but they only last, like, probably at most two weeks, maybe two and a half weeks. So like in the CS1 context, that’s called a long assignment. But it sounds like that CS2 not only had two plus week-long assignments, but those assignments might be integrated with each other potentially.
Amogh[13:37] Right.
Kristin[13:38] Okay. Got it.
Amogh[13:40] Yeah. And so that is in terms of the, just the CS1, CS2 contrast that with some other contrasts as well. So like, I mentioned, we also kind of asked instructors this question of what did you tell the TAs before the course began? How to interact with students on these forums? And the instructors who said that, oh, I explicitly told the TAs to not directly give the answer, but kind of indirectly give hints or kind of give what you call Socratic questioning. So just ask the student, what do you think? Why do you think this is happening? Or How do you think we can fix this? So those courses, we did see an increased rate of such indirect responses as opposed to just directly giving out the answer. But this was, I think, like a philosophy that most people tend to follow. It’s just kind of reinforcing that behavior helps socially. And the third one is, I think, what you mentioned about the help-seeking behavior as well, which is the number of debugging questions, which is kind of the main juicy bit of the forums, is pretty high on the two courses which follow a traditional classroom approach, but I think this was the Duke one which followed a flipped classroom approach and that we noticed that there were fewer debugging questions. And I don’t know for sure if that’s the case, but it could be that students do most of the debugging help-seeking during the actual class or maybe in the synchronous office hours. So it does seem to have a role.
Kristin[15:25] Like I’m trying to remember because this will be CS1 data from fall 2020 or like spring 2021.
Amogh[15:35] Yeah, this was actually CS1 Matlab.
Kristin[15:37] Oh, okay because the Duke CS1 class is not flipped right now.
Amogh[15:42] Yeah.
Kristin[15:43] But that class does get a lot of office hours help for the students, and it’s in the evening when students are more free though I would never do evening office hours, but the undergrad TAs are fine with evening office hours.
Amogh[15:58] Got it.
Kristin[15:59] So they do get a lot of help. One thing I’m curious, I don’t know if you actually have an answer to this question from your data, but like what is your gut response? For example, in our CS1 and CS2, to try and encourage student responses, there’s kind of like this soft rule of don’t answer a student question until it’s been there for 24 hours. And I think it’s for multiple reasons. One reason is to keep TAs to not feel like everything is urgent and to always be constantly on the class form, checking it to make sure everything’s fine and also to make students not feel like the class forum is their like oracle to answer all questions, and they should just do nothing until they get a response. So that’s kind of like one of the philosophies behind that. However, like on my data science class, I don’t have that rule, and that’s partially because I feel like students don’t use the class forum as much, and I don’t want them to feel like there’s no point to my asking my question because I’m not going to get an answer very quickly. While it seems for the CS1 and CS2, students have a lower barrier of entry to go and ask a question, but maybe that’s also because of law of large numbers because the data science is kind of big, but it’s not as big as the CS1 and CS2. So I don’t know. What is your sense of that vague question?
Amogh[17:29] Yeah. So I think the size factor is definitely playing a role, and in terms of the delay, I think that is I think not just in terms of the responses that we got from the instructors but also from several prior literature where people have actually tried to find the optimal time before the TA jumps in to answer. So it’s like this tradeoff between not just preventing the TAs from going into firefighting mode, but also to kind of let other peers know that, hey, the moment that TA answers, they’re like, hey, what’s the point of me even trying to answer this, it’s already been answered? So they don’t even try. So having that sweet spot between waiting too long for them to answer and letting them think about it or having other peers answer it. That’s kind of an open question. But I think it’s something that instructors can play around with to see how well it works for their specific context.
Kristin[18:28] So I remember from the paper that you had different categories for the question students were asking, as well as the kind of responses that people were giving. Could you go into some detail about that?
Amogh[18:47] Well, our six categories was based on this question of not just what their students are talking about, but also how they are asking the questions. So we had six categories for student questions and six for instructor responses. So just quickly go through them. So you have the debugging questions, which is when they have an issue with the specific piece of code. Then we have the, what we call the general questions, which are kind of conceptual. So it’s not necessarily tied to a piece of student’s code, but something like can arrays be expanded dynamically or something like that? And then there are the dev tool issues, which are, hey, my IDE is not setting up, or my style checker is throwing me an error and all that stuff. Then there are, like, logistical questions which form a pretty significant portion, like things about grading and class schedule. Then we have the how-to questions, which are very straightforward, like there is no reason to make the student think about it. It’s just, how do I import a library, or how do I plot a graph? And then, finally, we have clarification questions. So this is just students clarifying the requirements of the project or the assignment.
Kristin[20:06] So, I’ve got a question. From your sense which categories were most common? And was there a difference between the courses at all?
Amogh[20:14] So the most common ones were the debugging ones, the logistical, and the clarification questions.
Kristin[20:23] Okay. I’m trying to think of like would I consider that bad air quotes that that is the most common thing? I’m like, just trying to think through what would my ideal mix of categories be? And I don’t actually know. Now that I’m thinking about it.
Amogh[20:40] Yeah. So that’s a good point. And I think, maybe one flip side question that I had was, is it bad to have a lower number of general questions, which is kind of the, in some ways, like the gold standard of questioning, which is you abstract out your specific problem into something more general and then ask it. So that made up only like 5% to 8% in all of the three forums. So I don’t know if the other distribution of the other questions is bad, but I would certainly want to shore up this number.
Kristin[21:18] Yeah. Like, I was thinking through that. Like, I think general questions sounds like the thing that you would most want to happen in the class forum because it’s like students showing that they’re thinking more deeply about the content, and they’re extrapolating or they’re transferring their understanding to new contexts and everything. But then another part of me is like, but is that really the point of the class forum? I don’t know. I think it partially depends on how you use the class form because like, for example, I use the class forum also as a back channel to my lecture. And so what that means is I literally have a post per lecture with the links to, like the peer instructions. So it’s kind of like a useful thing for the students to have too. And then I tell the students that, like, if you have a question during class, you post it as a follow-up to this lecture post. And it also gets the students on the class forum so they know it exists, they can find it, they know how to use it, they’re familiar with it. It’s like there’s a lot of other reasons why I use it as a back channel. But part of me is kind of like, I think there is potentially where I would want the most general questions because it’s like it’s in lecture, the students are asking like, but what if you do X, Y and Z? And then I’d be like, okay, let’s find out if you X, Y and Z. But would I want a student to ask that question while sitting down and doing the homework? I don’t know because I don’t know if I’m really expecting them to have that skill of abstraction and then articulate in their question at the kind of more abstraction level that you mentioned, as opposed to the debugging level of like I need to do X and it’s not doing X. Please help. So yeah, I don’t know.
Amogh[23:00] Yeah. That’s a good point. And I wonder if maybe the joint distribution of the responses along with the students’ questions might be more, I guess, we can probably make more opinionated comments on that as opposed to just the type of questions. And that’s also one thing that we looked at, which is the kind of conditional distribution of a response, given a type of question.
Kristin[23:30] So now you’re talking about also the types of responses students got from questions, right?
Amogh[23:36] Yes.
Kristin[23:37] Okay. So what were those categories?
Amogh[23:38] So those would be the first is the direct answer, which is the TA directly giving the solution in very straightforward terms. Then there is the explanation, which is the TA says, hey, this is why you are getting an error, but I’m not going to tell you how to fix it. Then there’s the elaboration, which is the students are really just asking a simple question, but the TA decides to kind of give them an opportunity to learn by elaborating even more. Then there’s like a hint, which is a suggestion on how to proceed further. Then the counter questions, which is like questions that are sent back to the student. This is the kind of Socratic questioning I was talking about. And finally, like a close-ended response, which is just a simple yes or no.
Kristin[24:29] So you mentioned doing the conditional probabilities of question type versus response type? Any significant results there? Because if I had to guess, I would say that clarification questions or logistical questions got a lot of direct answers or close-ended replies.
Amogh[24:49] That’s right.
Kristin[24:50] Okay. But any that surprised you?
Amogh[24:54] So we saw that a significant portion of the answers to the debugging questions were explanations, and in the CS2 java course, some 48%, and this is in contrast to 28% in the other two CS1 courses.
Kristin[25:17] Oh, interesting. Do you have a sense as to why?
Amogh[25:21] I feel like from what I remember of looking at the data when the questions in the CS2 course were longer, so they would be like, kind of more descriptive about what’s happening. And then it’s, it’s not how-to questions so much like how do I fix it, as why is this complex code throwing an error? And typically, the response to that is like it’s because you’re getting a null pointer exception because blah blah blah. And I feel like the questions in CS1 seemed to be, for the lack of a better term, simpler. So it’s easy for a TA to just say, oh, this is why, this is how you can just fix it, or, you know, something like that. And also, I feel a hint may not be super appropriate in the CS2 context. Like if I say I have the null point of exception again in this giant piece of code like I think about blah blah blah. I don’t know if that would help in significantly progressing in the debugging as opposed to saying this is where it’s happening, but this is why it’s happening. But again, that’s just like my anecdotal remembrance of some of the poster.
Kristin[26:44] Like, I’m just thinking back to the CS1 and the common debugging questions that I got for that material. And the one that comes to mind is the first time the students are working with multiple files. And often, quite a few students always get the same error where they’re writing a function that is being called by starter code so when they run the whole program, they claim the starter code is broken because that’s where Python throws the error. In reality, what happened is that their function had returned None. And the starter code couldn’t handle that. And so for debugging, we often did resort to explanation with some elaboration without telling the solution, by just kind of going the reason why this is happening is your function is returning None. If you look at your code, you can see that there is a possibility of an input returning none rather than actually returning a string. Which is why it’s not the starter code that’s broken, it’s your code that’s broken. In the end, we actually decided that rather than having to repeat this explanation every single semester, we put a try block inside the starter code to catch the error and then print out to the student that if you see this message, that means your code is returning None. Just to make them stop claiming the starter code was broken. And we purposely added a unit test that would check to see if their code would return none. And for every permutation we’ve seen of occurrences where this could happen, but like the hard part was, so this was like our transform assignment, so basically it’s like they get text and they have to spit out like a transformed version of that text. So this could be for encryption, or it could be something like piglatin. Like you’re taking normal text, and you’re converting it to pig Latin. But the hard part with that kind of function is you don’t really know where a particular student’s code is going to return none. Like you can guess, but otherwise, you can’t figure it out easily. It was much easier for us to just put a try block in there and go like here, right? If you’re seeing this message, your code’s returning None.
Amogh[29:07] I guess it’s one of those things where, you know, giving them a hint is not like going to make them understand the concept better or something. So it’s just better to just fix it and move on.
Kristin[29:17] Yeah, like this is not the point of the assignment. I will just give you the answer.
Amogh[29:21] Yeah.
Kristin[29:23] All right. So I think that as we’re drawing near the end of the podcast, I think what I want to touch on next is kind of like a longer TL; DL: too long didn’t listen where what would you say would be the main takeaways from this work that would inform an instructor’s use of their class forum. Like what should they be thinking about as they sit down and start like figuring out how they want to incorporate the class form into their course, how they want to write the syllabus, or some other documentation of the students. That’s a very broad question. So try putting your teacher hat on and think about what would a teacher want to think about. Or the questions they ask themselves. Given your work on how to use the class forum in their class.
Amogh[30:17] Yeah, that’s a good question because I think at the end of the day, we want to have actionable insights that teachers can actually use in the real world. So from our results, I think I can try to divide it up into like three parts. I think one of the most important things that an instructor can do is collect as much raw data as possible and be very data-driven because at these larger class sizes, it’s almost imperative that you cannot manually go through each and every post, so you kind of need aggregate statistics. So Piazza, like for instance, provides a tool to export post data and all the kind of other metadata that’s happening on the forums. So maybe if you are a teacher, you can export all the previous courses and try to use that as a starting point for some analysis. And there’s also, I think, a lot of room for using natural language processing techniques to automatically cluster the questions and pick up on the common topics.
Kristin[31:22] Okay. What if I don’t have that kind of time?
Amogh[31:25] I think the burden of this should lie on people developing tools. So I think the community should think about developing a very easy-to-use out-of-the-box kind of open-source tools. I just export Piazza, and like I get a nice visualization or something about what is happening on my class. I think that would be like a super great tool for people to have. And this whole project was possible because of this kind of data collection.
The second one is from a student’s perspective. So here I think, like I mentioned, that you just can discourage just pasting a stack trace kind of questions. Encourage them to phrase their questions in abstract terms wherever possible. And I also think that taking a page off, you know, some of these GitHub repositories where that is kind of scaffolding when you’re creating a new issue. Similarly, I think including questions like what is the issue that you have? What do you think is causing the error? Did you try fixing it? What did you try? So having them fill out these forms, that would be an interesting idea.
Kristin[32:32] And so, to interrupt for that real quick, we’ve moved from Piazza to Ed discussion, and Ed discussion actually lets you create templates for different categories. And I have definitely gone in and added that to some of our categories for the class. So like, it forces students to choose a category, and if they pick that particular category, it pops up with, here’s the template you’re supposed to fill out. Technically, the student could like select all and delete it, but it’s at least a starting point. And students usually kind of like acquiesce to the template and start filling it out. And I think that does help. And the other piece for the CS1, especially when we were still using Piazza, that was hard to enforce because we didn’t have templates, but we provide to them kind of like a do’s and don’ts when you are posting a question on Piazza about your code. And try to get them to think about what questions they should be answering in their post because otherwise, the teacher is supposed to respond with please update your post and follow these do’s and don’ts guidelines.
Amogh[33:40] Yeah. Yeah, I would be very curious to see what the template looks like that you created.
Kristin[33:46] Sure, happy to share.
Amogh[33:49] Yeah, and finally, I think in terms of the teaching assistants explicitly, you know, giving them instructions, I think that usually is a great thing to do. Telling them to kind of hold off on answering really quickly until someone else makes a contribution. And I think this is maybe divided up into, like, if it’s a simple logistical question or a simple how-to question, then I think there’s no point in waiting to answer those. But the content-related questions, I think there is some value in holding off and thinking about how to indirectly respond to those, and those can have pretty good impact on the learning.
Kristin[34:27] What’s funny is I actually would say even for logistical questions, let the students sit there for an hour because usually a logistical question is answered in the syllabus or on the website’s calendar or something. And, it’s like, in some ways, some people claim there’s no such thing as a stupid question. And like my pushback to that, as always, but if they could have answered the question with a brief Google search or something, it’s kind of a stupid question because, like, if you had spent 5 minutes to answer it yourself, you probably would have answered it. And, like, I think if the student was going to ask that question, they needed to show a bit of homework saying like, I tried doing X, Y, and Z, and I couldn’t answer my question anyway like that at least shows they tried to answer it on their own. While, I purposely, when I teach, actually, even for logistical questions, unless it’s something that clearly only I know the answer to because it’s only in my head and I didn’t write it down anywhere, then I’ll answer it immediately. But otherwise, I’m kind of like, you know, if you just looked at the calendar, you’d know when the homework was due. Why are you asking on the class forum? And so that, I am waiting to see if a student will respond. And every once in a while, a student will be like, it’s right here on the syllabus, y’all.
Amogh[35:40] Like, yeah. And like the NLP engineer in me is like, can we come up with like an automated way to, you know, respond to those questions from the syllabus document
Kristin[35:50] Let’s see if I can summarize what you said. So for the student, it’s trying to kind of be more verbose, basically, in their post. And for the teacher, the teacher can actually support that by providing templates the students should fill out when they’re asking a question on the class forum. And from the platform perspective, finding ways to kind of use NLP or something to auto-summarize what is going on in the class forum that is not just simple quantitative metrics.
Amogh[36:28] Yes.
Kristin[36:29] So let’s wrap up because we’re definitely near the end here. So how about a proper TL;DR: too long didn’t listen. What would you say is the most important thing you would want our listeners to get out of our conversation?
Amogh[36:42] I think teachers should really try to get as much data as possible from the courses, just, you know, find ways to extract all the raw information as possible from all the modalities and use that to be data-driven in, you know, making decisions for the course. They can also influence how students do help-seeking by having templates in the question form. They can also have TAs kind of wait to respond and ask counter questions and make the students reflect on how to solve the problem. And yeah, I think those were the main three things. And I guess it’s like a constantly iterative process. This is just like general guidelines, but I think you have to like tweak it obviously to your own course and see what works well.
Kristin[37:37] Awesome. Well. Thank you so much for joining us, Amogh.
Amogh[37:41] Thank you so much for having me. I really enjoyed the conversation.
Kristin[37:46] And this was the CS-Ed Podcast hosted by me, Kristin Stephens-Martinez, and produced by Amarachi Anakaraonye. And remember, teaching computer science is more than just knowing computer science. And I hope you found something useful for your teaching today.