Episode 4

Recall Reinforces Learning: Evidence for Retrieval Practice with Jeff Karpicke

[MUSIC PLAYING] STEVEN ROBINOW: This is Teaching for Student Success. I'm Steven Robinow. Today,  my guest is Dr. Jeffrey Karpicke, the director of the Cognition and Learning Laboratory at Purdue University. Dr. Karpicke will discuss his research on learning and memory, particularly the power of information retrieval on long-term learning. All educators and learners should listen to this episode. Understanding the research on retrieval-based learning and retrieval practice might change how you think about teaching and learning. Stick around and learn about an important approach that can help your students succeed in your courses.

Dr. Karpicke's research sits at the interface between cognitive science and education with the goal of identifying effective strategies that promote long-term learning and comprehension. Dr. Karpicke's research has been funded by the National Science Foundation, the National Institutes of Health, the James S. McDonnell Foundation, and the Institute of Education Sciences at the US Department of Education.

Jeff was the recipient of the Janet Taylor Spence Award for Transformative Early Career Contributions from the Association for Psychological Science. He also received a National Science Foundation career award and the Presidential Early Career Award for scientists and engineers. Jeff's Cognition and Learning Laboratory conducts basic and applied research on how people learn. Welcome, Jeff. Thank you so much for joining us on Teaching for Student Success.

JEFFREY KARPICKE: Thank you for having me, Steve.

STEVEN ROBINOW: I'm excited to talk to you. You have a PhD in psychology. Given the breadth of this field, how and why did you decide to focus on how people learn?

JEFFREY KARPICKE: Yeah, my PhD is in experimental psychology. And within that, I'm a cognitive psychologist. And cognitive psychology is a broad field that studies all sorts of things about human cognition, so how we attend, how we think about things, how we make judgments and decisions and solve problems. Honestly, I guess it's tough for me to say exactly why I really got interested in how people learn and how memory works. But I've just always found memory to be a really fascinating ability.

Memory really touches on so many aspects of our lives and our everyday experience in ways that we maybe are not always aware. We have a vast amount of knowledge just about the world. That's all memory in some respect. And we have this incredible ability to mentally travel in time too, which is what we call episodic memory. So we can think back to particular events in our life. It's always been fascinating to me to ask questions like, how do we go about doing that? How does the human mind accomplish that amazing feat?

STEVEN ROBINOW: That's great. Thank you. It is fascinating. It's amazing. Can you tell us a bit about your institution and your student population, particularly students in your undergrad courses?

JEFFREY KARPICKE: Oh, sure, yeah. So I'm at Purdue University. This is my 15th year now at Purdue. We're a large research university. We have—I was just looking up the numbers this morning. We have over 37,000 undergraduates now and something like 12,000 graduates. So all told, we're almost at about 50,000 students. Right now, I'm also the head of my department. So I'm often like an ambassador for the whole department and speaking on behalf of what we do, so I could probably talk for a long time about what I think is so great about psychology at Purdue. But I'll try to keep it brief.

I think that our undergraduates are getting the opportunity to learn from faculty who are doing really cutting-edge, a lot of faculty who care about the undergraduate students too. And so I think we do a very good job of conveying what is really exciting about psychology and what's really contemporary, cutting-edge stuff. We do a good job of giving undergraduate students research opportunities in our laboratories, which is a lot of fun. It's fun for faculty too to have undergrads who come in and are very enthusiastic and excited about getting involved in research.

Your other question was about class sizes, and they tend to be large. Especially introductory-level classes that first-year students are taking can be pretty large. But we'll get into this when we start talking about learning strategies in a little bit because I think there are ways that instructors can make those really good, meaningful learning experiences for students.

STEVEN ROBINOW: Let's get right into that. So let's talk about your research and what you've been studying. I'll just say that we're going to talk today about two of your papers at least a bit. You can take those wherever you like to go. There's a 2006 paper on test-enhanced learning and a 2016 paper on does providing prompts during retrieval practice improve learning. I will say that I've been very impacted by the earlier paper describing the testing effect and introducing me, at least, to the concepts and power of retrieval-based learning. The only reason I wasn't impacted by the second was that by that time, I was an administrator, not teaching so much. So I haven't been able to bring that to the classroom. And we can talk about being chair sometime too. I think that is one of the hardest jobs on campus.

JEFFREY KARPICKE: I'll agree with that.

STEVEN ROBINOW: So congratulations to you on that. So the floor is yours. Could you talk about these papers a bit and describe what you did and what you've learned?

JEFFREY KARPICKE: Yeah, sure thing. If I may, I'd like to zoom out a little bit or step back a little bit, because I still think—even though I've been doing this research for a while now and there's a lot of evidence, there's a lot of research on testing effects and what we call retrieval-based learning, I still think that it's worth it to step back and think about why it's interesting and surprising that testing or retrieval enhances a person's learning. And I think it has to do with the way that most people tend to think about how their minds work and what their minds are.

We tend to think about—and I think about this on a day-to-day basis myself as well—we tend to think about our minds as places in our heads that store a lot of knowledge. And so if that's what a learner thinks, then they might be thinking, well, gosh, I really need to take this knowledge that's out there in my class, in a book, in a lecture, and I need to figure out, gosh, how do I get that into my head? It is a legitimate problem. But for some students or some learners, that might be the only thought or the only problem is how do I get this information in my head. There's not much consideration, I guess, given to, well, how do I get it back out of my head when I need it later on? That would be the retrieval part of it.

And usually in education, we give students tests as a way to see what they know, to measure what's in their mind. And then this is where what we've been doing now for 15 or so years gets surprising, is that the test is doing much more than just measuring what's in a person's mind. Every time that a person is testing themselves or that a person is retrieving some knowledge, that knowledge is actually changing. And that's what's so surprising and still, honestly, mysterious about what we call retrieval-based learning.

We've been able to leverage this fact and use it to—I guess I would say harness the power of retrieval, to say, well, retrieval is really producing a lot of learning. How do we take that and use it so that it benefits the students? What are the effective ways to do that in a classroom with a large number of students, or what are effective ways that a learner can do that on their own?

I wanted to zoom out, I guess, and give that big-picture bit about why retrieval—the fact that retrieval produces learning can still be surprising. Because there is a lot of evidence on it now. And so in my field, we've gotten to the point where no one's really debating the fact that retrieval is improving learning and that it's really a pretty powerful tool. Now we've gone beyond that into the more specific questions about, well, what if you do it this way, or is this way more effective, and so on?

STEVEN ROBINOW: Great. Fantastic. Since this is an evidence-based program, maybe you could talk about some of the basic data from your papers and about how you know that retrieval-based learning works.

JEFFREY KARPICKE: Sure, yeah. The paper that you cited from 2006 is, I think, a pretty straightforward example of the power that retrieval can have for learning. So in that study, we had—and this was done with college students. The students were reading brief educational texts, kind of like what a student might be doing when they learn on their own, but in a miniature version since this was all done in a psychology experiment in a laboratory. The students are reading these texts. And we had one group of students that read the text repeatedly.

So after they were done reading it, they just kept on reading it, and they could do it a number of times in a row. A second group or a second condition involved the students putting away the material and then trying to write down as much as they could remember without looking at it. So it involves retrieval. It involves thinking back, going into your mind and actively trying to retrieve and reconstruct that knowledge. And we actually had a condition where we had the students do that repeatedly and without any feedback. So the students are writing down whatever they can.

And I actually tested these students myself when I was a graduate student. And so they would write down whatever they could. And I took the sheet away from them and said, OK, here's another blank sheet. Go ahead and do it again and again. Yeah, it's not ideal, I would say, definitely not something you would want to do in an actual education setting. But we were doing it because we were matching the total amount of time that the students were spending with the material and just purely varying whether they were spending that time reading or spending it actively recalling.

And any effect that we saw would be due to just the process of recalling itself. Because if we had given students some feedback or let them reread, that would have been great for learning. But then you've got a mixture of things going on. You've got the retrieval part. You've got the learning from feedback part. We were just interested in seeing, what did the learning gains look like when people are purely retrieving the knowledge?

And what was so interesting, I think—well, I guess there were a number of things about that study that turned out to be interesting. When we brought people back after a delay, so a few days or a week later, and we had them try to recall the material again, the students who had practiced recalling the first time during learning recalled much more later on than the students who spent the same amount of time with the material but just rereading it. In my view, the other thing that was really interesting about that study is that during that first session in the laboratory with us, we asked the students at the end of that learning session, how well do you think you're going to remember this material?

And even though the students who practiced retrieval, who were repeatedly tested on the material, even though they remembered a lot more after a week, almost all the students said that they were going to remember more after they had repeatedly read and repeatedly studied. That's getting at a fundamental challenge that we still face, which is there's something about being repeatedly exposed to material, or repeatedly reading it, or now—now that we are in the video learning era, I think, in education, it might be repeatedly watching something on a video—that tends to inflate people's self-assessment of how much they have learned even though it may not be nearly as effective as what students think.

STEVEN ROBINOW: What I find interesting there, of course, is that students have grown up, at least through high school, being taught to read their notes, read the book, rewrite your notes. That's the method they've been taught. And they believe that that's successful because maybe there's some evidence that it is, because they're in your class.

They've gotten to where they are. So they've been successful with this method of just reading. Now you've introduced a new method that they have no idea about. It's foreign to them. It's uncomfortable for them. And they go with a thing they know, right? This is what I know, and this is what I'm going to do. Of course I'm going to learn better this way.

And students in the uncomfortable paradigm have never experienced this, although, of course, they have experienced this in other ways in their life because you don't learn a sport by watching videos about it over and over. You go out and practice it. So the same sort of thing. You actually have to do it. So the engagement is fascinating.

JEFFREY KARPICKE: Yeah, that's exactly right. This is a very reliable pattern. When people are repeatedly reading or repeatedly experiencing some content, they tend to grow increasingly overconfident. So their self-assessments tend to be larger than what their actual learning looks like. And as students do repeated retrieval, they tend to grow increasingly underconfident.

There's something about retrieval and, I think, the challenge, the effort involved in it. You said maybe that it's not as comfortable or certainly that it's different, maybe, from what people are doing. It could be all of those factors tend to make students a little bit less confident. And by that, I mean, their predictions of how well they're going to do just don't match up as well with how much they have actually learned. So it's a big challenge, yeah.

STEVEN ROBINOW: In the book, The Invisible Gorilla, the authors, Simons and—

JEFFREY KARPICKE: Chabris.

STEVEN ROBINOW: Chabris—thank you—talk about this expert versus novice situation. And as you were saying this, it occurred to me that our students are novices in the field of learning. They've been learning, but they're novices in understanding how learning works and what is impactful for their learning. And what they talk about in this chapter on expert versus novices is that novices always overestimate or tend to overestimate how they're going to do. And as you become more expert, you become very accurate at understanding where you are in the spectrum.

So I wonder if students who have been exposed to retrieval-based learning, if, after they've experienced it, if, after they've had an opportunity to talk to you about it cognitively—I mean, what's going on and why it works, and if, after that, after some time, after they begin to understand how it works and understand what's going on, if they then become more accurate in what they do and don't understand as a result of this practice. That was a complicated question.

JEFFREY KARPICKE: Well, that's a fantastic research idea. Someone should do that work. [LAUGHS]

STEVEN ROBINOW: OK.

JEFFREY KARPICKE: Oh, yeah, absolutely. This is great because there's a lot of contemporary attention on this issue right now, which is, OK, we know that retrieval is an effective learning tool. I guess one question that I brought up briefly earlier is, well, what's the most effective way to do it? But the second question that you're getting to is, how do you get learners to engage in this kind of strategy and persist with it?

I'll give you the honest answer, which is not satisfying, but the honest answer is we don't know yet. We haven't identified a gold standard, effective—I guess you could call it an intervention strategy or a way to communicate this in the most effective, surefire way to students to ensure a large percentage of adoption of this kind of strategy in normal studying. But I think you're right that there's something to experiencing the benefits in a shift from being a novice to toward more expertise. That's going to be really important in answering that question.

STEVEN ROBINOW: Really interesting. So that brings us to your 2016 paper where you looked at free retrieval, unstructured retrieval versus a structured retrieval pattern in which you ask students—well, maybe you can describe what you did and maybe, perhaps, the surprising results of that, in fact.

JEFFREY KARPICKE: Yeah. Well, in that work—again, let me zoom out a little bit and provide some background.

STEVEN ROBINOW: Absolutely.

JEFFREY KARPICKE: I think where the action is with retrieval practice is any activity where you're asking learners to actively reconstruct their knowledge is involving some retrieval-based learning to some extent. So there are so many different activities that we do in all walks of education—and really, at all levels of education, there are so many activities that we do that could be infused with some retrieval. They could be even converted into a retrieval-oriented kind of activity.

And the other bit to it is that when retrieval is really going to work well, the learner has to be successful at retrieval. So if you give someone tests or retrieval opportunities, and they just can't successfully retrieve the knowledge, that's not going to produce a whole lot of learning. That's not the most effective circumstance. On the other hand, you don't want to make the situation so trivially easy that it doesn't involve some kind of challenge or some kind of effort.

So there's this really tough balance between having learners be successful while they're retrieving and also making the retrieval activity challenging or effortful. And I wanted to lay all that out as some general thoughts because that was really the background for the stuff that we were doing in that paper in 2016, which is that if you give people some material to read and learn about and then you say, OK, just write me a summary of what you just learned about, I would call that free recall. You're not giving people a whole lot of support.

There's a whole lot of stuff that they have to output or to produce. They can probably recall stuff in any order, although people usually try to—if you read something, you tend to recall in a narrative order. But it's a free-recall method. And there are all kinds of other ways that we might be able to give people support or prompts or cues to try to boost their ability to be successful and hopefully not short-circuit the effort that's involved. Keep the good, effortful, challenging nature of it, but also give people some support.

So we tried that in a few experiments where we were comparing free recall to retrieval tasks where we gave the learners prompts. And we were thinking, boy, this is going to be so great. [LAUGHS] We're going to give these prompts, and it's going to be amazing, and it's going to be so effective. And it ended up that we didn't find any clear-cut benefit, at least from the particular way that we were giving people the prompts during recall. We didn't find clear-cut benefits beyond just having learners freely recall material.

I'm not saying that that means that prompts never work in any way, shape, or form, but I did find that surprising. And it does speak to the broader point that whenever you're thinking about, how could I get my learners, my students to engage in some more retrieval, there are these competing demands. You want to give them an activity where you're not killing the students. They're going to be successful in some way, shape, or form. But it's also got some challenging elements to it. And that's a tough balance to strike.

STEVEN ROBINOW: Both paradigms, though, whether you gave them a free-recall prompt or a structured-recall prompt, using either method to activate retrieval practice was still better than not practicing at all.

JEFFREY KARPICKE: Oh, that's right.

STEVEN ROBINOW: It's just that you couldn't discriminate between the two.

JEFFREY KARPICKE: Right.

STEVEN ROBINOW: So both are effective.

JEFFREY KARPICKE: Right, yes.

STEVEN ROBINOW: The hypothesis was that a structured prompt was going to have better results than a free-recall prompt. And that didn't come out to be both prompts, free-recall and structured-recall, significantly improved long-term learning.

JEFFREY KARPICKE: Yeah, that's exactly right.

STEVEN ROBINOW: In both cases, you are activating retrieval practice. You're just using two different prompts to activate retrieval.

JEFFREY KARPICKE: Yeah, exactly.

STEVEN ROBINOW: The unstructured prompt uses a single open question versus the structured prompt, which uses a series of questions intended to activate specific memories and recall.

JEFFREY KARPICKE: That's right, right.

STEVEN ROBINOW: And what you talked about before is the notion of not making it too hard, not making it so hard that they fail, because you're really not learning very much when you're failing or constantly failing. So the repeated failure isn't helpful, but you're putting them in a zone of proximal development where it's close, or it's a zone where they can struggle, but they can do it. And that's what you are suggesting is the magic place that you're trying to find.

JEFFREY KARPICKE: Yeah, always. And it's always going to depend on all sorts of factors, like what the learner already knows and what the nature of the material is. The example that I always give about making retrieval effortful versus successful is I give the example of when you meet a new person, and they tell you their name. You can say their name over and over to yourself in your head, which is kind of like retrieval.

Every time, you're retrieving it in order to maintain it in your mind. And if you do that, that's what we would call massed practice. And what's going to happen is a few minutes later, you're going to be like, I have no idea what that person's name was. You've completely forgotten it. We've all had that experience where you meet someone, they say their name, and moments later, we feel like we've forgotten it.

That's because you were engaging in some sort of retrieval, but it didn't have any sort of challenging or effortful kind of element to it. It was as easy of a retrieval as possible. So we make this distinction in the cognitive psychology literature between massed and spaced practice, spacing or distributing study or distributing retrievals over time. And that's just an example of how a massed condition can make retrieval very successful, almost not effortful at all, and produce really poor memory later on. You want both of these things. You want retrieval to be successful but also to involve some effort or some challenging.

And I like the way you put it. It's similar to the idea of having a zone of proximal development—that would be the developmental way of saying it—but a zone of proximal learning where there is some kind of space there that you want to hit where it is challenging. And because it's challenging, that's going to produce a lot of learning.

STEVEN ROBINOW: Could we talk a bit about the implementation? How do you or how do people you know—how do they implement this practice to help their students learn, and therefore, help their students succeed?

JEFFREY KARPICKE: I think that the most common way that people are implementing retrieval in higher education, at least, in college classrooms, is giving no-stakes quizzes. I would call them no stakes because you're not really doing it necessarily for the purpose of collecting a grade. You're doing it for the purpose of providing a retrieval opportunity.

Yeah, of course it matters whether students are correct on the quiz, but you don't need to tie points to it or any sort of—I don't know—anxiety-producing aspect to it. It's just, I'm going to give you this opportunity in my class to test yourself to retrieve what we just learned, or maybe retrieve, at the beginning of class, what we talked about during our last class meeting. And just a little activity like that engages some retrieval on the part of our students. And there's actually very solid evidence now that that's an effective way to implement retrieval-based learning in the classroom.

What I do—so I teach a course on learning and memory that's at the upper level for undergraduate students. So most of the students in that class are juniors or seniors. I will end every one of our class periods with a brief writing period. And I've told them at the beginning of class all about retrieval and why it's good for their learning, and so on, so they kind of know why I'm asking them to do this. I tell them, put your notes away. And I tell them I count it, I think, as points for completing it, but it's not going to be graded for content, per se.

But I give them just a few minutes, and I do give them some kind of a specific prompt or question about what we talked about. And I'm telling them just to write down what they remember about this topic that we learned about. Sometimes it's more open-ended. Sometimes it's a more specific question. But it is ending each one of our meetings with an opportunity to stop and retrieve what we just talked about.

It's interesting. Students actually tell me, at least—maybe they're lying to me—but they tell me that they enjoy doing that because it gives them a sense of like how well did they pick up on what we talked about. But they also know, because we've been talking about this in my course, that this is something that's solidifying their own learning, and it's going to help them to remember, in general, what we talked about the next time around. Those are a couple really quick—and they're pretty simple ways to infuse some retrieval into the classroom.

STEVEN ROBINOW: And that's what we're looking for. We're looking for simple practices that people can easily implement that have significant impacts on student success. So this is exactly why I invited you. I really wanted to talk about this. Again, that raises so many interesting questions—the other, as you said, that you're talking about students at the end of the class to write a reflection on what they've learned or whatever exactly the prompt is, but you ask them to write. You don't ask them to stop and think about it. It isn't like, hmm, think about that in your head.

Alternatively, of course, you could have asked them to have a discussion with each other. I don't know this literature, but there must be buckets of literature out there talking about the importance of how students engage in the thinking, whether it's written or verbal. It's got to be completely different than just, oh, think about this in terms of consolidating their learning. That's a question. I know nothing about this literature.

JEFFREY KARPICKE: Well, you said buckets, but there's actually not as much as you might expect. Just making that comparison between I'm going to retrieve this by writing or I'm going to retrieve it by speaking it or by having a discussion with another person—maybe it's because that research, at least the research in terms of having a discussion with another person, it's maybe a little bit more challenging to do. But it's kind of surprising there hasn't been more in terms of direct comparisons.

I ask the students to write, and I think that writing things out is actually a very effective way to do retrieval because it helps you literally to look at what it is that you've retrieved. This is what the few studies have shown as well. When you have someone write down what they're retrieving versus speaking it—and maybe this is very obvious. It seems kind of obvious to me. It's a lot easier to keep track of what you have recalled and retrieved when you've got a written record of it right there in front of you versus when you are doing it, and you're speaking it, and then the moment the words come out of your mouth, they're gone, and you don't have that same kind of record.

So it's much easier to monitor what you have retrieved when you're writing it down. That's why, when I'm telling students to try to use retrieval on their own, I encourage them not just to read something and then think about it in their head, because they may not be retrieving as completely as they could. They may be retrieving in a less than accurate fashion if they're just kind of retrieving all inside their own head. I encourage them to write things down and then to take what they've written down and compare it to whatever it was that they were trying to retrieve in the first place. But I guess I will say that even stopping, pausing, and retrieving in your own mind is probably better than nothing.

STEVEN ROBINOW: Probably somehow, I think that when students stop to do that—just internally think without writing it down—I think they deceive themselves about what and how well they actually know something. And when suddenly you actually have to put real words to it, you have to use the language that you want them to learn, that I want them to learn. Because there is a vocabulary here in addition, that I think somehow, putting it on paper or verbalizing it is different than just, oh, yeah, I know that. Mental check. Mental check. Mental check. Students go down the mental checklist, and they say, I know that. I know that. I know that.

Well, go ahead and try to describe it to your mom. Can you go to your mom's and sit at the kitchen table and describe it to her? And if you can't, then you don't really know it. But with a simple checklist in your head, you haven't really tested yourself. You haven't really demonstrated. You think you have internally, but it's not done.

JEFFREY KARPICKE: Yeah. I think you're right. I think you're on to something there.

STEVEN ROBINOW: OK. So we're going to be working together in the future now.

JEFFREY KARPICKE: [LAUGHS]

STEVEN ROBINOW: Is there anything that I didn't ask that you'd like to talk about with the people that are listening?

JEFFREY KARPICKE: Well, you did mention—I believe earlier, you mentioned that you were going to, I hope, talk to my colleague Robert Ariel.

STEVEN ROBINOW: Robert Ariel will be on the show in the coming months. I don't have an exact release date yet. I am excited to talk to him in the near future and put that on the air.

JEFFREY KARPICKE: That study was trying to get into this question of how do you get learners to engage in more retrieval. And I guess I won't steal Robert's thunder too much.

STEVEN ROBINOW: Don't give it away.

JEFFREY KARPICKE: The good news is that even just a very simple instruction about here's what this is, and it's effective, and you should really—this is a good idea. Come on, guys. This is good. You should do it. That simple instruction worked really well for encouraging learners to engage in more retrieval. And whenever—I guess I will say something that I'm not sure Robert would say, because whenever I talk about that study, I convince people, or I try to convince people, that it's actually plausible that that instruction would not work at all, that people would just not be receptive to this kind of, yeah, this is good for you, you should do it. Because how often do you hear, well, exercise is good for you. You should do it. Or you should eat your vegetables. Broccoli is good for you. You should eat it. That doesn't mean that people are going to do it.

But it was really nice that that simple instruction moves the needle pretty significantly in how often people were engaging in retrieval. It's maybe just not as bad as having to eat a bunch of broccoli. [LAUGHS]

STEVEN ROBINOW: That's fascinating. This is going to tie in so well with another episode this season with Harriet Schwartz talking about connected learning. Because as I suspect, as you said, it might not have worked at all. But one of the reasons it undoubtedly works is because—I don't know Robert, never met him, but I suspect he has a good relationship with his students.

They trust him. And so when he tells them something, they, the bulk of the class, is willing to believe him. I suspect that somebody who has a terrible relationship with their class can say the same thing and would have no impact at all. So it also demonstrates, I suspect, that the relationship you have with your students is fundamentally important to how you're perceived and how they take in what you tell them. And we'll talk about that with Harriet.

JEFFREY KARPICKE: That's a very, very good point.

STEVEN ROBINOW: Well, one of the interesting things about active learning is that people publish papers on different activities or different things to do all the time. And of course, they do it extremely well. And so it has a positive learning impact for students. Other people can take the same practice, and it doesn't work for them. And it can be frustrating. The issues are so much more complicated because we're dealing with people. Our students are people. They have lives. We have relationships with them. And it changes everything. It's super interesting.

So I'd like to spend the rest of our time today talking about your students, you, your motivation, and perhaps a few interesting stories. The first question: How has your work in learning and memory and education reform—because that's what you do, right, education reform—how has that impacted your identity or how you view and interact with the world outside of academia?

JEFFREY KARPICKE: Wow. So these are the deep questions. [LAUGHS]

STEVEN ROBINOW: Well, I don't know. It depends on how you answer.

JEFFREY KARPICKE: When I first went to graduate school, it was in a cognitive psychology, experimental psychology program. And that really is where my heart lies, is in randomized controlled experiments in laboratory settings. But given all the work that I've done trying to make the research relevant to education, I guess you could say that's changed my identity. It certainly has changed the way we do our research because to make this kind of—as you said, educational reform-oriented research, to make that effective, it really needs to be a partnership with the teachers and stakeholders that you're working with in education.

I guess I talked a little bit earlier about a lot of the research that we've done has been with college students or in college classrooms. We've done a fair amount in elementary school classrooms as well, and that's where the partnerships—that's what I think has really changed me and the way that I've thought about research over the years, is I say, hey, here's this great finding from the lab, but how are you going to make it relevant to the day-to-day life of a fourth grade teacher? That's a great challenge. That's a challenge that we've worried a lot about. And that really does change the way that you think about what's valuable in research and how are you going to talk about what you do in a way and make it meaningful to the people who could use the outcomes.

STEVEN ROBINOW: So let me take that in two directions. First, back to postsecondary education. Have you looked at impacts on equity gaps, also called opportunity gaps or performance gaps? They are the performance of your majority students versus your historically excluded students or minoritized students and/or first-gen students. Have you looked at the impact of retrieval-based learning on the opportunity gap?

JEFFREY KARPICKE: I have not myself. There is at least one study that I can think of—I think it came out of University of Texas maybe five years ago or so—that did look at including retrieval-based learning activities like quizzing or testing and achievement gaps and showed that it was, as you would hope, an effective way of evening things out, of—I don't know if eliminating—eliminating might be too strong of a word, but substantially reducing achievement gaps. And I've never done any kind of research like that.

But I do think that one of the ways that cognitive science can be really impactful is by identifying these effective strategies and then figuring out ways to get them in the hands of all students. And so yeah, there may be some students who, by virtue of their circumstances and their upbringing and their privilege, have a lot more tools and a lot more strategies that they come to college with.

How do we make sure that all of our students who come to college, and then I would hope all students at earlier levels of education as well, are getting the opportunity to have their hands on some of these effective strategies? Have someone explain to them that this is an effective way to—and I'm speaking well beyond just retrieval practice right now, but explain this is an effective way to go about studying. This is an effective way to go about organizing your time in the classroom or with all of the classes that you have to be taking and learning about.

That's a major challenge that the entire field should be thinking about and addressing.

JEFFREY KARPICKE: Absolutely—getting people to adopt the practice, getting both the students to adopt the practice and the faculty to develop the class structure that supports student learning to adopt these counterbalancing practices that support these structures. So the second thing I want to ask—in what you do on a daily basis, when you're walking down the street and not on campus, how do you see the world differently? How does what you do professionally impact your view of the world and how you interact with it?

JEFFREY KARPICKE: Over the years, as I've thought a lot more about our minds and our memories and what they are and what they're actually designed to help us do and accomplish in the world, that does kind of change your perspective on what is this ability we have in our brains to retain something about the past and use it right now in the present in order to help us get around in the world. That's, to me, what memory is really all about.

And that is something that I think about periodically in everyday life, walking around, talking with people who may remember events very differently, talking with people who may be try—or you're trying to remember something, and you're going, oh, my gosh. I can't remember this at all. I'm completely failing. Those are maybe the small, everyday moments that, as a memory researcher, I think about in a different way than I did long ago before I started down these paths.

STEVEN ROBINOW: What has been your greatest moment? Is there a moment that stands out in your mind? Perhaps you saw a student when the light bulb went on. Or for the class, is there a moment that sticks with you that helped support you as a faculty member? Because it's hard. It's a hard job.

JEFFREY KARPICKE: It is, and it's a complex job because there's so many different dimensions. I say this a lot talking with—well, all faculty members, but especially—now I've got junior faculty members on my brain right now. And I think about our faculty members who are coming in, and they're starting up a laboratory. They're starting teaching. And eventually, they're going to be expected to do something to serve. It's multidimensional and complex. So I certainly resonate with what you just said there.

From a teaching standpoint, I've been very fortunate to work with a lot of great students in my lab, undergraduates especially. I've had many great graduate students who've got their PhDs. And we were just talking earlier. Robert Ariel was one of a number of postdoctoral researchers who was really wonderful in my lab. But it's a lot of fun having undergraduate students come in and take an idea and run with it in a way that maybe is less inhibited than a graduate student who might be concerned about, oh, is this going to turn into a publishable thing, or is this going to turn into a master's thesis-style project? That's been a lot of fun.

And the undergraduate students, too, will often have great what-if questions. What about this thing I observed from students studying at the student union? Does that really work? Could we actually do a laboratory experiment to see if it works? And that's what really stands out.

STEVEN ROBINOW: I think this is a good point to end our discussion today. Jeff, I'd like to thank you so much for the time you've spent with us. We look forward to your future work on retrieval-based learning, student engagement, and student success.

JEFFREY KARPICKE: Thank you for having me.

STEVEN ROBINOW: For more information about Jeff Karpicke, his research and favorite papers, please go to our website, teachingforstudentsuccess.org. Thank you for spending time with us. To keep up on the latest releases, please subscribe to Teaching for Student Success on your favorite podcast app. Thank you for caring about your teaching and your students. Thank you also to the growing army of education researchers out there working to improve the learning experiences and learning environments for all students with the goal of providing opportunities that help all students succeed, an honorable and important goal.

Those of us at Teaching for Student Success would love to get your feedback. Please contact us through our website at teachingforstudentsuccess.org. I'd like to give a quick shout out to one of our listeners, Andy, who is mowing his lawn today, hopefully not the hedges. Teaching for Student Success is a production of Teaching for Student Success Media. Let's end this podcast with some music by JuliusH. Some of Julius's music can be found on Pixabay.

[MUSIC PLAYING]