Kasia Muldner: Artificial Intelligence and Student Learning

As part of our AI in your Community Series, I recently had the opportunity to sit down with Kasia Muldner, an assistant professor in the Institute of Cognitive Science at Carleton University. She works with intelligent tutoring systems to better understand student learning, problem solving, and creativity and the factors that affect them.

Tara Chklovski: Tell me a little bit about what kinds of problems you’re working on.

Kasia Muldner: I work in the field of learning and cognition, with applications to intelligent tutoring systems. My research focuses on student learning, including both cognitive and affective components. I’m particularly interested in factors that influence student learning and ways to improve it, which is where technology comes in.

Tara Chklovski: Can you explain what you mean by cognitive and affective components? What are the differences and how do you define them?

Kasia Muldner: Sure. Cognitive factors have traditionally been linked to domain knowledge – like the knowledge needed to isolate a variable in an algebra equation by subtracting some value from both sides of the equation.

Affect on the other hand is commonly used to refer to feelings, moods, or emotions – although these terms have distinctive definitions, they are often used interchangeably. When it comes to computer tutors, the field used to focus on designing support for cognitive factors, like having the tutor give the student domain hints. However, there is a lot of evidence that how students are feeling when they’re learning really influences what they learn and even whether they learn at all. So there is now a lot more work developing tutors that can both detect how students are feeling and respond to that emotion.

Tara Chklovski: What are some problems that you are working on now?

Kasia Muldner: Some of the projects involve building tutoring systems, both for pedagogical reasons and to help us answer certain questions. For instance, one of the tutoring systems we recently built with three of my honors students looked at how different levels of assistance within a tutoring system influenced learning outcomes from interacting with the system, and also influenced how they felt while working with the system.

For example, in one of the projects, we gave people problems to work on and gave them examples to help them – so the examples were the “assistance”. Some students were given examples that were very similar to the problems they were working on, while others had examples that were still helpful but weren’t so closely aligned with the problems being solved – this made copying from them more difficult. Our hypothesis was that the less similar examples would be more helpful for learning because it would make students more constructive and engaged with the material. This is indeed what we found. We also found that the lower levels of assistance didn’t have a negative influence on affect (how people were feeling about the problems or subject matter). So we measured both affect and cognition in that study.

Tara Chklovski: That’s really cool. What are some other questions that you are testing using these tutoring platforms?

Kasia Muldner: I work with artificial intelligence systems but I also work with more traditional instructional materials. So for example one project that my students and I have been working on looks at the impact of mindset interventions–encouraging students to develop a different mindset towards their learning abilities–we are focusing on programming activities in this study.

I teach programming and I love it. But while programming can inspire a lot of interest it can also inspire a lot of anxiety. Some people shy away from it because they believe that they just don’t have the ability to learn it and there’s nothing they can do to change that. This relates to Carol Dweck’s mindset theory on fixed and growth mindsets, where if you have a fixed mindset and believe that your ability cannot be improved then you also believe there’s no point in continuing that activity because you’ll never be good at it. That can have all kinds of detrimental effects, so it is important to shift people towards a less fixed mindset. There has been work in the past showing that mindset interventions, ones trying to encourage students to develop a mindset where you believe that effort will improve your ability, has helped students in traditionally challenging domains like math and programming.

Tara Chklovski: That’s so interesting. We reference Carol Dweck’s work in our parent trainings to help them encourage their children to develop more of a growth mindset. How did you do those sorts of interventions in this project?

Kasia Muldner: We ran a lab experiment where we compared what happened when people received an intervention designed to shift people towards an incremental mindset, compared to when people didn’t receive any encouragement. We found that after receiving the encouragement people were willing to invest much more effort in a programming activity (in other words, more time), but that it didn’t have a positive impact on their outcomes. So they weren’t necessarily better at the activity even though they were investing more time into it – we are following up that work with qualitative analysis to dig into why we got the results that we did.

Tara Chklovski: That’s interesting. Did you also do work where you videotaped novice programmers and you looked at their frustration levels?

Kasia Muldner: Prior to coming to Carlton I did some work with various collaborators in the United States looking at how you can measure emotion, like boredom or frustration, using various sensors as well as data from students’ interactions with educational technologies. We built models that aimed to automatically detect how students were feeling without having to ask explicitly for that information, so that the system would not be intrusive. The goal behind these models was to integrate them into the tutoring systems and so if students were feeling particular emotions like frustration or anxiety, the system could intervene and offer assistance as needed.

Tara Chklovski: What type of sensors did you use, and what did you find out?

Kasia Muldner: We used a skin conductance bracelet, designed to measure galvanic skin response. Our skin provides a lot of information about our physiological responses to certain events and related emotions. So if you’re feeling very frustrated, or if you’re feeling super excited you’re going to have bigger spikes in your galvanic skin response, which we measured with these bracelets. We also had a number of other sensors, like a pressure sensor and a chair pad to see how much people were shifting around. I also used an eye tracker in one of the studies to observe how people’s pupils were reacting – a larger pupillary response was associated with moments of delight during learning.

We compared the models built using data from these sensors and data coming from interaction with a system to models that detected affect using only information coming from a student’s interaction with the system. For instance, if someone is entering a lot of incorrect answers quickly you might infer that perhaps they were frustrated. We found that if we added information from the sensing devices, the accuracy of the models increased.

But there are some disadvantages for using sensing devices as well. For instance they are expensive and they’re not scalable yet. But I think it is important to explore their utility, particularly since they are becoming more accessible, so exploring what they can do for human computer interaction is important – it’s also fun because they give additional information about the user that we would normally not have access to..

Tara Chklovski: Right. And I think that’s exactly why these devices are becoming accessible even to children and to laypeople. So just a couple more questions. What inspires you?

Kasia Muldner: I’m fascinated by puzzles – not just the traditional kind but also problems in general without a clear answer. That’s probably why I’m also so interested by mechanisms of learning, both human and artificial . The cool thing about AI is we get to conceive of intelligence in a more general way – we’re not limited to only thinking of intelligence from a human perspective, but we can think about other forms of potential intelligence and how they can be realized in different AI applications.

Tara Chklovski: Answering big open problems is not easy – what types of difficulties have you encountered and what helps you get through them?

Kasia Muldner: One of the things I find really helpful when I get stuck on a problem is stepping away from it. It’s hard for me to do that because I get really stubborn about trying to solve it, but stepping away from it, and for instance going for a walk is really beneficial.

But more speaking more broadly, the difficulties I face are often related to the fact that I’m really interested in finding ways that technologies (and artificial intelligence in particular) can help education. However, it takes a great deal of effort to implement the software underlying these systems. Doing so used to require a very strong technical background – you had to know how to program, and it would take hours and hours of development time. That made it less accessible for people who just wanted to prototype these types of systems more quickly. But in the last five to ten years different platforms have been created that allow us to rapidly create and test tutoring systems. For instance, in our recent work we used one called CTAT, which stands for Cognitive Tutor Authoring Tools. It’s a platform for developing web-based tutors developed by Vincent Aleven and his colleagues at Carnegie Mellon University. They’re online which makes them very accessible as well. So in theory, once you launch your tutor, anyone can use it.

Another strategy is to build more pared down systems. For instance, building a pretty basic system that doesn’t have AI capabilities, and then exploring what AI capabilities might be useful by mimicking them with a human operator mimicking the machine actions. That helps determine if that capability is actually useful and worth building a complex system.

Often it’s not only building the technology that we’re interested in, but also the kinds of research questions we can answer with it. How can the technology inform how people learn and how people feel during that learning process? These technologies have strong potential to help answer those types of questions because they can log process data – we can then that use machine learning on that data to identify all kinds of interesting events, like moments of learning or moments of delight during learning, and variables influencing these events.

Tara Chklovski: That’s very helpful.

Kasia Muldner: One of the things I find really inspiring is that alongside this proliferation of machine learning, tools have been made available that allow pretty much anyone to go out there and play with machine learning technologies, which is amazing.

WEKA is one popular package that doesn’t require any technical background. It lets you put in a data set and use either supervised or unsupervised machine learning to build various models that, for instance, can act as classifiers for your data, and they actually learn from the data.

Tara Chklovski: Interesting. And where do you get the data sets from?

Kasia Muldner: So that’s a good question. There are a lot of data sets out there, but you can also develop your own. Last year, I had some students in one of my upper level classes who were interested in machine learning problems related to their own interests. For instance, several students were interested in music, and specifically analyzing the affective content of their favorite music artists. So they build their own data sets based on textual content of songs of their favorite artists – they used a natural language processing tool developed by Scott Crossley and his colleagues called SEANCE (which stands for Sentiment Analysis and Social Cognition Engine). SEANCE pulls out affective content from text like a song, and the students selected the songs they wanted to focus on, analyzed them with SEANCE, and compiled a file which they put into WEKA – WEKA then let them answer questions about things like affective content of songs by certain artists.

In general, you can answer whatever questions you want. The data is already out there because there is so much everywhere. I recently read about a 16-year-old woman in India who developed a neural-net classifier that can diagnose whether you’re at risk for developing blindness from diabetes. She did this by accessing data available online, and then she 3-D printed a device you can put on your iPhone to scan your eye. The fact that we can do these kinds of things is really inspiring and pretty amazing.

Tara Chklovski: Exactly. That’s why we want to open up this world of possibility and invention. So the last question: how do you think AI will strengthen communities?

Kasia Muldner: Personally, I’m really hopeful and excited about the prospect of AI improving education. One of the inspirations for this dream are based on a science fiction book called Diamond Age: Or, A Young Lady’s Primer, by Neal Stephenson which I read when I was younger but that I still find inspirational. In the book, there’s an orphaned girl who finds a textbook, which is essentially an AI technology that becomes her mentor and her teacher and guides her through learning everything she needs to learn. That story really highlights the potential for artificial intelligence to improve individual peoples’ experiences with various educational topics. I think we have the opportunity to make a real impact in people’s lives.

Tara Chklovski: Absolutely. Thanks so much Kasia.