Internet of Things

DIAGRAM: Are you researching the Internet of Things in the classroom?

SD: We work on part of the IoT vision, the sensing part. We’re interested in what kind of information can be gathered in classrooms and how it can be interpreted in useful ways for teachers and students.

In one of our projects, we’re recording what teachers are saying using a simple, unobtrusive microphone. We can get good enough recordings to support automatic speech recognition, so we can give teachers useful feedback on their pedagogy. For example, we can tell when a class session includes discussing authentic, open-ended questions with students and when it doesn’t. We can also tell how often teachers are using the topic-specific language students need to hear. We’re working towards an app that can give teachers real-time feedback on things like this.

DIAGRAM: What about other sensors?

SD: Other groups have tried to do something similar with video of classroom activities, but it’s hard to make sense of what you get. Also, we’d like to be able to process what students are saying, but we’ve not been able to develop a microphone setup that can give us interpretable data. I don’t see a breakthrough on that for at least five years.

On the other hand, we are making good progress with eye tracking. Lab studies have shown that eye-tracking data that tells what learners are looking at when they are working on an activity on a computer can tell you when a student’s mind is wandering.

DIAGRAM: Is that important?

SD: It can be very important. If a student hasn’t paid attention to one part of a lesson, and they try to push on to new material, they can get lost. Letting them know they need to look at something more carefully can help them stay on track.

DIAGRAM: You mentioned lab studies. Can you get the right kind of data in a classroom setting?

SD: It looks as if you can. Inexpensive eye trackers are now on the market, and we’ve shown that students can set these up and calibrate them on their own, following instructions we provide. We’ve also shown, though so far with adults, rather than children, that we can collect useful data over periods of as much as two weeks without needing to intervene or recalibrate the equipment. So, we think we’re close to being able to do this in regular classrooms.

DIAGRAM: What about cost?

SD: We think it’s going to be inexpensive. Eye tracking has become popular among gamers, so eye tracking might be built into laptops in a few years.

DIAGRAM: Sensors for lots of other things are now available, like heart rate monitors in smart watches and much more. Do you see possibilities there?

SD: We’re working with adult information workers, as well as with students, and we’re using a wider range of sensors, including smart watches and location beacons.

DIAGRAM: What are location beacons?

SD: They are devices that software on someone’s phone can detect when nearby. Participants place these in key locations in the workplace, or on something portable, like a laptop. Recording starts when the phone detects a beacon and lets us map people’s movements during the day and other aspects of their activities. In addition, a wrist-worn wearable device can give us measures of physical activity, sleep, and stress.

DIAGRAM: Would these kinds of data be relevant in school settings?

SD: Very possibly. Information about sleep could be quite important, for example. If students are having trouble focusing on what they are reading, it may be that they are tired, not uninterested. Similarly, we may be able to use the data help students get a healthy amount of physical activity.

DIAGRAM: Are there concerns about these kinds of sensors being too invasive?

SD: Absolutely! Some previous investigations have received a lot of pushback, with teachers feeling that their privacy was being violated and technology was being used to evaluate them. And parents are justifiably concerned about what data are being gathered on their children and who has access to it.

DIAGRAM: Are there ways to deal with those issues?

SD: We think so. One approach is to arrange settings so that data are given to the person it’s about and not to others. So, teachers could get feedback on their classroom conversations, but the feedback wouldn’t be seen by anyone else. Similarly, students could get prompts that they need to go back over something that they’ve just read, without that information being reported to anyone else. Another approach is to let people opt out of any data gathering they aren’t comfortable with.

Incidentally, we’ve found ways to process the video in our classroom work in such a way that video from which someone could be recognized isn’t stored after the facial expression information is extracted. That process controls the amount of data from which people could be identified, and that data would need to be protected.

DIAGRAM: Does any of your work have particular value for learners with disabilities?

SD: Yes, in two ways. First, our work on assessing how well students are attending to what they are reading is likely to be more useful for students for whom the material is more challenging. And because there is often a kind of multiplier effect, where falling a little behind now gets you farther behind later, a little boost can make a big difference if it helps you keep up with the material.

The second point is something I didn’t mention earlier, when we were talking about analyzing what teachers are saying in the classroom. We are looking at the language teachers are using when referring to their students. We think we can help teachers talk in a more inclusive way about students from historically underrepresented groups, or students with learning differences.

I’ll also mention some new work we’re excited about. As you know, a lot of assessments are being done using tablets. This technology makes it possible to gather data not just on the answer’s students provide, but also on the process they go through in completing the assessment. We’ve been invited to consult on how this process data can be analyzed and interpreted, specifically for students with disabilities. The focus is on math assessments, and we think there are some real opportunities.

DIAGRAM: Do you have any further thoughts on what’s coming regarding the role of technology in education?

SD: Continuing that last point about assessments, the trend is to move assessment into the learning process, so that we’re able to measure how well students are progressing as they learn, rather than having a separate testing process. The better picture sensors can give us of what’s happening during learning, the better we can make the assessment process work.

There’s more than sensor technology involved here. If we can track the content of students’ work, we can ensure that they get enough practice to solidify their basic knowledge and skills without having to give them concentrated drill on these basics. Avoiding concentrated drill can help maintain engagement.

Also, we can provide more individualized instruction, the more we know about what a student is doing. Kids with differences are the kids for whom standardized, one-size-fits-all instruction works least well. So, these kids stand to gain the most from progress on individualized instruction.

In the longer term, we’re looking forward to being able to provide students with their own learning assistants. The work we and others are doing on understanding what’s being said in a classroom will be important in a couple of ways. First, not everything kids need to learn to do can be done with paper and pencil or a tablet; talking about something is a crucial skill. Second, the most natural way to interact with a learning assistant will be a conversation, in many cases. As I said earlier, we have a lot of challenges in being able to process student conversations, even at the level of just recording them, but we think we can get useful results along the way.

DIAGRAM: Thank you for sharing your thoughts with us!

Note: After the conversation, Prof. D’Mello provided some references so readers can further explore the ideas he discussed.