In the emerging field of learning analytics – the use of technology and large-scale data to, in essence, learn how learners learn -- Ryan Baker, Associate Professor of Cognitive Studies, is a bona fide star. As a graduate student at Carnegie Mellon University, he designed the first-ever automated detector of disengaged learning behavior based on moment-to-moment data. At Worcester Polytechnic Institute, where he taught before coming to TC, Baker studied intelligent tutoring systems for middle-schoolers that create sets of practice problems and provide immediate cues to those who don’t answer correctly. He has since conducted an eight-year study of the teen-agers who used those systems, and he’s also taught a MOOC (massive open online course) on learning analytics that enrolled nearly 50,000 students.

Yet Baker will be the first to tell you that that since he joined TC’s faculty in 2012 and proposed the world’s first graduate degree program in learning analytics, his vision of the field and what students need to know has evolved significantly.

“My departmental colleagues had a lot of really great ideas that significantly enhanced my original idea,” he says. “The result turned out to be something much broader, more grounded in theory and more geared toward the real world.”

For example, one of the required courses in the program, which launched this past spring, is “Visual Explanations,” taught by psychologist Barbara Tversky, whose expertise includes spatial language and thinking, event perception and cognition, and diagram production and comprehension. If the connection isn’t immediately apparent, consider that smart tutoring systems can generate and aggregate reams of data about groups of students ranging from a single class to entire school districts and regions. That data then needs to be given to teachers, principals, or superintendents in a form that helps them quickly find out what they need to know.

“If you design a visual representation of data, you need to understand how the user perceives that data, and what visual presentation strategies – say, a dashboard for teachers – may aid their comprehension,” Baker says.

Another course, taught by Larry DeCarlo, Professor of Psychology & Education, introduces students to the basics of psychometrics – the field of test design and performance measurement.

“If you’re in learning analytics, you’ve got to be able to communicate with folks with a more traditional measurement background, particularly to determine where each field’s methods are appropriate,” Baker says. “Traditional psychometrics is very successful in doing measurement where you can control the conditions of measurement. Learning analytics is useful in messier situations – for example, to measure science inquiry skills when it’s not yet fully established what ‘good inquiry’ is, or what factors matter most in achieving it.”

And still another course in the program, “Normative Perspectives,” designed and taught by Gary Natriello, TC’s Ruth L. Gottesman Professor of Education Research, looks at how society thinks about analytics.

“All these dimensions of learning analytics are really important for when our graduates will go out into the field to take jobs at the New York City Department of Education, or at the Educational Testing Service, or at McGraw-Hill, or one of the other big education companies,” Baker says.

Baker says he also gained understanding from speaking with Vice Provost Bill Baldwin, who advised him on the steps necessary to propose the program. As a result, Baker didn’t lose heart during a year of absorbing feedback from a variety of stakeholders, navigating a round of adjustments after the Academic Program Subcommittee of TC’s Faculty Executive Committee weighed in, and then fielding still more changes from the New York State Education Department. Instead, he emerged with a broader perspective not only on the content of his program, but also on marketing it.

“While we were in the process of planning the program, I taught a MOOC [massive open online course] on big data in education,” he says. “It was good advertising – I’ve had several students since tell me that they came to TC because of taking the MOOC.” Baker subsequent taught another MOOC and is now re-teaching the first one, and he’s also used materials from the big data MOOC to re-tool the pedagogy in his class “Core Methods for Educational Data Mining.”

“The previous semester, a number of students taking the course for credit switched to merely auditing it, I think because they were finding the math too difficult,” Baker says. “There’s a lot of unfamiliar math, and I didn’t have that much time to cover it for those who were unprepared.” So, Baker decided to have the students view, prior to class, his MOOC lectures, which went into depth on math concepts such as the formulas underlying Bayesian Networks, so that they could navigate more effectively the collaborative work in the classroom. This time, almost no one switched to auditing.

“It’s a hard course, but TC students aced it,” he says.

One could say that of Baker as well.

Published Thursday, Aug. 27, 2015