PDF PDF    printable version of page Printer-friendly page

Focus On Basics

Volume 7, Issue C ::: March 2005

Focus on Research

Research on Factors that Shape Engagement

Rutgers-based NCSALL researcher Hal Beder and his research team have been conducting a study that explores what factors shape whether adult basic education (ABE) students engage in instruction. They define engagement as “focused effort on instructional tasks.” They are also examining the relationship between engage ment and whether a learner drops out of a program or not. They expect the study to provide guidance to teachers on how to engage learners and how to maintain engagement.

Although engagement is not the same as learning, it is a necessary step in learning, explains Beder. “You can’t learn unless you engage. So if we can understand engagement, we will know a lot more about the process of learning.” Engagement is synonymous with trying hard to learn, or “working hard.” The literature from k-12 has a great deal of information on engagement but, to Beder’s knowledge, this is the first engagement study in adult literacy. Some of the k-12 literature looks at engagement broadly and includes such things as engagement in school as an institution, while other traditions in the k-12 literature focus on engagement as mental activity. However, this literature is not a particularly useful guide for Beder’s study because of the great differences in context between k-12 and adult literacy. For example, in contrast to children, whose “work” is to go to school, adults are voluntary learners, independent from parents and teachers. Adult learners therefore have different motivations than children do.

The outcomes of the engagement study are not yet ready for release, nonetheless, the team is finding that the students who persist in ABE show high engagement while in class. The team’s analysis of the data is also suggesting a finding that will be of particular interest to adult basic educators: teacher conduct — a factor over which educators have direct control — seems to be a factor in engagement. See data analysis.

The Research Site

For the past two years, the Rutgers research team has been collecting data via classroom observations recorded on video, followed by stimulated recall interviews, in which the team interviews students after the st udents have viewed themselves on video. The team videos in six classrooms — three basic and three secondary level — at the Rutgers lab site, a partnership between NCSALL- Rutgers and the New Brunswick Public Schools Adult Learning Center (NBPSALC). In the lab environment, the research team gets to know the students, teachers, and program context very well, and the team feels t hat this familiarity enriches the study. The disadvantage of using a lab site is that it affords less variation in the con text than would research conducted in randomly selected programs.

The classrooms in which the research is being conducted use what the Rutgers team calls individualized group instruction (IGI; more commonly known as individualized instruction). In this mode of instruc tion, students work on their own, using materials geared to their academic levels. When t hey finish a set of materials, the teacher corrects the exercises and gives them more materials. The materials do most of the teaching; the teacher is there to help.

Preliminary Findings

Learners in these classes are very highly engaged, mostly because they’re highly motivated. Motivation is closely linked to engagement, Beder reports. Students engage for a purpose, because they want to achieve a goal, he says. “It is the achievement of that goal that provides the motivation. So motivation and engagement are like different sides of the same coin.”

“When we first started to interview learners in terms of motivation,” explains Beder, “the statements were so glowing we didn’t believe it. These are voluntary learners. So if they’re not motivated, there’s no reason for them to come to class. What we’re hearing from the interviews is a high degree of motivation, pretty concrete goals, and the desire for postsecondary education, whether or not they actually go [to college].”

The team hypothesizes that the self-paced nature of the individualized instruction model makes the model learner friendly. A substantial a mount of the data from the interviews indicates that learners really appre ciate that. One of the things these learners say about their experience in k-12 is that they fell behind and felt belittled. In the interviews, Beder explains, “we often asked them what they thought of IGI. That they didn’t have to worry about falling behind came out several times as a response.” Learners also point out that they can pick up where they left off in the materials, explains Beder, so they don’t miss anything if they can’t come to class, as would happen in group instruction.

The Teacher’s Role

The researchers are also finding that how the teacher interacts with learners is an important factor in engagement. Very few of the teachers in the study had experience with individualized instruction models before they became ABE teachers. They had to create their own role definition and an identity as an individualized instruction teacher. Because of this, the classes vary a great deal. For example, all the teachers in the study systematically encouraged their students, and this supported motivation. However, how teachers encouraged students differed from class to class.

Some teachers see their role as one in which they “correct and direct”: correct student’s materials, assign new materials, and end with praise. Other teachers are more interested in whether the learners comprehend their work, in addition to its being correct. There is variation in terms of how well the teachers actually understand the materials themselves. Some teachers know them backwards and forwards, explains Beder, and know where learners are going to have problems understanding the materials. Talking with and observing the teachers showed the research team that other teachers are less familiar with the materials and are less able to predict where learners will have problems.

In IGI, teachers have to make an important decision that affects engagement: whether to spend less time helping each learner and thus reach more learners during a class, or to give more in-depth help and reach fewer learners. If a teacher decides to spend less time and reach more learners, fewer learners have to wait for help. But spending less time with each learner makes it more difficult for the teacher to diagnose learners’ problems thoroughly and help them overcome those problems. If the teacher spends more time with each learner, however, then more learners disengage because they are waiting for help. Obviously the size of the class is a factor. In all the classes in the study, the learners had to wait for the teacher to get to them, although the wait time varied a lot. The lesson is that teachers need to do things that make wait time productive. Encouraging learners to help each other is one strategy that has merit, suggests Beder. Providing alternative work that learners can do while waiting is another commonly used strategy.

Individualized Group Instruction Model

The engagement research is taking place in a program that uses the individualized group instruction mode of delivery: students working on their own, with materials chosen for them by the teacher, based on results of assessment tests. The instruction is primarily provided by the materials. Although this model was not the focus of the research per se, the high level of engagement evident in the preliminary findings did surprise the research team because of common criticisms that individualized instruction is boring for learners. Some researchers, including, for example, Robert Kegan and the Adult Development Research Group (2001), have found that group instruction provides social interaction and the cohort formed helps keep students attached to the program. However, Beder explains that the assumption that group experience is intrinsically better is not supported by any evidence they have been able to find. “These people [in the study] are progressing: they’re attending, they’re participating, they’re doing all these things,” he says. In the individualized instruction model, learners have a lot of control over their own instruction, which is something learners don’t have in group instruction.

Beder also points out that one of the arguments against individualized instruction is that it doesn’t teach critical thinking and problem solving. He is not so sure that is true. “I think the learners pick up other skills, such as self direction and problem solving in terms of their strategies for working through materials,” says Beder. “I suspect that there are skills learned [indirectly] beyond what’s taught in materials, although we don’t have any e vidence of that.” Oral skills, he admits, are not taught in individualized instruction. Beder adds, “I don’t think our work has made us proponents of individualized instruction but neither do we think it’s the evil some people make it out to be. Our objective should be to make it better, not to ban it. We hope our study will be helpful in this regard.”

Another Component to Come

Another component to the study will examine the relationship between engagement and learning outcomes. This is a quantitative study and it employs two ways to measure engagement. One is via the survey instrument the team created. “When we developed the survey we were worried that low-level learners would not be able to complete it, but when we piloted the survey, we were pleasantly surprised to find that even the low-level learners had little trouble,” explains Beder. The other measure is observational: a team of trained researchers will rate learners’ level of engagement as they view video clips of learners working in class.

For outcome data, they will use teachers’ assessments of student progress, gains on the Test of Adult Basic Education (TABE), and persistence in the program. The Rutgers team is working with researchers from the University of Georgia on this study. The Engagement Project has three connected studies: the qualitative study that uses video and stimulated recall, and two quantitative studies. The research team thinks that looking at engagement from three different directions will give a much more complete picture than a single-component study. Look for the findings in Focus on Basics a year from now.

–Barbara Garner

 

References
Adult Development Research Group (2001). “The power of a cohort and of collaborative groups.” Focus on Basics, 5B, 15-22.


Data Analysis

The research team videotaped students in six classrooms, then reviewed the videos for student behaviors that were of theoretical interest, such as when they engaged, how they engaged, and with whom they engaged. The team noted these episodes. Next, the team showed the episodes to the students who were in them, and interviewed them as they watched the video episodes. The team asked questions of the students such as: Why were you doing that? and What were you thinking when you did that? This enabled the team to couple their observations about students’ behaviors with the students’ own thoughts about them. These are called stimulated recall interviews. Teachers are also in the videos, and their perspectives were recorded when they participated with the research team in data analysis, but stimulated recall interviews were not conducted with teachers.

The research team analyzed the video and stimulated recall data using grounded theory, a methodology that uses constant comparison. For example, if one teacher acted in one way and another teacher acted in another way, the researchers asked themselves what might account for the difference. The answer became a point of analysis that they investigated as they analyzed more data.

When they reviewed the data, they looked for evidence of engagement. They knew someone was engaged if they saw eye movements, hand movements, and turning of pages in the videos. They report that it is fairly obvious when someone is engaged. If the data included an oral encounter — a discussion between the teacher and student, or between two students — they examined the nature of the dialogue to see if the topic matter was related to class. They found that the discussion in class between students is highly directed toward the work of the class. They report seeing very little enjoyable socializing. It was all very businesslike, with a substantial amount of the dialogue involving students helping each other, says Beder.

The researchers were concerned at first that their presence might cause the students to behave differently than they would have had they not been subjects of a study. But the team reports that the videos make it evident when someone is grandstanding for the camera. They do, they admit, have several episodes where it looks like that’s happening. But students quickly got used to being videotaped. Very few episodes reveal someone being influenced by the researchers’ presence.


Updated 7/27/07 :: Copyright © 2005 NCSALL