I had posted recently about a Gallup poll on student engagement. Essentially the poll showed that student engagement dropped precipitously (though as I wrote, not as starkly as their graph indicated) as students moved from elementary to high school. My friend, Gaurav responded with a comment, as follows:

Engaged for how much percentage of the school day? Most of the day may be boring, but some parts may be engaging for many students. I haven’t seen the original poll that you highlight, but the data that you picked doesn’t say anything, and as you said, the graph is misleading there too.

This got me to thinking as to how engagement was measured in the first place. A bit of digging around led me to a couple of pieces by Dr. Shane Lopez.

1. A report titled: Strengths based education and student engagement
2. An article in the Kappan: The highs and lows of student engagement

Each of these articles provides a bit of context of how Gallup measures student engagement. As it turns out, the Gallup Student Poll Engagement Index consists of 5 items as follows:

• I have a best friend at school
• I feel safe in this school
• My teachers make me feel my schoolwork is important
• At this school, I have the opportunity to do what I do best every day
• In the last seven days, I have received recognition or praise for doing good schoolwork

What do you think? Are these 5 items a good measure of what we would call “student engagement.” Surprisingly there is no item for bordum (or lack thereof) – which was the thrust of Gaurav’s question. Also surprising to me is the lack of specific context of what is or is not engaging in school. As Gaurav said, “most of my day may be boring, but some part may be engaging for many students.”

Students may be engaged in school due to factors other than academics – as indicated to by the question on having a best friend in school or feeling safe there. I like the question on having “the opportunity to do what I do best every day” but am less sure about “My teachers make me feel my schoolwork is important.” Would it have been better to directly ask “I feel my schoolwork is important.” I know from my kids that some of their teachers do emphasize the value of what they are learning, but I am not sure my kids do. For instance, when speaking with my son about this survey, he said something as follows, “everything we need to learn to survive day-to-day has been learned by middle school. The stuff I am doing now (chemistry, pre-calculus) is for some future, which at this time makes no sense to me.”

I do not want it to appear that I am criticizing this survey per se (having been involved in educational research I understand how difficult it is come up with good measures and how easy it is to poke holes in any study). I would rather like to make a broader point about the need for critically evaluating the reports of scientific studies we often get in the media. If anything this should be seen as an example of how the results of a study lose nuance and detail by the time the reach the consumer. These two blog posts point to the need for us (as educators and citizens) to pay greater attention to the details, whether they be of the representation of the data (as in the previous post) or in how the data was collected in the first place (the broader theme of this one).