Let’s address HOW to deal with the test scores that you get.
Why?
It turns out that the way school leadership, as well as the staff, thinks about, discusses, and frames the conversations about test scores actually affects future scores.
How does this happen and how should a staff debrief the testing?
The Research
The way that your staff frames their results and frames their work is critical to the ongoing success at your school.
A “framing effect” is usually said to occur when varied, but usually equivalent descriptions (of a product/experience/decision or problem) lead to very different decisions. We’ve all known this as, “It’s not what you say, but how you say it.”
New research done at the University of Michigan by Juth and Helgesson (2012) suggests that your expectations and predictions shape your future efforts via the “framing effect.”
If we started a hypothetical group of elementary children, all earning the same letter grades (ex. A, B, … F), here is how their expectations matter. In those children expecting to become a teacher, an engineer, or a nurse when they grew up, this study successfully predicted that they’d work harder in school.
In this same study, nine out of ten children expected they would attend at least a two-year college, but less than half saw themselves as having an educational degree-dependent job. This is why it is so important to tie their dreams to an actual job, not just to college.
At the secondary level, researchers presented two different options of information to two groups of students. They heard about either:
1) potential education-dependent earnings of college degree recipients, such as a biologist, or,
2) potential earnings of actors, musicians, and sports figures.
The kids who saw how adult earnings were related to education were eight times more likely to do the homework as those who saw the presentation showing adult earnings independent of the amount of education needed.
This shows that a small, but powerful, intervention of showing how much education matters will likely have a major effect on the likelihood of that child’s investment in schooling.
Wonder why some teachers get kids to work harder for them? Very subtle cues can influence academic performance. Failing to see connections between adult identities and current actions puts children at risk of low effort in school. Do not wait until low-income and minority children are in high school. Start these connections early…in lower elementary school!
Whether your testing results are superb or disappointing, there’s a lesson in there for you. But the opportunity to move forward in your pursuit of excellence may depend not on the facts, but on your perception and analysis of the facts.
Framing has been a major topic of research in the psychology of judgment, understanding, thinking and decision-making and is widely viewed as carrying significant implications.
For example, low scores can be framed as, “(Bleep!) We knew it. There’s nothing you can do with these kids.” Low scores could also be framed as, “Now that’s feedback – that what we’re doing is not working. Let’s rethink this and come up with a better plan.”
In addition, if you ask your colleagues, “Where do you realistically expect our school to be (academically, in terms of student achievement) in 3 years?” Those that expect the school to be much better and even achieve noteworthy status will work harder to help your staff get there. In fact, they are more likely to see low test scores as evidence that they must put in more effort, better strategies or a stronger attitude. This is a great practical idea!
Those with lower future expectations about your school are more likely to see any additional time, meetings, accountability or preparation as a burden. They are less likely to do what it takes to make a miracle happen.
So, if your test scores are high and the school conditions are stable, some staff will see this as “proof” that what they are doing is working. They’ll believe they’re on the right track and want to continue.
Others will see this as “proof” that they have “made it” and it is no longer necessary to have a daily urgency to their work. That’s a mistake.
Accentuate the positive or accentuate the negative? The literature has been mixed. There appear to be many types of framing effects. One is the risky choice framing effect (ex. 90% chance of living or a 10% chance of dying). At school, “We have a 40% chance of success or a 60% chance of failure.”
Another framing effect is attribute framing, which affects the evaluation of object or event characteristics. “It’s those damnable tests, which are never fair!” Or, “Those tests are our friend. As we get better, the tests will like us more!”
The third is goal orientation, which effects the power of communication. “You do want to live another year, don’t you? Then take this medicine!” You’ll see how to use that in the next (Practical Applications) section.
In short, things are NOT the way they are. They are mostly, the way we think they are, or the way we feel about them, or the way we are vested in a certain outcome. In short, framing is no trivial strategy; it’s the narrative of our lives.
Practical Applications
There are many ways to apply this information.
First, let’s apply this research to your staff debriefing of test data. You’ll want to structure this meeting with a timekeeper!
Before you start any staff meeting, ask yourself, “What is the lens through which we want our staff to understand and respond to the school data?”
Remember from the section above: There were three types of framing mentioned: 1) risky choice, 2) attribute, or 3) goal orientation.
Let’s use the last one, goal orientation. Ask the KEY question, BEFORE the staff can look at the data and before you start the debriefing: “Where do we expect our school to be in 3 years?” This is critical! If any staff member does not expect your school staff to reach the stated goals, NOW is the time to have a conversation with them.
Ask the staff members that are “iffy” about reaching your school goals, “What kind of evidence would you need to see and when would you need to see it to decide if it’s possible for us to reach our school goals?” The answer to this is the key.
Let the staff share what specifically would tell them that it’s worth their time, energy and motivation to make massive changes in the way they do their job. If your school can’t meet their criteria for them “believing”, there needs to be a separate (fierce) conversation with these staff members, outside of the meeting.
Once that is squared away, here are the debrief steps for your meeting.
1. You set up the meeting tone with your framing first.
2. Review the results and have each staff member share their first impressions from their first data review (surprises or disappointments). Determine the proficiency cut-off for scores for all students in the grade level.
3. Divide the total group into three performing levels so that intervention efforts can be planned more efficiently.
4. Celebrate the students that have made strong progress.
5. Then, focus on the lowest-performing of the three groups, first. Analyze one student within this group at a time. Begin each individual case looking at testing patterns, and find which standards the student mastered and did not master.
6. Review of the possible issues with each of your kids, such as Attendance, English Language Learners, etc.
7. Begin to formulate a plan. Review the materials, strategies, and time as well as learning problems or issues. Design intervention strategies based on which standards need to be taught to mastery, how best to meet the academic needs of the students, when to provide intervention or reteaching, etc. There’s a good chance you’ll build attitudes, but most likely executive function skills.
8. Next set targets: who will be working with each student, what should be done differently to support each, set the academic goals, and identify how the goal will be measured. Figure out what assets you’ll need (software, PD, etc.) and how and where to get them. Discuss necessary resources with the principal.
9. Repeat this process with the remaining two performing groups (middle and high).
10. Set up regular team meetings and goals. Start with small miracles. Wash, rinse, and repeat.
Now, let’s apply the University of Michigan (above) research to students in your classroom.
Say to your kids, “Think about yourself as an adult. What are some jobs you think you’ll have? What will you be doing in 5, 10,15 years?” Next, ask kids to write down how much education they will need to get the job they envision. This extra thinking will help become a motivational driver for your kids who will need the college education.
REMINDER: if you work at a Title 1 school, with 80% or more of the kids on free and reduced lunch, then listen carefully. I am looking for schools that are struggling. If your test scores are in the bottom 25% in your district, I may be able to help you. Please contact me at eric@jlcbrain.com. Thank you.
Your partner in learning,
Eric Jensen
CEO, Jensen Learning
Brain-Based Education
________________________________________
REFERENCES
Kahneman, D. (2000). “Preface.” In Choices, Values, and Frames, ed. Kahneman and Tversky. Cambridge: Cambridge University Press.
Lynöe N, Juth N, Helgesson G. (2012). Case study of a framing effect in course evaluations. Med Teach. 34(1):68-70.
Destin M, Oyserman, D. (2010). Incentivizing education: Seeing schoolwork as an investment, not a chore. J Exp Soc Psychol. Sep 1;46(5):846-849.