How do you measure what takes years to accomplish?
In Chapter 5 of Teacher Evaluation that Makes a Difference, Toth and Marzano consider two ways that scores within student growth and teacher practices can be assembled. The goal of assembling these variables is to create an aggregate or omnibus score for a teacher; how the data is assembled also helps administrators decide how such scores should be interpreted. There are two basic approaches to creating an aggregate or omnibus score for multiple forms of data: the compensatory approach and the conjunctive approach. In this series of posts we’ll be comparing the two approaches and applying them to three different types of scores.
Today we’re taking a look at the Compensatory Approach. In short, this method allows for high scores on some variables to compensate for low scores in other variables.
In Chapter 2, Toth and Marzano recommended that student growth or Value Added Measures be collected from five sources: state tests, benchmark or end-of-course assessments, common assessments, SLOs, and student surveys. They also recommend that these various measures be translated to standardized scores to ensure they are all expressed according to the same metric. Using the compensatory approach, you can combine students’ scores according to a weighted formula that leaves you with just one score.
When the compensatory approach is applied to scores on teacher practices or Marzano Domains 1-4, teacher scores on each element of each domain are averaged. An unweighted average score might be computed for each domain for each teacher, and these scores would then be aggregated using a weighted average.
Marzano Domain 1 includes 41 of the 60 elements of the model, so it would receive the most weight of the 4 domains. The authors recommend that Domain 1 be weighted at 68.3%, Domain 2, 13.3%, Domain 3, 8.3%, and Domain 4, an even 10%.
Let’s say I’m a World History teacher, and I’m a pretty average teacher. My unweighted scores were as follows:
• Domain 1 = 3.22
• Domain 2 = 2.44
• Domain 3 = 2.41
• Domain 4 = 3.11
If you do the math, my weighted average score across Domains 1-4 equals 3.03. Pretty good, right? If you use the scale below you’ll see that I’m an Effective teacher. Whereas if my supervisor had not used the compensatory approach, and just averaged my scores I would be at 2.75.
• Highly Effective: 3.25-4.00
• Effective: 2.50-3.24
• Needs Improvement: 1.25-2.49
• Unsatisfactory: 1.24-0.00
It’s been estimated that about 10,000 hours of deliberate practice are required for one to become an expert. In fact, one of the defining characteristics of expertise is that it takes time to develop.
So how do we measure something that will take years to accomplish? Chunk it into digestible bites. Growth can be measured relatively easily by comparing teacher’s score at the beginning of the year to a teacher’s score at the end of the year in a specific domain. Simply take the two scores, subtract the first score from the second, and hope you end up with a positive number.
A teacher can also measure his or her growth by tracking progress on different types of goals:
: “I will increase my skill at having students track their progress on learning goals to the Applying level or higher.”
: “I will increase my skill at having students preview content to the Applying level or higher.”
: “I will increase my skill at enhancing student engagement by using academic games to the Developing level or higher.”
Putting All Scores Together
At the end of the school year, we have three combined scores to consider for each teacher: one for VAMs (Student Growth), one status score for Domains 1-4 (Teacher Practices), and one growth score related to teacher growth goals. Each of these can be expressed in terms of proficiency levels:
• Highly Effective
• Needs Improvement
It’s important to keep in mind that different standards for classification in these categories should be used for teachers with different levels of experience.
In the next post we’ll be talking about another way to classify these categories for teachers: the Conjunctive Approach.
Buy the book: Teacher Evaluation that Makes a Difference here.