August 02, 2013

Computing and Reporting Teacher Status and Growth, Part 2: The Conjunctive Approach


Marzano Center Staff
Learning Sciences Marzano Center

Growing pedagogical skill, just like growing an Oak Tree- takes time.

How many hours does it take to grow a teacher?

In part 1 of this article we discussed the various ways that scores within student growth and teacher practices can be assembled to obtain an aggregate or omnibus evaluation or growth score for a teacher, as well as how such scores can be interpreted. There are two basic approaches to computing an aggregate a score using the same data: the compensatory approach (basically, averaging the scores) and the conjunctive approach (which makes allowances for a single low score). In this series of posts we’ll be comparing the two approaches and applying them to different types of scores.

Today we’re taking a look at the conjunctive approach. In this approach, minimum scores are established for each proficiency level, but as we will see, teachers do not necessarily suffer for a weakness in one area if they have sufficient strengths in others. This approach values the idea that teachers may be skilled in different areas and still achieve high levels of expertise – they just take individual paths to achieve desired results.

But first…a bit about score inflation (and new teachers).
We might say that there has been a tendency in K-12 education to inflate the competency scores of some teachers, who may well benefit from more accurate feedback to improve their pedagogy and classroom practice. One way we’ve found to counteract this tendency toward teacher score inflation is to use different standards for different levels of experience—that is, holding teachers in the early stages of their careers to different criteria than those in the later stages. It makes sense: You wouldn’t grade a third-grade student on the same material you would grade a seventh-grader.

It’s been estimated that about 10,000 hours of deliberate practice, or 1,000 hours each year for 10 years are required for one to become an expert. Needless to say, teacher evaluation systems based on development, don’t expect teachers to reach the highest levels of effectiveness quickly.

Based on what we know about the development of teacher expertise, we can predict that inexperienced teachers will have relatively low teacher evaluation scores compared to more experienced teachers. After all, it takes a long time to become good at a complex endeavor like teaching. The conjunctive approach helps avoid punishing new teachers who wouldn’t be expected to have developed the same level of skill as more accomplished senior staff.

When it comes to creating an overall evaluation score using the conjunctive approach, the observer uses cut scores. These are standard scores the school or district chooses that defines a score range for each proficiency level.

Why use the conjunctive approach?
As we have noted, a number of the elements in Domain 1 are not necessary conditions for improving student learning. For example, a teacher might stimulate exceptional learning without using strategies like like academic games or friendly controversy. You wouldn’t want it to count against that teacher if you gave them a low score on those elements, even though they are mastering Domain 1 effectively.

Our teacher evaluation system allows an individual teacher to become highly effective by crafting his or her own strengths profile,which may include a large number of strategies in which they are highly skilled along with a much smaller number of strategies in which they are not very skilled.

If a teacher establishes three growth goalsat the beginning of the year and follows up with those goal at the end of the year, his or her growth can be computed for each goal. The overall growth across the three goals would constitute the teacher’s growth score for that year.

A teacher’s overall growth score does not lend itself to a compensatory (averaging) approach, as we have said, because a single low score in one design question or element will drag the teacher’s score down. (This is because growth using the scales in this approach is hard to translate to a common metric. Teacher pedagogical growth due to deliberate practice has not been studied enough to ascertain common expectations, but trust us, we’re working on it.) Until more research is available, it’s best to use the conjunctive approach to establish an overall growth score for a teacher.

Putting It All Together

Cut Scores for Proficiency Levels

Highly EffectiveEffectiveNeeds ImprovementUnsatisfactory
All Domains: 75% of scores at Level 3 or higher AND >50% of scores at Level 4Domains 1&2: 65% of scores at Level 3 or higher AMD >20% of scores at Level 4 Domains 3&4: 65% of scores at Level 3 or higher AND 25% of scores at Level 4All Domains: 85% of scores at Level 2 or higherAll Domains:<85% of scores at Level 2 or higher
Take this first-year physics teacher for example. Below are the scores from her first observation: Domain 1= 4 Domain 2= 4 Domain 3= 3 Domain 4= 3 Using a conjunctive approach, that teacher is rated Highly Effective. If the observer were to use thecompensatoryapproach, instead, using an unweighted average, that teacher’s summary score would be 3.5, classifying her as Effective. We hope these methods will help you score the teachers in your school or district fairly, accurately and reliably as they build their expertise over time. Tell us what methods you have for organizing teacher growth scores in the comment box below. We’d love to hear your thoughts. This post is adapted from Chapter 5 of Robert Marzano and Michael Toth’s new book, Teacher Evaluation that Makes a Difference.




Name: * Required Field

Email: * Required Field

Location: * Required Field

Remember my information

Notify me of follow-up comments