Teacher Evaluations At The Heart Of Education Reform Are Flawed (By Jonathan Kantrowitz)

Jonathan Kantrowitz writes a blog for the Connecticut Post and the Hearst Media Group.  You can count on his articles to be astute, thoughtful and extremely informative.  If you haven’t bookmarked his blog, you should.

Kantrowitz’s latest piece examines the significant problems associated with the faulty teacher evaluation programs that are being pushed by the corporate education reform industry and their political allies.

Unfortunately, Connecticut Governor Dannel “Dan” Malloy has been one of the nation’s leading proponents of these inappropriate and inaccurate teacher evaluation systems.

As Jonathan Kantrowitz explains,

This week, two new studies have reported on significant flaws at the heart of teacher evaluations:

standardized tests don’t really measure teacher’s value added,

and classroom observations are often biased.

Here’s a summary of the first report:

“There are very weak associations of content alignment with student achievement gains and no associations with the composite measure of effective teaching…the tests used for calculating VAM are not particularly able to detect differences in the content or quality of classroom instruction. Empirical and conceptual work illustrates that these kind of assessments tend to be, at best, weakly sensitive to carefully measured indicators of instructional content or quality…

At a minimum, these results suggest it may be fruitless for teachers to use state test VAMs to inform adjustments to their instruction. Furthermore, this interpretation raises the question— If VAMs are not meaningfully associated with either the content or quality of instruction, what are they measuring?”

Key findings and resulting recommendations from the second report include:

* Under current teacher evaluation systems, it is hard for a teacher who doesn’t have top students to get a top rating. Teachers with students with higher incoming achievement levels receive classroom observation scores that are higher on average than those received by teachers whose incoming students are at lower achievement levels, and districts do not have processes in place to address this bias. Adjusting teacher observation scores based on student demographics is a straightforward fix to this problem. Such an adjustment for the makeup of the class is already factored into teachers’ value-added scores; it should be factored into classroom observation scores as well.

* The reliability of both value-added measures and demographic-adjusted teacher evaluation scores is dependent on sample size, such that these measures will be less reliable and valid when calculated in small districts than in large districts. Thus, states should provide prediction weights based on statewide data for individual districts to use when calculating teacher evaluation scores.

* Observations conducted by outside observers are more valid than observations conducted by school administrators. At least one observation of a teacher each year should be conducted by a trained observer from outside the teacher’s school who does not have substantial prior knowledge of the teacher being observed.

* The inclusion of a school value-added component in teachers’ evaluation scores negatively impacts good teachers in bad schools and positively impacts bad teachers in good schools. This measure should be eliminated or reduced to a low weight in teacher evaluation systems.

The new reports provide ample evidence that the Connecticut’s teacher evaluation system that Governor Malloy pushed through needs to be repealed and replaced with a far more appropriate program that will ensure that our students have the most effective effective teachers.

By clicking on the follow links, you can read additional articles that Jonathan Kantrowitz has written about the teacher evaluation issue,

State tests used for VAM ae not meaningfully associated with either the content or quality of instruction

Improvements are needed in how classroom observations are measured for teacher evaluation

  • R.L.

    One of my first evaluators once told me how he did evaluations. This was just before the NCLB nonsense started warping our education system. He told me, “You know what I look for? I go into a classroom and think to myself, would I want this person teaching my kids?” How easy, real, down to earth, and inexpensive is that?

    • jonpelto

      I’d want you to teach my kids.

      There we just saved the state funds on “calibrating” the evaluators.

      And for the record what does calibrating even mean????

      War is peace

      • CTedFromTheTrenches

        calibrating… or indoctrinating…

  • Wendy Lecker

    These are just two of the MANY studies showing that VAM is junk science. While it is important to hammer home that VAM is worthless, it is even more important to remember that as bad as VAM is, CT does not even use VAM and in fact uses a system that is WORSE- SGP (student growth percentiles).

    VAM attempts-but fails- to isolate the miniscule teacher effect on student test scores. SGP does not even try to isolate the teacher effect.

    For a comprehensible explanation of the differences in these two very bad systems, try Bruce Baker http://schoolfinance101.wordpress.com/2011/09/02/take-your-sgp-and-vamit-damn-it/

  • JMC

    BTW, on Monday Gov. Rick Scott of Florida signed legislation to limit the effects of CC and putting off the new teacher evaluations for a year. In the legislation was language not allowing CC to drive textbook selection.

    • JMC

      And the RI Senate has approved a 3-year moratorium on high stakes testing. The bill awaits approval from the House and Gov. (From D. Ravitch’s site)

  • Bluecoat

    The Race to the Top Program has specific language that says: “Equitable distribution of teachers and Principals”.

    This can only be done when the teacher evaluation system is working in conjunction with the Student Assessment Programs.
    So in essence, you can keep your teacher if you like your teacher, as long as you are willing to follow that teacher since they will be moved to another school district. Why? Because it would be unfair for one town to have teacher’s who continue to score high on these electronic evaluations and another town that does not.
    For the Teacher’s out there, how does it feel to be the object of the progressive movement to help “spread the wealth around”?

    • guest

      There is nothing “progressive” about this teacher evaluation system. And a student’s score has nothing to do with a teacher’s effectiveness. And, in fact, there should be an equitable distribution of experienced and qualified teachers- not by “moving teachers around.” By improving teaching and learning conditions in our most impoverished districts to attract and retain teachers.

    • CTedFromTheTrenches

      Jeb Bush and Rupert Murdoch both support this ‘progressive’ movement.
      This is not about GOPs vs DEMs. It’s about the #Establishment vs #Grassroots.
      However, I do agree with you on several points.

  • Bluecoat

    So while the teachers keep barking up the tree that the teacher evaluations are junk science, when will the teachers speak up with the same loud voice in regards to the personal and private data that is being stolen from our kids within Smarter Balanced Assessments and included in CT’s Statewide Longitudinal Data Base, and shared with 3rd party vendors without parental consent?
    This should be the number one concern for all, but as usual our kids are really not our kids, are they? As long as the rights of teachers come first, everything else is thrown to the side.
    The Smarter Balanced Assessment program and the Statewide Longitudinal Data System must be stopped immediately.
    But all our state legislature gives a crap about is banning chocolate milk. We are the dumbest generation ever!

    • CTedFromTheTrenches

      Teachers ARE upset by student data collection; they’re also upset with teacher data collection. Despite the negative feelings that many have towards unions, I can tell you that CEA is actively pursuing initiatives to protect student data.

  • Bluecoat
  • Bluecoat

    Why do I feel like in the end, what only matters is the well being of the teaching establishment, and that the rights of parents and students will always get steam rolled.
    I still see a lot of Retire NEA Lawyer Bob Chanin’s ideology when some teachers speak and comment.
    How many people remember this:

    “Despite what some among us would like to believe it is not because of
    our creative ideas. It is not because of the merit of our positions. It
    is not because we care about children and it is not because we have a
    vision of a great public school for every child. NEA and its affiliates
    are effective advocates because we have power.”

    “And we have power because there are more than 3.2 million people who
    are willing to pay us hundreds of millions of dollars in dues each year,
    because they believe that we are the unions that can most effectively
    represent them, the unions that can protect their rights and advance
    their interests as education employees.”

    “This is not to say that the concern of NEA and its affiliates with
    closing achievement gaps, reducing dropout rates, improving teacher
    quality and the like are unimportant or inappropriate. To the contrary.
    These are the goals that guide the work we do. But they need not and
    must not be achieved at the expense of due process, employee rights and
    collective bargaining. That simply is too high a price to pay.”
    – Retired NEA Chief Council Bob Chanin- retirement speech July 2009

    I suggest, go find the video of this speech, watch as the teachers and administrators in the crowd give this guy a standing ovation when he said the above remarks.

  • cindy

    Remember the test scores are only 45% of the evaluation. The other incredibly messed up part is the 40% teacher observation. It isn’t wrong to observe a teacher, but this new method takes what was a simple practice and turned it into a circus of scripting lessons, tagging rubric-based indicators, looking for evidence that may or may not exist in a lesson, a 20+ page rubric and in the end, teachers are told they may visit “perfect” but they will never liver there as a final score. Set up to fail from day one.

    • cindy

      Silly me…it is a 105 page rubric:

      https://www.teachscape.com/…/2011-fft-evaluation-instrument.pdf

      • CTedFromTheTrenches

        Our rubric is also extremely lengthy and detailed.
        It’s quite disgusting how EVERTHING that actually measures the effectiveness of a good teacher, only amounts to 40% of one’s evaluation.
        The evaluation piece is certainly the most convoluted part of this whole ‘reform’ package.