Malloy’s Teacher evaluation system is fundamentally and fatally flawed

In her latest MUST READ commentary piece, fellow public education advocate, Wendy Lecker, lays out the facts about Governor Malloy’s unfair, inappropriate and fatally flawed teacher evaluation system.  Like the junk bonds that helped take down Wall Street, Connecticut’s teacher evaluation system is based on junk science and false assumptions.

The question is not whether the state should have a comprehensive teacher evaluation system, but whether the corporate education reform industry will continue to stand in the way of developing one.

While recent attempts by the teacher unions to “correct” the flaws in the Malloy administration’s teacher evaluation system are laudable, the fact is that Malloy’s system cannot be fixed because it is inherently dependent on standardized test scores that fail to evaluate teachers on factors that are within their control.

As Wendy Lecker explains, the present system is not only unfair but is actually a useless waste of  a massive amount of time and money.  It needs to be repealed and replaced by a teacher evaluation program that actually enhances the quality of education in our schools.

Wendy Lecker’s piece from this weekend’s Stamford Advocate and Hearst Media Group papers is entitled, Solution to failed tests is not more tests

Fact: Connecticut’s teacher evaluation plan, because it relies on student standardized test scores, is fundamentally flawed. Student test scores cannot measure a teacher’s contribution to student learning. In fact, the president of the Educational Testing Service recently called evaluation systems based on student test scores “bad science.”

Rather than admit failure, the Malloy administration is trying futilely to “fix” the fatal flaw. Last week, PEAC, the panel charged with developing Connecticut’s teacher evaluation system, working under the direction of Commissioner Stefan Pryor, approved a change which calls for more standardized tests to be included in a teacher’s evaluation.

The commissioner’s “solution” is to add interim tests to a teacher’s rating. Determining what tests will be used, how they will be aligned to the standardized tests, and how all the test scores will be rolled into one “score” for teachers, will likely render this change completely unworkable.

However, there is an even larger issue at play. Will the addition of more tests in a teacher’s evaluation help us measure whether a teacher is effective?

According to the Connecticut Supreme Court, Connecticut’s public schools must prepare children “to participate in democratic institutions, and to prepare them to attain productive employment and otherwise to contribute to the state’s economy, or to progress on to higher education.”

Thus, we want our children to acquire the skills and knowledge that will enable them to succeed in college and in life. We want teachers who will help our children develop these skills.

Standardized tests have no bearing on college success. Moreover, although standardized tests are supposed to measure cognitive skills, research from MIT has shown that increasing test scores does not increase cognitive skills.

Even more striking is that cognitive skills, while important, are not the most important skills in determining success either in college or in life after college. Research has shown again and again that non-cognitive skills such as self-discipline, taking responsibility, and listening skills are more critical.

A recent comprehensive study by Northwestern Professor Kirabo Jackson found that children with teachers who help them develop non-cognitive skills have much better outcomes than those who have teachers who may help them raise test scores. Jackson found that every standard deviation increase in non-cognitive skills corresponds to a significant decrease in the drop-out risk and increased rates of high school graduation. By contrast, one standard deviation increase in standardized test scores has a very weak, often non-existent, relationship to these outcomes. Test scores also predict less than 2 percent of the variability in absences and suspensions, and under 10 percent of the variability in on-time grade progression, for example.

Increases in non-cognitive abilities are also strongly correlated with other adult outcomes, such as a lower likelihood of arrest, a higher rate of employment and higher earnings. Increased test scores are not.

In short, focusing on non-cognitive abilities, those not measured by test scores, are more important in predicting success in high school and beyond.

Jackson also found that a teacher’s supposed effect on test scores is not related to how well that teacher can improve non-cognitive skills.

Moreover, a new statement by the American Statistical Association reminds us that ranking teachers based on test scores does not even work for measuring their effect on cognitive skills.

ASA notes that teachers account for 1-14 percent of the variability in student standardized test scores. The majority of variability in test scores results from “system-level conditions”; meaning everything affecting a student outside the teacher’s control: the child’s socio-economic status, parental background, language barriers, medical issues, student mobility, etc. Rating systems cannot eliminate the “noise” caused by these other factors.

ASA further states that test scores at best “predict only performance on the test.” This conclusion confirms Jackson’s results, i.e that tests cannot predict how well a student will succeed in school or life.

In the context of this evidence, what does the PEAC change mean?

By adding more tests of the same skills in the same subjects, PEAC merely added more meaningless “noise.” This addition will not give us any better picture of how well a teacher teaches.

Worse still, adding more tests increases the focus on tests, increases the frequency of testing, and distracts us from considering the skills teachers should be helping children develop. And since Connecticut’s evaluation system completely ignores these non-cognitive skills, they will be de-emphasized in school.

Meaningful evaluations systems can be developed, but relying on faulty measures is simply rearranging deck chairs on the Titanic. Connecticut’s students, parents, teachers and taxpayers deserve better.

You can read the full commentary piece here: http://www.stamfordadvocate.com/news/article/Lecker-Solution-to-failed-tests-is-not-more-tests-5449394.php

  • guest

    The new teacher evaluation experience for many of my colleagues has left us bitter and drained.

    If the evalutors were supposed to be “calibrated” last summer to use the new system then the “calibration” failed. Miserably.

    For example – There has been no consistency between those making the evaluations. In some cases the “calibrated evaluator” made more forms to fill out in addition to the plethora of forms already dished out to us. Some of the “calibrated evaluators” (from now on C.E.) even demanded to “see” all 26 or so evaluation descriptors in a single formal evaluation. This included “seeing” a parent teacher phone call just so they can check that off their “list.” How professional would it have been if I just said, “Hey, kids. I need to make a parent/teacher phone call just so my “C.E.” can check off that I am doing my job. Any volunteers??!! Bueller? Bueller? So hold on for a sec while I make that call and by the way you will hear every word I say!

    Oh, and let’s not forget to smile, smile, smile because if you don’t you will get downgraded.

    And then there is the infamous, “I am going to give you 2’s even though you did everything so that on the next evaluation I can show you had growth over time and can give you 3’s.”

    How about this? Why did so-and so not improve his test scores? “Why it is simple. So-and-so doesn’t do the homework, doesn’t participate in class and is L.A.Z.Y.” So said the “C.E.” “So what would you do differently?” HUH??!!

    A student misses 6 weeks of class because they are out on vacation and the teacher is supposed to do……what?

    Kids are too lazy to even take their calculator out of their backpack for some math calculations as one self admitted to me, “Naaa. I am too lazy to get it out.” And I am supposed to do…..what?

    Yes, the teachers know that this is one messed up – FUBAR – piece of legislation. The amount of paper work we have to fill out is crap. The “data” we have to collect and analyze is BS.

    We are all hoping that this new FUBAR eval will “go away”.

    • ReTired

      Unfortunately as long as RTTT initiatives exist, and big business leads the charge, this type of evaluation will not go away. Even unions can’t pull this out of the water until legislatively teacher evaluation language becomes a mandatory subject of bargaining. CEA didn’t even think about that when they pushed local presidents to sign on to RTTT 1 and 2. Teachers were pushed under the bus! Now look where we are. Always funny for me seeing them praise their governance, leaders, and oh yes, teachers whom they urge to write letters when they finally get the concept! Hindsight is a wonderful world.

    • guest

      What evidence is there of the teacher’s unions trying to correct the flaws? My SEED book thanks the Gates Foundation for funding the SEED evaluation process – both the CEA and AFT-CT, under the national organizations didn’t refuse the funds of hedge funded billionaires.

      We’ve had numerous, excellent teachers non-renewed for unexplained reasons (though we all knew they rocked the boat, were too expensive, or blew the whistle on wrongdoing). Those who asked to explanations and hearing were/are being subject to more retaliation.

      Even if testing is never tied, the current CT Teacher Evaluation process is not being used to evaluate – it’s being used to terminate, threaten and harass in certain districts. Plus we have to make room for all those TFA recruits coming in.

      Too many teachers have already lost jobs so the standardized testing piece is irrelevant. It will just get worse. Unless we bring a class action lawsuit with evidence of improper use.

      • brutus2011

        Yes, teacher evaluation is to be used for cover to eliminate teachers.

        It is a fiction or actually it is a lie.

        The fact that our so-called education leaders are leading the charge on this is an indictment on where are society’s values are at the moment.

        And, our union management (local, state, national) are either silent or agreeing with this travesty.

        And, where is our media–print and online? Are they blind, deaf and dumb or just plain bought?

        Teacher evaluation is flawed, immoral (yes, I went there), and just plain wrong!

        • guest

          The HFT just sent out emails to members touting the Houston AFT affiliate as leading a class action lawsuit regarding improper use of Teacher Evaluations.

          Well Hartford is clearly not Houston and who knows how invested Houston is in the actual lawsuit or if it’s mere PR for the AFT.

          Every teacher in Hartford is told that no one who receives a non-renewal ever wins the fight to retain one’s job. And that’s told at the get go!! Encouraging eh? But let’s go Houston – whoo-hoo!

          So to send that email to Hartford teachers, especially those non-renewed, is pouring salt in the wound. The irony is not lost given how many teachers hire outside counsel from state to state. And we pay dues for????

          I’d rather have private liability insurance like other professionals.

          However when people are used to steamrolling with no accountability, mistakes are always made because who has investigated prior since those who complained or felt they could succeed, went in alone to face the tribunal – but times are changing.

          The colonists didn’t go to King George for help in breaking free from tyranny and unfair oppression- they sought freedom and independence on their own. Easy? No. Bloody? Yes. But to win autonomy and the right to rule themselves was well worth it.

          It’s time for teachers to take back teaching and mobilize. I never met a dedicated teacher who didn’t self-evaluate more intensely than the Danielson, SLOs, etc ever could (which we’ve been so doing on our own for years). And I never came across a teacher worth his/her salt who didn’t welcome anyone into the classroom to come and take a good hard look.

          In fact, come right into my class because I’m proud of what my students are doing and I want parents, admins, and everyone to see what we are accomplishing.

          But if you evaluate us improperly or with punitive intent – well we will collectively fight back. It’s not just a group of one anymore.

          And no we are not using the unions.

  • guest

    Struck me as ironic that recent Hartford Courant story re: teachers of the year didn’t lead with – or even MENTION – their students’ test scores. In fact, I don’t think I’ve EVER seen an article about ANY teacher of the year or teacher of the year candidate that says teacher was nominated because of rise in test scores.

    • guest

      What a great point. In my opinion, the unions are complicit. They are hailing this change as fantastic. They could have opposed the system from the start. That would have been in the best interest of kids and teachers. But they were too interested in keeping their “seat at the table” or whatever. What we need is some of those non-renewed teachers to sue like they are suing in Houston and have sued in Rochester NY. It is an arbitrary system that denies teachers their due process!

  • Linda174

    Oh yes and we will visit exemplary but we will never live there. The CT SDE slogan programmed into the minds of all calibrated evaluators.

    If you use Bloomboard you’re probably aware of the chiclets.

    Here are some exemplary chiclets for all. #evaluatethat

    Sorry it’s upside down, can’t seem to flip it.

  • jrp1900

    I pity the children in the public schools. Their education is being sacrificed at the altar of corporate school reform. The people enforcing sacrifice are zealots, who lack the creativity and intelligence to approach matters in a different fashion. Wendy Lecker underlines a point that has been repeatedly made in the scholarly literature: standardized tests do not measure a student’s native ability and nor do the tests measure a teacher’s contribution to a child’s learning. But the corporate zealots have no interest in truth. Their motivation is purely ideological.

    The zealots favor standardized tests because the tests generate numbers–“hard data”–that can be used to assess “performance” and “competitiveness.” Underpinning the use of numbers is the spurious doctrine of “value added” measure. The corporate zealots assume that there is an accurate way to assess what any given teacher contributes to the performance of her students. But as Wendy Lecker reports, the ASA utterly rejects this notion. Tests measure test-taking; they do not measure the complete personhood of the individuals taking the test. People who push for teacher evaluations on the basis of standardized tests are being disingenuous or cynical: either they don’t know what they are talking about or they know they are talking rubbish, but have other fish to fry and so are quite content to advance a policy they know is wrong. Either way, teaching is degraded, and the education of children greatly suffers.

    The “value added” model of teaching is not just bad science, it is also deliberately bad politics. If we can lay all blame on supposedly ineffective teaching, then we don’t have to address “system level conditions” in causing educational underachievement. This is very convenient for those political reactionaries who like to blame the poor for their own poverty, and who are adamant that nothing can be done, at the governmental level, to reduce social and economic inequality. In this regard, it is quite astonishing that so many “progressive liberals” have fallen for the narrative of “school failure” and are right onboard the corporate effort to privatize public schooling. Some of these “liberals” are so deluded, there is hardly any difference between them and the reform zealots. I believe that even Stefan Pryor considers himself a Democrat!

    Wendy Lecker is right to warn us that Malloy’s teacher evaluation system is “fundamentally flawed” and that it cannot be fixed by a little tinkering, here and there. The evaluation system pushed by the zealots is bad for teachers, but it’s even worse for children: their days will become nothing but an endless round of mindless testing just to generate “data” that can be used in “assessments.” Schools really are becoming factories…

  • mookalaboona

    I was convinced years ago the teacher’s unions, although they say work for the teachers, do not in many cases. I remember the CEA president looking for re-election 9 years ago ran on indoor air quality, rather than running on a platform of securing guaranteed rights for teacher retirement funding. When challenged, she changed her platform, and pushed hard for a contractual right for teacher’s retirement, and they were able to get it that year because she pushed the teachers while she was in office to call legislators, etc. to complain. I know, because I challenged her and my platform was teacher retirement. I didn’t run to win, I ran to get her to change her platform. And it worked. Now, they are running around the PEAC waste of time, kissing up to Malloy and Pryor. Glad I’m retiring this year.

    • ReTired

      You won’t reject exiting now. Best decision I made!

      • mookalaboona

        My BOE made it easy….up to four years of paid health care with a small cost share. Was a no brainer. But I think I would have tried to leave anyway with what this idiot governor and educationer commissioner are doing.

  • R.L.

    It’s just starting out, but……..
    http://cape-teach.org/

  • Philip Stull

    As the issues keep coming forth like the Budget and the growing realization that this administration’s educational policies are one and the same with the corporate educational policies that are profit driven, I don’t see how the Democrats will retain power in Hartford. It seems that we will have divide government for awhile. CT can live with that but had we had a real Democrat in the Governor’s office things could have been different. We’ve experienced worst times in CT but these last few years have been right up there with my worst years. Many of our highly educated children can’t fine appropriate work without moving 100’s of miles away. Our homes have lost 10’s of thousands of dollars in value, some in our area more than $100k. Small businesses have difficult times getting started and inadequate government support. Some corporations are making out big time with generous government support, with CEO’s pay at $80K a week, that’s right, and some make much much more.
    With Jon running we have the opportunity to bring some real progressive issues forward in the debate,and when he wins, he can govern like a progressive and restore confidence in the belief that government can make things better for the all and not just the few. If by bad luck Jon is not elected, the Overton window will have been moved and in future contests these progressive policies will again be part of any debate. People want change and they should be given multiple choices. We are all tired of only two choices and voting for lesser of two evils. What kind of a political system is that? Maybe that is why so many do not register and have given up on government.
    There can be good government but today it is harder and harder to find. We must continue to be part of the efforts to restore our state and nation back in the better direction. Our job here is to make the future than the present, so each step moves us forward.

    • CTedFromTheTrenches

      JON PELTO 2014

    • JMC

      Some of us think we’re seeing the natural result of an all-Blue legislature.

  • CTedFromTheTrenches

  • Guest

    Teacher evaluation is now nothing more (or less) than a witch hunt. Hysteria is building. Scare tactics are being used. Teachers are the scapegoats and the pawns for those with ulterior motives. Watch out!

    • brutus2011

      It pretty much always has been. In my experience, admins use teacher evaluation as a cover to cover their rear ends.

      Why do they cover their rears?

      To protect their 6 figure salaries and pensions.

      This money incentive for admins is a significant force behind this whole conundrum.

      Eliminate this incentive and our schools will not only continue but will flourish due to greater cooperation among the school personnel.

  • Pingback: Jonathan Pelto Reports on Wendy Lecker’s Critique of Governor Malloy’s Failed Teacher Evaluation System | Diane Ravitch's blog()

  • Pingback: Jonathan Pelto Reports on Wendy Lecker’s Critique of Governor Malloy’s Failed Teacher Evaluation System | Educational Policy Information()