CMT Scores and Teacher Evaluations – But Wait – That’s Like Comparing Apples and Tomatoes

Apr 30

jonpeltoEducation Reform, Malloy, Stefan Pryor , , , 17 Comments

Governor Malloy, Education Commissioner Stefan Pryor and the rest of the “education reformers” continue to claim that Connecticut needs a “one-size-fits-all” approach to teacher evaluations in which teachers are, at least in part, rewarded, promoted or let go based on how well their students do in Connecticut’s standardized tests.

Malloy, now famous for his -” I’d don’t mind if they teach to the test as long as the test scores go up” – statement has been leading the mob mentality that is claiming that it is imperative that 20-40% of a teacher’s annual evaluation be based on their student’s annual test scores.

What is never articulated is what counts as a “good” or a “bad” change in test scores.

Thinking of the following example as if it was a question on a standardized test;

Teacher A is a 4th grade teacher in New Britain.  This year, 25.7 percent of teacher A’s class scored “at goal” on the Connecticut Mastery Test in reading (up from 22 percent last year).

Teacher B is a 4th grade teacher in Hamden.  This year, 56 percent of teacher B’s class scored “at goal” in the CMT in reading (up from 54 percent last year).

Teacher C is a 4th grade teacher in Fairfield.  This year 78 percent of teacher C’s class scored “at goal” in the CMT in reading (down from 79 percent last year).

Presently, in New Britain, 22 percent of 4th graders are at goal in reading and 27 percent are at goal in math.  On the other hand, in Hamden, where poverty and language barriers are not as great as in New Britain, 54 percent of 4th graders are at goal in reading and 58 percent are at goal in math. Finally, in Fairfield, 79 percent of 4th graders are at goal in reading and 84 percent are at goal in Math.

Do any of the three teachers deserve a merit bonus?  Do any of the teachers need some extra professional development support? Do any of the teachers need to be put on the “watch list” for unsatisfactory performance?

If the number of students testing a goal is going up – is that a sign of the teacher’s success?  If a teacher maintains test scores is that good or bad?  What about a teacher whose sees the number of students testing at goal actually drop?

In this case, the New Britain teacher saw a 15 percent increase in the number of students testing at goal, the Hamden teacher had a 5 percent increase and the Fairfield teacher saw a slight decline.  Which teacher is succeeding?  Which is failing?

Of course, without knowing the total number of students taking the test in each class we can’t even be sure the information is statistically significant.  It may be that in all three situations the change is within the standard margin of error and therefore no conclusion can be reached in any of the cases.

Meanwhile, what would we do if one 4th grade class in Meriden has a 7 percent drop in the number of Latino students and sees a 5 percent increase in the number of students at CMT goal, while the same sized class in an elementary school across town has a 10 percent increase in the number of Latino students and the number at goal in that class drops by 2 percent?  Which teacher has done a better job?

These are very real issues.  In New York, the failure to account for these issues has destroyed the entire credibility of their new teacher evaluation effort.

But don’t let the details stand in the way of progress.

While the “reformers” continue to yap about the need to link test scores to teacher evaluations, they still haven’t begun to articulate how including standardized test scores in teacher evaluations are going to help determine who gets a bonus in pay or who gets punished.

What is clear is that among those who profess to know that attaching test scores to teacher evaluations is definitely the way to go is the Connecticut Association of Public School Superintendents.

The spokesperson for the Superintendents has repeatedly joined in Malloy’s claim that teachers must be held accountable for their students standardized test scores — despite the fact that test scores are driven by wide range of factors far beyond the teachers’ control.

The logically absurd claims being made by the superintendents, and other reformers, got me wondering about how they could consistently get away with comparing apples and tomatoes without ever admitting that the comparison is fundamentally bogus.

Then again, maybe they are on to something…

These superintendents are paid big bucks to run their local school systems.

True they are supporting legislation that undermines the rights of their own boards of education, local elected officials and taxpayers but they must know what they are doing.  They are all certified to be superintendents (well all but one).

So, perhaps this whole apple vs. tomato approach might also serve as a useful mechanism to judge the effectiveness of Connecticut’s superintendents

Let’s look at the data.

The following chart shows what taxpayers are getting for the money they pay superintendents.

The data measures the superintendents’ cost per student, their cost per school employee, their cost per poor student (that is students who receive free or subsidized lunches) and the cost per students who don’t speak English.

We might say that it is a good way to determine how superintendents are allocating their resources.  There will certainly be differences from town to town, but the fundamental cost per unit has to be somewhat similar, right?

Like mastery tests, these costs per unit measurements will provide the state (and taxpayers) with an opportunity to determine which superintendents are doing well and should be rewarded for their efficient operation of services, which need a dose of professional development to help them get a hold of their financial operation and which need to be removed for their failure to get their job done correctly.

The data tells us;

  • A superintendents’ cost per student ranges from a low of $9 in Waterbury and $11 in Hartford to a high of $85 in Weston and $62 in Wilton.
  • A superintendents’ cost per school employee ranges from $60 in Waterbury and $73 in Hartford to $537 in Wilton and $463 in Brookfield.
  • A superintendents’ cost per low-income student ranges from $11 in Hartford and $12 in Bridgeport to $6,396 in Weston and $5,831 in Wilton and.
  • And a superintendent’s cost per non-English Speaking Student ranges from $61 in Hartford and $87 in Waterbury to $16,071 in Darien and $13,591 in Weston.
Town Annual Salary Salary per Student Salary per Employee Salary per Low Income Student Salary per non-English Speaking Student
Avon

181,000

$50

$186

$1,215

$3,694
Brookfield

177,568

$60

$463

$1,032

$7,102
Bridgeport

229,000

$12

$77

$12

$88
Danbury

200,778

$18

$115

$43

$97
Darien

225,000

$47

$302

$4,411

$16,071
Hartford

225,000

$11

$73

$11

$61
Killingly

149,171

$56

$272

$137

$2,528
New Fairfield

170,980

$59

$434

$601

$5,699
New Haven

226,920

$11

$68

$14

$68
New London

148,500

$50

$307

$52

$230
New Milford

188,000

$42

$313

$231

$1,352
Norwalk

200,000

$18

$124

$183

$159
Norwich

136,400

$35

$291

$53

$334
Region #15 (Southbury, Middlebury)

185,000

$43

$308

$740

$5,286
Stamford

215,000

$14

$98

$36

$105
Waterbury

169,580

$9

$60

$12

$87
West Hartford

223,966

$22

$141

$123

$355
West Haven

163,110

$26

$154

$58

$298
Weston

217,484

$85

$537

$6,396

$13,591
Westport

279,487

$49

$254

$1,827

$4,110
Wethersfield

208,677

$55

$342

$342

$1,280
Wilton

268,256

$62

$387

$5,831

$13,413


Now, while it is true that all this may be comparing apples and tomatoes, certainly there is validity in the saying that what’s good for the goose should be good for the gander.

If teacher evaluations are going to be dependent, at least in part, on standardized test scores, then certainly superintendent evaluation should be dependent, at least in part, on how well they do handling standardized per unit expenditures.

Let’s face it, are you really telling me that the Darien superintendent should be spending $16,000 for a student who doesn’t know English when Hartford is only spending $61?

Be Sociable, Share!

  • http://www.facebook.com/kristen.record Kristen Record

    Welcome to my world.  Thank you for sharing the realities of my professional life with others!

  • Mbracksieck

    I’m beginning to think that the superintendents who are supporting Malloy’s anti-tenure/ anti-union plan are doing it because they don’t want to do their job (or they don’t know what their job is).  Firing a teacher is a (necessarily) long, drawn-out and uncomfortable process.  If the superintendents could just shift the responsibility for that process over to a faceless “objective” test, that would relieve them of a significant burden.  

    Any superintendent who says they cannot fire bad teachers is either unclear about their responsibility or unclear about their authority.

    After Cirasuolo’s recent comment in the Post about the union preventing progress in schools which needed help, I suggested that he find another line of work.  Now I would add that any community which finds itself with a superintendent who says that they need Malloy’s bill to pass so they can fire the bad teachers may want to find a new superintendent because the one they currently have is just trying to pass the buck.

    • Linda174

      From my experience Superintendents do not even hold their principals accountable. They want to deal with the board, the budget and manipulate the community as budget time rolls out. They don’t want to deal with teaching and learning…..that is delegated as everything else is. They have a sweet deal with the original SB 24 – their contracts are extended to five years and they take a lot of power away from the BOE. Their biggest cost is the teachers/salaries and if they can get rid of those at the top (over time) they can save a lot of money.  These groups (CCER, CABE, CBIA, ConnCon, CAPSS, etc) are using the reform movement for their own selfish purposes.

  • Mbracksieck

    “Some Necessary Context
    Everyone who supports public education believes that only effective teachers should be in the classroom; ineffective teachers who can’t improve should lose their jobs. Accomplishing this requires a sound method for evaluating teachers and a fair process for firing. In the current system, school principals have the responsibility to assess teachers’ performance and dismiss ineffective ones. Making sure that principals do this well is the district superintendent’s responsibility (not the teachers’). The system works if administrators at all levels and school boards do their jobs.”

    Read more:
    http://www.dissentmagazine.org/online.php?id=504

    • Magister

      I just noticed this:

      http://www.ctnewsjunkie.com/ctnj.php/archives/entry/op-ed_in_support_of_malloys_education_bill/

      I am disappointed that a president of a university that produces a fair amount of CT teachers could be shortsighted enough to support this.

      • guest

        This little piece of support for the Gov and his Network comes approximately 2 weeks after Malloy gave Elsa Nunez a nice promotion.  He sort of made her a Superintendent/CEO of the CSU-Community College System, with a big fat raise.
        I think it opened her eyes to the Governor’s original school reform package, don’t you?
        And Nunez also sits on a board that is going to be assisting those low-performing Commissioner’s Network schools get help with swallowing SB 24.
        Thanks for posting this, Magister.  But there are no surprises.  She is becoming the Adamowski/Cirasuolo of the CSUs.

      • Magister

        I was hoping she was merely being an unrigorous thinker rather than a mercenary.

      • guest
      • savage

        Yep. Also about two weeks ago, Nunez was singing the praises of Gov. Malloy’s original education reform package when the gov. went to ECSU to participate in a men’s march protesting violence against women. What a cozy little family the gov. is putting together for himself, no? Wow, as a Puerto Rican emigre (her words) you’d think that Nunez would resent the fact that Malloy’s proposal will funnel funds from the poorest children (many of whom, in Willimantic at least, are also Puerto Rican immigrants) into the pockets of the corporatists who hope to profit off of charter schools. Guess money trumps ethnic pride, huh?

      • Follow the Money

         Oh how conveeeeeenient…

      • guest

        By collapsing positions into each other, and paying fewer people more money, the powers that be can create loyal servants… and they have fewer employees/lackeys whose arms they might need to twist.  But piling on the money usually obviates the need to twist arms.

    • Linda174

      Another great excerpt from the same article and exactly what his happening here in CT.

      Does this sound familiar? ConnCon, students First, GNEPSA, add salivating “reform” group name here.

      For most ed reformers, better a train wreck than no reform. They want as much change as possible as fast as possible in order to take advantage of momentum and the favorable political climate. The rush to pass state laws has provided their greatest opportunity so far. In response, they’ve skillfully built state campaigns, spending millions on organizers who advise lawmakers and legislative staffs, generate grassroots support, and run ad campaigns. A pipeline of private money—most of it from large private foundations—funds their state operations just as it funds almost all the ed reform movement’s activities.

      http://www.dissentmagazine.org/online.php?id=504

  • guest

    It used to be that Superintendents were vaguely authoritative “eminences grises” respected in the community to which they had close ties and where they occasionally caught hell for not calling a snow day right… Now they are “CEOs” and educational leadership graduates who have never spent a day in the classroom and who probably don’t even decide on snow days anymore (outsourced to a drone monitoring a weather satellite somewhere in south Asia). 
    I am starting to think that we could run school districts just fine without Superintendents or Special Masters.  We need to return to the system of someone teaching long enough to get tenure; who becomes a principal and runs a school fairly; who then considers further education and applying for a superintendent’s post.  No standardized test scores required.

  • Peg

    Congratulations from a stats instructor on your absolutely correct remark: “It may be that in all three situations the change is within the standard
    margin of error and therefore no conclusion can be reached in any of
    the cases.”  Most people would incorrectly label such a situation a “statistical tie,” of which there is no such thing.  (http://blogs.wsj.com/numbersguy/whats-a-statistical-tie-anyway-234/)

    • jonpelto

      Wow, thank you. Now I’m beaming with pride…

      And to this day my stats instructor swears I wasn’t even listening those few times I even bothered to show up. J

  • ConcernedTeacher

    All about the money.
    Average lifespan of a superintendent – 3-5 years.
    Average salary – $200,000 ($199,494 according to the chart above)

  • Follow the Money

    For a clue as to how misguided this is, look only toward New York City, and the Post’s unfortunate decision to take those test scores, rank teachers accordingly and publish the information, then interview the two teachers who were at the top and bottom of the list. The best and the worst. The only problem was that when they attempted this again, that teacher on the bottom had risen to the top quarter and the guy on top had fallen. Such is the life of a classroom teacher. Every year’s class is an entirely different makeup of students and abilities. If it could somehow be guaranteed that the raw material entering the classroom each year was identical to last year, then the problem would be solved. It would be easy to use test scores to evaluate teachers. But this isn’t a factory setting, or manufacturing operation, or is it? This so-called “business model” doesn’t work with human beings.