Advocate for the Right Tools in Teacher Evaluation

The blog post that follows is lifted (and slightly modified) from a lengthy comment I put on the Accomplished California Teachers Ning site, which is our members-only online platform.  A Los Angeles teacher asked me to elaborate on a prior comment I’d made suggesting that I think teachers need to fight against the use of value-added measures in teacher evaluation.

Okay – you asked for it!

It's important to have the right tools for the job.

Before I post the particulars, please note: standardized tests are not well-designed diagnostic tools when applied to individual students, let alone teachers.  They are mainly designed to give an indication of school, district, or state level performance.  They may have some value at the individual student level, but even that breaks down if you get to the subtest level (see J. Cizik, 2007).  So, if I had my way, any debate on this issue would begin (and moments later, end!) with the following exchange:

Q: Are these tests you’re using in our evaluations designed for that purpose, or validated for that purpose?

A: Um, no.

Q: What measures should we use then?

This is not just my opinion.  I’m relying on the policy position of the nation’s leading educational research organizations and professional associations: the American Psychological Association, National Research Council, American Education Research Association, and National Council for Measurement in Education all have said that tests and VAM are not up to the task of teacher evaluation.  Period.

Sadly, that won’t win the argument.  They’ll come back at you and say, “but data shows that effective teachers can raise scores…” or “we’re only using the tests for 20%…” or “other methods are not objective or reliable either.”  To which I would say:

– statistical correlation is not causation.  Data can also “prove” that effective 5th grade teachers raise their students’ 3rd grade scores.  Of course, we know that doesn’t happen, so this falsification test demonstrates the the test data are influenced by unseen factors (J. Rothstein, 2010).

– the percentage is arbitrary and subject to change, but since we know the data are faulty, they don’t belong in the evaluation.  At all.

– Other methods like observation, portfolios, self/peer evaluation, etc., may have reliability issues used in isolation and infrequently.  However, used in concert and multiple times, they are the best option we have.  James Popham, professor emeritus at UCLA and a pre-eminent expert in assessment, says that ultimately, the “professional judgment of well-trained colleagues” is the best option we have for teacher evaluation systems.  (When he says “colleagues” I think he means educators, but not necessarily immediate peers in your school; they might be former teachers, principals, etc.).

Here are some of my blog posts about VAM in evaluations, in which I elaborate on some of the items above, and link to more sources:

• Turning the Tables: VAM on Trial – This is my favorite prior post on the topic, in which I imagine an attorney shredding a VAM advocate in a trial cross-examination.  Inspired by an actual LAUSD lawsuit.
• Big Apple’s Rotten Ratings – Let’s learn something from the huge VAM mess New York City just stepped into.
• Evaluating Teachers with VAM: Variable Ambiguous Mistake – includes citations from an important VAM report by the Economics Policy Institute.
• Bleeding the Patient: VAM Nauseum – As the title suggests, I’ve tired of hearing VAM defended with the type of unsubstantiated arguments that made bleeding of patients seem like a good idea at the time.
• Open Letter to California Public Officials – A blog post that led to this slide in a presentation I made to the San Mateo County School Boards Association.

Original image by David B. Cohen. May be used with attribution.

I’m glad to talk more about this with you or any of your colleagues.  To be frank, I think some unions and some teacher leadership groups have made strategic errors by entertaining the idea of compromise on VAM in teacher evaluation policies.  Think how much stronger our advocacy for teaching and learning would be if teaching groups like Accomplished California Teachers, Educators for Excellence, Teach Plus, UTLA, NewTLA, and others could all agree and present a united vision to LAUSD!  Let’s include evidence of student learning, but make it student learning that matters: authentic, meaningful, substantive, and richly integrated with the curriculum.  But as long as VAM remains part of any evaluation proposal, I would advocate rejection.  Of course, it helps to have an alternative, not just a rejection, which is what teacher groups are thankfully getting around to in recent months and years.