Accountability Missing at L.A. Times
EDIT/UPDATE (8/18/10) – Today, I had the opportunity to discuss teacher evaluation as a guest on the radio program Forum, on KQED 88.5 FM San Francisco. Clicking on the link will take you to their web page, where you can hear the show and see the large volume of comments, some of which I replied to.
What were they thinking? I have yet to see any researchers come to the defense of the L.A. Times hit piece using VAM (edit: “value-added measurement”) as the sole measure of teacher effectiveness; meanwhile, criticisms of their work are pouring in from around the country – from researchers, teachers, and a variety of leaders in education. See the comments section of this blog post for some of the those responses, and feel free to add others.
Almost one year ago, the Times ran an article about the use of value-added measures in San Diego. I spotted some flaws in that report, also written by Jason Felch and Jason Song – two of the authors of this recent fiasco. So, I wrote them an email outlining some of the problems I saw. (They did not reply to my email). Below, I am publishing the full and exact text of that email. It is not a comprehensive take on the issue of VAM (for example, I didn’t include the NCME/APA/AERA statement about not using student achievement tests as teacher effectiveness tests since they haven’t been validated for that purpose), and I would trust that that the authors have even better resources to turn to. However, I’m making this letter public now to share at least one example of how the reporters should have known better when putting together their report on teacher effectiveness in LAUSD.
from: David B. Cohen
to: email@example.com, firstname.lastname@example.org
date: Tue, Oct 20, 2009 at 11:45 PM
subject: value-added measurement
I am a leading member of Accomplished California Teachers (
http://nbrc.stanford.edu/act/ ) and a National Board Certified
Teacher, and I’d like to share some thoughts about your recent
education article. Thank you for your reporting on the important
developments and issues in education reform. I read your article
about movement towards the use of value-added measurement in San
Diego, and I was disappointed by some lack of information.
The main issue is that you seem to accept at face value claims about
VAM and seem to accept as fact that the standardized test data used in
VAM means what the test-publishers say it means. I say seem because
the only opposing view you offer is from teachers and unions (who are
rightly skeptical of VAM), but you do us a disservice by neglecting to
mention that VAM has serious flaws that have been pointed out by
researchers, statisticians, economists, etc. You distort the issue if
you only go so deep as to repeat tired clichés about anti-reform union
politics and entrenched teachers who seem to reject accountability.
In fact, if you look more carefully at other research and writing on
the topic of VAM, there is no consensus that it works. You should
also note that Sanders use of VAM for profit pushes it further than
many other experts consider to be wise or appropriate. He uses a
proprietary, unexamined model for his calculations. I’m not saying
that makes it wrong – he’s entitled to sell his work. But you should
be reporting that doubts about his methods do not come exclusively
from teachers unions and his work can’t be independently
Here’s some information you might find useful in future coverage of this issue.
Gerald Bracey has written this on the topic ( from
“Aside from William Sanders and his Tennessee Value Added Assessment
System (TVAAS), those working in VAA (Henry Braun, Howard Wainer, Dan
McCaffrey, Dale Ballou, J. R. Lockwood, Haggai Kupermintz, for
example) acknowledge that it cannot permit causal inferences about
individual teachers. At best, it is a first step toward identifying
teachers who might need additional professional development or low
performing schools in need of technical assistance.
“The model also presumes that the teacher ‘effect’ persists-like a
diamond, it lasts undiminished forever. This has not been
And finally, if I may, here’s a couple links providing my own take on the issue:
http://www.sacbee.com/1190/story/2156879.html (A Sacramento Bee op-ed
from last month)
(From Teacher Magazine. Registration required. In case you aren’t
registered, or aren’t interested in registering, the main idea of the
piece is that there is absolutely no way to reliably isolate and
measure “teacher effect” using test scores, when you understand the
realities of how schools and teachers function and how students live
and learn. For what its worth, I do believe in “teacher effect” if we
can develop a more robust and authentic set of “data” using multiple
For your future information, Accomplished California Teachers has
members in the Los Angeles and San Diego area who would be glad to
speak with you on these matters, when you want to get into the messy
details of real teaching and learning underlying the test scores,
unions, and school board politics.
David B. Cohen
Accomplished California Teachers