Teacher Evaluation as Legal, Political Football
photo by the author
[EDIT: 12/5/12 – also see Lisa Alva’s comment below regarding UTLA vote]
Los Angeles Unified School District and United Teachers of Los Angeles have reached a tentative agreement on new teacher evaluation practices and policies. The union will take up the agreement in a meeting of its governing body later today, and I expect that there will be some tough questions for the union leadership regarding compromises on the use of student testing results. A legal ruling against the district held that, under current law, student growth had to be a factor in teacher evaluation. That may be the law, but it still doesn’t make sense from an educational measurement or research standpoint. Judges, lawyers, and legislators cannot change the facts, even though they can ignore them: tests designed and validated for the purpose of measuring student learning cannot be deemed valid for other purposes like teacher evaluation.
The favored approach in policy and politics these days – even though it doesn’t work – is to use “value-added” measures of student learning, trying to isolate the effect of a given teacher on each student. LAUSD has its own formula, called “academic growth over time” (AGT). Writing in EdSource Today, John Fensterwald summarized the agreement’s handling of AGT:
[LAUSD Superintendent] Deasy had proposed using it as a significant factor in evaluations; UTLA opposes it as unsound. In the agreement, AGT scores of individual teachers cannot be part of a teacher’s evaluation, although school-level AGT data, thought to be more reliable, can be included among many factors.
However, a more surprising development is that, apparently, students’ “raw” test scores will be factored into evaluation, as described by Chad Aldeman from EdSector:
Late last week the Los Angeles Unified School District and its teachers union (UTLA) reached an agreement on including student test scores in teacher evaluations. The agreement does not include all of the details (UTLA has posted a summary and the full provisions), but it is clear that they will not be using a teacher’s individual value-added score. Instead, they will be using school-level value-added data and a teacher’s raw performance data. Using raw performance data in a teacher’s evaluation is an incredible mistake, and one that both reformers and union leaders should loudly denounce.
The reason for alarm regarding that approach is that, even more than value-added formulas, this approach would distort or falsely attribute student performance to teachers. Standardized test scores track most closely to socio-economic status and English language acquisition, and students who excel or flounder in any given year are likely to produce similar results the following year. Under this approach, there’s an obvious penalty incurred by any teacher unlucky enough to teach students who are poor, or still learning English. What percentage of LAUSD teachers and students are we talking about here? A significant majority, I think.
Classroom observations will continue to be the more significant part of teacher evaluations, though LAUSD is still in the midst of working out new approaches to this component of evaluation. Lisa Alva Wood wrote about the 63-point rating system and the time involved in the process for her as a volunteer in the pilot-program for new evaluations. So, at the moment, it seems there’s nothing certain, nothing stable, when it comes to teacher evaluation in the state’s largest district.
Not surprisingly, teachers I’ve been in touch with in Los Angeles are not enthused about this agreement. Among multiple teachers I corresponded with, Jordan Henry, a leader in the NewTLA group within UTLA, took this view:
[The tentative agreement] basically slides the use and application of student data into Article X, which is the current evaluation protocol – the same protocol that has not been occurring inside LAUSD for decades. The plaintiffs in Doe/Deasy argued successfully that a flaw of the current evaluation system was the lack of student data – but those of us in LAUSD know and have known for some time that the dominant flaw of the evaluation system was the actual lack of observation/evaluation – and the joint planning before it, the joint discussion afterwards, all dictated in Article X. So for the time being, we’ve added a missing piece to a puzzle hardly used.
I don’t know how Los Angeles teachers will vote on this agreement, but I do know that, once again, education policies are being negotiated and debated under the wrong conditions. Rushed negotiations under threat of legal action, changing almost everything at once in procedures, neglecting the underlying lack of system capacity, and on the cusp of transitions to new Common Core standards and assessments – this is no way to improve education.
ACT Guest Blog Posts
Martha Infante
David B. Cohen
Kelly A. Kovacic
Lisa Alva Wood
InterACT 
When I heard about the agreement, I was shocked. I am afraid that this will set a precedent for many large districts, mine included which have a considerable amount of EL students. We just had a discussion at my site today about the problems EL students have accessing secondary core content, especially when moving towards the Common Core.
It is quite clear that not only do EL students struggle with language, but then to be evaluated on some of the Common Core strands is a tall mountain for them to climb. For many of them to expect them to have mastery in the Common Core is analogous to building a house without a foundation. Some of my EL students are failing my class and are also classified as basic to below basic on the CST LA. If my district would follow LAUSD’s lead I would be evaluated on the 100 EL students and the 150 standard students achievement, even as a percentage of EL students who are in middle school remain at the same CST level from year to year (this I just looked up on our own district data system). And to think I could be evaluated on this segment is incomprehensible.
I hope that teachers are aware of the precedent this might set. As an 8th grade history teacher, I can’t even imagine what type of VAM would be used to evaluate me when my students are tested on three years of content from 6th grade Ancient Civilizations, 7th Grade World History, and then 8th Grade US History. Am I to take the fall for the previous two years since I am the teacher of record? I find this whole agreement professionally insulting and a band-aid to previous failed evaluation systems.
Chris, if we assume best intentions in those who propose using test scores for evaluation, I think they’d say teachers should be held accountable for growth and improvement, and so if your students struggle but improve, you wouldn’t be penalized. Of course, those VAM-for-eval advocates never have an educationally sound justification for using invalid and unreliable measures in teacher evaluation, so it’s increasingly hard to assume good intentions. In fact, back in September CTA tried to get Asm. Fuentes to insert language in his bill to require tests to be validated for evaluation before being used for evaluation, and at the leader of at least one educational advocacy organization called that a “poison pill” that would ruin test-based evaluation. If validity is poison, I think we can assume that the speaker has an interest other than improving teaching and learning.
I think the argument that this use of test data is a mistake is an argument we can win, and I do not think that many of the proponents of this mistake have nefarious motives. I think they are often trained in business school reasoning and want to apply it to education, and have very little experience with the day-to-day realities of school-based education.
The UTLA House of Representatives ratified this agreement tonight by a 2/3 majority. Warren Fletcher presented the TA positively, stating pointing out the fine print about the lack of a percentage or a number attached to the incorporation of AGT, individual or schoolwide. He also discussed being under pressure of a deadline and a judge who was ready to impose the unequivocal use of VAM and AGT on every teacher. Warren clearly stated that every California teacher would be affected by the decision.
John Perez, UTLA past president, spoke about the fact that teachers in other large urban areas, and even local non-LAUSD teachers, are governed by data at evaluation time, and that this alternative, which was derived from “negotiating with a gun to (our) head,” was a better fate than theirs.
Warren did point out that teachers with ten or more years’ experience and consecutive positive evaluations could, by mutual agreement with administration, would be evaluated every three, four or five years, and that the final qualitative descriptor would be binary (meets/exceeds vs. not does meet) as opposed to the four-level descriptor that LAUSD was seeking in order to take the next step towards merit pay. He and others indicated that our affiliates in Sacramento were supportive of the agreement.
The debate included most of the concerns you address, David.
Taken on face value, the agreement does include many of the elements proposed by UTLA in the first place, and spells out very clearly when, how, and which data will be used. There are many, many details that to me, seem open to interpretation. Having piloted the proposed evaluation system, I am interested to see how this will play out in the real world of fewer people doing more with far far less.
Thank you for your very thoughtful report, David. I continue to reference ACT documents to inform my thinking and actions as time goes by.
Hmm. The part I don’t understand is why the lack of a percentage or number is a good thing. That seems to leave the door open for teachers to have as many raw standardized test scores incorporated into their evaluations as their administrators can pressure.
It’s difficult to tell how this will play out, but I do know that we work in a culture of fear. I teach inner-city kids whose raw CST scores have been low since elementary school. I can only imagine the disaster to come with Common Core. 20 out of 25 kids in my advisory got a legal notice to appear at a meeting with their parents regarding truancies today. Few are academic, few really care, and exceptions exist but rarely. I’ve already quit teaching English due to this nonsense, and after a 13 year stint teach highly gifted kids in the Westside, I wonder if it’s time to regret my decision to teach this hardcore population and return to the more affluent to save my job, even though the inner-city kids need me more. I don’t know what to think or how to vote EXCEPT that if NewTLA is against it, perhaps I should consider voting yes.
Oh. I see that I made lots of end-of-a-long day grammatical errors, but did not mean to imply that NewTLA is for or against anything. We don’t have a position on anything, we only exist to encourage teacher participation in a democratic process. I wonder why you would be for something that NewTLA would purportedly be against? Please get in touch, CJ, if you’d like to share a perspective with me.
BTW, I have started addressing common core in my urban classroom, and am having better, measurable learning results this year, and more fun and more connection and interest in everything, than I ever have before. Common core isn’t anything to really be afraid of, if you ask me. I have taught in my high-poverty, home school of attendance school for 12 years now, and this is the best year I’ ve had in a really long time, despite the fact that this population is the most challenging that I’ve experienced. The new plethora of charter and other school choices in our community means that our 100 year old neighborhood school is the default. No matter. I enjoy my classroom children, and even using current grade level standards, got three perfect essays this year, and great results. They read at elementary levels, but they think just like regular sophomores. My children just need a little more support in their writing and conventions. When they get it, it’s all good. Really good!
I’m not an educator, per se, and a lurker to boot, but as a parent and a researcher, I wanted to comment on this note to add a different perspective to the conversation.
If we wish to talk about the role of unions in education and the possiblity that teacher evaluation will play a role in (activity), why not use professional sports as a baseline? I this environment, unions play a role in mediating the relationship between owners and producers, mainly to prevent exploitation of producers and owners hogging all the profit. In return for being allowed to come to the game, athletes are subjected to innumerable statistical evaluations. Yes, some data may seem arbitrary and others may have enormous consequence in a player’s ability to continue playing the game. Why is it automatically a bad thing to have more data collected on teacher performance?
In the sport analogy, the consumers are the fan and spectator. In education, it is the student. In both cases, the consumer does not receive a direct finanancial gain from the experience. All of the money fought over by owners and players ultimately comes from investments made by the consumers. Do we want education to become the next NHL?
Kradak, thanks for your comment. The statistics gathered on one player are more akin to the statistics for one student. Are you going to judge the quality of a manager based on the compiled statistics of the players? Player A had the following batting averages in the past three years: .270, .285, .300 – so is it bad managment if he doesn’t hit .315 or better this year? Last year the team had a 81-81 record: if their record this year is 91-71, is that better management? Last year’s team ERA was 3.00, and this year’s was 2.8 – is that good management? Any sports fan knows there are far too many variables to make snap judgments, and no one delights more than sports fans in arguing about the meaning of the numbers. And compared to education, those are robust numbers about simplistic concepts. In education we have much less data about much more complicated situations with much higher stakes. But I do agree with you that management and labor would be better off not fighting so that the consumer/investor – i.e., the general public – retains confidence in organization. But then again, “management” in this case is asking for teachers to subject themselves to invalid and unreliable evaluation tools, and this approach will not help students. So, by fighting managment’s position, we believe we’re serving the public better, pointing the way to a superior evaluation system that can improve teaching and learning.