Next Steps for LAUSD Teacher Evaluations: You, Me, and AGT
[edit: see end of post]
An interesting thing happened at a Teach Plus event in Los Angeles last month. LAUSD Superintendent John Deasy engaged in a “conversation” with about 100 public and charter school teachers and teacher-leaders on the district’s new Teacher Evaluation System (TES). After Deasy’s opening remarks, moderator Celine Coggins, CEO of Teach Plus, asked the assembly how we felt about evaluation and what constituted an “effective teacher.” Three disparate comments followed:
One teacher declared that it is high time that variations in students’ school experiences and subsequent performance were reduced. The woman stated that teachers needed to “get over” their reluctance to identify what they were striving for in the classroom and for individual students, and then actually measure it. This, she said, would make teachers more effective by revealing what actually worked.
Another teacher responded by saying that creativity and imagination cannot possibly be measured, and that education was always going to have that element of art and mystery. In a later personal conversation, she did not clearly explain how she planned, graded and reflected on her students’ progress, but she did state that they all performed extremely well on their California Standards Tests.
A third teacher said we would measure what we thought to be important, and educators need to define what this is themselves. Dr. Deasy “respectfully disagreed,” with her, saying we have “standards clearly defined” at every grade level, and why wasn’t that obvious?
Will our new, computerized Teacher Evaluation System be able to find any middle ground among the three viewpoints?
The Los Angeles Unified School District rolled out its technology-based teacher evaluation system in August. Seven hundred fifty “pioneer” teachers volunteered to test the new system, featuring a newly-negotiated set of teaching and learning standards. The new framework was devised with input from over 1,000 educators working in small groups. To begin the process, teachers grade themselves (from “ineffective” to “highly effective”) on 63 teaching standards, then fill in a lesson plan template, identifying which of the 63 standards their lesson addresses, and to what degree. Trained observers download the lesson, observe the teacher using it, and enter their own data. Everything but the observation itself is managed online.
Teachers at the September conversation showed a real willingness to reform their own approach to evaluation; not one spoke up to say they would not participate, or be against the new system. Of course, many stated historical concerns: did we really expect that a new system would foster collaboration, when the current system supposedly depends on collaboration but doesn’t produce enough of it? What about principals who are not experts in the teacher’s content area? Are we really going to continue using standardized test scores, when there is so much evidence that many learning gains happen in ways the tests cannot measure?
October’s “pioneer teachers” session included a lesson on AGT – Academic Growth over Time. This measure, the facilitator explained, factored in the intangibles we worry about: students’ language abilities, past performance, poverty factors, etc. The baseline is three, a comparison number that equals zero on the AGT number line, but allows comparing without anyone being in negative territory. If my students perform (that is, if I perform) below the district average, I’m not minus-one, I’m at “two.” Underneath my own performance number is a line stretching from my possible lowest score (one) to my possible highest score (five), called the “confidence interval.” In a perfect world, my own AGT would be above the district norm, showing my students had gained more than others, and the confidence interval would be very, very short, showing supreme confidence in that result.
In reality, while my AGT was slightly higher than the district average (good! good?) the confidence interval went from about 1.5 to about 4, because the results were based on twelve students. Which twelve? What happened to the rest?
All right, we’re piloting a new system. We’re helping work out the glitches. That’s fine. And I understand that my AGT number has been run through an algorithm designed by experts, and that efforts have been made to ensure that it’s fair. I’m comforted knowing that gender, race, homelessness and special education status have been factored in. That kind of takes some of the wind out of my protests, but actually, but I still have a question: when and where can I measure for the standards I am teaching, in a way that’s appropriate for these students, at this time?
We teach differently from one class to the next, from one year to the next, and we do it because we are paying attention to our students’ specific needs. I have asked my district, my administration, my network partner and my district’s foundation partner for a way to track and measure my own metrics, and though I finally found one possibly solution in an online grading program, no one who returned my calls or emails could really help me. Now the question becomes: when can I be evaluated on my actual practice rather than student results from an annual high-stakes test? It’s a question I continue to ask and encourage my union to pursue. If you’re a teacher, you should too.
[Note: This blog post was originally posted as an ACT Guest Blog Post. Lisa has since become a regular contributor to InterACT, and her old “guest” posts have been modified to reflect her authorship, and have had the original introduction removed. – David Cohen]
Trackbacks
- Friday On My Mind » #5 More Classroom Visitations – All I Want for Christmas: My 2011 Education Wish List
- Wrapping Up 2011: Best Guest Posts on InterACT « InterACT
- Teaching With Heart in the New Millennium « InterACT
- Budget Doublespeak: Less Money, More Change « InterACT
- LAUSD Ruling Clear on the Law, Won’t Shake Up the State « InterACT
- Deflating the Rhetoric in the Evaluation Debate « InterACT
ACT Guest Blog Posts
Martha Infante
David B. Cohen
Kelly A. Kovacic
Lisa Alva Wood
InterACT 
Interesting report. Your last question is extra important.
“when can I be evaluated on my actual practice rather than student results from an annual high-stakes test?”
But isn’t that what the “old” system was suppose to do? evaluate you on practice? An observation-based approach, ideally led by highly experienced administrators (who presumably have had years of successful teaching under their belts). It obviously failed miserably for the majority.
The administrators are for the most part not successful teachers. They, for the most part, avoid classrooms, and teachers teach in isolation.
Many of the problems that the new TES is suppose to fix could be addressed just by fixing poor implementation of the old system. And if poor implementation is the primary problem, then TES, no matter how fancy it is, risks the same half-assed implementation.
I’m not saying TES isn’t useful. But if you don’t fix poor implementation, you don’t fix 90% of the problem.
Thank you for your thoughtful comments. You are absolutely right — we should simply implement our current system! As a serious and reflective professional I appreciated the exercise of grading myself on 63 standards. As a taxpayer I question the validity of the TES expense.
Lisa, thank you. But I want to be clear: I absolutely believe we need to find some intelligent and relevant way to connect student outcomes to the work teachers do each day to improve those outcomes. But this is actually a small piece of a healthy evaluation system (in my view). The authentic relationship built up between a good manager/principal/teacher leader and a teacher – well, that is at the center, it’s the thing that’s been neglected, and it’s 90+% of a strong evaluation system.
I agree with Mike’s comments about the loss of traction in implementation. We are well-known for this in LAUSD. I would also direct your attention to this recent video of UC Berkeley Assoc. Prof. Jesse Rothstein on reforming teaching. I don’t think that I have seen a better encapsulation of the challenges inherent in our current mania about evaluating teachers.
Neglected to post the link: http://www.calchannel.com/channel/viewVideo/3071
You indicated that the LAUSD Teacher Evaluation System has been newly negotiated. I wonder with whom it has been negotiated. United Teachers Los Angeles has developed various ideas for a new Teacher Evaluation system and the district has refused to negotiate anything with the union. This system that they have bribed less than 0.01% of teachers to try out is not an item negotiated with the union but a political stunt by Superintendent Deasy to further his own goals.
As a parent I ask: If an adult, trained teacher should not be held accountable based on ONE high-stakes standards test, then why are we subjecting children to this standard? Teachers rightly complain that it doesn’t take into account the varying ability of kids in the class year to year, etc. The same can be said from the student’s perspective. Teacher quality varies from year to year. Kids develop at different rates. Yet, I hear few of my children’s teachers make the same fuss about using these tests on kids as they do about being held accountable based on their student’s performance. Some do see the all-around uselessness of these tests. But most others seem to like the simple one-off evaluation that a computer can do for them.
Jesse – I think if we’re talking about state tests, most teachers would agree with you that they are suitable neither for teacher nor student evaluation of any substantive nature. Many teachers are making a huge “fuss” about the misuse of tests both for students and for teachers. Yes, we see the uselessness and we do address it – but at the moment, politics, the media, and public awareness are not tipped in our favor to take down the testing juggernaut. I’m not sure I understand where you’re coming from with the last comment, however. The state tests are not a “one-off [student] evaluation that a computer can do for them.” Teachers are evaluating students all year long, and even if teachers liked using the state tests for that purpose, the results don’t come back during the school year. Furthermore, most teachers do not teach state-tested subjects and/or grade levels.
Jesse – I agree with David’s reply to your comment. I have been teaching for 29 years. In the “old days”, state tests were not high-stakes tests. They were just one of many snapshots of a child’s growth. With NCLB things changed. Tests became very high-stakes. Teachers complained bitterly. We knew that some of our students knew the material but did not perform well on tests. We were forbidden to talk to parents about their right to have their child excused from the test for that or any other valid reason. There may be a few teachers who think these tests are valid and fair. ALL the teachers I know agree that it is very unfair to rate a student, a teacher, a school, or anyone on just one test, taken on one day. We understand that a child may not be good at tests, may have a cold that week, may have just experienced a recent crisis in the family,etc. We have, and continue to discuss and rant about this problem. We also have learned that we should fear for our careers as the value of test scores continues to be elevated in the eyes of politicians, the media, and the public. We would love to have students and teachers evaluated by multiple measures. The way this will happen is if parents and teachers continue to cry out about the faults inherent in these tests.
clicky teachers hand pick their students for the following year…..
“artful” or “mysterious” teachers always get “problem students” cause as you say “they can handle it.” Then, two weeks before cst tests, four or 5 top performing students from the “mysterious” teacher are placed into other teachers classes to spread the numbers around. Happens every year. What you describe as a “artful mysterious” teacher will always have far below basic students all day long in every class while the others have the mid range and honors classes all day long, for decades.
If Duffy is working at a Magnet, what does that say about our Union? I’ve known always they have bad legal advice but I don’t think they understand historical significance of Unions, or history of what a Union is. They don’t return phone calls and if you do get someone on the phone, they are only interested if you did something wrong. WHen administrators or Beaudry big wigs belittle you and say derrogatory things in front of students and fellow staff, WHere is the Union you may ask? Sure, they’re defending sexual predators. What started at a Haymarket in Chicago died with Los angeles teacher’s union. Their actions will have national ramifications for all educators.
“my data” is not accurate. The scores that my students from last year recieved on their Cst report is different than the scores in “my data.” my honors class from last year is not in “my data.” 5 of my best students were placed in other classes and their data is not in my cst report, it’s in another teachers’ who hand picks his students. “my data” is a big waste of taxpayer money. It has good intentions but there is too many holes in the bag.
My last evaluation said that my students read and write too much. Said they needed more discussion. I discuss things with students all the time, where the hell do we live when “work ethic” hard work, students working in groups and actually having a hard challenging work load is bad?
Good teachers transcend all the yahoos who look down on them. If evaluations have nothing good to say, than they should shut the hell up……
Obama wants a national agenda for education? Make all the administrators and cubicle trolls teach for a period or two a day! How can they evaluate me when they haven’t been in a classroom as a teacher for three decades? The other 4 periods they can work their “administrator” functions, and that way when they talk about brilliant lesson plans, we can say, “oh cool, I’d love to see you teach that.” If they want to be experts, they need to proove that they are expert. It would solve all the financial woes of the district, the teachers do all the work yet they are paid next to nothing. Others spend three hour lunches downtown or go golfing all day with everyone in their office (I have pictures and evidence).
I work too damned hard.