Today’s guest blog post comes from Chris Miraglia. Chris is an intermediate school US history teacher from Santa Ana USD. A prior contributor and frequent commenter at this blog, Chris also writes at his own blog and is active on Twitter. Part Two of this post, by guest contributor Chandra Goodnough, is now online as well.
Last summer, a select group of teachers met at the California Teacher Association headquarters in Burlingame as part of the newly formed Teacher Leadership Cohort. The group was composed of teachers of varied levels of experience, as well as coming from diverse school districts. The goal was simple and twofold: tackle an area in education where change is needed, and take on a teacher-leader role (possibly not yet defined) to address that need. The philosophy behind the effort is based on Andy Hargreaves and Michael Fullan’s book, Professional Capital: Transforming Teaching in Every School.
In the two-day session last summer, twenty-plus educators discussed the basic tenets of the book, focusing on implementing change. Teacher empowered change was the buzzword. With assistance from CTA staff and with hours of discussion and team building, each teacher began to formulate a project addressing identified needs in their respective districts. Leaving the two-day session, empowered by the energy generated by these teacher leaders, each individual began the process of identifying the stakeholders in their projects, areas of resistance they might encounter, where they could find support, and what was the timeline for implementation of their goal.
For me, teacher evaluations have always had an appeal, so I set out to make a change in our process at the district level. For teacher evaluations and observations, my district has utilized an archaic form that does not serve its intended purpose. The first step in the process was meeting with my local president and CTA representative who were informed of my undertaking. With their support for the venture, we collaboratively worked out a strategy in which I visited school sites in order to survey their needs in area of evaluation, then presented findings to the union’s local representative council, and compiled a survey which will hopefully be used as jumping off point with our district in negotiations.
While this process moved forward in my district, our CTA Region 4 teacher leader cohort met monthly, online or in person, to discuss the status of the projects. Each member of the cohort facilitated a session and provided valuable input for each of our projects. Although our CTA cohort process technically ended in March, my project, as well as those of many of my colleagues, will continue past that point, as we are fully committed to change. Since teacher evaluations are a negotiable item, the outcome of my work may not be readily evident until next year or later. However, the knowledge gained from the experience was invaluable.
Teacher leadership roles have typically consisted of department and committee chairs, union positions, and only a few other identifiable positions. However, by empowering teachers to take additional roles in shaping the profession, in professional development, parent engagement, or changing evaluation policies and practices, a new activism is born.
For teacher-generated ideas about improving teacher evaluation, see the evaluation policy report produced by Accomplished California Teachers – on our Publications page.
At EdSource Today, a recent headline caught my eye (actually, thanks to Educate Our State for pointing it out):
Unfortunately, this problem is one of several indications that California’s policy makers don’t understand or don’t value the connection between “the whole child” and the classroom: we’re similarly underfunded when it comes to school counselors and nurses.
Many people hold an outdated image in mind concerning librarians, and they don’t know how much a professional librarian adds to the quality of a school. In response to comments at EdSource Today suggesting we could hire some “librarians” on the cheap to take the place of professionals, I composed the comment below – and encourage you to add your own comments here, and at EdSource.
Glad this is getting attention. It should be noted that librarians are teachers. In fact, a good librarian has to be the best teacher on a campus, because this is typically the only teacher whose responsibility extends to every student. We’re not talking about glorified book clerks and “Shhhh”-ing duty here. At Palo Alto High School, our librarian is incredibly pro-active. She anticipates student needs by familiarizing herself with the school-wide curriculum, to a degree possibly matched (but certainly not surpassed) by counselors and administrators. Knowing the curriculum allows the librarian to anticipate teacher needs as well. Collaboration between these teachers means that students (and even the classroom teachers) have better content for study, and learn the most up-to-date research skills and informational tools. Our librarian makes continual efforts to promote reading and literacy – not just by acquiring and checking out books, but also by finding out what students want to read, what form they prefer (audio, e-book, print), and what their individual tastes and interests are. She identifies what’s timely, relevant, important, popular. She has improved instruction by teachers, and made a remarkable difference in the academic careers – perhaps even the lives – of many, many students.
As much as I admire her, she is not Superwoman. This is not heroism – it’s professionalism. And she couldn’t do that job without having her own support. That means she has a budget to spend, authority to spend it, and classified staff supporting her.
My question is, why is the norm for Palo Alto, and not every other school in California?
Teacher evaluation and value-added measures have both been topics of frequent consideration here at InterACT. This re-blogged post by Ben Spielberg makes some excellent points about both, and provides a fine explanation of why we need to be cautious about relying on outcomes to evaluate teaching. Ben writes at 34justice.com where his bio notes: “Ben is currently a math instructional coach at two traditional public schools (one middle school and one high school) in San Jose, CA. He partners with all the teachers in both schools’ math departments to deliver high-quality instruction to students. ”
Originally posted on 34justice:
Imagine I observe two poker players playing two tournaments each. During their first tournaments, Player A makes $1200 and Player B loses $800. During her second tournament, Player A pockets another $1000. Player B, on the other hand, loses $1100 more during her second tournament. Would it be a good decision for me to sit down at a table and model my play after Player A?
For many people the answer to this question – no – is counterintuitive. I watched Player A and Player B play two tournaments each and their results were very different – haven’t I seen enough to conclude that Player A is the better poker player? Yet poker involves a considerable amount of luck and there are numerous possible short- and longer-term outcomes for skilled and unskilled players. As Nate Silver writes in The Signal and the Noise, I could monitor each player’s winnings…
View original 845 more words
The Common Core transition reached me on a more personal level this week when my 11-year old son came home and asked, “Dad, do you know anyone who made the Smarter Balanced test?” When I replied that I do not, he said, “That’s good. It’s a bad test.” My older son chimed in as well, giving the test mixed reviews.
Some people who know me might think I’d opt my children out of standardized testing, but so far, I haven’t. While I don’t care for standardized tests, I haven’t felt the need for my sons to opt-out because I don’t believe they have been over-tested, or had their time wasted on test prep; to my knowledge, no one in their school, or in our district, has suggested we should put those test result to any inappropriate use for the children or teachers.
I might have considered opting out anyways because I don’t like the use of tests to rank and punish schools, but even that objection has faded for now, with California moving away from its prior accountability program and entering a transitional period to something new. To be honest, I don’t fully understand the new system yet; schools can report the same rankings or ratings this year that they had last year, as we pilot test the Smarter Balanced assessments, and the eventual accountability measures will involve more local decision-making about a variety of measures beyond testing.
My sons had a few specific critiques of the test questions, user interface, and other issues. The whole family got a good laugh out of my 11-year old telling us about his essay. Due to a glitch in the system, he reports, his full response turned out like this:
Sorry I cannot write more than a line or it deletes itself. :(
My point in sharing this anecdote is not to criticize. Next year, the test will probably be improved, and from my sons’ perspective, the novelty will be long gone. What’s too hard this year may seem normal next year. Maybe not. It’s too soon to say.
But I will say this: I think my son’s response was perfect for the situation. And I think this anecdote perfectly illustrates how stupid it would be to evaluate his teacher based on results from a first-ever administration of a flawed assessment. (Even using the old tests, the problems of VAM in teacher evaluation are, at this point, insurmountable). Thankfully, in California, no one will even see the results from the first run-through. And why should we? No one would be able to say reliably what the results even mean, and it will take at least a few iterations before year-to-year comparisons have even a chance of offering any real insights.
I don’t think any independent expert in educational measurement or assessment is ready to go on record vouching for the validity of value-added measures in teacher evaluation if the inputs come from brand new assessments – tests that were never validated for that purpose in the first place. It’s mainly politicians and certain “accountability” enthusiasts in the education bureaucracy or think tank crowd who are ready to plunge recklessly into these unknown waters. Of course, many of these individuals, and their districts and states, are reacting to the pressure from the Education Department to take these unwise steps. Despite the legal ambiguities around his approach, and the deficiencies in research and reasoning, Secretary Arne Duncan continues to play at carrots and sticks to push VAM into teacher evaluation.
My sympathies to those of you living and working with the consequences; for the time being at least, in my state and district, the imperfections are being handled perfectly. Who knows? California’s slow and sane approach just might work.
A more philosophical post than usual – for what it’s worth.
This morning I had the opportunity to listen to a talk by Elane Geller, a Holocaust survivor originally from Poland, and now a resident of Southern California. I’ve heard Holocaust survivors speak a number of times in my life, and it’s always a profound experience, but there were two particular take-away ideas I thought would be worth sharing in this space.
The first was an observation Geller offered about Holocaust education, particularly for Jewish children. She commented that any learning about the Holocaust should be considered in a wider context (my words, not hers), when she talked about a cycle, “from joy, to pain, and back to joy again.” In other words, Holocaust education would not be a starting point for Jewish education; instead, you should start with the positivity of traditions and living culture. Then, yes, it is necessary to understand the negative history of the Holocaust, to confront evil directly and name it. And finally, you wouldn’t want to neglect the importance of closure that brings the child back to a sense of joy and positivity about the future.
It struck me that this cycle could apply to any learning experience, within the span of day, week, month or years. If academics are relevant to our students’ lives as members of a challenged society, then we must confront challenges openly, honestly, in ways that are sensitive to individuals and cultures while also academically focused. On the personal side, it makes sense to establish a sense of joy about learning, a degree of comfort among people in an academic community. Then, it should be safe to move into content that may be uncomfortable for some, but absolutely necessary. Such work can be done in an age-appropriate way that is still academically “honest” and true to the core of an academic discipline, and able to respect and honor the personal, emotional side of learning. The cycle is complete when our uncomfortable or challenging learning experiences are brought to a positive conclusion, with a sense of agency and purpose, and clear evidence of new learning.
The second observation that stuck with me this morning occurred when Geller talked about a sense of mutual responsibility, and even a sense of mutual peril in looking at world events. It’s a sentiment that has been expressed many times in many cultures, that a threat to human rights anywhere is a threat to human rights everywhere. The word ubuntu, found in multiple languages and dialects in southern Africa, identifies a similar concept – that my humanity is bound up in your humanity.
Given the scope of the humanitarian crises right now in places like Syria, South Sudan, central Africa, and North Korea, American education policy debates begin to look relatively minor. But on the other hand, the United States is not exactly leading the world in efforts to avoid a humanitarian crisis of its own (though of a different nature). The overall poverty rate in the United States is shameful, given our overall economic output. The childhood poverty rate is an embarrassment, and a blight that should speak to all of us on a personal, moral level. The potential social and economic upheaval that awaits us if we continue down this path should give us all pause, and then, prompt us to act.
Considering the severity of the poverty problem and the obvious deleterious effects of poverty on children’s health, social and academic development, it’s frankly troubling to me that philanthropists, politicians, and others supposedly dedicated to children’s welfare can remain relatively silent about economics and broader social policies, while dedicating considerable time, money, and energy to vigorous battles over policies that have questionable chances of producing minor improvements in children’s lives. I’m not trying to seize the Holocaust or other vast social problems as a high road to attack the positions of people whose education policy ideas I disagree with; setting aside the merits of any specific policy position, I will go so far as to say that those focused on marginal issues while ignoring essential issues lack credibility when they try to seize the moral high ground.
Let’s have those debates. But maybe those debates would be less contentious and more productive if we had more ubuntu.
Teacher evaluation has been a frequent topic in this space: Accomplished California Teachers (ACT) first coalesced as a teacher leadership group in large part to produce a report on evaluation that would feature teacher voice regarding current practices and promising reforms for California schools. I’ve also written frequently about an evaluation method that stands out as the worst popular idea out there – using value-added measurement (VAM) of student test scores as part of a teacher evaluation. The research evidence showing problems with VAM in teacher evaluation is solid, consistent, and comes from multiple fields and disciplines – most recently, statisticians (more on that in a moment). The evidence comes from companies, universities, and governmental studies. And the anecdotal evidence is rather damning as well: how many VAM train-wrecks do we need to see?
On the relevance of student learning to teacher evaluation, the ACT team that produced our evaluation report was influenced by the fact that many of us were National Board Certified Teachers. Our certification required evidence of student learning – after all, teaching without learning is merely a set of word or actions. Board certified or not, our team members all agreed that an effective teacher needs to be able to show student learning, as part of an analytical and reflective architecture of accomplished teaching. It doesn’t mean that student learning happens for every student on the same timeline, showing up on the same types of assessments, but effective teachers take all assessments and learning experiences into account in the constant effort to plan and improve good instruction.
Value-added measures have a certain intuitive appeal, because they claim the ability to predict the trajectory of student test scores, theoretically showing the “value” added by the teacher if the score is higher than predicted. This deceptively simple concept sounds reasonable, especially for non-teachers, and even more so for policy makers. They often seem eager to impose on teachers and administrators what is essentially one-way “accountability” for the success of schools; stagnant or declining scores bring negative consequences, so the public can be reassured that insecure school personnel will be compelled to do their jobs. Meanwhile, policy makers often ignore (because the voters and media allow them to ignore) what should be their share of accountability for the conditions of schools, and even the outside-of-school conditions that all the experts agree outweigh teacher effects on standardized test scores. Yes, you read that correctly: most of the variation in students’ test scores can be accounted for by factors outside of school – factors like family wealth, educational attainment, health care, and similar.
If you care to look at some of my prior posts on the topic of VAM in teacher evaluation, you’ll find that education researchers, economists, scientists, mathematicians, and experts in psychometrics (the measurement of knowledge) have all weighed in against the idea. Some offer stronger objections than others, but most agree that VAM is not stable or reliable enough for high-stakes usage. It has also been noted by multiple professional associations that measures validated for one purpose (measuring student knowledge) cannot be assumed valid for other purposes (measuring teacher effect). The main proponents of VAM use for high-stakes personnel decisions all seem to be economists (Hanushek, Chetty, Kane), or researchers with some vested interest in finding what they end up finding (Gates Foundation, William Sanders).
Well, the latest professional group to weigh in on the topic was the American Statistical Association. The ASA is not against the concept or use of VAM, but they do caution that VAM should only be used under a whole set of circumstances that are quite unlike the circumstances found in schools and districts using VAM. For example, VAM should be used by experts, with clear information regarding formulas and margins of error, and careful analysis of how sensitive statistical models are when the assessment changes.
Here are some choice quotes from their April 8, 2014 report:
VAMs typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model.
Ranking teachers by their VAM scores can have unintended consequences that reduce quality.
VAMs are only as good as the data fed into them. Ideally, tests should fully measure student achievement with respect to the curriculum objectives and content standards adopted by the state, in both breadth and depth. In practice, no test meets this stringent standard, and it needs to be recognized that, at best, most VAMs predict only performance on the test and not necessarily long-range learning outcomes.
[Regarding studies that have found some predictive ability in VAM scores by teachers, with "correlations generally less that 0.5]: These studies, however, have taken place in districts in which VAMs are used for low-stakes purposes. The models fit under these circumstances do not necessarily predict the relationship between VAM scores and student test score gains that would result if VAMs were implemented for high-stakes purposes such as awarding tenure, making salary decisions, or dismissing teachers.
I should also note that there’s a portion of this report I disagree with, regarding the potential use of VAM to evaluate teacher training programs:
A VAM score may provide teachers and administrators with information on their students’ performance and identify areas where improvement is needed, but it does not provide information on how to improve the teaching. The models, however, may be used to evaluate effects of policies or teacher training programs by comparing the average VAM scores of teachers from different programs. In these uses, the VAM scores partially adjust for the differing backgrounds of the students, and averaging the results over different teachers improves the stability of the estimates.
It’s unclear to me how VAM within schools or districts is recommended for observing correlation, but when extended beyond schools to involve even more complex interactions of variables known and unknown, we’re now talking about evaluating effects (causation, rather than mere correlation). While I understand the value of larger sample sizes in reaching stronger conclusions about data, I question the ability of anyone undertaking such an evaluation to control for the differences among schools. The quote above mentions only the “differing backgrounds of the students.” However, different teacher training programs develop different relationships with schools and districts. Teachers are not randomly distributed to schools or communities after their training, and the school’s and community’s effects on the teachers would seem highly relevant. There are studies that show the effects of principals on test scores, the effects teachers on teachers, effects of class period length, effects of tutoring that may or may not be available, effects of libraries that may or may not even be open, etc. My open letter to California policy makers on this topic argued, and would stand by the argument, that there are simply too many interacting variables to reach any reliable conclusions that depend on value-added measures.
Teachers are tribal people. We build a nice little fortress and stay inside as much as possible, defending the gates when necessary. This tribalism has saved us in an ever-shifting landscape, but it’s got its limitations and it may soon be the cause of our demise. The two big tribes now are Old Generation and Next Generation teachers, and the most obvious place of need is our local unions. The Next Gen teachers are attracted by the glamour of the reform groups and the promise of an amplified voice; the Old Gen prefers the lunchroom and the union hall. Over the last school year I’ve gotten the chance to hang out with 58 Next Gen teachers from 17 states, courtesy of the NEA. I’ve learned that we’re lined up on either side of a generation gap that is worlds apart, but that each side is a critical part of the equation. We need each other more than we will benefit from sticking with our tribe.
Because the origins of unions are rooted in staying alive in a hostile environment, the Old Generation totally gets the need for protection, collective bargaining and the need to jump up and down once in a while. The Old Gen safeguards everyone’s rights whether they like it or not – experience has taught them that careers are subject to the whim of the public, politicians, and now philanthropists. We once did believe that our good work would be our protection from harm or unfair practices; we never saw the need for pensions or fully-funded health benefits as youngsters. And then life taught us something – we’re treading faster for less money and less respect, and sometimes one of us gets caught in the machinery and goes down. Working conditions, pay raises, benefits, retirement, the bread and butter issues, these are the spears and bagpipes of teachers who are nearing the end of their tenure and see the world falling apart around them. They sometimes see the Next Generation of teachers as an opposing camp, vulnerable to the influence of outside agencies. But these are the people we need to protect us, our profession, and public education and we need to make friends now.
The Next Generation doesn’t seem to register the Old Gen to the same degree. The Old Gen are just irrelevant, cranky strangers in red t-shirts standing outside the board of education, waving signs. These younger folks see step and column pay scales as ridiculous, and can’t understand why anyone would ever have a problem with being evaluated or using student outcome data as part of that evaluation. The reform groups give them opportunities to meet policymakers, be on panels, write policy papers – that’s what drew me in and that’s what we need to do to compete. In the last three years in LA, Next Gen issues have been career pathways, evaluation and new pay structures. They don’t care about retirement right now. But the Next Generation needs to understand that without involvement in the union today, there won’t be a retirement later, or anything else resembling stability. Time to end the tribalism.
So how do unions take responsibility for educating their younger siblings, and why should it be incumbent upon the Next Gen to cooperate with the Old Gen? Union policy is often the result of great ideas for what other people should do. Someone should form a committee! Someone needs to do outreach! Someone should plan an event! Our locals, with the support of state and national affiliates, need to offer professional development, discussions and social events. Our locals need to admit that a panel with dinner and drinks is way more interesting than parliamentary procedure, and then have panel discussions around Next Gen issues with union leadership. Finally, our locals need members like you and me to create opportunity for the Next Gen to learn the relevance of the union, and to be relevant in it. We know there are interesting things happening in our locals even if we have to dig around for them. Find one of those opportunities and take along a young friend.
Our future depends on it.