Skip to content

Do You Understand My School? (Part Two)

April 22, 2010

Yesterday, I posted “Do You Understand My School (Part One)” and argued that policy makers place unwarranted faith in state test scores as a tool for teacher evaluation.  In large part, I believe that happens because they don’t sufficiently understand how schools operate.  The argument continues:

To further complicate matters for the test advocates, they have no formula to account for the students who change classes, teachers, or schools during the year.  You can’t just weight the test score according to the percentage of instructional time spent in a class, since classes do not cover the same content and skills all year long.  That’s not a trivial matter affecting a small number of students.  In some school communities, student transiency is a huge factor; additionally, in some schools, students can drop a class, or be removed from it for disciplinary reasons, but the student still must take the state test in that subject.  In some schools, semester long courses mean teacher changes for an entire student body at the middle of the year.  How do you use state tests to evaluate a teacher’s performance across semesters when that test is given before the completion of the second semester, but months after the first?

If my school expects me to take on a different teaching assignment, new curriculum, or new methodology next year, am I allowed a one-year grace period on producing test results?  After all, the first year of teaching a new curriculum presents some unique challenges during a period of adjustment.  Or, should I fight any change to my duties, in order to protect my livelihood?  If I’m deemed effective as a teacher of juniors but my principal asks me to teach freshmen, that change might involve some adjustment, or I just might not be as good with one age group as I am with another.  Why take a chance?  If I teach seniors, they don’t even take a state test, depriving me of a chance to (supposedly) prove my effectiveness.

If I teach courses for two grade levels, will I be evaluated with student test results separated by grade levels? If so, we should be concerned about sample size – how much weight is given to each single student’s results – and the likely volatility that results from small samples.  If not, it seems likely an average might mask evidence of strengths and weaknesses.  (By critiquing both approaches, my point is not that teachers should therefore ignore student growth; we need multiple and more robust measures of student growth – a topic I will return to in later blogs, and which will be addressed in an upcoming policy report by Accomplished California Teachers).  Further complicating the matter of grade levels, how do you measure effectiveness when students in the same class are in different grades?  Is it reasonable to expect the same growth from freshmen that you expect from sophomores, or even juniors, who might all be enrolled in the same math, language, or arts class?

An additional problem too often overlooked is that students are not randomly distributed. I might teach the exact same course at the same grade level as one of my colleagues, but differences in the students’ schedules and activities will affect the make-up of the class.  If computer science classes are only offered at certain times of the day, you increase the odds of grouping those students in the same math classrooms, while the same math course offered at other times, with another teacher, will have a low concentration of students currently enrolled in computer science.  If my English classes are mostly in the morning, my students will be present for more instructional time, while my afternoon classes are thinned out when athletes are excused for competitions.  Then, at every school you will find differences in the ways students with special needs are placed in mainstream classes, further complicating efforts to compare results.

What about extra support?  If one-third of my students have a study skills class to help them keep up, and that class is effective, would you expect to see performance results on state tests?  And if so, what does that have to do with my teaching?  Consider also that some students are repeating a class, though we have no means of figuring out precisely which content and skills they picked up in each time they attempted the course.  What about students being tutored, or those who have better academic support at home?  We have no means of tracking that information reliably, no way to measure the tutoring effect, and again, no method to ensure that well-supported and under-supported students are distributed randomly among teachers.

Just to be safe, maybe I’ll start offering more personal assistance after school.  That would help my students achieve better results.  Students may be more likely to come in for extra help if they come with friends, but I’d better not help the friends if they are not in my class, lest I make myself appear less effective relative to their teachers.  I certainly shouldn’t promote the idea of having a team of teachers operate an after school program, because well-distributed results do little to enhance my evaluation in a competitive atmosphere, or increase my job security when layoffs are looming, and seniority no longer matters (another currently popular goal of education reformers).

Education reformers, especially those who want to weaken teacher unions, like to tell us that schools should be run according to the needs of the students – not the teachers.  I agree.  So, why do they argue for policies that encourage teachers to neglect the best interests of students and put self-preservation first?

It’s because they don’t understand schools.

They might say I’m indulging in hypotheticals, or presenting exceptions as if they are the rule.  That would be further evidence of their misunderstanding.  Here’s the thing:  exceptions – collectively – are the rule.

6 Comments leave one →
  1. April 22, 2010 8:34 am

    The two blogs, read together, represent familiar turf for most teachers. It’s worth remembering that you’re teaching in a “functional” school, too. From your detailed synopsis of the daily operations of schooling, it only goes downhill for most teachers, vis-a-vis having their work measured by testing.

    Your blog should be required reading for policymakers, of course–but it should also be required reading for researchers. I have met a number of researcher-promoters of value-added / growth-model statistical measures, who believe they have bypassed the measurement difficulties inherent in judging teacher practice via numeric student achievement data through the use of sophisticated statistical modeling.

    When you start telling them that the kids are re-arranged throughout the day/week/year for reading (something that virtually every elementary school does)–or that kids often learn the most about reading for comprehension from their science teacher or media specialist, not their LA teacher–they grow agitated. When VAM guru Bill Sanders tried to measure NBCT effectiveness in North Carolina (where the data pool was large and available), he ran into all kinds of difficulties in tracing kids back to their NBCT teachers. That didn’t keep him from making grand pronouncements, however.

    Researchers have their own agendas (continued federal grant funding, for example). Stories like yours only confound their goals. Keep writing, keep writing. Another great piece from InterACT.

  2. David B. Cohen permalink*
    April 22, 2010 9:44 am

    Thanks for reading and commenting, Nancy. I appreciate your notes about the importance of recognizing how my views are shaped by my context. I hope that the final line seems universal to teachers though. When you really know your students, none of them seem “average” anymore. My goal is not to say that because they’re all exceptional in some way that I reject in any way my responsibility for their academic growth. It’s just that these tests aren’t the proper measure. It would all be distorted in too many directions – some false “good” results and some false “bad.” Some of my current students might perform better this year because their family life is stabilized, their illness is under control or gone, their personal life has improved, any number of reasons. If they failed to show what they know on a test last year and shine this year, it’s not necessarily evidence that I’m good, and I don’t want useless information tied to my evaluation – whether it’s supposedly good or not.

    On the research front, I hope researchers aren’t just relying on teachers to point this out. AERA, APA, and NCME are the organizations that set standards in this area ( http://teststandards.org/ ), and their guidelines argue against the moves politicians are embracing. If politicians reject research standards, well… we can be disappointed, but not surprised anymore. If researchers reject those standards, I’d worry.

  3. April 26, 2010 8:56 am

    David, a very thoughtful posting on what you’re seeing from the perspective of a teacher. My fear has been that policymakers who seek to break the grip of a status quo that no one supports–who thinks achievement gaps are acceptable?–are moving into areas they do not understand well. They want to improve teacher evaluation–a laudable goal–but feel student test scores should be the primary driver of this strategy. My fear is that an accountability system that does this work on the cheap could be far, far worse than what we have now. You ground that fear in daily reality.

  4. David B. Cohen permalink*
    April 26, 2010 11:30 am

    Thanks for confirming my fears, I guess! I appreciate your comment Claus, and all the work you do along the same lines. I think you had a recent post about worries, and mentioned that from the political side, there’s a push for changes that are large-scale and immediate, which makes it urgent for us to raise the issues, loudly, and constantly, and now. Not the best way to operate in examining broad and complex systemic reforms…

  5. David B. Cohen permalink*
    April 27, 2010 10:37 pm

    For an expert opinion from the research realm, check out this blog post by Kathryn McDermott, Associate Professor of Education and Public Policy and Lisa Keller, Assistant Professor in the Research and Evaluation Methods Program, both at the UMass, Amherst.

    http://shar.es/m5CfP

Trackbacks

  1. 2010: InterACT in Review « InterACT

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: