The Evil Economics Of Judging Teachers

The Times and a host of other publications heralded last week’s new study extolling the lifelong money-earning benefits of having a good primary/middle-school teacher. Oh, yay! Let’s do what these economists from the National Bureau of Economic Research suggest, right?

Actually, ugh, no. What economists Raj Chetty and John N. Friedman of Harvard and Jonah Rockoff of Columbia want to do, apparently, is to identify and fire “weaker” teachers, for the sake of a barely perceptible increase in students’ “lifetime income.” Nobody has actually tried this yet; the report doesn’t describe an experiment. It’s just the conclusion they draw from their analysis of massive amounts of data gathered from public schools in New York City and cross-referenced against IRS records and the like.

Here’s a bit from the summary of the original paper. Note that a “high-VA” (“value-added”) teacher is a “good” one—meaning by this, solely, that the teacher in question has succeeded in raising standardized test scores.

Students assigned to high-VA teachers are more likely to attend college, attend higher- ranked colleges, earn higher salaries, live in higher SES neighborhoods, and save more for retirement. They are also less likely to have children as teenagers. Teachers have large impacts in all grades from 4 to 8. On average, a one standard deviation improvment [sic] in teacher VA in a single grade raises earnings by about 1% at age 28. Replacing a teacher whose VA is in the bottom 5% with an average teacher would increase students’ lifetime income by more than $250,000 for the average classroom in our sample. We conclude that good teachers create substantial economic value and that test score impacts are helpful in identifying such teachers. (© 2011 by Raj Chetty, John N. Friedman, and Jonah E. Rockoff. All rights reserved.)

Great! So how much more likely are these students to attend college, according to this paper? One-half of one percent more likely. And check out the forked-tongued flim-flamming of the alleged $250,000-for-the-average-classroom increase in “lifetime income.” If there are 25 kids in this average classroom, that means about ten grand per kid, per lifetime. Say they work for forty years: that comes to $250/year.*

The aggregate figure is estimated and rejoiced over, but the teacher in question is an individual, not an abstract quantity. This is not enough money to be crowing about in the newspaper; still less is it enough to be contemplating firing anybody over. That the authors would deliberately spin the number in this way seems like headline-baiting.

The authors might argue that they are aiming to influence policy by focusing on the accrued financial benefits to millions of students over time, rather than suggesting that each individual student might make a little more money in “x” or “y” circumstances. But analyzing past data doesn’t necessarily mean you can isolate and amplify whatever bit of the results you like. And the big question, is this even a legitimate method of guiding education policy? Monetizing not eyeballs, but minds?

Here’s what they told Annie Lowrey at The Times:

The authors argue that school districts should use value-added measures in evaluations, and to remove the lowest performers, despite the disruption and uncertainty involved.

“The message is to fire people sooner rather than later,” Professor Friedman said.

Professor Chetty acknowledged, “Of course there are going to be mistakes—teachers who get fired who do not deserve to get fired.” But he said that using value-added scores would lead to fewer mistakes, not more.

That is easy for Professor Chetty to say. He’s 32 years old, and some kind of a wunderkind who is apparently very comfortable dictating the fates of lesser souls. I can’t help but think that someone with just a little more imagination would have recommended instead, if these claims are true, that it might be a good idea to find out exactly how the “better” teachers achieve their results, and then teach those techniques to everybody else. But no! They want to “fire people sooner, rather than later.”

I would just love to know how Chetty, Friedman and Rockoff would like it if economists, too, had to be standardized-tested, and stood to lose their jobs every year if their recommendations didn’t result in a measurable increase in GDP, or if their predictions were off by half a percent. Would we have even one economist left? Keynes maybe, oh whoops, he’s dead.

What is glaringly obvious to those of us who’ve actually spent some time in schools is that teachers in this country are already hamstrung by excessive testing requirements and all the rest of the crazy demands of a Kafkaesque bureaucracy that does our students far more harm than good. To these already ludicrous teaching conditions we are meant now to add a new burden, the fear that you’re going to get canned because you aren’t raising test scores year after year.

Plus which, where does Prof. Chetty propose to get all the “high-VA” teachers he’s going to need in order to replace the so-called “lowest performers”? Out of a vending machine?

Whatever, let’s finish poisoning the work environment for these people who are teaching our children, because obviously $250 bucks per year per kid adds up to a lot of sugar, if you’re talking about millions of kids, and of course there won’t be any unintended consequences to terrorizing teachers in this egregious fashion. Yes, and let us also send some more rabbits to Australia!

Whether even a large increase in income is a legitimate indicator of a successful life is a question that is going to have to wait for another day, because there is so much wrong with this study and the response to it that there isn’t a hope of getting to the real philosophical underpinnings of educating young people, the part we ought to be thinking and talking about.

But let’s get a few things out of the way, just for context. I’m an ordinary parent and I do not give .5% of a tinker’s curse whether my kid earns an extra $250 a year or not. I know many parents who feel as I do about this. What we want for our kids is for them to grow up to be adults who are happy, well-informed, engaged citizens. Love their work and be good at it. Have a lot of pleasure and love in their lives, eyes open, good friends and good companions. Responsible, kind, reliable, open-minded people. Good parents themselves, maybe, someday. If anyone is going to be envisioning education reforms, then let him reckon with these principles first.

It is a real disgrace that this NBER study should have been quoted so widely and so approvingly, particularly when we already have some rock-solid indications of what should be done instead.

Which brings us to Finland.

***

The Finns didn’t always have an educational system that is the envy of the world. For the fourth time in a row, though, they posted sky-high scores on the latest PISA tests that rank 15 year olds’ reading, math and science literacy by country. (The tests began in 2000, and are held every three years.) So how do they do it?

We trust our teachers,” says Reijo Laukkanen, for over three decades a member of Finland’s National Board of Education. Only imagine the likes of Arne Duncan, Margaret Spellings, Bill Gates, Michelle Rhee or any other American crusader for School Reform ever saying such a thing.

The Finnish reforms leading to these results were developed in the 1950s and 60s, put into place starting around 1971 and concluded in 1984. Before that, the Finns had a relatively heterogenous school system (though by 1907 they had already developed “an educational autonomy not to be found in the Baltic States or in Poland,” as David E. Whittaker put it in a 1983 article for Comparative Education [via JSTOR]). This is important because, though Finland has only been formally independent since 1917, the Finns have long viewed education as a “tool for sustaining national identity, basic literacy, and essential political freedom.” That is to say, there is a distinctive pride and unity in the Finnish approach to educating their kids, kind of in opposition to the oppression they’d faced in the form of Swedish and Czarist colonization. The key feature of the Finnish educational reforms was therefore to equalize and maximize opportunity for all Finnish children, regardless of class, race or other factors.

The Basic School Act of 1970 instituted a universal nine-year common school, peruskoulu, to unify all education between the ages of 7 and 16. Beyond that, Whittaker notes that an effort was made “to stress the equivalence of vocational education, so that it should not be regarded as a second-best alternative to continued (academically superior) general education.”

There are no private schools in Finland. No for-profit schools, no charter schools, no vouchers, no “school choice” at all. Everyone gets the same compulsory education. Also, they spend less per kid per year, on average, than we do in the U.S. There is the barest minimum of standardized testing in Finland, either of individual students, or of schools, of principals or of teachers. Linda Darling-Hammond compared the two systems in the NEA newsletter in November 2010:

The process of change has been almost the reverse of policies in the United States. Over the past 40 years, Finland has shifted from a highly centralized system emphasizing external testing to a more localized system in which highly trained teachers design curriculum around the very lean national standards. This new system is implemented through equitable funding and extensive preparation for all teachers. The logic of the system is that investments in the capacity of local teachers and schools to meet the needs of all students, coupled with thoughtful guidance about goals, can unleash the benefits of local creativity in the cause of common, equitable outcomes.

An excellent précis of the difficulties faced by Americans attempting to understand the Finnish system appeared last month in The Atlantic. It was written by Anu Partanen, a Finnish journalist.

Since the 1980s, the main driver of Finnish education policy has been the idea that every child should have exactly the same opportunity to learn, regardless of family background, income, or geographic location. Education has been seen first and foremost not as a way to produce star performers, but as an instrument to even out social inequality. [...] In fact, since academic excellence wasn’t a particular priority on the Finnish to-do list, when Finland’s students scored so high on the first PISA survey in 2001, many Finns thought the results must be a mistake. But subsequent PISA tests confirmed that Finland — unlike, say, very similar countries such as Norway — was producing academic excellence through its particular policy focus on equity.

That this point is almost always ignored or brushed aside in the U.S. seems especially poignant at the moment, after the financial crisis and Occupy Wall Street movement have brought the problems of inequality in America into such sharp focus. The chasm between those who can afford $35,000 in tuition per child per year — or even just the price of a house in a good public school district — and the other “99 percent” is painfully plain to see.

Stunningly, the comments to this piece are bristling with what amounts to white-supremacist nonsense, suggesting that there is Something (White) About the Finns that other, more diverse nations just can’t compete with. This assertion flies in the face of facts as stated in the article itself and elsewhere. (For example, Norway, which is demographically very similar to Finland, but which uses American-style standardized testing methods, ranks near the U.S.’s mediocre level in the PISA results.)

Any Americans who wish to maintain their white-supremacist fantasies should have a look at the consistently excellent results produced by American military schools, which outperform public school results year after year. Military schools are exempt from the requirements of No Child Left Behind, you may be surprised to hear. “Even more impressive, the achievement gap between black and white students continues to be much smaller at military base schools and is shrinking faster than at public schools,” wrote Michael Winerip in The Times last month.

Here are things that Finnish schools and U.S. military schools have in common:

• No standardized testing of kids
• Egalitarian treatment of kids
• Teachers have autonomy and are trusted
• No standardized testing for teachers, principals or schools
• Smaller class sizes
• Smooth relations between teachers and administrators

***

The National Bureau of Economic Research, or NBER, is famous for letting us know when recessions begin and end. That is a government gig, but the NBER is not a government agency. It’s a private nonprofit, funded by grants and government contracts. There is a summary of their 2010 numbers online.

Though the NBER claims to be a nonpartisan group, its president for thirty years, until 2008, was the hugely eminent Martin S. Feldstein, a hardcore supply-side Republican who served both the Reagan administration and that of Bush II (the latter scarcely noted for its ideological diversity). Private foundations funding the NBER over the years have included the Olin Foundation, the Bradley Foundation and the Sarah Scaife Foundation.

In 2002, David Leonhardt wrote a not-entirely-flattering profile of Feldstein in The Times:

Mr. Feldstein has shown little taste since the 1980’s for straying from the Republican Party line. In 1992, he predicted that the Clinton administration’s tax increase would stifle economic growth and do little to erase the deficit. An article he wrote for The Wall Street Journal in 2000 was headlined ”Bush’s Tax Plan Is Even Better Than the Campaign Says.” (Mr. Feldstein helped create the plan.)

For all his apparent qualification as a Low Value Added Economist, Martin Feldstein is very highly regarded among fellow-economists and politicians. During his watch, the NBER always hired “the best and the brightest” young economists from all sides of the political spectrum. Paul Krugman and Steven Levitt both benefited from NBER’s largesse. In the Times profile, Leonhardt quoted Robert Reich, labor secretary under Clinton and a famously self-avowed liberal, describing Feldstein: ”He’s very supportive of people who disagree with him ideologically, of people who are to the left of him. Basically, he’s an honest intellectual.”

There’s no particular reason to doubt this, but it’s striking that so many economists still skew so far to the right in their public pronouncements, despite whatever avalanche of facts may be found to contradict their positions. How can there be a supply-side economist left, for example, when after thirty long years nothing whatsoever has “trickled down”? When even Henry Blodget has finally figured out that there is also a Demand Side, surely, the writing is on the wall.

The results don’t seem to count, they haven’t ever counted; these people don’t know how to change course. The reason for this may have something to do with politics, and still more to do with being the Best and Brightest.

The authors of this education study, the “best” young economists hired by the country’s premier economic think tank, are not the kind of people who ever got a “B” in school. Such people can hardly help coming to think of themselves as superior to the common man. They’ve spent their whole lives proving that they are not “ordinary”—which, if the rest of us had any sense at all, should utterly disqualify them from influencing policy for ordinary people. I’m not being flippant; this is a specific problem.

These guys have swallowed completely the axiom among Business Administrators that “if you can’t measure it, you can’t manage it.” Naturally, since they are focused entirely on economic activity and climbing to the top of the slippery pole, they don’t even pause to wonder whether higher test scores or a higher income might not be the be-all-and-end-all of success in education or in life.

This tendency is not just a deficiency of logic, or even of principle; it’s a deficiency of character. Somewhere in the course of establishing their “brilliant” careers, many eminent people seem to lose sight of their essential humanity. They stop being able to see the human story as a single narrative, of which they themselves are a tiny part. It’s the same deficiency of character that leads, for example, the rich to imagine that wealth inequality in this country isn’t a problem worth addressing.

In so many of our institutions, those in charge are not making decisions based on the idea that those decisions will affect people exactly like themselves. Their decisions, on the contrary, are sent down from on high, to affect people on a distant and distinctly lower plane. What is “good enough” for these lesser creatures is not at all necessarily “good enough” for those who reside in the rarefied air above. So private schools, business class and barfing at Per Se for some; for-profit universities, television and McDonald’s for the rest.

Because we lesser ones aren’t fit to decide for ourselves, goes this reasoning, we need someone from Harvard or Kleiner Perkins or Goldman, Sachs or Gibson, Dunn or the NBER to sort us out. There are various problems with this model but the principal one is that it has proved ruinously false often enough that we should discard the model altogether. Still more problematic is the fact that such people so often appear to be lacking in the most basic humanity or imagination, embalmed as they are in the idea that they are the Special Ones.

***

My younger daughter’s preternaturally gifted third-grade teacher, Amanda, once told me a wise thing that I never forgot. The fourth grade had brought a colossal letdown for me in the form of Dave, a very sweet, spanking-new teacher whose command of the fourth-grade curriculum was alarmingly shaky. One day I went to Amanda’s room after some unbelievable episode or other and whined and complained and moaned about Dave.

“Not everyone is good at their job,” Amanda replied, in such a gentle, appealing way, causing me to gape like a goldfish. This was an attempt to counsel me to accept, to try to take up the slack myself, to understand that the cookie can’t be made of all chocolate chips. Maybe it would be a better idea to help Dave and stop screaming? This turned out to be the case, in fact.

It isn’t healthy what we are doing to kids, smashing their curiosity and sense of play. Making everything about Achievement with a capital A. By high school they’re often facing four or six or more APs, SAT prep classes, plus sports, music, church, Boy Scouts, whatever. They don’t put this kind of pressure on kids in Finland, or even in American military schools, where they seem to understand that you take your last SAT in this world around age seventeen, after which point life begins to arrange itself along other lines entirely.

But what we’re doing to teachers is far worse. There are real, longstanding problems with identifying and removing really terrible teachers, but anyone who works in education can tell you that none of this is as simple as it looks. A teacher friend decocted the issues for me perfectly.

Unions tend to resist merit pay and firing based on student scores, and some of that is sheer protectionism, but there’s more to it, as everyone knows there is a big political dimension beyond the numbers. Principal evaluation, peer evaluation, student evaluation, these are all about feelings. But even when a numerical measure is involved, if your supervisor likes you, you don’t get the problem class, you get the resources you need, etc. The system is manipulated to keep the ones that are favored, and lean on the ones who aren’t. But too often known bad ones are tolerated because it’s too much of a pain to replace them and there’s no guarantee that the next ones will be any better.

Teachers now have zero time to think about how to encourage a specific kid because they are laden down with a crippling amount of bureaucratic claptrap and test preparation. They can’t get to know their kids because they have to conform to a regimented nonsense make-work politically-motivated schedule every second. There are crazy parents to attend to, staff meetings, testing, testing, testing. Somewhere in there are lesson plans to develop and work to grade. There’s not enough money for anything whatsoever because of budget cuts. They have to worry about every syllable that comes out of their mouths in case some fool goes all haywire over their views on politics or whatever. The stuff you see in even a really good public school would curl your hair, seriously.

When we should be giving them every support, the best training, great salaries, all the honor and gratitude we have, and a teacher’s lounge even a little more like the teachers’ lounge at Vaajakumpu, Finland.


* Update—a clarification from Prof. Raj Chetty:

I just wanted to clarify one arithmetic error in your note, for the record. You report that the gain per student from a better teacher would be $250 per year.

This is a basic arithmetic error due to not adjusting for discounting. We discount all gains back to age 12 at a 5% interest rate in order to put everything in today’s dollars, which is standard practice in economics. Your calculation requires the undiscounted gain (i.e. summing the cumulative earnings impact), which is $50,000 per student for a 1 SD better teacher (84th pctile vs 50th pctile) in one grade. Discounted back to age 12 at a 5% interest rate, $50K is equivalent to about $9K. $50,000 over a lifetime is itself not a very large amount, but it would be implausible that a single teacher in one grade could do more than that. So the magnitudes strike me as reasonable yet important. It sounds like many readers make this discounting mistake, so it might be helpful to correct your calculation so that your readers have the facts right (the paper also provides these calculations if you’d like to read it).



Maria Bustillos is the author of Dorkismo and Act Like a Gentleman, Think Like a Woman.

Photo of “Classroom with Three Figures” by Lavern Kelley; photo taken by cliff1066™, via Flickr.