Thursday, January 12th, 2012

The Evil Economics Of Judging Teachers

The Times and a host of other publications heralded last week's new study extolling the lifelong money-earning benefits of having a good primary/middle-school teacher. Oh, yay! Let's do what these economists from the National Bureau of Economic Research suggest, right?

Actually, ugh, no. What economists Raj Chetty and John N. Friedman of Harvard and Jonah Rockoff of Columbia want to do, apparently, is to identify and fire "weaker" teachers, for the sake of a barely perceptible increase in students' "lifetime income." Nobody has actually tried this yet; the report doesn't describe an experiment. It's just the conclusion they draw from their analysis of massive amounts of data gathered from public schools in New York City and cross-referenced against IRS records and the like.

Here's a bit from the summary of the original paper. Note that a "high-VA" ("value-added") teacher is a "good" one—meaning by this, solely, that the teacher in question has succeeded in raising standardized test scores.

Students assigned to high-VA teachers are more likely to attend college, attend higher- ranked colleges, earn higher salaries, live in higher SES neighborhoods, and save more for retirement. They are also less likely to have children as teenagers. Teachers have large impacts in all grades from 4 to 8. On average, a one standard deviation improvment [sic] in teacher VA in a single grade raises earnings by about 1% at age 28. Replacing a teacher whose VA is in the bottom 5% with an average teacher would increase students’ lifetime income by more than $250,000 for the average classroom in our sample. We conclude that good teachers create substantial economic value and that test score impacts are helpful in identifying such teachers. (© 2011 by Raj Chetty, John N. Friedman, and Jonah E. Rockoff. All rights reserved.)

Great! So how much more likely are these students to attend college, according to this paper? One-half of one percent more likely. And check out the forked-tongued flim-flamming of the alleged $250,000-for-the-average-classroom increase in "lifetime income." If there are 25 kids in this average classroom, that means about ten grand per kid, per lifetime. Say they work for forty years: that comes to $250/year.*

The aggregate figure is estimated and rejoiced over, but the teacher in question is an individual, not an abstract quantity. This is not enough money to be crowing about in the newspaper; still less is it enough to be contemplating firing anybody over. That the authors would deliberately spin the number in this way seems like headline-baiting.

The authors might argue that they are aiming to influence policy by focusing on the accrued financial benefits to millions of students over time, rather than suggesting that each individual student might make a little more money in "x" or "y" circumstances. But analyzing past data doesn't necessarily mean you can isolate and amplify whatever bit of the results you like. And the big question, is this even a legitimate method of guiding education policy? Monetizing not eyeballs, but minds?

Here's what they told Annie Lowrey at The Times:

The authors argue that school districts should use value-added measures in evaluations, and to remove the lowest performers, despite the disruption and uncertainty involved.

"The message is to fire people sooner rather than later," Professor Friedman said.

Professor Chetty acknowledged, "Of course there are going to be mistakes—teachers who get fired who do not deserve to get fired." But he said that using value-added scores would lead to fewer mistakes, not more.

That is easy for Professor Chetty to say. He's 32 years old, and some kind of a wunderkind who is apparently very comfortable dictating the fates of lesser souls. I can't help but think that someone with just a little more imagination would have recommended instead, if these claims are true, that it might be a good idea to find out exactly how the "better" teachers achieve their results, and then teach those techniques to everybody else. But no! They want to "fire people sooner, rather than later."

I would just love to know how Chetty, Friedman and Rockoff would like it if economists, too, had to be standardized-tested, and stood to lose their jobs every year if their recommendations didn't result in a measurable increase in GDP, or if their predictions were off by half a percent. Would we have even one economist left? Keynes maybe, oh whoops, he's dead.

What is glaringly obvious to those of us who've actually spent some time in schools is that teachers in this country are already hamstrung by excessive testing requirements and all the rest of the crazy demands of a Kafkaesque bureaucracy that does our students far more harm than good. To these already ludicrous teaching conditions we are meant now to add a new burden, the fear that you're going to get canned because you aren't raising test scores year after year.

Plus which, where does Prof. Chetty propose to get all the "high-VA" teachers he's going to need in order to replace the so-called "lowest performers"? Out of a vending machine?

Whatever, let's finish poisoning the work environment for these people who are teaching our children, because obviously $250 bucks per year per kid adds up to a lot of sugar, if you're talking about millions of kids, and of course there won't be any unintended consequences to terrorizing teachers in this egregious fashion. Yes, and let us also send some more rabbits to Australia!

Whether even a large increase in income is a legitimate indicator of a successful life is a question that is going to have to wait for another day, because there is so much wrong with this study and the response to it that there isn't a hope of getting to the real philosophical underpinnings of educating young people, the part we ought to be thinking and talking about.

But let's get a few things out of the way, just for context. I'm an ordinary parent and I do not give .5% of a tinker's curse whether my kid earns an extra $250 a year or not. I know many parents who feel as I do about this. What we want for our kids is for them to grow up to be adults who are happy, well-informed, engaged citizens. Love their work and be good at it. Have a lot of pleasure and love in their lives, eyes open, good friends and good companions. Responsible, kind, reliable, open-minded people. Good parents themselves, maybe, someday. If anyone is going to be envisioning education reforms, then let him reckon with these principles first.

It is a real disgrace that this NBER study should have been quoted so widely and so approvingly, particularly when we already have some rock-solid indications of what should be done instead.

Which brings us to Finland.


The Finns didn't always have an educational system that is the envy of the world. For the fourth time in a row, though, they posted sky-high scores on the latest PISA tests that rank 15 year olds' reading, math and science literacy by country. (The tests began in 2000, and are held every three years.) So how do they do it?

"We trust our teachers," says Reijo Laukkanen, for over three decades a member of Finland's National Board of Education. Only imagine the likes of Arne Duncan, Margaret Spellings, Bill Gates, Michelle Rhee or any other American crusader for School Reform ever saying such a thing.

The Finnish reforms leading to these results were developed in the 1950s and 60s, put into place starting around 1971 and concluded in 1984. Before that, the Finns had a relatively heterogenous school system (though by 1907 they had already developed "an educational autonomy not to be found in the Baltic States or in Poland," as David E. Whittaker put it in a 1983 article for Comparative Education [via JSTOR]). This is important because, though Finland has only been formally independent since 1917, the Finns have long viewed education as a "tool for sustaining national identity, basic literacy, and essential political freedom." That is to say, there is a distinctive pride and unity in the Finnish approach to educating their kids, kind of in opposition to the oppression they'd faced in the form of Swedish and Czarist colonization. The key feature of the Finnish educational reforms was therefore to equalize and maximize opportunity for all Finnish children, regardless of class, race or other factors.

The Basic School Act of 1970 instituted a universal nine-year common school, peruskoulu, to unify all education between the ages of 7 and 16. Beyond that, Whittaker notes that an effort was made "to stress the equivalence of vocational education, so that it should not be regarded as a second-best alternative to continued (academically superior) general education."

There are no private schools in Finland. No for-profit schools, no charter schools, no vouchers, no "school choice" at all. Everyone gets the same compulsory education. Also, they spend less per kid per year, on average, than we do in the U.S. There is the barest minimum of standardized testing in Finland, either of individual students, or of schools, of principals or of teachers. Linda Darling-Hammond compared the two systems in the NEA newsletter in November 2010:

The process of change has been almost the reverse of policies in the United States. Over the past 40 years, Finland has shifted from a highly centralized system emphasizing external testing to a more localized system in which highly trained teachers design curriculum around the very lean national standards. This new system is implemented through equitable funding and extensive preparation for all teachers. The logic of the system is that investments in the capacity of local teachers and schools to meet the needs of all students, coupled with thoughtful guidance about goals, can unleash the benefits of local creativity in the cause of common, equitable outcomes.

An excellent précis of the difficulties faced by Americans attempting to understand the Finnish system appeared last month in The Atlantic. It was written by Anu Partanen, a Finnish journalist.

Since the 1980s, the main driver of Finnish education policy has been the idea that every child should have exactly the same opportunity to learn, regardless of family background, income, or geographic location. Education has been seen first and foremost not as a way to produce star performers, but as an instrument to even out social inequality. [...] In fact, since academic excellence wasn't a particular priority on the Finnish to-do list, when Finland's students scored so high on the first PISA survey in 2001, many Finns thought the results must be a mistake. But subsequent PISA tests confirmed that Finland — unlike, say, very similar countries such as Norway — was producing academic excellence through its particular policy focus on equity.

That this point is almost always ignored or brushed aside in the U.S. seems especially poignant at the moment, after the financial crisis and Occupy Wall Street movement have brought the problems of inequality in America into such sharp focus. The chasm between those who can afford $35,000 in tuition per child per year — or even just the price of a house in a good public school district — and the other "99 percent" is painfully plain to see.

Stunningly, the comments to this piece are bristling with what amounts to white-supremacist nonsense, suggesting that there is Something (White) About the Finns that other, more diverse nations just can't compete with. This assertion flies in the face of facts as stated in the article itself and elsewhere. (For example, Norway, which is demographically very similar to Finland, but which uses American-style standardized testing methods, ranks near the U.S.'s mediocre level in the PISA results.)

Any Americans who wish to maintain their white-supremacist fantasies should have a look at the consistently excellent results produced by American military schools, which outperform public school results year after year. Military schools are exempt from the requirements of No Child Left Behind, you may be surprised to hear. "Even more impressive, the achievement gap between black and white students continues to be much smaller at military base schools and is shrinking faster than at public schools," wrote Michael Winerip in The Times last month.

Here are things that Finnish schools and U.S. military schools have in common:

• No standardized testing of kids
• Egalitarian treatment of kids
• Teachers have autonomy and are trusted
• No standardized testing for teachers, principals or schools
• Smaller class sizes
• Smooth relations between teachers and administrators


The National Bureau of Economic Research, or NBER, is famous for letting us know when recessions begin and end. That is a government gig, but the NBER is not a government agency. It's a private nonprofit, funded by grants and government contracts. There is a summary of their 2010 numbers online.

Though the NBER claims to be a nonpartisan group, its president for thirty years, until 2008, was the hugely eminent Martin S. Feldstein, a hardcore supply-side Republican who served both the Reagan administration and that of Bush II (the latter scarcely noted for its ideological diversity). Private foundations funding the NBER over the years have included the Olin Foundation, the Bradley Foundation and the Sarah Scaife Foundation.

In 2002, David Leonhardt wrote a not-entirely-flattering profile of Feldstein in The Times:

Mr. Feldstein has shown little taste since the 1980's for straying from the Republican Party line. In 1992, he predicted that the Clinton administration's tax increase would stifle economic growth and do little to erase the deficit. An article he wrote for The Wall Street Journal in 2000 was headlined "Bush's Tax Plan Is Even Better Than the Campaign Says." (Mr. Feldstein helped create the plan.)

For all his apparent qualification as a Low Value Added Economist, Martin Feldstein is very highly regarded among fellow-economists and politicians. During his watch, the NBER always hired "the best and the brightest" young economists from all sides of the political spectrum. Paul Krugman and Steven Levitt both benefited from NBER's largesse. In the Times profile, Leonhardt quoted Robert Reich, labor secretary under Clinton and a famously self-avowed liberal, describing Feldstein: "He's very supportive of people who disagree with him ideologically, of people who are to the left of him. Basically, he's an honest intellectual."

There's no particular reason to doubt this, but it's striking that so many economists still skew so far to the right in their public pronouncements, despite whatever avalanche of facts may be found to contradict their positions. How can there be a supply-side economist left, for example, when after thirty long years nothing whatsoever has "trickled down"? When even Henry Blodget has finally figured out that there is also a Demand Side, surely, the writing is on the wall.

The results don't seem to count, they haven't ever counted; these people don't know how to change course. The reason for this may have something to do with politics, and still more to do with being the Best and Brightest.

The authors of this education study, the "best" young economists hired by the country's premier economic think tank, are not the kind of people who ever got a "B" in school. Such people can hardly help coming to think of themselves as superior to the common man. They've spent their whole lives proving that they are not "ordinary"—which, if the rest of us had any sense at all, should utterly disqualify them from influencing policy for ordinary people. I'm not being flippant; this is a specific problem.

These guys have swallowed completely the axiom among Business Administrators that "if you can't measure it, you can't manage it." Naturally, since they are focused entirely on economic activity and climbing to the top of the slippery pole, they don't even pause to wonder whether higher test scores or a higher income might not be the be-all-and-end-all of success in education or in life.

This tendency is not just a deficiency of logic, or even of principle; it's a deficiency of character. Somewhere in the course of establishing their "brilliant" careers, many eminent people seem to lose sight of their essential humanity. They stop being able to see the human story as a single narrative, of which they themselves are a tiny part. It's the same deficiency of character that leads, for example, the rich to imagine that wealth inequality in this country isn't a problem worth addressing.

In so many of our institutions, those in charge are not making decisions based on the idea that those decisions will affect people exactly like themselves. Their decisions, on the contrary, are sent down from on high, to affect people on a distant and distinctly lower plane. What is "good enough" for these lesser creatures is not at all necessarily "good enough" for those who reside in the rarefied air above. So private schools, business class and barfing at Per Se for some; for-profit universities, television and McDonald's for the rest.

Because we lesser ones aren't fit to decide for ourselves, goes this reasoning, we need someone from Harvard or Kleiner Perkins or Goldman, Sachs or Gibson, Dunn or the NBER to sort us out. There are various problems with this model but the principal one is that it has proved ruinously false often enough that we should discard the model altogether. Still more problematic is the fact that such people so often appear to be lacking in the most basic humanity or imagination, embalmed as they are in the idea that they are the Special Ones.


My younger daughter's preternaturally gifted third-grade teacher, Amanda, once told me a wise thing that I never forgot. The fourth grade had brought a colossal letdown for me in the form of Dave, a very sweet, spanking-new teacher whose command of the fourth-grade curriculum was alarmingly shaky. One day I went to Amanda's room after some unbelievable episode or other and whined and complained and moaned about Dave.

"Not everyone is good at their job," Amanda replied, in such a gentle, appealing way, causing me to gape like a goldfish. This was an attempt to counsel me to accept, to try to take up the slack myself, to understand that the cookie can't be made of all chocolate chips. Maybe it would be a better idea to help Dave and stop screaming? This turned out to be the case, in fact.

It isn't healthy what we are doing to kids, smashing their curiosity and sense of play. Making everything about Achievement with a capital A. By high school they're often facing four or six or more APs, SAT prep classes, plus sports, music, church, Boy Scouts, whatever. They don't put this kind of pressure on kids in Finland, or even in American military schools, where they seem to understand that you take your last SAT in this world around age seventeen, after which point life begins to arrange itself along other lines entirely.

But what we're doing to teachers is far worse. There are real, longstanding problems with identifying and removing really terrible teachers, but anyone who works in education can tell you that none of this is as simple as it looks. A teacher friend decocted the issues for me perfectly.

Unions tend to resist merit pay and firing based on student scores, and some of that is sheer protectionism, but there's more to it, as everyone knows there is a big political dimension beyond the numbers. Principal evaluation, peer evaluation, student evaluation, these are all about feelings. But even when a numerical measure is involved, if your supervisor likes you, you don't get the problem class, you get the resources you need, etc. The system is manipulated to keep the ones that are favored, and lean on the ones who aren't. But too often known bad ones are tolerated because it's too much of a pain to replace them and there's no guarantee that the next ones will be any better.

Teachers now have zero time to think about how to encourage a specific kid because they are laden down with a crippling amount of bureaucratic claptrap and test preparation. They can't get to know their kids because they have to conform to a regimented nonsense make-work politically-motivated schedule every second. There are crazy parents to attend to, staff meetings, testing, testing, testing. Somewhere in there are lesson plans to develop and work to grade. There's not enough money for anything whatsoever because of budget cuts. They have to worry about every syllable that comes out of their mouths in case some fool goes all haywire over their views on politics or whatever. The stuff you see in even a really good public school would curl your hair, seriously.

When we should be giving them every support, the best training, great salaries, all the honor and gratitude we have, and a teacher's lounge even a little more like the teachers' lounge at Vaajakumpu, Finland.

* Update—a clarification from Prof. Raj Chetty:

I just wanted to clarify one arithmetic error in your note, for the record. You report that the gain per student from a better teacher would be $250 per year.

This is a basic arithmetic error due to not adjusting for discounting. We discount all gains back to age 12 at a 5% interest rate in order to put everything in today's dollars, which is standard practice in economics. Your calculation requires the undiscounted gain (i.e. summing the cumulative earnings impact), which is $50,000 per student for a 1 SD better teacher (84th pctile vs 50th pctile) in one grade. Discounted back to age 12 at a 5% interest rate, $50K is equivalent to about $9K. $50,000 over a lifetime is itself not a very large amount, but it would be implausible that a single teacher in one grade could do more than that. So the magnitudes strike me as reasonable yet important. It sounds like many readers make this discounting mistake, so it might be helpful to correct your calculation so that your readers have the facts right (the paper also provides these calculations if you'd like to read it).

Maria Bustillos is the author of Dorkismo and Act Like a Gentleman, Think Like a Woman.

Photo of "Classroom with Three Figures" by Lavern Kelley; photo taken by cliff1066™, via Flickr.

45 Comments / Post A Comment

deepomega (#1,720)

Was wondering when your Finland tweets would turn into a piece! I'm super skeptical of comparisons btwn the US and Finland, though, for a few reasons. One, centralizing the education of a nation of 300 million is not the same as doing so to a nation smaller than LA. The LAUSD is larger than Finland's school system! And but then, probably more significantly, Finland is way more economically and culturally homogeneous. Maybe making each state a mini Finland might help, but I'm not sure we'd be happy with how much intelligent design happened.

SeanP (#4,058)

@deepomega I'm not sure what to make of the "cultural homogeneity" and size arguments, though. Surely Norway is of a similar size and just as homogeneous as Finland, but if Maria is correct, that didn't help them any.

Obviously there could be something else holding back Norway, but it seems clear that there's more to it than just cultural and size factors.

LL Smooth J (#174,779)

@deepomega wouldn't America's size and not-homogeneousness be an argument in favor of not applying the same standardized tests everywhere and letting teachers figure out what's best for their situation?

deepomega (#1,720)

@SeanP Finland and Norway have hugely different economies, though. I'd be interested to see how the oil economy of Norway translates to different socioeconomic breakdowns. And really, I'd say that extrapolating from Finland is just as silly as extrapolating from Norway.

@LL This is interesting, because in finland there is a very strong centralized structure, just one that doesn't like standardized tests. But note that schools are required to have masters degree educated teachers, which means fewer teacher candidates and more cost per… maybe getting away from standardized tests has to be paid for with more expensive teachers? Which doesn't explain why Finland spends less per student.

SeanP (#4,058)

@boyofdestiny Holy crap. I'm glad I'm not a student in New Hampshire right now. That sounds like a recipe for total chaos.

turd_sandwich (#5,660)

@deepomega My reading is that Finland has strong, centralized standards, but that they are some absurdly small fraction of the length of U.S. Dept of Ed (ED) and individual state standards. Finland's standards don't focus on tiny minutiae of curricula and, instead, use broader concepts about skills and reasoning abilities as achievement benchmarks. Maybe I'm wrong, but that's what I took from a lecture by Darling-Hammond.

If ED published standards like that, I think that there would be a lot more flexibility for state and local departments to innovate and create programs that make the most sense from a regional perspective.

DoctorDisaster (#1,970)

@deepomega As LL points out, your argument seems to line up with the ideas in the article more than it contradicts them. Rather than No-Childing single standardized tests for the entire country and basing all our textbooks on the Texas curriculum, states (and even overpopulated hellholes like LA) should get more leeway to figure this stuff out on their own. Break it up into Finland-size chunks, let them all try whatever crazy stuff they want, and eventually you'll be pointing to, say, Michigan the way we point to Finland now (and with fewer cultural differences to worry about). Instead, the power of the purse has been abused to enforce some really cockamamie 'reforms' at the federal level.

And, please: the "homogenous" line is dumb, dumb, dumb, don't go down that path. Cherry-picking a single quality of the winner and deciding that's how they won is uncut Gingrich-grade analysis failure. What the Norway counter-example proves is that if you really want to prove that hypothesis, you need to take all the countries competing in this PISA thingy, rank them by level of homogeneity on the x-axis, and graph their scores. If you want to get really fancy, find a line of best fit so that you'll be able to see exactly how much of Finland's success is thanks to Heimdall being played by a white actor and, more importantly, how much you've yet to account for.

deepomega (#1,720)

@DoctorDisaster So talking about getting out of education at the federal level is… not that different from republicans arguing that we should abolish the Department of Education. A lot of the cost would be on liberal causes in education – sex ed, gays in history, etc. (And i know how graphing and cultivable analysis works, bro. Relax.)

DoctorDisaster (#1,970)

@deepomega I didn't mean to pick on you, I just get annoyed with the general level of confirmation bias you run into when discussing these things. There are way too many people who react to ANY news out of Scandinavia with "I warned you about brown people/socialism, bro!" The entire region is like a crystal ball in which some people can see nothing but PROOOOOF of their most odious opinions. That's not you, but you said a thing that sounded like them, and it set me off.

DoctorDisaster (#1,970)

To answer your point though, I'm not saying "get out of education." Obviously I would rather have federal dollars spent on teachers than useless shit like invasions. What I'm saying is that we should be way more selective about the strings we put on that money. You can say, "Hey, teach kids about this thing and we'll give you some more money" and leave lots of leeway for experimentation. "Your school will go bankrupt if your kids don't do well on this test" has no wiggle room.

And at the end of the day, I'm a lot less worried about "liberal causes" than finding what works. I'm in favor of comprehensive sex ed because it has a powerful positive effect on teen pregnancy and STI rates. Hope I'm not offending anybody, but I genuinely don't think "gays in history" has any business in federal legislation. I think any history teacher who wants to cover that should have the right to do so, but to prohibit or mandate specific topics like that at a national level is ludicrous.

Morbo (#1,288)

One thing not mentioned here is to be a teacher in Finland, you need to hold a master's degree, at the minimum. I would wager that if we made that a qualification in America (and hence, had to compensate them for that level of education), the educational achievement profile of this country would change.

Class sizes are smaller in Finland, the school day is longer, and they have more recess. It takes a good number of dedicated and qualified individuals to make this happen.

However, in America, we continue to under-dedicate resources to public education, and then inefficently allocate those resources (what economists are concerned about, and express in a ham-fisted manner with this study).

Let us not also look at the shortcomings of the Finnish system – they are over-educated, have a bubble in higher education (in the future, some colleges will have to consolidate/close), and their ability to focus on this generation of kids comes from the fact that their population pyramid is currently inverted, and can concentrate their resources of their (relatively) precious snowflakes.

"There are no private schools in Finland." SYK is still technically a private school, and has competitive entrance exams to get in.

deepomega (#1,720)

And when we start talking like a lot of contradictions pop up. How do we require a masters for teachers without jacking up per student costs even higher?

Morbo (#1,288)

@deepomega Very true…..especially since in the US, individuals pay for college, as opposed to Finland, where there is little cost to the student.

Once again, if we decided as a nation to devote more resources to education, some of these problems would alleviate, but not all of them.

Comparing our country to Finland isn't going to do much good until we realize they are the Brentwood of the developed world, and the US is more like Anaheim.

boyofdestiny (#1,243)

@Morbo Can someone translate this analogy for an East Coast audience?

Morbo (#1,288)


Brentwood -Finland- Relatively homogenous population, tighter income distribution, very high standard of living, almost universal emphasis on education. Kids come to school prepared to learn.

Anaheim – US- More economically diverse, dominated politically by a right wing that doesn't like to acknowledge needs of others, dealing with different groups that have different attitudes about education and where the responsibilities lie.

iantenna (#5,160)

@deepomega we could start by not spending a bazillion dollars on private teaching consultants whose value to teachers and, in turn, students, families, and the general public is highly suspect.

grape (#206,042)

@deepomega Consider just how much $ is being spent on designing tests, materials for test prep, grading tests, etc. How about redirecting that money toward teacher education (both in terms of upping requirements for teachers and ongoing teacher education through professional development, inquiries, etc.)

boyofdestiny (#1,243)

I know every column on education reform can't be simultaneously about every problem with our schools today, but I'll play the role of the guy reminding folks that there are a litany of non-school factors that trump teacher quality when it comes to student performance. Which is not to say that we shouldn't talk about teacher quality; just that great teachers aren't a magic pill that wipe away the effects of poverty, hunger, homelessness, poor health, disengaged parents, etc etc etc.

turd_sandwich (#5,660)

@boyofdestiny second, third, and fourth this. when i read the first long quote from the original article, i immediately wanted to know how many other variables these guys looked at in relationship to income later in life. it would shock the shit out of me if high-VA teachers were more strongly associated with income that many of the items on your list.

iantenna (#5,160)

yeah, this. i mean, does this study make any attempt to account for the other factors involved before concluding it must be the teachers fault?

Morbo (#1,288)

@iantenna From Page 60 of the Harvard paper:

"Predicted score is based on the fitted values from a regression of test score on mother’s age at child’s birth,
indicators for parent’s 401(k) contributions and home ownership, and an indicator for the parent’s marital status interacted
with a quartic in parent’s household income (see Section 4.3 for details). All three figures control for the following
classroom-level variables: school year and grade dummies, class-type indicators (honors, remedial), class size, and cubics
in class and school-grade means of lagged test scores in math and English each interacted with grade. They also control for
class and school-year means of the following student characteristics: ethnicity, gender, age, lagged suspensions, lagged
absences, and indicators for grade repetition, special education, limited English."

iplaudius (#1,066)

The current and upcoming generations of economists think they can play with statistics and bypass discipline-specific knowledge and expertise.

It's offensive and stupid. Economists aren't statisticians, so for the most part they don't understand the mathematics that produces the figures they use in their analyses. They don't know anything about the disciplines they're criticizing, so they frame the questions crudely and draw inaccurate conclusions from their analyses.

Here's an economic argument: If you want better teachers, make the field more competitive and pay teachers more. A lot more. It's a simpler economic hypothesis and easier to support with argument and empirical data.

tastylatkes (#204,627)

Plus there's good evidence that reward/punishment systems actually make people perform worse, which may be a partial explanation for why our schools continue to tank with No Child Left Behind.

A Good Question (#182,018)

Bustillos says:
"I can't help but think that someone with just a little more imagination would have recommended instead, if these claims are true, that it might be a good idea to find out exactly how the "better" teachers achieve their results, and then teach those techniques to everybody else."

Reasons why this isn't being done:
1. Teachers are no longer considered professionals. They're the new servant class, and who cares what the servants think?
2. The reports and recommendations are coming out of boardroom culture. It's less important to get the right solution than for the CEO figure to come in and take charge, and for everyone else to fall in line.

C_Webb (#855)

@A Good Question Read the comment section to any Times piece on higher education, and you'll see that people are starting to regard college professors the same way. How long until ETS worms its slimy way into college classrooms?

C_Webb (#855)

@C_Webb BTW, I wonder why there has never been (to my knowledge, feel free to prove me wrong) any solid, in-depth investigative journalism about ETS. They have an unbelievable amount of power, and they're growing all the time.

maddieD (#9,798)

@C_Webb Good ole' Ralph Nader investigated the ETS in the late 70s, I believe. His report is floating around somewhere. And then Nicholas Lemann wrote an excellent book a few years back called The Big Test: The Secret History of the American Meritocracy. Well worth reading.

Tulletilsynet (#333)

I got a lot out of the first version of this book by David Owen from 1985 (I have not read the completely reworked & updated version), which was very effective at debunking standardized tests. Owen did look into what you mention, in particular the organizational biases of the ETS and about the ideological history of the doctrine that tests like the SAT or General GRE can measure something called "aptitude" apart from content. (Short answer: it was the GI Bill.)

I actually think standardized tests are just dandy for some purposes, but even 30 years ago, they were part of a bogus religion in need of debunking. And the most effective debunkers were not investigative journalists, but Stanley Kaplan and later on the Princeton Review people, who proved by the give-me-your-money-and-you'll-see method that the ETS was lying when it said you couldn't prep kids for a test of "pure aptitude."

omaha (#204,653)

I think this work addresses most of the concerns voiced above. The hope in using "statistics" to evaluate anything is that it gives you some objective criteria to help make decisions. The point of this paper was to find out if teacher VA is one such measure. For example, does it penalize teachers who are given a troubled class or a great class? They found that it does not. Test scores are not the "end all be all" so they looked at many other adult outcomes: income, chance of going to college, quality of college, quality of the neighborhood you live in, likelihood of having a 401(k) retirement plan, likelihood of being a teen mom.

lotsoftreble (#2,715)

The thing to remember and emphasize is that the effectiveness of a school (relative to whatever preparation the students might have) is the cumulative result of a whole bunch of decisions made by individual teachers. Teachers can act in isolation and out of fear. They can also act in a network of support and trust. Networks of support and trust come from real monetary investment in the people doing the work. Etc. Etc Etc.

deb quinn (#204,827)

Diane Ravitch once said "imagine that a town decides that crime rates have to go down and if the police FAIL to bring down crime rates, then they start firing the cops." It's her analogy for the teaching-test score insanity. So if test scores don't go up, let's not help the teachers, let's just fire them. Because THAT will absolutely help the students, right? Here's another analogy that highlights the short-sightedness that has become the hallmark of much US educational policy:

Nice piece, Maria.

As a first-generation immigrant whose parents depended highly on the direction and guidance of public schools, I often feel like I'm a victim of the misplaced teleology of the American public education system that you describe above. I still have to actively quell the achievement-for-the-sake-of-achievement-driven aspect of my mind that was instilled and sharpened by that system.

"They've spent their whole lives proving that they are not "ordinary"—which, if the rest of us had any sense at all, should utterly disqualify them from influencing policy for ordinary people."

I've just started reading The Power Broker, and this seems to sum up precisely what drove Robert Moses to make New York City into the 1%-driven city that it's now become. Bloomberg has achieved success mostly in improving upon Moses's foundations.

Ack, I'm a first-generation American, not immigrant.

Wow, your logic is stellar and I really appreciate the gonzo approach. Few teachers being targeted for dismissal are being dismissed for professional incompetence. On the contrary, most of the rubber room teacher's are honored veteran educators, whistle blowers, plaintiffs in civil cases against district, targets for reprisal by autocratic administrators. These people use students as pawns by overloading teachers with special Ed, EL and behavior problems. This often fails as do witch hunts that make students complicit in their crime less often than you'd think. Kids may hate a teacher but not enough to help the man ruin her. In these cases, the principal just makes stuff up to drive the teacher out. Bad evaluations–FRAUD–, discipline, ambushes, punitive scheduling. The admin want cronies to insulate them. A teacher who's mature, ethical and student centered cannot be tolerated in the emerging school culture.
With many baby boomers scheduled to retire , there is a window of opportunity that privatizing pirates are eager to exploit. It is clear they have no intention of improving public schools. Class size expands, teachers are abused and denied access to contractual concessions, due process! And TFA will fill their positions to have student loans forgiven before moving on to a more lucrative industry. Teacher's are going to be temps. No more answering the call: no more academic freedom, no more democracy. Thanks for this.

What the theorists do not take into account is the deplorable political situation in school districts. Cheating on testing, and attempted cheating, orchestrated by school district administrators, was definitely an issue in the past decade, due to pressures to perform and no independent supervision of testing, such as we see with SAT's. Teachers who played ball with these plans, or were intimidated into doing so, became favorites. These would be kept while the honest teachers were harassed and driven out. Who out there thinks that superintendents and principals are honest? Are you kidding? School districts in America are easily corrupted political systems, rampant with corruption and complete lack of independent accountability, run like a fifedom. Do you think that a teacher should avoid kids who underperform for fear of being "judged" when they don't excel? In our area, many of these were from families who didn't speak English at home or in the neighborhood. We want the best for all of them, but to punish teachers is just plain wrong. It invites corruption of the system and does not allow teachers to use the skills they do have. FYI, I reported my principal for cheating and the CA Dept of Education got together with my school district and they put HER (the cheater) in charge of the investigation. Go figure.

You comment "Norway, which is demographically very similar to Finland, but which uses American-style standardized testing methods, ranks near the U.S.'s mediocre level in the PISA results." suggests you are not very familiar with Norway. If you would like to know how it works (and how it could be improved) please take a look at OECD Reviews of Evaluation and Assessment in Education Report for Norway
Evaluation and assessment for students, teachers, schools and systems are complex processes that need to work together coherently to support the goal of improving outcomes for all students. To learn more about these and see what other countries do, see
By the way, in 2009, Finland's foreign born population was 4.4 per cent compared with 10.9 in Norway (and 12.7 for the US). This is just one of the many ways in which Finland and Norway are different.

Scum (#1,847)

I can think of a wide variety of problems in looking to Finland as a model for US policy. They're probably similar to the ones you'd come up with if I were to argue for the loosening of gun laws by highlighting the far lower incidence of gun crime in Finland as compared to the US.

Sassquatch (#205,676)

Curiously both South Korea and Finland have largely phonetic languages, it probably helps that children spell and read in a non-sadistic written language.

Read "Beware of Smoke and Mirrors: On Bloomberg and the State of the NYC's Education System, Part 1"
on ""

Galdes (#205,724)

Not mentioned in any of these articles is that in the last few years administrators and educators in New York, Maryland, Kentucky, Minnesota, California, Texas and Florida have been implementing new programs and many more districts nationwide are moving toward grade –reform programs especially in middle and high schools.
The questions they have been asking are; who deserves high marks?, what standards are judging students on?, and what goal are we trying to move students toward?
At this time all states now mandate that teach certain curriculum, and 45 have begun implementing a national list of education standards know as Common Core. Schools are realizing that if you don’t have standard-based grading, you can’t measure, or communicate, how well students are actually learning the material as opposed to merely being “exposed” to it. In this system, homework doesn’t count toward the main “knowledge” grade, which is determined primarily by scores on end-of-unit tests. A separate mark is given for work habits and more subjective measures, like attitude, effort and citizenship. The key being to separate the product (what the student learns) from the process (attitude, ability to function well in the classroom). Either of these skill sets might require help, but you can’t focus in on what needs work unless the two are evaluated separately. When Kentucky’s Oldham County school district started standards-based grading parents were concerned when then they saw their kids’ grades drop. And some of the students who were involved in athletics had to shift tactics to keep their grades up so they remained eligible to participate in sports. In the past they added an “extra-credit project” to shore up their poor grade, but under the new system they had to buckle down and study in order to do well on classroom tests. An added feature is that standards-based grading allows students to retake tests (same subject, different questions), the point being to master the material, not necessarily on the first try, because children not all children learn at the same rate, and the emphasis remains that they learn the material and that they are supported in that effort.
Districts that have adopted mastery grading say it has changed, for the better, the way teachers and parents talk about students. Being able to focus on specifics allows the parents, teachers to put their heads together and figure out how to help. A standards-based system also allows teachers to spot hidden talents that often go undiscovered, especially among those that do not have the social skills that some others have.
The first year Minnetonka, MN implemented mastery grading, students received 7% fewer A’s-what their administration viewed as a “market correction.” But in the subsequent years marks have been on an upward climb. In the last 5 years ACT subject test scores for Minnetonka have risen, as have the number of students who are named National Merit Scholars and the percentage of those who pursue the rigorous International Baccalaureate degree has increased. In 2009 147 students at Minnetonka HS received F’s, in two years that number was down to 54. The success of this change has school administrators from other districts and states calling to find out how they did it. They are all told the basis is grading the students on what they know, not on how well the “do school.”
There are many things that need to be address within the educational system in the US. Things that need to change and even be eliminated. After working within education, for an educational publisher for 25 years, I believe this approach can help in all aspects of education. First and foremost it focuses on children learning, and simultaneously provides a format to address the myriad problems in both learning and being successful in the classroom. It could be the foundation for changing education and educational standards in the US. Because of the size of our systems and the diversity found in them we have to develop answers that focus on ways for all students to learn to their real potential. Education is the foundation for life and well-being in this country. I may not be in the majority, but I believe that everyone deserves a good foundation.

Warm and verticle (#206,259)

"Governor Brewer’s education plan includes initiatives to:

Produce a searchable database so that every parent can research the license and any disciplinary actions taken against their children’s teachers, and reform the teacher decertification process."

I don't think I'm going into teaching.
First, where is my database to check up on the child or parents' history?
Secondly, after spending 4 plus years of life and extraordinary amounts of money to get ready to pass a States test for licensing, I'll be subject to being fired at whim… maybe because I don't drink after school with the principal.
Third, added value to a teacher is accumulated when after years of in-service, additional course work and other experiences over the duration of teaching. This is separate from seniority.

Sofia – Bulgaria
True. Absolutely true about the Finnish educational system.We should simply copy it if we really would like any improvement in our educational institutions.
I happened to attend a training in Cambridgein 2010 and one of our group's colleagues was from Finland.
Her course project was entitled: How to make our students happier at school?
It makes one think, doesn't it?

Post a Comment