Philip Tetlock (screen shot from high-res Seminar video available to members)
Philip Tetlock recently presented a Seminar About Long-Term Thinking to the effect that confident forecasters ought to be ignored. Despite his research showing the profound unreliability of such speculation, it’s rare to find even a moderately systematic evaluation of political forecasts in the popular media. In this category comes an article from the online version of Radar magazine earlier this year, “The Iraq Gamble”, which scrutinises journalistic commentators’ predictions about the war in Iraq. In it, writer Jebediah Reed tracks the fortunes of predictions against those of the predictors, and is disturbed by what turns up.
[M]aybe something is amiss in the world of punditry. Are the incentives well-aligned? Surely those who warned us not to invade Iraq have been recognized and rewarded, and those who pushed for this disaster face tattered credibility and waning career prospects. Could it be any other way in America?
…
So we selected the four pundits who were in our judgment the most influentially and disturbingly misguided in their pro-war arguments and the four who were most prescient and forceful in their opposition.
…
Then we did a career check … and found that something is rotten in the fourth estate.
The suggestion is that American meritocracy seems to be in trouble when it comes to the careers of political pundits. Eight journalists’ profiles are presented under two headings: “Getting rich by being wrong”, and “Right but poor”. As one of those drafted into the latter category, Jonathan Schell, remarks: “There doesn’t seem to be a rush to find the people who were right about Iraq and install them in the mainstream media.” Meanwhile, certain confident voices have prevailed despite the question marks over their track record (a similar point is made in an article from FAIR.org last year, addressing the Iraq-related statements of columnist Thomas Friedman, who also comes into the firing line in the Radar piece).
Whether or not one agrees with the argument as applied to this politically contentious topic — and a larger sample would be required to draw firm conclusions — it seems clear that improving accountability (i.e., correspondence of reputation and record) for predictions made by public figures remains a major challenge in politics and the media alike. It’s an aspiration embedded in the Long Bets project, and any readers who think they can do as well as, or better than, these guys — on any topic of public interest — is encouraged to test their long-term prescience by placing a wager on the site.
