In an age where experts are a dime a dozen, willing to pontificate at the drop of a pin, it is hard to tell whom to believe, and whom NOT to believe. In comes Phillip Tetlock, an academic who has made it his mission to evaluate the prognosticators! This is described in his book:

Tetlock, P.E. (2005). Expert political judgment: How good is it? How can we know? Princeton: Princeton University Press.

I recently came across a review written by him, titled Reading Tarot on K Street (in the September/October 2009 issue of The National Interest) and I thought it captured his work in this area quite nicely (and would be worth preserving).

When we score the accuracy of thousands of predictions from hundreds of experts across dozens of countries over twenty years, we find the best forecasters tend to be modest about their forecasting skills, eclectic in their ideological and theoretical tastes, and self-critical in their analytical styles.1 Borrowing from philosopher Isaiah Berlin, I call them foxes—experts who know many things and are not finicky about where they get good ideas. Paraphrasing Deng Xiaoping, they do not care if the cat is white or black, only that it catches mice.

Contrast this with what I call hedgehogs—experts who know one big thing from which likely future trends can be more or less directly deduced. The big thing might be any of a variety of theories: Marxist faith in the class struggle as the driver of history or libertarian faith in the self-correcting power of free markets, or a realist faith in balance-of-power politics or an institutionalist faith in the capacity of the international community to make world politics less ruthlessly anarchic, or an eco-doomster faith in the impending apocalypse or a techno-boomster faith in our ability to make cost-effective substitutes for pretty much anything we might run out of.

What experts think—where they fall along the Left-Right spectrum—is a weak predictor of accuracy. But how experts think is a surprisingly consistent predictor. Relative to foxes who are less encumbered by loyalties to an all-encompassing worldview, hedgehogs offer bolder forecasts and, although they hit occasional grand slams, they strike out a lot and wind up with decidedly poorer batting averages.

The implications for people who make projections about technology and schools and learning is quite obvious to me. It is the hedgehogs we need to be careful of, mainly because of the vehemence of their beliefs which can sometimes override our “foxy” nature. I say inherent because I think that educators, for the most part, are pragmatists, sensitive to the limits of arm-chair theorizing and big ideas. A hard nosed approach to reality, that recognizes its complexity, that demands multi-faceted problems solving approaches is what is needed, not being wedded to one, just one overarching idea.