Keeping tabs on the experts

by | Tuesday, September 08, 2009

In an age where experts are a dime a dozen, willing to pontificate at the drop of a pin, it is hard to tell whom to believe, and whom NOT to believe. In comes Phillip Tetlock, an academic who has made it his mission to evaluate the prognosticators! This is described in his book:

Tetlock, P.E. (2005). Expert political judgment: How good is it? How can we know? Princeton: Princeton University Press.

I recently came across a review written by him, titled Reading Tarot on K Street (in the September/October 2009 issue of The National Interest) and I thought it captured his work in this area quite nicely (and would be worth preserving).

When we score the accuracy of thousands of predictions from hundreds of experts across dozens of countries over twenty years, we find the best forecasters tend to be modest about their forecasting skills, eclectic in their ideological and theoretical tastes, and self-critical in their analytical styles.1 Borrowing from philosopher Isaiah Berlin, I call them foxes—experts who know many things and are not finicky about where they get good ideas. Paraphrasing Deng Xiaoping, they do not care if the cat is white or black, only that it catches mice.

Contrast this with what I call hedgehogs—experts who know one big thing from which likely future trends can be more or less directly deduced. The big thing might be any of a variety of theories: Marxist faith in the class struggle as the driver of history or libertarian faith in the self-correcting power of free markets, or a realist faith in balance-of-power politics or an institutionalist faith in the capacity of the international community to make world politics less ruthlessly anarchic, or an eco-doomster faith in the impending apocalypse or a techno-boomster faith in our ability to make cost-effective substitutes for pretty much anything we might run out of.

What experts think—where they fall along the Left-Right spectrum—is a weak predictor of accuracy. But how experts think is a surprisingly consistent predictor. Relative to foxes who are less encumbered by loyalties to an all-encompassing worldview, hedgehogs offer bolder forecasts and, although they hit occasional grand slams, they strike out a lot and wind up with decidedly poorer batting averages.

The implications for people who make projections about technology and schools and learning is quite obvious to me. It is the hedgehogs we need to be careful of, mainly because of the vehemence of their beliefs which can sometimes override our “foxy” nature. I say inherent because I think that educators, for the most part, are pragmatists, sensitive to the limits of arm-chair theorizing and big ideas. A hard nosed approach to reality, that recognizes its complexity, that demands multi-faceted problems solving approaches is what is needed, not being wedded to one, just one overarching idea.

A few randomly selected blog posts…

49 Amazing moments of STEM: New article

49 Amazing moments of STEM: New article

The universe is made up of stories not atoms — Muriel Rukyeser (Image © punyamishra) Every educator has had an amazing teaching moment. It is that magical moment, when the topic comes to life and the energy in the classroom is palpable. These are moments that we...

Best of SkyMall

I love browsing through the SkyMall catalog when I am flying. I never cease to be amazed by human ingenuity - the range of things we have built, irrespective of how useful (or useless they may be). Anyway, someone has now listed the 10 best (or worst, depending on...

Google, teaching & creativity

Mike DeSchryver and I recently presented a paper at AERA titled "Googling creativity: An investigation into how pre-service mathematics teachers use the Web to generate creative ways to teach." The abstract is as follows: This study examined teacher creativity and its...

Stop motion fun

My daughter, Shreya, had some friends over yesterday and they created a short stop-motion animation film with the new setup in our basement. Enjoy [youtube]http://www.youtube.com/watch?v=TTkhuEfTAnk[/youtube] More videos made with my kids can be seen by clicking...

New Orleans (photos)

I took a couple of hours off to walk around New Orleans in the French Quarters taking pictures. Here they are... Click on the image for more pictures...

Summer travel 2008 photographs

I have been taking photographs as I travel around Asia (what I have previously described as my multi-national TPACK tour) and uploading them onto Flickr as and when I can. Go to the photographs

Cybersecurity & the Future of Education

Cybersecurity & the Future of Education

I was recently interviewed by David W. Schropfer for his DIY Cyber Guy podcast. David is an expert on cybersecurity and, and that is the focus of his podcast. I am clearly not an cybersecurity expert, so I was somewhat surprised at being invited to his show. What...

TPACK Newsletter #26, February 2016

TPACK Newsletter #26, February 2016

TPACK Newsletter, Issue #26: February 2016 Welcome to the twenty-sixth edition of the (approximately bimonthly) TPACK Newsletter! TPACK work is continuing worldwide. This document contains recent updates to that work that we hope will be interesting and useful to you,...

TPACK and online learning

Matt Koehler and I just completed a talk on TPACK and online learning for the Faculty Seminars in Instructional Technology. The Faculty Seminars are a semi-annual two-day series of short courses for faculty who want to learn about using technology in instruction....

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *