Rethinking Ed Tech Research…

by | Friday, April 29, 2011

I have been a huge fan of Don Norman ever since I first ran into his book on the Psychology of Everyday Things (which he later renamed as The Design of Everyday Things, and the story behind that name change is worth reading as an excellent example of design). Don Norman also was the inspiration behind my collection of examples of good and bad design, something that ended up in the CEP817, Learning Technology by Design seminar.

Recently I got to read an essay by him titled “Why Design Education Must Change.” Essentially he argues that design education, as it is done today, does not prepare designers for the challenges of the present or the future. As he says:

Where once industrial designers focused primarily upon form and function, materials and manufacturing, today’s issues are far more complex and challenging. New skills are required, especially for such areas as interaction, experience, and service design. Classical industrial design is a form of applied art, requiring deep knowledge of forms and materials and skills in sketching, drawing, and rendering. The new areas are more like applied social and behavioral sciences and require understanding of human cognition and emotion, sensory and motor systems, and sufficient knowledge of the scientific method, statistics and experimental design so that designers can perform valid, legitimate tests of their ideas before deploying them.

Many of the things he wrote about resonated with me as an educator with an interest in technology. I have often argued for seeing education through the lens of design and in fact have written extensively about it (too lazy to list/link these publications here).

Over the past few years I have become somewhat disenchanted with the nature of educational technology research and its value to practitioners. The top journals seem to be biased towards specific kinds of research (quantitative, experimental, control group kinds of studies). The research and publication process just takes too long. It can take years from the start of a research study to its final publication (going through the stages of conceptualizing a study to collecting and analyzing data, to writing and submitting it for publication and responding to reviewers comments). This process was ok when the world we lived in was stable. But in a world where technology changes pretty much every day, a publication can be out of date even before it is published. The goals of this process were more to be “scientific” rather than to impact practice. Qualitative approaches have often been offered as a response but they have their own challenges of experimenter bias, generalizability and so on.

To cut a long story short, I have been struggling, in often an inchoate kind of way, with these issues. So it was with great pleasure that I read Don’s article – even though it did not deal directly with educational research. And somewhat towards the end a couple of paragraphs caught my eye – that seemed to offer, very broadly, a way forward. I have cut and pasted these paragraphs below, with one change – replacing the word “designer” with “educator.” Take a look…

Educators are practitioners, which means they are not trying to extend the knowledge base of science but instead, to apply the knowledge. The educator’s goal is to have large, important impact. Scientists are interested in truth, often in the distinction between the predictions of two differing theories. The differences they look for are quite small: often statistically significant but in terms of applied impact, quite unimportant. Experiments that carefully control for numerous possible biases and that use large numbers of experimental observers are inappropriate for educators.

The educator needs results immediately, in hours or at possibly a few days. Quite often tests of 5 to 10 people are quite sufficient. Yes, attention must be paid to the possible biases (such as experimenter biases and the impact of order of presentation of tests), but if one is looking for large effect, it should be possible to do tests that are simpler and faster than are used by the scientific community will suffice. Designs don’t have to be optimal or perfect: results that are not quite optimum or les than perfect are often completely satisfactory for everyday usage. No everyday product is perfect, nor need they be. We need experimental techniques that recognize these pragmatic, applied goals.

Education needs to develop its own experimental methods. They should be simple and quick, looking for large phenomena and conditions that are “good enough.” But they must still be sensitive to statistical variability and experimental biases. These methods do not exist: we need some sympathetic statisticians to work with educators to develop these new, appropriate methods.

What do you think? What would some of these new experimental methods look like? It seems to me that this is a design problem that should really be at the forefront of what we educational technology researchers do.

 

 

A few randomly selected blog posts…

My favorite Internet meme (and how it almost died)

I have been tracking the Hitler-Downfall parodies for over two years now and it seems that they keep getting better and better. But over the last few days comes the news that Constantin films, which owns the rights to the original movie asked YouTube to find and take...

The futures of higher ed with Phoebe Wagner

The futures of higher ed with Phoebe Wagner

The Center for Science and the Imagination at ASU runs a series of short stories and virtual gatherings that explore issues related to transformative change. Essentially they solicit and publish a (super-short) short story that explores “themes of community,...

Creativity, Digitality, and Teacher PD

It has been almost 5 years since my research interests shifted formally to issues related to creativity, technology and teacher education. This line of work (though less influential than my TPACK related work) has led to multiple journal articles and research studies....

LanguageART: Meaning making through type & image

LanguageART: Meaning making through type & image

I love collecting quotations—usually related to learning, design, and creativity. Over the past couple of years I started trying to visualize these quotations, playing with type and image, to tease apart their meanings, sometimes to undermine, sometimes to enhance. I...

Contextualizing TPACK within systems and culture

Contextualizing TPACK within systems and culture

Melissa Warr and I were recently asked to write a afterword to a special issue of the journal Computers in Human Behavior. The focus of the special issue was on the kinds of knowledge, skills and attitudes (KSA) teachers need to successfully integrate technology in...

Darwin Day & A new Gallup Poll

Charles Darwin 12 February 1809 – 19 April 1882 On this day, it is sobering to read the results of the latest Gallup Poll: On Darwin’s Birthday, Only 4 in 10 Believe in EvolutionOn the eve of the 200th anniversary of Charles Darwin's birth, a new Gallup Poll shows...

The hitchhiker’s guide to online doctoral programs, SITE2013

We finished our second symposium yesterday. Titled the The Hitchhiker’s Guide to Hybrid and Online Doctoral Programs the symposium included presentations by faculty members from Michigan State University and University of North Texas. Ann Thompson was our...

TPACK in the land down under

I recently received an email from Debra Bourne, IT Coordinator at St. Paul's International College in Australia informing me about some work related to TPACK being done in Queensland. Specifically she mentioned a paper to be presented at the upcoming Australian...

TPACK in EDTECHNICA

TPACK in EDTECHNICA

I have been a huge fan of EdTechBooks for a long time. Their philosophy of making quality textbooks freely accessible for all resonates with me deeply. It is no surprise that I was excited to hear of their latest initiative: that of creating a living encyclopedia of...

6 Comments

  1. Israel Valenzuela

    Teachers as designers–what a concept! My experience as a grad student at Pepperdine’s MA in Ed Tech (now Learning Technologies) opened my eyes to being an educator and a designer of learning. We created a Design Library. And there was nothing wrong or weird about it. Today the program includes design challenges and design circles. Pretty awesome stuff. Thanks, Punya.

    Reply
  2. Punya Mishra

    @Suzanne Good point regarding shifts in mind-sets. I would like to see a more explicit approach to training educators in the design way of looking at the world. I do that in a master’s seminar I teach and the results are often fascinating to watch.

    @Bob The tension between “scientific” knowledge and “practitioner” knowledge is important – and finding a bridge between them is what I am (and I think Don Norman) is talking about.

    Reply
  3. Bob Reuter

    I have felt like you Punya for the last few years while working in our initial teacher training program at the University of Luxembourg, where an EMPHASIS on educational research has been introduced. I’ve been struggling to try to make my students (future teachers) understand why they need to have knowledge of and about scientific inquiry for their future teaching practice… and most of them did not really see the value of scientifically grounded knowledge for everyday teaching practice… and, reading your post, I somehow think they were right to be skeptical, because most of what scientific research on education has to give is not really that relevant for those everyday problems that they do have to address… I still think that it’s wiser to use scientific knowledge to ground our decisions than to go with personal wisdom… but I also do see the value of practitioner knowledge, because specific contexts require specific “solutions”, that general solutions, extracted from many specific contexts, cannot give us.

    Reply
  4. Suzanne

    Interesting post. I had one thought. Most individuals who would identify with being an educator (e.g. teachers) don’t also identify themselves as designers. If the did, and then applied the research tools of say action research with prototype development and then evidence-based practice, I think we’d see movement in mindsets and orientations.

    As ever more teachers are forced or asked to become “designers” with and without training and support, I think we’re seeing shifts already in mindsets. At our institution, SDSU, faculty who participate in our course design institute, in order to put course online, begin to think and work like designers.

    It’s an interesting process to observe.

    Reply
  5. Punya Mishra

    I think action research / teacher inquiry are surely part of the solution. The issue really is two fold. The first of valuing practitioner knowledge and the second of developing practitioners who can participate in this knowledge building. There are ways of representing our practice that goes beyond merely describing – in ways that connect to the larger scholarly conversation. ~ punya

    Reply
  6. Holly Marich

    Where does action research/teacher inquiry fit into the possible solution? Mini inquiries could be accomplished in days – although, the inquiry communities I’ve facilitated have been over the course of a school year. I don’t know that an appropriate method has not been developed, it’s more a question of the larger scientific community valuing such methods. What do you think?

    Reply

Leave a Reply to Suzanne Cancel reply

Your email address will not be published. Required fields are marked *