Cleaning and coding Interviews with AI

by | Monday, May 08, 2023

I have previously written about how AI can possibly help with qualitative research AND how how AI has given me a superpower which is the ability to write computer programs. Well this post is an extension of both of these topics.

To provide some context, for the past 10 years the Deep-Play Research Group (DPRG), has been writing a series for the journal TechTrends around the broad and intersecting themes of reimagining creativity, technology, and learning. Since 2016 we have focused on this series on conversations with renowned creativity scholars.

What that has meant is lots of zoom interviews which we then have to get transcribed for analysis and writing. Though Zoom provides us with transcripts they are quite messy to work with, since they break the conversation up, insert in time-stamps and more, and cleaning it up often not worth the effort – so we typically have sent it out for transcribing. I remember doing a search for such an online tool that would help us clean up these transcripts, a year or so ago, but came up with nothing that I could use, which was surprising just given the number of zoom transcripts being generated these days.

But now we have generative AI and given that ChatGPT has helped me write code, I wondered if I could write a small app whose sole job would be to clean up zoom transcripts. Could I create a drag-and-drop interface where I could drop in a zoom generated .vtt file and get a cleaned up .txt file at the end of it.

Well last Saturday I gave myself a few hours to try this out. And boom! I now have a small app (creatively titled: Zoom-Transcript-Cleaner-APP) on my computer that does exactly that. How awesome. For instance, here is a random zoom transcript I found on the web and what I get when I drop it onto the app.

How cool is that!

And that’s not all. In a previous post about AI and qualitative research I had mentioned using a website (anysummary.app) to analyze an interview with Chris Dede (more about that interview here). And just this morning I learned about AILYZE (www.ailyze.com) a website devoted to qualitative analysis. The claim is that this tool can automatically identify themes/ codebook, summarize documents/ transcripts, answer questions, and more… And, most important, they have a free version. So I dumped Chris’ transcript into the website and asked it to identify key themes, provide 2-3 sentence descriptions of each theme, and a key quote from Chris that exemplified the theme. And it churned on the task for a minute or two and produced a perfectly formatted table (see below).

Before I share what the AI analysis came up with – let me raise a question that I had asked at the end my previous post:

This makes me wonder, just how these AI tools are going to change qualitative research? Should we be ready to have qualitative data coded by AI coders? How soon before we see journal articles reporting Cohen’s Kappa for data that has been coded by humans AND AI systems?

I think the day is already here.

Below is the table that AILYZE created based on the transcript. Is it perfect? Not really. But that’s missing the point, isn’t it?

ThemeDetailed DescriptionQuote
Emphasis on AI ReplacementAI can complement human capabilities, and the real potential of AI lies in its ability to work together with humans.“What I’ve always been interested in is what people in AI can do together based on complimentary strengths and the whole being more than the sum of the parts.”
Human-AI CollaborationThe challenge is to understand what human beings bring that AI cannot replicate, and to explore what a human-AI combination can do that neither can do alone.“What can a human AI combination do that neither the human nor the AI alone can do?”
Tension between AI and Human CapabilitiesAI is not a weak form of human intelligence but rather a complementary one, and it cannot replicate human values and spiritual beliefs.“AI is not going to get to…the knowledge of what it means to have spiritual values.”
Ethical Decision-MakingAs AI systems enter into our lives, they will be forced to take moral decisions, and AI-based assistance can potentially result in better ethical decision-making.“Wise use of that knowledge requires all sorts of things that AI cannot do.”
Preparing Students for AI IntegrationThe focus should be on preparing students to work with AI and to understand its potential and limitations, and to recognize that AI can assist with writing but cannot replace creativity and critical thinking skills.“We need to see ourselves in that light. No, not really.”
Limitations of AI in Creative EndeavorsWhile AI can be useful for repetitive tasks, it cannot replace the creativity and nuance of human artists.“That’s the difference between AI and the human creator.”
Need for Foundational KnowledgeWhile AI can perform many tasks, it is important for humans to have a deep understanding of foundational concepts.“If I can look it up quickly, I don’t need to memorize it.”
Role of Bias in AIAI can be biased in several ways, and it is important to be aware of these biases and to work to mitigate them.“Understanding…the challenge [of bias] and not just saying, well, if it’s on the internet and you average across everything, you’re gonna come out with an unbiased thing. Nothing could be farther from the truth.”
Importance of Narrative in EducationNarrative and creative exposition distinguish great stories and ideas, and educators should challenge students to turn descriptive stories into compelling narratives.“Narrative is what distinguishes a descriptive story from a great story, narrative that puts the story in a larger context that tells a story that people can relate to.”
History of AIAI has made significant progress, but it has not yet reached the level of sophistication that some media outlets have portrayed.“I’ve lived through many cycles of advances in AI and been impressed by the progress. I’ve lived through many hype cycles of AI and been depressed by the, um, glowing, uh, endorsements that somehow never worked out.”
Definition of Generative AIGenerative AI refers to AI generating performances that were previously thought to be limited to human beings.“Generative AI can make that prediction because it can pull data from topological databases, from meteorological databases, from other kinds of databases, and come up with a forecast that’s beyond just sort of gluing together stuff on the web.”
Limitations of AI IntelligenceAI does not have the kind of intelligence that involves common sense, understanding what it means to have a body, culture, or ethical system.“AI does not have that kind of intelligence. It can do calculative reckoning where you’re working with data of different kinds and combining them in powerful ways to produce a forecast.”
Role of Culture and Embodiment in CognitionKnowledge, information processing, and cognition are deeply embodied in our bodies and the broader cultural and social matrix, and AI will never have that kind of knowledge and understanding.“And I would say that to some extent it is intelligent, but it’s not what, what I would think of as a strong definition of intelligence, which involves common sense, which involves understanding what it means to have a body, what it means to have a culture, what it means to have an ethical system, and so on.”
Role of AI in EducationAI can be used to build intelligent assistance for university professors, and it is not a generalized intelligence but rather a very narrow and specific kind of intelligence.“I’ve been talking about, well, you know, reporters may have an AI partner and doctors may have an AI partner. Well, I may have an AI partner quite soon.”
Potential for AI to Assist in LearningAI can be used to build question-answering instructional assistants, tutoring instructional assistants, a library instructional assistant, and a laboratory social instructional assistant that helps students connect with other students.“Suppose that I’ve got all those assistants sitting around me a couple of years from now, I can easily be de-skilled by those assistants where they’re doing much of the work and I’m in a sense working for them.”
Importance of Domain-Specific AIAI is better suited for hard sciences than soft sciences, and it struggles with questions about human behavior.“If you ask an AI to explain something in the hard sciences, like what are the different cooling mechanisms, you’re going to get a probably really good explanation, maybe a better explanation than the typical high school science teacher. However, if you ask a question about human behavior, AI is at sea in something like that.”
Challenges of Assessing Human BehaviorAssessing human behavior is a challenge because psychometric tests are on the AI side of the equation, while performance simulations are on the human side.“Psychometric tests are really on the AI side of the equation, while performance simulations are very much on the human side.”
Need to Balance AI with Human SkillsEducation should prepare people to use AI as a servant as opposed to making AI their master, and to recognize that there are things that need to be learned but cannot be taught.“We need to change education to maximize the human side of what we prepare people for and to prepare people to use AI as a servant as opposed to making AI their master.”

A few randomly selected blog posts…

You have been terminated: A case for humane design

You have been terminated: A case for humane design

Good design cares about details. Good design is humane. Bad design is neither. Designers must bring this attention to detail and humanity to every aspect of their work. And this applies even the invisible parts. This, caring for the "invisible" details, is captured in...

Money for nothing, and your clicks for free!

I knew that website developers would go to great lengths to rise in Google rankings. What I didn't know was just how far people were willing to go, till I received this email. Note: I have deleted all the links and names, since that would be giving free publicity to...

TPACK @ AERA, New York

In a few weeks TPACK will be going to AERA. There are a couple of events scheduled: First, is a symposium titled: Technological Pedagogical Content Knowledge (TPCK): A Conceptual Framework With Examples for Integrating Technologies Into Teacher Education [download...

TPACK Vanity (v. 2.0)

Back in 2006 Matt and I took a bunch of work that we had been doing in the area of technology integration for teaching and pulled it together into one broad theoretical framework and published it in TCRecord. The TPACK framework as it has come to be known has been...

Forget MMORPG, its time for MMLSG

NYTimes article titled, Storming the Campuses on the next big thing on college campuses: GoCrossCampus! This new kind of a game (and game genre) has been described as Multiplayer Locally Social Gaming and the way it is spreading, it may soon need to add "Massively" to...

Partial to PartiallyClips

I Stumbled Upon PartiallyClips, a web-based comic strip based on clip art. The rules are simple, "No changes to the art from frame to frame ... Never use the same clip in two strips. No repeating characters." It it amazing just how well this works, despite these...

Photos from SITE08

Matt has Flickrd photos from SITE08. Some of these photos are taken by me, but the rules are that the owner of the camera automatically gets the bragging rights 🙂 and since I didn't take my camera along, he takes credit for all the pictures. Given that a bunch of...

Untangling a decade of creativity scholarship

Untangling a decade of creativity scholarship

How do we capture a program of scholarship in an image? This is particularly complicated when the work is a tangled web of connections between research, teaching and practice, spread out over multiple publications, presentations and people. One attempt to do...

Hello Hong Kong, goodbye Hong Kong

A short and sweet trip to Hong Kong, one full day, two nights, fly in fly out. I was a guest of the University of Hong Kong and gave a talk there yesterday, the last in my series of TPACK talks. I had a good audience, the talk went well, though I did not manage my...

1 Comment

  1. Liz

    Zoom Transcription Cleaner — are you willing to share? That would be SO HELPFUL.

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *