Using AI to digitally clone myself (AKA creating a Puny-Punya)

by | Sunday, March 19, 2023

Note: The photo-manipulated image of me holding my own head was created almost 20 years ago by Paul Kurf, a student in my learning by design, class! Image design & layout, Punya


Ethan Mollick is a professor at Wharton and he has been doing some of the most interesting work in playing with ChatGPT3 and other generative language models, particularly exploring their role in teaching. If you haven’t been following him, and are even peripherally interested in these issues, do check out his substack: One useful thing.

One of his experiments involved creating video of himself that was entirely AI generated (see A quick and easy guide to cloning yourself).

Of course I had to try this for myself to create my own mini-Me (or should it be Puny Punya).

I decided to create a short video of myself introducing an episode of Just an Hour—a series of informal conversations that Betty Gee and I co-host every Friday for faculty and doctoral students. This felt appropriate for two reasons. First, that particular session was devoted to generative AI in higher education, and it seemed right to provide an example of what this could look like. Second, I knew was going to miss that particular episode, since I was flying back from the SITE conference, so it made sense to share a short video introducing the session and sending my regrets for missing the meeting.

What that in mind, I asked ChatGPT3 to write a short script on my behalf, in first person. In parallel I had trained Eleven Labs to generate my voice by reading it 2 – 3 minutes or prose from my website. Eleven Labs took that recording and created a virtual voice for me, which could then read out any prose it was given, in that voice. I gave it the prose created by ChatGPT3 and it read it back to me in my “digitally generated” voice. Finally, I uploaded a photo of myself and the MP3 to the D-id website. (I had previously used this website to create the video for my learning styles blog post),

Within minutes I had a video I could download.

So just to be clear, the video was created by text, audio and video all generated by AI (with minimal input from my side.

So does this video look and sound like me? I would ask you to be the judge of that but before we get to the videos, a few things to note.

  1. This entire process took me around 30 minutes (and that includes time it took to creating the accounts, training Eleven Labs on my voice, taking a picture, uploading it, and so on.
  2. Also, all this was done by spending just one dollar! Eleven Labs needed me to sign up and pay for the service but since they were having a sale I got to use it for that amazing price!
  3. The final product is sort of weird, and does not look extremely realistic. But just the fact that it even exists is amazing. I am sure if I had spent more time and money I could have had a much better product.
  4. I also experimented with creating a low-res version of the video and strangely enough it actually seems more realistic and believable than the hi-res one. It is almost as if the graininess of the video makes us ignore the other glitches.
  5. Last but surely not the least, Eleven Labs totally messed with my voice and accent. if you have heard me speak, or recognize my voice, you will immediately realize that my voice has been changed quite dramatically, removing almost all traces of my my Indian accent. Again, given my previous experience with these technologies, I am not surprised at all at this!

With that, here are the two videos, first the hi-res version followed by the low-res version. What do you think?

The Hi-Res version of the video.

The Low-Res version of the video.

The key question of course that we should all be asking ourselves: What does it mean when the cost of creating content like this drops to zero.


Addendum

For the record, below is the image that Paul Kurf created for me almost 20 years ago!

Despite my complaints about how Eleven Labs messed up my accent, it clearly does a better job with other voices, as evidenced by this story I read today about how an AI generated voice allowed someone to break into a bank account.

A few randomly selected blog posts…

Robert Pirsig, 1928 – 2017

Robert Pirsig, 1928 – 2017

Zen and the Art of Motorcycle Maintenance was one of those books that have influenced me deeply. I read it when I was in high school and read it again and again, almost obsessively for a while. It was my companion through college, graduate school and beyond. I...

Jugaad, educational toys from Junk (TPACK at work)

I had written earlier about the idea of Jugaad, the quintessential Indian idea of situational creativity. One of the masters at this is Arvind Gupta. Check out his website for tons of wonderful science toys and experiments that can be made from stuff we typically...

Goodbye 2020 (whew), welcome 2021

Goodbye 2020 (whew), welcome 2021

2020 has been a heck of a year... and maybe in hindsight (hindsight, of course, being 2020) it will all make sense. But, I think we can all agree that it is time to let it go. A lot has changed this past year but one tradition we wanted to keep alive was the short...

Creativity and the urban STEM teacher

Creativity and the urban STEM teacher

I have written previously about the MSUrbanSTEM project and what it has meant to me. Over the past couple of years we have also published about this line of work (most prominently in a special issue of The Journal of Computers in Mathematics and Science Teaching)....

New ambigrams for a new blog!

What do you think this is? Take a guess...Well, it is the top half of a lake-reflection ambigram. What this means is that if you reflect what you see along a horizontal line at the bottom of the image, the picture you will then get will spell a word. Can you figure...

Emailing a plagiarist

I am sending the following email to David Jiles, Ph.D. whose plagiarism I have documented in this posting: David Jiles, Ph.D., Creativity Expert, Plagiarist. The email is as follows: Dear Dr. Jiles -- I have some questions and concerns about your book "Creativity and...

Computational Thinking paper wins Outstanding Paper

Computational Thinking paper wins Outstanding Paper

A paper co-authored with Jon Good and Aman Yadav, building on Jon's practicum study has received the Outstanding Paper Award at the SITE 2017. Complete reference, link to article and abstract given below. Good. J., Yadav. A., & Mishra, P....

Spring break 2008

Our first family vacation in over three years! New Jersey to visit relatives, Delaware to visit friends, and New York city for the big city excitement! Hectic but great fun. I took over 500 photographs, got back home and deleted around 200 of them - the remaining are...

Creativity Symposium at SITE2013

We just completed our symposium at SITE titled: Breaking Disciplinary Boundaries in 21st Century Learning: Creative Teaching with Digital Technologies. The symposium consisted of 7 presentations followed a summary by Teresa Foulger (of Arizona State University). In...

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *