Using AI to digitally clone myself (AKA creating a Puny-Punya)

by | Sunday, March 19, 2023

Note: The photo-manipulated image of me holding my own head was created almost 20 years ago by Paul Kurf, a student in my learning by design, class! Image design & layout, Punya


Ethan Mollick is a professor at Wharton and he has been doing some of the most interesting work in playing with ChatGPT3 and other generative language models, particularly exploring their role in teaching. If you haven’t been following him, and are even peripherally interested in these issues, do check out his substack: One useful thing.

One of his experiments involved creating video of himself that was entirely AI generated (see A quick and easy guide to cloning yourself).

Of course I had to try this for myself to create my own mini-Me (or should it be Puny Punya).

I decided to create a short video of myself introducing an episode of Just an Hour—a series of informal conversations that Betty Gee and I co-host every Friday for faculty and doctoral students. This felt appropriate for two reasons. First, that particular session was devoted to generative AI in higher education, and it seemed right to provide an example of what this could look like. Second, I knew was going to miss that particular episode, since I was flying back from the SITE conference, so it made sense to share a short video introducing the session and sending my regrets for missing the meeting.

What that in mind, I asked ChatGPT3 to write a short script on my behalf, in first person. In parallel I had trained Eleven Labs to generate my voice by reading it 2 – 3 minutes or prose from my website. Eleven Labs took that recording and created a virtual voice for me, which could then read out any prose it was given, in that voice. I gave it the prose created by ChatGPT3 and it read it back to me in my “digitally generated” voice. Finally, I uploaded a photo of myself and the MP3 to the D-id website. (I had previously used this website to create the video for my learning styles blog post),

Within minutes I had a video I could download.

So just to be clear, the video was created by text, audio and video all generated by AI (with minimal input from my side.

So does this video look and sound like me? I would ask you to be the judge of that but before we get to the videos, a few things to note.

  1. This entire process took me around 30 minutes (and that includes time it took to creating the accounts, training Eleven Labs on my voice, taking a picture, uploading it, and so on.
  2. Also, all this was done by spending just one dollar! Eleven Labs needed me to sign up and pay for the service but since they were having a sale I got to use it for that amazing price!
  3. The final product is sort of weird, and does not look extremely realistic. But just the fact that it even exists is amazing. I am sure if I had spent more time and money I could have had a much better product.
  4. I also experimented with creating a low-res version of the video and strangely enough it actually seems more realistic and believable than the hi-res one. It is almost as if the graininess of the video makes us ignore the other glitches.
  5. Last but surely not the least, Eleven Labs totally messed with my voice and accent. if you have heard me speak, or recognize my voice, you will immediately realize that my voice has been changed quite dramatically, removing almost all traces of my my Indian accent. Again, given my previous experience with these technologies, I am not surprised at all at this!

With that, here are the two videos, first the hi-res version followed by the low-res version. What do you think?

The Hi-Res version of the video.

The Low-Res version of the video.

The key question of course that we should all be asking ourselves: What does it mean when the cost of creating content like this drops to zero.


Addendum

For the record, below is the image that Paul Kurf created for me almost 20 years ago!

Despite my complaints about how Eleven Labs messed up my accent, it clearly does a better job with other voices, as evidenced by this story I read today about how an AI generated voice allowed someone to break into a bank account.

A few randomly selected blog posts…

Tweaking the design

I have been blogging pretty seriously now for 10 months now and am quite enjoying it. I have made some changes to the design of the site that may be worth explaining. As I have blogged over the past few months, I have come to realize that I typically make three kinds...

What is this thing called text?

Steven Johnson has a great essay on the future of text title: The Glass Box And The Commonplace Book. I recommend reading the full thing but here is a quote that sort of captures his vision (though there is more, much more). Here is a great quote: WHEN TEXT IS free to...

TPACK Newsletter, Issue #13, December 2012

TPACK Newsletter, Issue 13, December 2012 Welcome to the thirteenth edition of the (approximately quarterly) TPACK Newsletter! TPACK work is continuing worldwide, and is appearing in an increasing diversity of publication, conference, and professional development...

Cybersecurity & the Future of Education

Cybersecurity & the Future of Education

I was recently interviewed by David W. Schropfer for his DIY Cyber Guy podcast. David is an expert on cybersecurity and, and that is the focus of his podcast. I am clearly not an cybersecurity expert, so I was somewhat surprised at being invited to his show. What...

Bye bye textbooks, buy buy laptops

Reuters story titled Technology reshapes America's classrooms. Couple of quotes worth noting: "Why would we ever buy a book when we can buy a computer? Textbooks are often obsolete before they are even printed," said Debra Socia, principal of the school in Dorchester,...

Capturing CAPTCHA or If it can be outsourced…

... it will. We have all see CAPTCHA's (aka Completely Automated Public Turing test to tell Computers and Humans Apart). They are images with somewhat garbled text on them that websites used to tell humans from automated programs. The idea is to prevent prevent...

Shattered: Myth, Metaphor & Gen AI

Shattered: Myth, Metaphor & Gen AI

A few weeks ago, I wrote a blog post about Tennyson's "Lady of Shalott" and its resonance with our AI age (The Mirror Cracked: AI, Poetry, and the Illusion of Depth). In that post I explored how our experience of the world is increasingly mediated by technology, AI...

Posting from an iTouch

typing on this keyboard is still kind of painful, though I am getting better every word I type.

Cool logos

I just came across this page of excellent logo designs. Some of the designs may need a moment or two to truly register. Check out 20 Unique and Creative Logo Designs. Here are couple of my favorites.

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *