Using AI to digitally clone myself (AKA creating a Puny-Punya)

by | Sunday, March 19, 2023

Note: The photo-manipulated image of me holding my own head was created almost 20 years ago by Paul Kurf, a student in my learning by design, class! Image design & layout, Punya


Ethan Mollick is a professor at Wharton and he has been doing some of the most interesting work in playing with ChatGPT3 and other generative language models, particularly exploring their role in teaching. If you haven’t been following him, and are even peripherally interested in these issues, do check out his substack: One useful thing.

One of his experiments involved creating video of himself that was entirely AI generated (see A quick and easy guide to cloning yourself).

Of course I had to try this for myself to create my own mini-Me (or should it be Puny Punya).

I decided to create a short video of myself introducing an episode of Just an Hour—a series of informal conversations that Betty Gee and I co-host every Friday for faculty and doctoral students. This felt appropriate for two reasons. First, that particular session was devoted to generative AI in higher education, and it seemed right to provide an example of what this could look like. Second, I knew was going to miss that particular episode, since I was flying back from the SITE conference, so it made sense to share a short video introducing the session and sending my regrets for missing the meeting.

What that in mind, I asked ChatGPT3 to write a short script on my behalf, in first person. In parallel I had trained Eleven Labs to generate my voice by reading it 2 – 3 minutes or prose from my website. Eleven Labs took that recording and created a virtual voice for me, which could then read out any prose it was given, in that voice. I gave it the prose created by ChatGPT3 and it read it back to me in my “digitally generated” voice. Finally, I uploaded a photo of myself and the MP3 to the D-id website. (I had previously used this website to create the video for my learning styles blog post),

Within minutes I had a video I could download.

So just to be clear, the video was created by text, audio and video all generated by AI (with minimal input from my side.

So does this video look and sound like me? I would ask you to be the judge of that but before we get to the videos, a few things to note.

  1. This entire process took me around 30 minutes (and that includes time it took to creating the accounts, training Eleven Labs on my voice, taking a picture, uploading it, and so on.
  2. Also, all this was done by spending just one dollar! Eleven Labs needed me to sign up and pay for the service but since they were having a sale I got to use it for that amazing price!
  3. The final product is sort of weird, and does not look extremely realistic. But just the fact that it even exists is amazing. I am sure if I had spent more time and money I could have had a much better product.
  4. I also experimented with creating a low-res version of the video and strangely enough it actually seems more realistic and believable than the hi-res one. It is almost as if the graininess of the video makes us ignore the other glitches.
  5. Last but surely not the least, Eleven Labs totally messed with my voice and accent. if you have heard me speak, or recognize my voice, you will immediately realize that my voice has been changed quite dramatically, removing almost all traces of my my Indian accent. Again, given my previous experience with these technologies, I am not surprised at all at this!

With that, here are the two videos, first the hi-res version followed by the low-res version. What do you think?

The Hi-Res version of the video.

The Low-Res version of the video.

The key question of course that we should all be asking ourselves: What does it mean when the cost of creating content like this drops to zero.


Addendum

For the record, below is the image that Paul Kurf created for me almost 20 years ago!

Despite my complaints about how Eleven Labs messed up my accent, it clearly does a better job with other voices, as evidenced by this story I read today about how an AI generated voice allowed someone to break into a bank account.

A few randomly selected blog posts…

Lego based Sudoku & Rubik Cube solving robots

Two robots made entirely using Lego Mindstorms NXT Retail-kit that can solve Sudoku problems and the Rubik's Cube! How totally cool is that. LEGO Mindstorms is a line of Lego sets combining programmable bricks with electric motors, sensors, Lego bricks, and Lego...

Facilitating collaborative design: New publication

Facilitating collaborative design: New publication

Essential tensions in design. Image designed by Punya Mishra Design facilitators play an important role in the open-ended collaborative design process. This becomes even more important as design based approaches expand to groups and teams that may not be as familiar...

On surviving a Ph.D.

I just discovered (H/T Daily Dish) Matt Might's website and his writings on graduate school, academia, and the professoriate. Matt is funny, cogent and most importantly insightful. I recommend his writing to anybody who is interested in getting into graduate school,...

Brilliant stop motion

Just came across this on Nina Paley's blog... and it just blew me away! [youtube width="425" height="355"]http://www.youtube.com/watch?v=2_HXUhShhmY[/youtube]

Facebook Username

I now have a facebook username! Hah! Check out http://www.facebook.com/punyamishra/

TPACK Newsletters (#28, #29 & 30)

TPACK Newsletters (#28, #29 & 30)

In the rush of summer and the move to Arizona I missed posting #28 and #29 of the TPACK newsletter, and before I knew it, #30  was here as well. Well here are links to the PDFs of all three newsletters Newsletter 28: May 2016 (pdf)Newsletter 29: July 2016...

Update on “The TPACK story” Or “Oops!”

Update on “The TPACK story” Or “Oops!”

I had recently posted a video of my talk fall Doctoral Research Forum for the Mary Lou Fulton Teachers College on the ASU West campus. As I had written in my post, "I thought it best to speak about the role of theory in research. This is something that...

Post-lunch session: Nancy Law

The last session of the day was led by Dr. Nancy Law, Director, Centre for Information Technology in Education, University of Hong Kong. Her session was titled Using ICT to support learning: lessons learnt from international studies I met Dr. Law at SITE 2008 and then...

Is a lecture just a lecture?

My mashup of a commercial has been on YouTube for a while and just yesterday I noticed that someone had left a very thoughtful comment... and that comment got me thinking... and hence this posting. To start with, if you haven't seen the videos here they are again....

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *