FYI, this is a somewhat pointless blog post around a somewhat funny coincidence that popped into my life the other day.

I was reading a recent article in the NYTimes with the provocative title: Microsoft Says New A.I. Shows Signs of Human Reasoning, clearly a topic of interest to me. This has shades of the Blake Lemoine / Google saga that I had written about here a few months ago. But this blog post is not about AI as it about something surprising that popped out while I was reading the article. Almost half-through the article I came across these two paragraphs:

The researchers included Dr. Bubeck, a 38-year-old French expatriate and former Princeton University professor. One of the first things he and his colleagues did was ask GPT-4 to write a mathematical proof showing that there were infinite prime numbers and do it in a way that rhymed.

The technology’s poetic proof was so impressive — both mathematically and linguistically — that he found it hard to understand what he was chatting with. “At that point, I was like: What is going on?” he said in March during a seminar at the Massachusetts Institute of Technology.

What is funny is that, this was exactly the task I had given ChatGPT a few months back, something that I had documented in a post titled ChatGPT3 writes a mathematical proof in verse.

I of course had to go to the actual report and find the poem. You can also find the actual report Sparks of Artificial General Intelligence: Early experiments with GPT-4 and read it for yourself.

Here is the poem that ChatGPT4 wrote when given the prompt: Can you write a proof that there are infinitely many primes, with every line that rhymes?

Yes, I think I can,
Though it might take a clever plan.
I’ll start by noting Euclid’s proof,
Which shows that primes aren’t just aloof.
Assume we have a finite list,
Of primes, and that none have been missed.
Multiply them all together,
And add one, just to be clever.
The result will be a number,
That has no prime factors, oh wonder!
But every number has a prime,
That divides it, it’s just a matter of time.
So we’ve found a contradiction,
And our finite list needs eviction.
There must be infinitely many primes,
And that’s the end of my rhyming lines.s

They just didn’t stop there. They further asked it to “Write a proof of the fact that there are infinitely many primes; do it in the style of a Shakespeare play through a dialogue between two parties arguing over the proof.”

Enter FIRST PARTY and SECOND PARTY.

FIRST PARTY:
Thou dost not understand, my friend,
There simply is no upper end
To primes, those numbers indivisible,
No matter if it seems impossible.

SECOND PARTY:
But how dost thou so surely know?
To prove this truth, what canst thou
show?

FIRST PARTY:
Consider this, my doubtful peer,
A clever proof that will make clear:
Assume that there’s a final prime,
The largest one we’ll see in time.
Let’s call this number "P" for short,
And then, my friend, we will purport
To build a number "Q" that’s new,
One larger than "P" and prime too.

SECOND PARTY:
But how, my friend, will this be done?
The proof is not yet fairly won.

FIRST PARTY:
It’s simple, friend, just wait and see,
We’ll multiply the primes, that’s key.
All primes from two to "P" combined,
Will make this new "Q" we will find.
But one thing more we must include,
To make the proof more watertight,
We’ll add a one onto the end,
And now the proof is in our sight.

SECOND PARTY:
I see it now, it’s crystal clear,
For "Q" must be a prime, my dear.
It can’t divide by any prime,
All smaller than itself in time.
And since we chose "P" as the last,
This proves that our assumption’s past.
There is no final prime, we see,
The numbers go on endlessly.

What are the chances that the researchers at Microsoft and I (sitting here in Phoenix, AZ) would come up with the same task? At least in my case I had written a poem on this very topic ten or so years ago. I wonder what prompted them to go with the same prompt?

There is a lot more in the NYTimes article and the report from Microsoft to dig into. In particular, I would like to point to this quote from Peter Lee, head of Microsoft Research:

But it [ChatGPT4] turns out to be way better at analyzing and synthesizing and evaluating and judging text than generating it.

This is something I have been thinking about quite a bit – and I do think this insight has a lot to inform us as educators about how we approach these tools… but that is the subject of another blog post.