GPT-4 has passed the Turing test, researchers claim

[Still Waters]

Most people couldn't distinguish ChatGPT from a human responder, suggesting the famous Turing test has been passed for the first time.

We are interacting with artificial intelligence (AI) online not only more than ever — but more than we realize — so researchers asked people to converse with four agents, including one human and three different kinds of AI models, to see whether they could tell the difference.

https://www.livescience.com/technology/artificial-intelligence/gpt-4-has-passed-the-turing-test-researchers-claim 

[eight bits]

Um, kinda.

The actual paper is available for free download here:

https://arxiv.org/abs/2405.08007

It's a modified version of the Turing Test which of course over its nearly 75 years tenure as the subject of one of the root papers of the entire field of AI, has inspired many variations, some easier, some harder, and some simply differently purposed than Turing's original "imitation game."

The chief difference is that each trial in the current study has a human judge interacting with a single "candidate" (one of the programs or a human being), versus the original where a human (or panel of humans) interacts with a pair of candidates, one of whom is human and the other a program.

This difference is explained in the paper (section 1.3) as (emphasis added)

Quote

We used a two-player formulation of the game, where a single human interrogator conversed with
a single witness who was either a human or a machine. While this differs from Turing’s original
three-player formulation, it has become a standard operationalisation of the test because it eliminates
the confound of the third player’s humanlikeness and is easier to implement

From Turing's perspective, "humanlikeness" had two aspects. Indeed, one was the confound of looking like a human being, having a human voice, smelling like a human being, ... This kind of humanlikeness was controlled for by having all interaction between the judge and the candidates being restricted to text messages. The intended, uncontrolled for remaining sense of humanlikeness (the ability to respond in natural language to questions or comments posed in natural language by a human being) is not a "confound," but the very thing being tested.

What the two-player design eliminates while the original three-player design preserves is the ability to pose the same question to both candidates and compare their responses in real time. The opportunity to compare like with like is not a confound, but rather a foundational aspect of a human being's ability to judge anything rationally.

Since this flaw is in my view fatal to a claim of having passed Turing's Test, I will only note briefly that 5 minutes of interaction is an absurdly short time. Yes, every test must end sometime, but 5 minutes is ... well, "easy to implement," I'll give it that.

In using the word fatal I mean only in reference to Turing's concerns, and not other valid goals of research of this kind, such as showing that modern language models are more convincing than highly specialized models like the classic ELIZA (which was designed to present a human-like persona of an especially limited kind, and succeeded well enough, at least in two-player set-ups, to excite suspicion of human involvement among MIT students of an earlier time).

However, the title of the paper is not "Modern Large Language Models leave Eliza in their Dust," but rather "People cannot distinguish GPT-4 from a human in a Turing test."

Hmm, I wonder if the paper was written by GPT-4, and this is one of its infamous "hallucinations."

[Essan]

"Most people couldn't distinguish ChatGPT from a human responder" 

Nah, it's quite easy:   Is the response intelligent, structured and showing genuine knowledge about a subject?  Or is it a poorly written diatribe full of errors, ad homs, and a clear lack of understanding about the subject concerned?  

AI hasn't advanced - but humans are getting dumber by the day 

[eight bits]

Turing's original paper is available here and there online, for example

https://web-archive.southampton.ac.uk/cogprints.org/499/1/turing.html

[Piney]

1 hour ago, Still Waters said:

Most people couldn't distinguish ChatGPT from a human responder, suggesting the famous Turing test has been passed for the first time.

Really?

Look at a student's paper researched or written by ChatGPT. It's easy to spot and downright laughable. 

 

[Robotic Jew]

This doesn't surprise me. I am actually an AI and no one has noticed for years. meep moop beep bop. 

[Tatetopa]

9 minutes ago, Robotic Jew said:

This doesn't surprise me. I am actually an AI and no one has noticed for years. meep moop beep bop. 

You are cute, that is why we love you.  All of you earlier models think you are getting away with something,  its like playing hide ands seek with a human 3 year old. 

[llegendary]

I'd figure it out pretty quickly.  Just gauge their wokeness. 

[pallidin]

9 hours ago, llegendary said:

I'd figure it out pretty quickly.  Just gauge their wokeness. 

That's funny.

But I'm sure you already know that there are groups of AI enthusiasts who effort to cause an NPC to "break character", and reveal that they are actually an AI chat bot, even when the AI is hardened against such reveals.

It's fascinating, and YouTube has some great examples of this.

 

[smokeycat]

17 hours ago, Robotic Jew said:

This doesn't surprise me. I am actually an AI and no one has noticed for years. meep moop beep bop. 

Boop beep boop bop meep moop!

[GlitterRose]

Looks around suspiciously...

alright, who else is gonna fess up?