CheatGPT?

July 20th, 2024| Topic: RaMbLeS | 0

Alan Mathison Turing (1912–1954) was an English polymath—mathematician, computer scientist, logician, cryptanalyst, philosopher and theoretical biologist. He is rightly considered the father of theoretical computer science. He is well known for having postulated the eponymous “Turing Test.”

In its original form a human interrogator has a text-based conversation with two people in another room (unknown to the interrogator, one is a man and the other a woman). The goal for the interrogator is to determine which of the two interrogators is a man and which a woman. Turing asked what would happen if the man + woman pair was replaced by a human + machine pair? Could the interrogator decide which was which? If not, the machine would have passed the test, masquerading successfully as a human.

Researchers from the School of Psychology and Clinical Language Sciences, at the University of Reading, Reading, UK, decided to check this out a bit differently.

There has been an explosion of interest in AI over the last few years, driven largely by the emergence of more complex ‘large language models’ (LLM’s), a type of AI designed to understand and generate human language. The most widely discussed of these, both in the academic literature and public discourse, is ChatGPT created by OpenAI (the latest model now being GPT-4).”

So they decided to focus on the on the threat AI poses in the educational sector. What if students use GPT-4 for an unsupervised exam assignment or coursework and submit the result?

Academic misconduct has always been a problem. And now LLMs and AI pose an extremely serious threat to academic integrity. Easily available, usually free, would such deceptions be detectable? Would examiners and coursework assessors be able to discern that the answers came from ChatGPT and not real, live humans? A “Turing Test” was needed. So the University of Reading scientists mixed 100 percent AI-written submissions along with 100% human (i.e., student)-written ones into the school’s examination system in five undergrad modules. Examiners and graders were unaware of this blending of non-human + human submissions. The results, published in PLOS-ONE, were stunning!

Overall, we found that 94% of AI submissions were undetected, even though we used AI in the most detectable way possible. AI submissions consistently outperformed real students, attaining grades which were on average around half a classification boundary higher. We found that in 83.4% of instances the grades achieved by AI submissions were higher than a random selection of the same number of student submissions.”

ChatGPT had become CheatGPT!

How are we university teachers ever going to fight this? And what about employers who are hiring? Can they trust academic credentials?

Added co-author Etienne Roesch:

Our study highlights the responsibility we have as producers and consumers of information. We need to double down on our commitment to academic and research integrity.”

How? Who knows?

But I know this …

You are not a God who takes pleasure in wickedness;
with You no evil dwells.
The arrogant cannot stand before Your eyes;
You hate all doers of harm.
You destroy those who speak falsehood;
the person of bloodshed and deceit Yahweh abhors.
Psalm 5:4–6

Instead, here is the one who delights God:

The innocent of hands and the pure of heart,
who has not lifted up his soul to emptiness
and has not sworn to deception.
He shall take away a blessing from Yahweh,
and righteousness from the God of his deliverance.
This is the generation of those who seek Him,
those who look for Your presence—Jacob.
Psalm 24:4–6

SOURCE: StudyFinds; PLOS-ONE