GPT-3 can reason about as well as a college student, psychologists report

[Kamaji]

GPT-3 can reason about as well as a college student, psychologists report

But does the technology mimic human reasoning or is it using a fundamentally new cognitive process?

Date:  July 31, 2023
Source:  University of California - Los Angeles
Summary:  The artificial intelligence language model GPT-3 performed as well as college students in solving certain logic problems like those that appear on standardized tests. The researchers who conducted the experiment write that the results prompt the question of whether the technology is mimicking human reasoning or using a new type of cognitive process. Solving that question would require access to the software that underpins GPT-3 and other AI software.

People solve new problems readily without any special training or practice by comparing them to familiar problems and extending the solution to the new problem. That process, known as analogical reasoning, has long been thought to be a uniquely human ability.

But now people might have to make room for a new kid on the block.

Research by UCLA psychologists shows that, astonishingly, the artificial intelligence language model GPT-3 performs about as well as college undergraduates when asked to solve the sort of reasoning problems that typically appear on intelligence tests and standardized tests such as the SAT. The study is published in Nature Human Behaviour.

But the paper's authors write that the study raises the question: Is GPT-3 mimicking human reasoning as a byproduct of its massive language training dataset or it is using a fundamentally new kind of cognitive process?

Without access to GPT-3's inner workings -- which are guarded by OpenAI, the company that created it -- the UCLA scientists can't say for sure how its reasoning abilities work. They also write that although GPT-3 performs far better than they expected at some reasoning tasks, the popular AI tool still fails spectacularly at others.

"No matter how impressive our results, it's important to emphasize that this system has major limitations," said Taylor Webb, a UCLA postdoctoral researcher in psychology and the study's first author. "It can do analogical reasoning, but it can't do things that are very easy for people, such as using tools to solve a physical task. When we gave it those sorts of problems -- some of which children can solve quickly -- the things it suggested were nonsensical."

*  *  *

Source:  https://www.sciencedaily.com/releases/2023/07/230731110750.htm