English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
 
 
DownloadE-Mail
  Using cognitive psychology to understand GPT-3

Binz, M., & Schulz, E. (submitted). Using cognitive psychology to understand GPT-3.

Item is

Files

show Files

Locators

show
hide
Locator:
https://psyarxiv.com/6dfgk/download (Any fulltext)
Description:
-
OA-Status:

Creators

show
hide
 Creators:
Binz, M1, Author           
Schulz, E1, Author           
Affiliations:
1Research Group Computational Principles of Intelligence, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_3189356              

Content

show
hide
Free keywords: -
 Abstract: We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3's decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3's behavior is impressive: it solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multi-armed bandit task, and shows signatures of model-based reinforcement learning. Yet we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. These results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.

Details

show
hide
Language(s):
 Dates: 2022-06
 Publication Status: Submitted
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: DOI: 10.31234/osf.io/6dfgk
 Degree: -

Event

show

Legal Case

show

Project information

show

Source

show