English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Prospective and retrospective temporal difference learning

Dayan, P. (2009). Prospective and retrospective temporal difference learning. Network: Computation in Neural Systems, 20(1), 32-46. doi:10.1080/09548980902759086.

Item is

Files

show Files

Locators

show
hide
Description:
-
OA-Status:

Creators

show
hide
 Creators:
Dayan, P1, Author           
Affiliations:
1External Organizations, ou_persistent22              

Content

show
hide
Free keywords: -
 Abstract: A striking recent finding is that monkeys behave maladaptively in a class of tasks in which they know that reward is going to be systematically delayed. This may be explained by a malign Pavlovian influence arising from states with low predicted values. However, by very carefully analyzing behavioral data from such tasks, La Camera and Richmond (2008) observed the additional important characteristic that subjects perform differently on states in the task that are at equal distances from the future reward, depending on what has happened in the recent past. The authors pointed out that this violates the definition of state value in the standard reinforcement learning models that are ubiquitous as accounts of operant and classical conditioned behavior; they suggested and analyzed an alternative temporal difference (TD) model in which past and future are melded. Here, we show that, in fact, a standard TD model can actually exhibit the same behavior, and that this avoids deleterious consequences for choice. At the heart of the model is the average reward per step, which acts as a baseline for measuring immediate rewards. Relatively subtle changes to this baseline occasioned by the past can markedly influence predictions and thus behavior.

Details

show
hide
Language(s):
 Dates: 2009-03
 Publication Status: Issued
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: DOI: 10.1080/09548980902759086
 Degree: -

Event

show

Legal Case

show

Project information

show

Source 1

show
hide
Title: Network: Computation in Neural Systems
  Other : Netw.-Comput. Neural Syst.
Source Genre: Journal
 Creator(s):
Affiliations:
Publ. Info: Bristol : IOP Pub.
Pages: - Volume / Issue: 20 (1) Sequence Number: - Start / End Page: 32 - 46 Identifier: ISSN: 0954-898X
CoNE: https://pure.mpg.de/cone/journals/resource/954925576018