Two steps to risk sensitivity

Dayan, P; Gagne, C

Local TagsRelease HistoryDetailsSummary

Two steps to risk sensitivity

Dayan, P., & Gagne, C. (2022). Two steps to risk sensitivity. In M. Ranzato, A. Beygelzimer, P. Liang, J. Vaughan, & Y. Dauphin (Eds.), Advances in Neural Information Processing Systems 34: 35th Conference on Neural Information Processing Systems (NeurIPS 2021) (pp. 22209-22220). Red Hook, NY, USA: Curran.

Item is Released

show all hide all

Basic

show hide

Item Permalink: https://hdl.handle.net/21.11116/0000-0009-8A1E-B Version Permalink: https://hdl.handle.net/21.11116/0000-000A-8245-5

Genre: Conference Paper

Files

show Files

Locators

show

hide

Locator:
https://proceedings.neurips.cc/paper/2021/file/ba530cdf0a884348613f2aaa3a5ba5e8-Paper.pdf (Publisher version) Open Access status unknown

Description:
-

OA-Status:

Creators

show

hide

Creators:
Dayan, P¹, Author
Gagne, C¹, Author

Affiliations:
1Department of Computational Neuroscience, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_3017468

Content

show

hide

Free keywords: -

Abstract: Distributional reinforcement learning (RL) – in which agents learn about all the possible long-term consequences of their actions, and not just the expected value – is of great recent interest. One of the most important affordances of a distributional view is facilitating a modern, measured, approach to risk when outcomes are not completely certain. By contrast, psychological and neuroscientific investigations into decision making under risk have utilized a variety of more venerable theoretical models such as prospect theory that lack axiomatically desirable properties such as coherence. Here, we consider a particularly relevant risk measure for modeling human and animal planning, called conditional value-at-risk (CVaR), which quantifies worst-case outcomes (e.g., vehicle accidents or predation). We first adopt a conventional distributional approach to CVaR in a sequential setting and reanalyze the choices of human decision-makers in the well-known two-step task, revealing substantial risk aversion that had been lurking under stickiness and perseveration. We then consider a further critical property of risk sensitivity, namely time consistency, showing alternatives to this form of CVaR that enjoy this desirable characteristic. We use simulations to examine settings in which the various forms differ in ways that have implications for human and animal planning and behavior.

Details

show

hide

Language(s):

Dates: Accepted: 2021-10Date issued: 2022-05

Publication Status: Issued

Pages: -

Publishing info: -

Table of Contents: -

Rev. Type: -

Identifiers: -

Degree: -

Event

show

hide

Title: Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021)

Place of Event: -

Start-/End Date: 2021-12-06 - 2021-12-14

Legal Case

show

Project information

show

Source 1

show

hide

Title: Advances in Neural Information Processing Systems 34: 35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Source Genre: Proceedings

Creator(s):
Ranzato, M, Editor
Beygelzimer, A, Editor
Liang, PS, Editor
Vaughan, JW, Editor
Dauphin, Y, Editor

Affiliations:
-

Publ. Info: Red Hook, NY, USA : Curran

Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 22209 - 22220 Identifier: ISBN: 978-1-7138-4539-3