English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Risk-sensitive planning and sampling in simple and complex generative models

Gagne, C. (2022). Risk-sensitive planning and sampling in simple and complex generative models. Talk presented at 5th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2022) Workshop. Providence, RI, USA. 2022-06-11.

Item is

Files

show Files

Locators

show
hide
Description:
-
OA-Status:

Creators

show
hide
 Creators:
Gagne, C1, Author           
Affiliations:
1Department of Computational Neuroscience, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_3017468              

Content

show
hide
Free keywords: -
 Abstract: In a world replete with past disappointments and future threats and dangers, an adaptive agent might ponder the causes of its miserable lot and ways of mitigating prospective problems. Pondering the past can be seen as a form of rumination, and pondering the future as a form of worry. Although worry therefore shares many characteristics with planning, and has indeed been described in exactly those terms by both clinical psychologists and those who worry frequently, there have been few attempts to formalize it in the modern language of model-based computation and simulation. In this talk, I will provide a formal framework for characterizing the foundations of adaptive worry, and show applications of this framework at two radically different cognitive scales. Prospective problems typically arise as unlikely, unpleasant, tail, events from distributions of possible outcomes. In our framework, these are conceptualized in terms of a generalized notion of worst-case risk (called CVaR), parametrized by a form of risk aversion. Rational risk-averse agents will plan to reduce worst-case risks as best as possible; we relate such planning in a very simple world model to a form of worry that involves thinking through problematic scenarios and their possible solutions. Given evidence that various forms of planning can occur as unconscious offline replay, we can predict that individual differences in the contents of neural replay will correlate with behavioral measures of risk-aversion. However, the simplicity of the world model precludes investigation of more naturalistic, conscious, worry, which is often verbal in nature. We therefore turn to what are increasingly being seen as the most sophisticated programmatic world models, namely large-scale language systems, and, as an example of contentful worry, show how we can use CVaR to capture the phenomena of catastrophizing, a type of worry characterized by progressively worsening ‘what-if’ chains-of-thought.

Details

show
hide
Language(s):
 Dates: 2022-06
 Publication Status: Published online
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: -
 Degree: -

Event

show
hide
Title: 5th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2022) Workshop
Place of Event: Providence, RI, USA
Start-/End Date: 2022-06-11
Invited: Yes

Legal Case

show

Project information

show

Source

show