Help Privacy Policy Disclaimer
  Advanced SearchBrowse





Interrupting Options: Minimizing Decision Costs via Temporal Commitment and Low-Level Interrupt

There are no MPG-Authors in the publication available
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)
There are no public fulltexts stored in PuRe
Supplementary Material (public)
There is no public supplementary material available

Lloyd, K., & Dayan, P. (2017). Interrupting Options: Minimizing Decision Costs via Temporal Commitment and Low-Level Interrupt. Poster presented at 3rd Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2017), Ann Arbor, MI, USA.

Cite as: https://hdl.handle.net/21.11116/0000-0004-DADC-F
Ideal decision-makers should constantly monitor all sources of external information about oppor-
tunities and threats, and thus be able to redetermine their choices promptly in the face of change. However,
perpetual monitoring and reassessment can impose substantial computational costs, making them imprac-
tical for animals and machines alike. The obvious alternative of committing for extended periods of time
to particular courses of action can be dangerous and wasteful. Here, we explore the intermediate option of
making provisional temporal commitments, but engaging in limited broader observation with the possibility
of interruption - effectively a form of option (Sutton et al., Artificial Intelligence, 112, 181-211, 1999). We
illustrate the issues using a simple example of foraging under predation risk, in which a decision-maker
must trade off energetic gain against the danger of predation. We first show that an agent equipped with the
capacity for self-interruption outperforms an agent without this capacity. Next, we observe that the optimal
interruption policy is particularly uncomplicated in our example, and show that performance is essentially
identical when using an approximation based on placing simple thresholds in belief space. This is consistent
with the idea that a relatively simple, low-level mechanism can prompt behavioural interruption, analogous
to the operation of peripherally-induced interrupts in digital computers. We interpret our results in the con-
text of putative neural mechanisms, such as noradrenergic neuromodulation, and diseases of distractibility and roving attention.