Bounded Rational Decision-Making in Feedforward Neural Networks

Leibfried, F; Braun, D

Local TagsRelease HistoryDetailsSummary

Bounded Rational Decision-Making in Feedforward Neural Networks

Leibfried, F., & Braun, D. (2016). Bounded Rational Decision-Making in Feedforward Neural Networks. In A. Ihler, & D. Janzing (Eds.), Uncertainty in Artificial Intelligence (pp. 407-416). Corvallis, OR, USA: AUAI Press.

Item is Released

show all hide all

Basic

show hide

Item Permalink: https://hdl.handle.net/21.11116/0000-0000-7A8E-8 Version Permalink: https://hdl.handle.net/21.11116/0000-0000-7A8F-7

Genre: Conference Paper

Files

show Files

Locators

show

hide

Locator:
Link (Any fulltext) Open Access status unknown

Description:
-

OA-Status:

Creators

show

hide

Creators:
Leibfried, F^{1, 2}, Author
Braun, D^{1, 2, 3}, Author

Affiliations:
1Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497794
2Research Group Sensorimotor Learning and Decision-Making, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497809
3Research Group Sensorimotor Learning and Decision-making, Max Planck Institute for Intelligent Systems, Max Planck Society, ou_1688138

Content

show

hide

Free keywords: -

Abstract: Bounded rational decision-makers transform sensory input into motor output under limited computational resources. Mathematically, such decision-makers can be modeled as information-theoretic channels with limited transmission rate. Here, we apply this formalism for the first time to multilayer feedforward neural networks. We derive synaptic weight update rules for two scenarios, where either each neuron is considered as a bounded rational decision-maker or the network as a whole. In the update rules, bounded rationality translates into information-theoretically motivated types of regularization in weight space. In experiments on the MNIST benchmark classification task for handwritten digits, we show that such information-theoretic regularization successfully prevents overfitting across different architectures and attains results that are competitive with other recent techniques like dropout, dropconnect and Bayes by backprop, for both ordinary and convolutional neural networks.

Details

show

hide

Language(s):

Dates: Date issued: 2016-06

Publication Status: Issued

Pages: -

Publishing info: -

Table of Contents: -

Rev. Type: -

Identifiers: BibTex Citekey: LeibfriedB2016

Degree: -

Event

show

hide

Title: 32nd Conference on Uncertainty in Artificial Intelligence (UAI 2016)

Place of Event: New York, NY, USA

Start-/End Date: -

Legal Case

show

Project information

show

Source 1

show

hide

Title: Uncertainty in Artificial Intelligence

Source Genre: Proceedings

Creator(s):
Ihler , A., Editor
Janzing, D., Editor

Affiliations:
-

Publ. Info: Corvallis, OR, USA : AUAI Press

Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 407 - 416 Identifier: ISBN: 978-0-9966431-1-5