English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  How biased are maximum entropy models?

Macke, J., Murray, I., & Latham, P. (2012). How biased are maximum entropy models? In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, & K. Weinberger (Eds.), Advances in Neural Information Processing Systems 24 (pp. 2034-2042). Red Hook, NY, USA: Curran.

Item is

Files

show Files

Locators

show

Creators

show
hide
 Creators:
Macke, JH1, Author              
Murray, I, Author
Latham, P1, Author
Affiliations:
1Gatsby Computational Neuroscience Unit University College London, UK, ou_persistent22              

Content

show
hide
Free keywords: -
 Abstract: Maximum entropy models have become popular statistical models in neuroscience and other areas in biology, and can be useful tools for obtaining estimates of mutual information in biological systems. However, maximum entropy models fit to small data sets can be subject to sampling bias; i.e. the true entropy of the data can be severely underestimated. Here we study the sampling properties of estimates of the entropy obtained from maximum entropy models. We show that if the data is generated by a distribution that lies in the model class, the bias is equal to the number of parameters divided by twice the number of observations. However, in practice, the true distribution is usually outside the model class, and we show here that this misspecification can lead to much larger bias. We provide a perturbative approximation of the maximally expected bias when the true model is out of model class, and we illustrate our results using numerical simulations of an Ising model; i.e. the second-order maximum entropy distribution on binary data.

Details

show
hide
Language(s):
 Dates: 2012-01
 Publication Status: Published in print
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: BibTex Citekey: MackeML2012
 Degree: -

Event

show
hide
Title: Twenty-Fifth Annual Conference on Neural Information Processing Systems (NIPS 2011)
Place of Event: Granada, Spain
Start-/End Date: -

Legal Case

show

Project information

show

Source 1

show
hide
Title: Advances in Neural Information Processing Systems 24
Source Genre: Proceedings
 Creator(s):
Shawe-Taylor, J, Editor
Zemel, RS, Editor
Bartlett, P, Editor
Pereira, F, Editor
Weinberger, KQ, Editor
Affiliations:
-
Publ. Info: Red Hook, NY, USA : Curran
Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 2034 - 2042 Identifier: ISBN: 978-1-618-39599-3