English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Book Chapter

Statistical Laws in Linguistics

MPS-Authors
/persons/resource/persons145764

Altmann,  Eduardo G.
Max Planck Institute for the Physics of Complex Systems, Max Planck Society;

/persons/resource/persons184524

Gerlach,  Martin
Max Planck Institute for the Physics of Complex Systems, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)
There are no public fulltexts stored in PuRe
Supplementary Material (public)
There is no public supplementary material available
Citation

Altmann, E. G., & Gerlach, M. (2016). Statistical Laws in Linguistics. In M. Degli Esposti, E. G. Altmann, & F. Pachet (Eds.), Creativity and Universitality in Language (pp. 7-26). Cham: Springer. doi:10.1007/978-3-319-24403-7_2.


Cite as: https://hdl.handle.net/11858/00-001M-0000-002B-AD21-5
Abstract
Zipf's law is just one out of many universal laws proposed to describe statistical regularities in language. Here we review and critically discuss how these laws can be statistically interpreted, fitted, and tested (falsified). The modern availability of large databases of written text allows for tests with an unprecedent statistical accuracy and also for a characterization of the fluctuations around the typical behavior. We find that fluctuations are usually much larger than expected based on simplifying statistical assumptions (e.g., independence and lack of correlations between observations). These simplifications appear also in usual statistical tests so that the large fluctuations can be erroneously interpreted as a falsification of the law. Instead, here we argue that linguistic laws are only meaningful (falsifiable) if accompanied by a model for which the fluctuations can be computed (e.g., a generative model of the text). The large fluctuations we report show that the constraints imposed by linguistic laws on the creativity process of text generation are not as tight as one could expect.