User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse




Conference Paper

Temporal Shingling for Version Identification in Web Archives


Schenkel,  Ralf
Databases and Information Systems, MPI for Informatics, Max Planck Society;

External Ressource
No external resources are shared
Fulltext (public)
There are no public fulltexts stored in PuRe
Supplementary Material (public)
There is no public supplementary material available

Schenkel, R. (2010). Temporal Shingling for Version Identification in Web Archives. In C. Gurrin, Y. He, G. Kazai, U. Kruschwitz, S. Little, T. Roelleke, et al. (Eds.), Advances in Information Retrieval (pp. 508-519). Berlin: Springer. doi:10.1007/978-3-642-12275-0_44.

Cite as: http://hdl.handle.net/11858/00-001M-0000-000F-1534-D
Building and preserving archives of the evolving Web has been an important problem in research. Given the huge volume of content that is added or updated daily, identifying the right versions of pages to store in the archive is an important building block of any large-scale archival system. This paper presents temporal shingling, an extension of the well-established shingling technique for measuring how similar two snapshots of a page are. This novel method considers the lifespan of shingles to differentiate between important updates that should be archived and transient changes that may be ignored. Extensive experiments demonstrate the tradeoff between archive size and version coverage, and show that the novel method yields better archive coverage at smaller sizes than existing techniques.