Help Privacy Policy Disclaimer
  Advanced SearchBrowse





Simplified Data Wrangling with ir_datasets


Yates,  Andrew
Databases and Information Systems, MPI for Informatics, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

(Preprint), 9KB

Supplementary Material (public)
There is no public supplementary material available

MacAvaney, S., Yates, A., Feldman, S., Downey, D., Cohan, A., & Goharian, N. (2021). Simplified Data Wrangling with ir_datasets. Retrieved from https://arxiv.org/abs/2103.02280.

Cite as: https://hdl.handle.net/21.11116/0000-0009-6679-D
Managing the data for Information Retrieval (IR) experiments can be
challenging. Dataset documentation is scattered across the Internet and once
one obtains a copy of the data, there are numerous different data formats to
work with. Even basic formats can have subtle dataset-specific nuances that
need to be considered for proper use. To help mitigate these challenges, we
introduce a new robust and lightweight tool (ir_datasets) for acquiring,
managing, and performing typical operations over datasets used in IR. We
primarily focus on textual datasets used for ad-hoc search. This tool provides
both a Python and command line interface to numerous IR datasets and
benchmarks. To our knowledge, this is the most extensive tool of its kind.
Integrations with popular IR indexing and experimentation toolkits demonstrate
the tool's utility. We also provide documentation of these datasets through the
ir_datasets catalog: https://ir-datasets.com/. The catalog acts as a hub for
information on datasets used in IR, providing core information about what data
each benchmark provides as well as links to more detailed information. We
welcome community contributions and intend to continue to maintain and grow
this tool.