English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Paper

Label-Descriptive Patterns and their Application to Characterizing Classification Errors

MPS-Authors
/persons/resource/persons229482

Fischer,  Jonas
Databases and Information Systems, MPI for Informatics, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

arXiv:2110.09599.pdf
(Preprint), 568KB

Supplementary Material (public)
There is no public supplementary material available
Citation

Hedderich, M., Fischer, J., Klakow, D., & Vreeken, J. (2021). Label-Descriptive Patterns and their Application to Characterizing Classification Errors. Retrieved from https://arxiv.org/abs/2110.09599.


Cite as: https://hdl.handle.net/21.11116/0000-0009-B127-3
Abstract
State-of-the-art deep learning methods achieve human-like performance on many
tasks, but make errors nevertheless. Characterizing these errors in easily
interpretable terms gives insight into whether a model is prone to making
systematic errors, but also gives a way to act and improve the model. In this
paper we propose a method that allows us to do so for arbitrary classifiers by
mining a small set of patterns that together succinctly describe the input data
that is partitioned according to correctness of prediction. We show this is an
instance of the more general label description problem, which we formulate in
terms of the Minimum Description Length principle. To discover good pattern
sets we propose the efficient and hyperparameter-free Premise algorithm, which
through an extensive set of experiments we show on both synthetic and
real-world data performs very well in practice; unlike existing solutions it
ably recovers ground truth patterns, even on highly imbalanced data over many
unique items, or where patterns are only weakly associated to labels. Through
two real-world case studies we confirm that Premise gives clear and actionable
insight into the systematic errors made by modern NLP classifiers.