Item

Abstract

The notion of individual fairness requires that similar people receive
similar treatment. However, this is hard to achieve in practice since it is
difficult to specify the appropriate similarity metric. In this work, we
attempt to learn such similarity metric from human annotated data. We gather a
new dataset of human judgments on a criminal recidivism prediction (COMPAS)
task. By assuming the human supervision obeys the principle of individual
fairness, we leverage prior work on metric learning, evaluate the performance
of several metric learning methods on our dataset, and show that the learned
metrics outperform the Euclidean and Precision metric under various criteria.
We do not provide a way to directly learn a similarity metric satisfying the
individual fairness, but to provide an empirical study on how to derive the
similarity metric from human supervisors, then future work can use this as a
tool to understand human supervision.