hide
Free keywords:
Statistics, Machine Learning, stat.ML,Computer Science, Computers and Society, cs.CY,Computer Science, Learning, cs.LG
Abstract:
Societies often rely on human experts to take a wide variety of decisions
affecting their members, from jail-or-release decisions taken by judges and
stop-and-frisk decisions taken by police officers to accept-or-reject decisions
taken by academics. In this context, each decision is taken by an expert who is
typically chosen uniformly at random from a pool of experts. However, these
decisions may be imperfect due to limited experience, implicit biases, or
faulty probabilistic reasoning. Can we improve the accuracy and fairness of the
overall decision making process by optimizing the assignment between experts
and decisions?
In this paper, we address the above problem from the perspective of
sequential decision making and show that, for different fairness notions from
the literature, it reduces to a sequence of (constrained) weighted bipartite
matchings, which can be solved efficiently using algorithms with approximation
guarantees. Moreover, these algorithms also benefit from posterior sampling to
actively trade off exploitation---selecting expert assignments which lead to
accurate and fair decisions---and exploration---selecting expert assignments to
learn about the experts' preferences and biases. We demonstrate the
effectiveness of our algorithms on both synthetic and real-world data and show
that they can significantly improve both the accuracy and fairness of the
decisions taken by pools of experts.
