The effects of mixing machine learning and human judgment

Reviewer: Jonathan K. Millen

Automated risk assessment systems are often used in situations that require human judgment. One motivation for doing this is to remove human bias. Even when the automated system has been shown to be more accurate than human assessments, a team combining system and human decisions has occasionally been proven to be better for some applications and collaboration modes. The presented experiment involves criminal recidivism assessments using a well-known algorithmic system, COMPAS. Human subjects were recruited according to their interest, rather than their expertise, in criminal justice. As expected, COMPAS was more accurate by itself than the humans, given the same data from real court cases. The pertinent question, however, is whether, and in what way, the human results are influenced by being told the COMPAS results prior to making their own assessments. In the first trial, humans were told the COMPAS recidivism risk scores, and their own scores were (on the average) different and less accurate. In the second trial, the experimenters investigated an "anchoring" effect by providing COMPAS results that were deliberately altered higher or lower. The average human scores differed significantly in the direction of the altered COMPAS scores that they were given. These results are not world shaking, and the article is short, less than seven pages. Yet it was as absorbing as a mystery novel, and raised all sorts of questions that aroused the hope of further work. For example, the cited success of teaming involves a feedback loop between the humans and the system, which was not tried here. Also, would the experiment have come out differently with expert humans on the team There is so much more to learn.