The effect of explanations on perceived control and behaviors in intelligent systems

Intelligent systems often violate fundamental usability principles, such as control and transparency. Explanations have been shown to have a positive effect on transparency in such systems but little research exists as to how it may affect control. We set out to investigate how explanations may impact upon users' perceptions of control in an intelligent system. We conducted an empirical study in which 15 participants carried out a qualitative data analysis task using an intelligent system. Participants were divided into two groups: with and without explanations. Participants could indicate agreement or correct the system. Our results show that participants without explanations display more control-exerting behaviors but that there is no difference between conditions in participants' perception of control. We discuss our findings and implications for future work.