Investigating whether people identify how suitable a data visualization is for answering specific analysis questions

Choosing the type of chart to represent data may be challenging. Various books and websites provide catalogs of data visualizations to help choose the correct chart, usually taking into account data types and general tasks. However, they often do not consider the analyst's goal, as expressed by specific questions. In this work, we studied how participants assess the suitability of certain data visualizations for answering specific analysis questions before and after being exposed to related guides that mimic what is often found in online catalogs. We also investigated whether they could assess suggestions (both textual and visual) to change the charts to better answer the questions or recommend suitable alternative modifications to improve them. We discovered that, for basic charts, the participants could identify whether the visualization is suitable to answer the posed question. However, when they found it unsuitable, they could not always recommend changes to improve them. When presented with alternative visualizations, the participants seemed to provide assessments in line with the literature more often than when presented with textual descriptions of recommended changes, but the difference was not significant. Our results showed that guidelines, either in a textual or visual format, may not help novices effectively relate analysis questions to specific chart properties, and may even confuse them and lead them to make mistakes they might not have made otherwise.