Evaluating the Recommendations of LLMs to Teach a Visualization Technique using Bloom's Taxonomy


Joshi A., Srinivas C., FIRAT E. E., Laramee R. S.

IS and T International Symposium on Electronic Imaging 2024: Visualization and Data Analysis, VDA 2024, California, United States Of America, 21 - 25 January 2024, vol.36 identifier

  • Publication Type: Conference Paper / Full Text
  • Volume: 36
  • Doi Number: 10.2352/ei.2024.36.1.vda-360
  • City: California
  • Country: United States Of America
  • Çukurova University Affiliated: Yes

Abstract

Large Language Models (LLMs) have demonstrated a huge impact on education and literacy in recent years. We evaluated the recommendations provided by two popular LLMs (OpenAI's ChatGPT and Google's Bard) to educate novices on the topic of Parallel Coordinate Plots (PCPs) using Bloom's taxonomy. We present the results of a human-expert evaluation of the recommendations provided by both the LLMs with experts from the visualization literacy field. Based on the analysis of the expert evaluation, we found that while both the LLMs provided some relevant and practical recommendations, some of the recommendations were either too difficult for novices or were in the wrong cognitive process (according to Bloom's taxonomy). In some cases, the hallucinations led to recommendations that were completely inapplicable to Parallel Coordinate Plots literacy.