A gaze path cued retrospective thinking aloud technique in usability testing

Abstract

The present study explores the differences and similarities of concurrent thinking aloud and gaze path cued retrospective thinking aloud techniques in usability testing. Eight users were asked to complete a set of tasks (usability testing) individually, whilst their eye tracking data was collected with a Tobii eye tracker. During the testing, four users were asked to verbalise their thoughts (thinking aloud) when they were carrying out the tasks and four users were not given instructions to do so. After completing the tasks, users were shown their individual gaze paths recorded during the test session and they were asked to verbalise their thoughts (retrospective thinking aloud) while they watched the replay. After the retrospective viewing, users were interviewed briefly. Firstly, the usability problems observed in the present study were observed and reported. The results were compared to an independent usability test conducted by students of a usability evaluation methods course. Secondly, operational comments produced by each user in concurrent and retrospective thinking aloud conditions were recorded and analysed. It was noted that the gaze path cued retrospective thinking aloud technique was no superior over traditional usability testing using concurrent verbalisation in terms of quality of the usability problems obtained. The second part of the analysis revealed that users did produce significantly more operational comments retrospectively than concurrently. The results suggest that the gaze path cued retrospective thinking aloud condition could be used in usability testing to reveal usability problems.

Keywords: Human-computer interaction, Usability testing, Thinking aloud, Retrospective thinking aloud, Eye tracking.