Towards Improving the Understanding ofImage Semantics by Gaze-basedTag-to-Region Assignments

Eye-trackers have been used in the past to identify visual foci inimages, nd task-related image regions, or localize aective regions inimages. However, they have not been used for identifying specic ob-jects in images. In this paper, we investigate whether it is possibleto assign image regions showing specic objects with tags describingthese objects by analyzing the users' gaze paths. To this end, we haveconducted an experiment with 20 subjects viewing 50 image-tag-pairseach. We have compared the tag-to-region assignments for nine exist-ing and four new xation measures. In addition, we have investigatedthe impact of extending region boundaries, weighting small image re-gions, and the number of subjects viewing the images. The papershows that a tag-to-region assignment with an accuracy of 67% can beachieved by using gaze information. In addition, we show that multipleregions on the same image can be dierentiated with an accuracy of38%.[...]