Skip to Main content Skip to Navigation
Journal articles

The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts

Abstract : Previous research has shown that listeners follow speaker gaze to mentioned objects in a shared environment to ground referring expressions, both for human and robot speakers. What is less clear is whether the benefit of speaker gaze is due to the inference of referential intentions (Staudte and Crocker, 2011) or simply the (reflexive) shifts in visual attention. That is, is gaze special in how it affects simultaneous utterance comprehension? In four eye-tracking studies we directly contrast speech-aligned speaker gaze of a virtual agent with a non-gaze visual cue (arrow). Our findings show that both cues similarly direct listeners’ attention and that listeners can benefit in utterance comprehension from both cues. Only when they are similarly precise, however, does this equality extend to incongruent cueing sequences: that is, even when the cue sequence does not match the concurrent sequence of spoken referents can listeners benefit from gaze as well as arrows. The results suggest that listeners are able to learn a counter-predictive mapping of both cues to the sequence of referents. Thus, gaze and arrows can in principle be applied with equal flexibility and efficiency during language comprehension.
Document type :
Journal articles
Complete list of metadata

https://hal-uphf.archives-ouvertes.fr/hal-03578505
Contributor : Julie Cagniard Connect in order to contact the contributor
Submitted on : Thursday, February 17, 2022 - 12:29:42 PM
Last modification on : Friday, February 18, 2022 - 3:36:42 AM

Identifiers

Collections

Citation

Maria Staudte, Matthew W. Crocker, Alexis Heloir, Michael Kipp. The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts. Cognition, Elsevier, 2014, 133 (1), pp.317-328. ⟨10.1016/j.cognition.2014.06.003⟩. ⟨hal-03578505⟩

Share

Metrics

Record views

3