A critical assumption of basic research is that what we measure in the lab with abstract and controlled stimuli can be generalized to natural human behavior. Especially, for basic oculomotor control, most paradigms measure eye movements to white points or Gaussian Blobs on a gray background. Such experiments provided great insights and allow to tap into individual mechanisms, but are far away from our typical complex natural input.

In a series of studies we tried to directly compare oculomotor control to stimuli varying in complexity, ranging from moving dots to naturalistic videos.

What drives predictive eye movements in naturalistic scenes

In our recent study, we took our ice hockey videos and manipulated them to investigate which factors drive the observed predictive advantage.

In the first experiment we manipulated two factors: The amount of peripheral information that was available, and the level of expertise observes had with the stimuli. We observed that the more peripheral information became available, the more predictive eye movements got. In addition, there was an interaction with the level of expertise: while there was no difference for the Disk-Condition, in the conditions with naturalistic context, expert ice hockey-fans showed even more preditive behavior.

In the second experiment, we investigated which type of information is used to make predictive eye movements. For that we manipulated the contextual cues about the player positions and the reliability of kinematic cues about player movements. When representing the player positions by squares, we did not observe a significant improvement over the Disk-Condition. The critical factors seems to be access to analyzing the movement of the players. When we impaired the causal structure of the video by playing it in reverse, oculomotor behaviour was strongly impaired. However, when flipping the video vertically, predictive eye movements remained comparable to the Video-Condition, showing that if the analysis of player movements and the understanding of the scene is still possible, predictive eye movements can occur.

Naturalistic stimuli guide predictive eye movements

How do we track a moving target? Due to internal delays in processing incoming information, we constantly need to predict to bring our eyes to the right place at the right time. In order to be useful, our oculomotor system must minimize delay with respect to the dynamic events in the visual scene. The ability to do so has been demonstrated in situations where we are in control of these events, for example when we are making a sandwich or tea , and when we are active participants, for example when hitting a cricket ball. But what about scenes with complex dynamics that we do not control or directly take part in, like a hockey game we are watching as a spectator?

Here we compared tracking of a complex target trajectory (the movement of a puck in an ice hockey game), once in a condition mimicking a typical lab experiment (a dot on a gray screen) and once in the natural context: during a video of an ice hockey game.

We then used a cross-correlation approach to estimate the delay between the eye and puck movement and observed that while in the disk condition, observers lagged behind the target movement by about 180 ms, in the video condition the average delay was close to zero. In certain situations, for example passes between players, gaze was even ahead of the puck already fixating the receiving player to assess what is going to happen next. Our results reveal fundamentally different eye movement strategies when viewing complex but meaningful action sequences compared to strategies seen when viewing simple target motion sequences predominantly used in prior eye movement studies. The integration of contextual cues that are present in naturalistic images into eye movement behavior is something that is typically missed in classical lab experiments and that can give some insights into the brain of a hockey fan.

Goettker, A., Pidaparthy, H., Braun, D. I., Elder, J. H., & Gegenfurtner, K. R. (2021). Ice hockey spectators use contextual cues to guide predictive eye movements. Current Biology, 31(16), R991-R992.

From Gaussian Blobs to naturalistic videos

Is tracking of a Gaussian blob the same as tracking a duck flying across a river? No? Even if we match the movement trajectory? Here we tried to assess exactly this question. Across a set of videos filmed in the real world, we compared how people tracked the movements of different targets (like the duck) with a Gaussian Blob that moved along the same trajectory. We observed some interesting differences: Pursuit was more accurate for the natural stimuli, but latencies were longer, presumably due to a more difficult target selection. Across both conditions, saccades and pursuit were most accurate when they were collinear. This suggest that our knowledge of the world, e.g. how a person moves or how a duck flies, is integrated into our eye movements.

Goettker, A., Agtzidis, I., Braun, D. I., Dorr, M., & Gegenfurtner, K. R. (2020). From Gaussian blobs to naturalistic videos: Comparison of oculomotor behavior across different stimulus complexities. Journal of vision, 20(8), 26-26.