The eyes have often been described as a window into the human mind. When we hear spoken language about things present in our immediate surroundings, our gaze (largely automatically) searches for items or persons that speakers refer to. When we speak to describe something that we see, our eyes fixate upon parts of what we perceive as our description unfolds. Since where we look is tightly linked to processing linguistic information about what we see, tracking eye movements has become a well-established tool for psycholinguists to learn more about the time course of listening and speaking.

When looking at a scene, the eyes regularly perform jumps, so-called saccades, between fixations, when the gaze rests upon a particular point. During a saccade, we do not perceive anything. What we perceive of as a stable image of our surroundings is basically an image computed in our brain from the multiple rests or fixations.

Our eye tracking lab is located at the "Haus des Hörens" audiological research center. We use an SR Research EyeLink CL Remote eye tracking device that can record eye movements and fixations at a speed of up to 1000 Hz. The device can be used with a headrest and in remote mode. Tracking the gaze of a subject with this system works through a high-speed image analysis analysis of an infrared video stream. The device consists of three parts: an infrared camera pointed at the subject, an infrared light source, and the computer running the analysis software. In order to estimate the position of the gaze, the system keeps track of the elliptical shape of the pupil. The infrared light illuminating the subject creates a characteristic reflection on the cornea, which is also recorded. After an initial calibration, the pupil position and shape information together with the position of the corneal reflection allow for a precise mapping of gaze onto the stimulus picture or scene shown to the subject.

The setup is currently used for language perception experiments where subjects listen to speech of varying complexity under different noise conditions. We show the subjects b/w scene drawings from our OLACS corpus that either match or do not match the sentence they hear and track the course of fixations on the visual display. In a different experiment we are looking for the distribution of gazes across depicted scenes while the subjects are describing the scene verbally.

Looking closely, speaking is quite a complex task. We have to think of what we want to talk about, plan an appropriate way of saying what we want to communicate, select the right words, build a syntactic structure and put the words into the right place, and finally coordinate the articulatory muscles to produce the planned words and sentences in the correct order without spluttering. Most of these processes happen automatically and within a few hundred milliseconds.

Speaking under noisy circumstances can be a bit more challenging. Speakers normally tend to counteract concurrent noise by changing pitch and intensity of their voice. We are interested in the possible effects (automatic or strategic) that adverse communication settings and sensorineural perception problems of speakers with hearing impairment might have on central cognitive processes implied in planning and formulating sentences. Do speakers perhaps "simplify" their language under noise?

In order to test this question empirically, we developed different experimental methods to elicit spontaneous and controlled speech under different acoustic background conditions. In our language production experiments, speakers are presented with a variety of materials, mostly pictures, and we ask the participants to describe the materials in different ways.

For example, we recently conducted a study where a couple of subjects with and without hearing impairment were asked to describe simple drawings from the OLACS corpus in one sentence. The acoustic background was manipulated using the communication and acoustics simulator (CAS) facility at the Oldenburg "Haus des Hörens" audiological research centre.