Project Overview

A gaze-contingent display modifies the rendered information based on data about the user’s gaze, which is gathered with the help of an attached eye-tracker that can sense and compute the gaze location and other metrics (e.g., blinks, fixations or saccades). The aim is to do this in a way that does not allow the user to perceive the system reacting to his gaze, but instead create an holistically changed impression of the display.

In the past this has mostly been proposed for performance gain (i.e., improve rendering times) by selectively omitting details in unattended parts of a display. Our goal, however, is to find perceptual modifications that augment the displayed information and thus create an enhanced viewing experience for
the user.

Example of a gaze-contingent multi-resolution image

Example of a gaze-contingent multi-resolution image rendered in two different states. In the left image the user is looking at the sheep in the left part of the image, while in the right image the user looks at at the mountain in the background. High detail is only rendered where the user is looking, while the rest of the image is rendered sparsely.

Enhancing Depth Perception

Blur in images can create the sensation of depth because it emulates an optical property of the eye; namely, the limited depth of field created by the eye’s lens. When the human eye looks at an object, this object appears sharp on the retina, but objects at different distances appear blurred. A GCD enables us to reproduce dynamic depth of field, providing an additional way of conveying depth. We investigated gaze-contingent depth of field as a method to produce realistic 3D images, and analysed how effectively people can use it to perceive depth.

Commonly used techniques to produce 3D effects (i.e., binocular disparity) are incomplete: many visual cues are missing. The lack of those cues can lead to visual fatigue and eye strain. Other depth cues, like DOF (see Fig. 1 for example), can be used to influence depth perception and remedy those issues. DOF is already used in static images and movies to convey depth. DOF has been shown to influence perception of depth in human vision, therefore simulating DOF might be beneficial for perception of depth and realism.

blur_wide

Example of a scene recorded with a camera at three different focal points, showing the effects of the limited depth of field.

Scene captured using an optical systems with limited DOF focused on different distances showing the resulting defocus blur. To answer the question of whether GC DOF is an effective and reliable method to convey depth information we designed a quantitative, controlled experiment that investigates depth perception accuracy through a depth comparison task. We found that GC DOF increases subjective perceived realism and depth and can contribute to the perception of ordinal depth and distance between objects, but it is limited in its accuracy. Full details of this investigation can be found in Michael Mauderer , Simone Conte , Miguel A. Nacenta and Dhanraj Vishwanath. Depth Perception with Gaze-contingent Depth of Field. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2014).

Enhancing Colour Perception

The human perception of colour is a complex topic and much effort has been put into determining how we perceive certain stimuli in varying conditions. While the actual physical stimulus, the spectral power distribution of the observed light, plays a huge role, it is not the only factor. This can clearly be seen when observing optical illusions that manage to trick our color perception. Context, spatial configuration, ambient light and texture all influence how we perceive colour.

simultanous_contrast_samples

Examples of simulations contrast. The centres of each column have the same colour, but can appear different through the varying surround.

Using a GCD it is easy to change the background of an object depending on the gaze location. By applying the observations and models from colour science, we can thus manipulate the perceived colour of a given stimulus in a predictable way. Using this technique it should be possible to extend the gamut of a display and make colours more distinguishable. This could be used to allow for better discrimination of colour based information, e.g., in information visualisation, or to display high dynamic range images.