The institute owns several eye movement measuring systems of the company SmartEye both in the fighter aircraft and in the helicopter simulator. These systems are used in the field of operator/pilot assistance to determine in real time where the operator is looking. This information can be used to generate task- and strain-adaptive assistance.

Measuring System

The measuring system consists of several cameras and infrared radiators. For a successful measurement the poses of the cameras and the infrared radiators as well as the poses of the screens are measured and entered into a 3D world model. During the measurement, the emitters emit infrared pulses which are reflected in the eyes of the surgeon. These reflections are captured by the cameras and the surgeon's viewing direction is derived from them. In the 3D world model, the penetration point of the line of sight vector with the screens can then be determined.

The resulting viewing position is made available in real time via a programming interface and can thus be further processed in our simulation environments. The coordinate can be used to determine the distribution of attention and individual viewing patterns. However, semantic information is missing in this measurement.

Since the semantic information is important for the above mentioned assistance, the coordinate of the eye movement measurement is combined with an abstract representation of the screen content. This makes it possible to determine in real time which buttons, chart elements or displays the pilot is looking at in the cockpit. These visual system interactions are essential information for Pilot Assistance Systems.