Eye Movement-Based Human Computer Interaction Techniques
Published on Feb 21, 2020
As with other areas of user interface design, considerable leverage can be obtained by drawing analogies that use people’s already-existing skills for operating in the natural environment and searching for ways to apply them to communicating with a computer.
Direct manipulation interfaces have enjoyed great success, particularly with novice users, largely because they draw on analogies to existing human skills (pointing, grabbing, moving objects in physical space), rather than trained behaviors; and virtual realities offer the promise of usefully exploiting people’s existing physical navigation and manipulation abilities.
These notions are more difficult to extend to eye movement-based interaction, since few objects in the real world respond to people’s eye movements. The principal exception is, of course, other people: they detect and respond to being looked at directly and, to a lesser and much less precise degree, to what else one may be looking at. In describing eye movement-based human-computer interaction we can draw two distinctions: one is in the nature of the user’s eye movements and the other, in the nature of the responses.
Each of these could be viewed as natural (that is, based on a corresponding real-world analogy) or unnatural (no real world counterpart):
• Within the world created by an eye movement-based interface, users could move their eyes to scan the scene, just as they would a real world scene, unaffected by the presence of eye tracking equipment (natural eye movement, on the eye movement axis). The alternative is to instruct users of the eye movement-based interface to move their eyes in particular ways, not necessarily those they would have employed if left to their own devices, in order to actuate the system (unnatural or learned eye movements).
• On the response axis, objects could respond to a user’s eye movements in a natural way, that is, the object responds to the user’s looking in the same way real objects do. As noted, there is a limited domain from which to draw such analogies in the real world. The alternative is unnatural response, where objects respond in ways not experienced in the real world. The natural eye movement/natural response area is a difficult one, because it draws on a limited and subtle domain, principally how people respond to other people’s gaze.
Starker and Bolt provide an excellent example of this mode, drawing on the analogy of a tour guide or host who estimates the visitor’s interests by his or her gazes. In the work described in this chapter, we try to use natural (not trained) eye movements as input, but we provide responses unlike those in the real world. This is a compromise between full analogy to the real world and an entirely artificial interface. We present a display and allow the user to observe it with his or her normal scanning mechanisms, but such scans then induce responses from the computer not normally exhibited by real world objects. Most previous eye movement-based systems have used learned ("unnatural") eye movements for operation and thus, of necessity, unnatural responses.
THE SKILLS NEEDED BY THE USER
This system is mainly developed for those who lack the use of their hands or voice. Only requirements to operate the Eyegaze are control of at least one eye with good vision & ability to keep head fairly still. Eyegaze Systems are in use around the world. Its users are adults and children with cerebral palsy, spinal cord injuries, brain injuries, ALS, multiple sclerosis, brainstem strokes, muscular dystrophy, and Werdnig Hoffman syndrome.
Eyegaze Systems are being used in homes, offices, schools, hospitals, and long term care facilities. By looking at control keys displayed on a screen, a person can synthesize speech, control his environment (lights, appliances, etc.), type, operate a telephone, run computer software, operate a computer mouse, and access the Internet and e-mail. Eyegaze Systems are being used to write books, attend school and enhance the quality of life of people with disabilities all over the world
Good control of one eye:
The user must be able to look up, down, left and right. He must be able to fix his gaze on all areas of a 15-inch screen that is about 24 inches in front of his face. He must be able to focus on one spot for at least 1/2 second.
Several common eye movement problems may interfere with Eyegaze use. These include:
Nystagmus (constant, involuntary movement of the eyeball):
The user may not be able to fix his gaze long enough to make eyegaze selections. Alternating strabismus (eyes cannot be directed to the same object, either one deviates): The Eyegaze System is constantly tracking the same single eye. If, for example, a user with alternating strabismus is operating the Eyegaze System with the right eye, and that eye begins to deviate, the left eye will take over and focus on the screen.
The Eyegaze camera, however, will continue to take pictures of the right eye, and the System will not be able to determine where the user's left eye is focused. When the left eye deviates and the right eye is again fixed on the screen the Eyegaze System will resume predicting the gazepoint. Putting a partial eye patch over the nasal side of the eye not being observed by the camera often solves this tracking problem. Since only the unpatched eye can the screen, it will continuously focus on the screen. By applying only a nasal-side patch to the other eye, the user will retain peripheral vision on that side
Inadequate Visual acuity:
The user must be able to see text on the screen clearly. If, prior to his injury or the onset of his illness he wore glasses, he may need corrective lenses to operate the Eyegaze System. If he's over 40 years old and has not had his vision checked recently, he might need reading glasses in order to see the screen clearly.
In most cases, eyetracking works well with glasses. The calibration procedure accommodates for the refractive properties of most lenses. Hard-line bifocals can be a problem if the lens boundary splits the image of the pupil, making it difficult for the system's image processing software to determine the pupil center accurately. Graded bifocals, however, typically do not interfere with eyetracking.
Soft contact lenses that cover all or most of the cornea generally work well with the Eyegaze System. The corneal reflection is obtained from the contact lens surface rather than the cornea itself. Small, hard contacts can interfere, if the lens moves around considerably on the cornea and causes the corneal reflection to move across the discontinuity between the contact lens and the cornea.
Diplopia (double vision):
Diplopia may be the result of an injury to the brain, or a side effect of many commonly prescribed medications, and may make it difficult for the user to fix his gaze on a given point. Partially patching the eye not being tracked may alleviate double vision during Eyegaze System operation.
Another occurrence associated with some brain injuries, as well as a side effect of medications, a blurred image on the screen decreases the accuracy of eye fixations.
Cataracts (clouding of the lens of the eye):
If a cataract has formed on the portion of the lens that covers the pupil, it may prevent light from passing through the pupil to reflect off the retina. Without a good retinal reflection the Eyegaze System cannot accurately predict the user's eye fixations. The clouded lens may also make it difficult for a user to see text on the screen clearly. Surgical removal of the cataracts will normally solve the problem and make Eyegaze use possible.
Homonymous hemianopsia (blindness or defective vision in the right or left halves of the visual fields of both eyes):
This may make calibration almost impossible if the user cannot see calibration points on one side of the screen
Much of that work has been aimed at disabled or hands-busy applications, where the cost of learning the required eye movements ("stare at this icon to activate the device") is repaid by the acquisition of an otherwise impossible new ability.
However, we believe that the real benefits of eye movement interaction for the majority of users will be in its naturalness, fluidity, low cognitive load, and almost unconscious operation; these benefits are attenuated if unnatural, and thus quite conscious, eye movements are required. The remaining category in unnatural eye movement/natural response, is anomalous and has not been used in practice