GMU:Max and the World/F.Z. Aygüler

AN EYE TRACKING EXPERIMENT

[link title]

This project is an eye-tracking experimentation with Max MSP. My experiment on eye tracking is inspired by historic findings of Alfred Yarbus on eye movements in the 1950s and 1960s. Yarbu’s device was a video-based system. He recorded the close-up videos of the eyes and then he edited the video frame by frame to calculate the gaze tracking data. I experimented with both video footage and web cam to track the eyeball. After targeting eyeballs using cv.jit library extensions, adaptive threshold method was used to turn eye image into a white dot on black background. The most difficult part was to capture eye balls with a web cam due to the low resolution. I used jit.lcd to draw lines based on the data coming from tracking. In the first video, you see a sample of my webcam and video-based eye tracking.

In the second video, you see me looking at my room while tracking my eye movements. During the quarantine, I have stayed at my family house and spent most of my time in my childhood bedroom. I had many moments when my eyes were darting around trying to pull the answer out of my head. Looking at a very familiar place, looking at objects that are not there or searching for a long term memory.

Although predominantly externally triggered by the vision, eye movements also occur in cognitive activities. This pattern of eye movement is mostly related to internal thought processes and don’t generally serve visual processing. It is called ‘nonvisual eye movements’ or ‘stimulus independent eye movement’ in most of the recent studies. Why people move their eyes when they are thinking or searching for a long term memory is not clear.

PROJECT DESCRIPTION

How the brain process information to create a representation of the external world? How do we recognize a face, reach an object, or appreciate a piece of art? Those questions go back at least to Aristotle who noted that our minds create images, “internal representations of the external world”. But The process of seeing is far from a reproduction of the images impinging the retina. It is rather the result of our unique interpretation of ambiguous sensory information. Conceptually, how we sense the world and what we consider to be ‘truth’ is something unique and personal.

EYE'S MOVEMENT AND ITS ANATOMY

There are roughly 130 million photoreceptors in the human eye, only in order of a million fibers in the optic nerve carry the signal to the brain. Eye movements are categorized in many different types of motion, two of which are most commonly studied: fixations and saccades. If the eye rests on a specific location for a certain amount of time, this non-movement is classified as a fixation. Fixations are those times when our eyes essentially stop scanning about the scene, holding the central foveal vision in place so that the visual system can take detailed information about what is being looked at. The movement from one fixation to the next is called a saccade. This is the fastest eye movement, in fact, the fastest movement the body can produce, with a duration between 30 to 50 milliseconds. Due to the fast movement during a saccade, the image on the retina is of poor quality and information intake thus happens mostly during the fixation period.

INSPIRATION - Alfred Yarbus’s Eye Tracking Experiment

EYE TRACKING TODAY

RESEARCH NOTES

TECHNICAL ASPECTS

REFERENCES Alfted L. Yarbus, Eye Movement and Vision, trans. Basil Haigh, New York Plenum Press “Yarbus, eye movements, and vision” Benjamin W Tatler, Nicolas J Wade, Hoi Kwan, Boris M Velichkovsky https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3563050/ How Do We See Art: An Eye-Tracker Study https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3170918/ Tracking - Pupil Tracking https://www.notion.so/Tracking-Pupil-Tracking-2371c7334752476f86f22e7ee6ffdf47 cv.jit | Computer Vision for Jitter https://jmpelletier.com/cvjit/ EYE TRACKING WITH CV JIT CENTROIDS https://www.youtube.com/watch?v=o4F34FL8BN4&ab_channel=ProgrammingforPeople Jit.lcd tutorials by Kimberlee Swisher https://www.youtube.com/channel/UCg8PkUJrxqsYaesUwhadzfw

ASSIGNMENTS

Week 1

Week 2

Week 3

Week 4

/Final project