Retina

The human eye is incredible. And our ability to use this to work out so much about the world is remarkable. It’s even more impressive when you realise that there are huge limitations and so much trickery involved, and that you don’t ‘see’ everything you believe you do. This post is about the ability for your eye to capture information in the world, for your brain to use. In later posts we will look at how that is processed into useful information in the brain and an understanding of the world.

Some surprising facts about your eye:
  • When you look at a scene, like a park on a sunny day, you feel like you take in a pretty accurate view of the whole scene in front of you. In fact the amount of your vision that is detailed is tiny – just the size of your thumbnail held at arms length away from you. Outside of this the detail reduces dramatically, and it takes constant scanning to take in detail from different parts of the scene.
  • This tiny disk of detail is swept around the scene in front of you in tiny, jerky, and mostly involuntary movements called saccades, During these movements you effectively see nothing.
  • You have over 100million photo receptors (think of light detecting pixels) in each eye that detect light, but only around 1 million neurons to transmit this information to the brain, so something pretty tricky is going on to transmit all even this limited detail.
  • The effort needed to process vision is huge, and a large amount of the brain’s capacity is given over to it. This is all very expensive in terms of energy, and your brain is very good at saving energy by avoiding any work that is not very important (according to its own priorities). This means that only parts of the visual information are ever processed at all, or given any attention.
Whole books struggle to explore the truly incredible faculty of human and animal vision. This humble post takes a look at one tiny aspect: the ability of your eye to detect light and information from the world, and some of the quirks of your ‘seeing’ equipment, before your brain even gets to do anything interesting with the raw information. We are going to focus (ha ha) on the retina, the name for the part of the back of the eye where light is detected by tens of millions of special cells called photo receptors. It’s easy to think of our eyes and vision as working like a camera, and that everything that is in the camera’s view can be instantaneously known by us. That can be useful to understand some concepts, but it is really not helpful in appreciating just how complex it all is. This camera idea misses so many complexities, compromises, and illusions.

How much of the world can we see?

Each of our eyes gives us a pretty huge field of view, and where these two fields overlap (giving us binocular, or stereo vision) we have around 120 degrees of view from side to side. Across both eyes you can see more than 180 degrees without moving your eyes (eye movement can increase this to 220 degrees). Top to bottom you can see around 130 degrees (with your eyes fixed in one position). Out of all of that field of view, you only see in high detail over a disk around 1.5-2 degrees across. Your ability to detect colour reliably is limited to a disk around 30 degrees across. Have a try at the following experiments to see what this means for yourself.

Experiments

Range of Vision
With both eyes open, hold out your arms either side of you at full stretch, and then look straight ahead (Try really hard not to move your gaze – harder than you might think). Wiggle your fingers. Hopefully you can see movement even if you cannot clearly make out the fingers themselves. If not, move them slightly forward. Given that your eyes are slightly in front of your arms, that’s more than 180 degrees. Not bad.
Now Close one eye , but keep looking straight ahead. The hand on the side of your closed eye has disappeared from view. Move it forwards until you can just make out the movement of your wiggling fingers. Keep it there, and now repeat with the other eye and arm. Open both eyes and this is the range of stereo view you have if your eyes are firmly facing ahead. This is probably at least 90 degrees, maybe up to 120 degrees.
Limits of detail
While still looking directly ahead, and not moving your eyes for a sneaky look (as tempting as it is), notice that your fingers are still pretty indistinct. If someone else held up some mystery object where your fingers are, you would struggle to identify it, without shifting your vision. Why is this? more on this in a moment.
Your laser beam of focus
Now hold out one arm in front of you at full stretch and then raise your thumb. That area, the size of your thumbnail is the extent of your most detailed vision. At around 1.5-2 degrees, this is the skinny beam of high-accuity vision that you have. Outside of the tiny spot your accuracy rapidly reduces.
Now stretch out your fingers at the end of your outstretched arm (still straight ahead of you), with fingers pointing outwards like the rays of the sun. Keep your gaze pointing at the centre of your hand. The disc made by your fingertips is around 16 degrees. Outside of this range you see very little detail. A little further out your colour detection becomes unreliable.
So what we have is a skinny beam of high detail and colour awareness, and then increasingly less detailed perception as you move out. This does not match our intuition about the world. Our experience suggests that we are able to take in whole scenes in an instant, like a photograph, to examine at leisure. We seem to be aware of detail across the scene.
Imagine if all you had was the thumbnail beam, and absolutely no other awareness at all – a tube 70cm long and 15mm across, and you had to scan this around the whole scene in front yo you, with no clues from any peripheral vision. How long would it take you to make sense of what is happening. How you catch a ball, or judge when it is safe to cross a road, or identify your family in a crowd? Since we are able to do all these things there is clearly some usefulness in the rest of our vision.

Photoreceptors: Our Pixels

The reason for our uneven powers of sight is the way that photoreceptors are laid out in the retina. At the centre of our retina (in an area called the Fovea) there is an incredible density of colour sensing photo receptors called cones (because of their shape). but this density reduces rapidly. How rapidly? At the very centre of the fovea, we typically have over 100,000 cones per mm2 or per degree of range of vision.
Just 1 degree (less than 1mm) away, this has dropped to around 20,000. At 15 degrees from centre, it is at or below 5,000 receptors per mm, and that level covers most of the rest of the eye. These receptors detect colour – each one detecting red or green (near the centre), or blue more towards the edge. The combination of these allows us to detect other colours by the amount that nearby red, green (and blue) cones are stimulated. We won’t say much more about these component colours, just take it that cones detect colour, and are mostly concentrated at the centre.
There is another type of photoreceptor that does not detect colour but is much more sensitive to light levels. These (called rods) are almost absent in the centre of the fovea, but then cluster in a ring around 6mm from the centre. The density of these then also reduces towards the edges, but not as quickly as the cones.
That’s a lot of numbers to take in. See the graph which shows density of cells (amount of detail we can see) as you get further from the centre of your retina.
 

Photoreceptor density

As you move away from the centre of the eye, the number of cells that pick up colour and light data dramatically reduce. Cones detect detail and colour in normal daylight. Rods detect light (but no colour) in low light conditions.

In truth, rods do not appear to contribute much to daylight vision. They are so sensitive to low light that daytime or artificial light levels wash out their ability to detect differences. In poor light, like in your home after dark when the lights are out, or in moonlight, when our cones are unable to pick up enough light, they are responsible for all of our vision. Most of our discussion that follows will concern the daylight and colour sensitive cones.

Retina Illustration

Let’s look at a simulation of cone density and what vision might look like if you could intercept the signals from these photo receptors.

How to use this illustration
This illustration shows a zoomed in view near the centre of our simplified retina, to show how quickly the level of detail drops away from the centre. The top panel on the right hand side, shows this detail area against your overall view of the scene. The panel below this lets you control where in the scene your retina is focused by dragging the orange disk. The bottom panel lets you switch between scenes, so you can see the effect on both images and text.

This simulated retina in this illustration has around six thousand cones compared to six million in a human retina.

In the illustration above, you can see the effect of very dense detailed vision right at the centre, but with detail rapidly dropping away. Change this scene to see how this effects your view of an everyday scene, and then text of different sizes. Note how when you move your focus around the scene or text, the unfocused area become much harder to make out. Note as well that for words, once they are unclear, you almost instantly forget or disregard them (a post on reading will follow at some point). Whereas in the scene with the toy monkeys, you retain some impression of the monkeys even when they are out of focus. This is to do with how your brain recognises and processes shapes and objects and adds meaning, which is way beyond the scope for this post (hopefully we can come back to that later).
It doesn’t seem as though our vision works like this. The brain does so much work from this raw sense stream to add more meaning, and make the most of all the data received from the world.

How Much Detail Can You See?

Here is another illustration of the limits of detail perception.
Initially the text is mostly obscured due to blurring. Rings of increasing clarity expand outwards,  revealing more and more of the text in high detail.
If you stare at the red spot in the centre, and resist the urge to scan and shift your eyes, note that even when the whole text is in sharp detail, you will struggle to be aware of words outside the red ring.
How to use this illustration
This illustration shows you an area of text that is mostly obscured. As the area of clarity expands out, keep your focus on the inner red spot, and try not to move your gaze. You will probably find that the only text you see clearly is right under the spot, but maybe you can make out some detail within the red ring. Beyond this, even when the text is perfectly clear, your retina is not able to make it out.
Reading really relies on your most focused and detailed parts of vision. There have been experiments to determine just how unaware you can be of text outside of that narrow band, and that this dictates how far our eye has to move to get the next “chunk” of detail, such as when reading.

Retina Rings

One way to think of the different levels of vision across the retina (if we simplify things a bit) is like concentric rings –
  • a central, small area of high detail colour perception
  • a ring with detail but less colour
  • a less detailed very low colour ring
  • just very scant light/dark detection over the rest of our visual field.
If we imagine that this set of simplified rings represents our range of vision, and that we can shift where the detailed part is focused, we get something like the following illustration.
How to use this illustration
This illustration shows you the joined up effect of different levels of detail and colour detection, simplified into rings. As you move your retina around each scene, you get an overall view made up of decreasing levels of detail and colour detection.

Use the first panel to control where your view is focused, the second panel shows the result. You can choose a different scene to view in the last panel.

Using the illustration, note that when viewing the monkeys, even when your focused detail is no longer giving you detail from other areas of the scene, you have built up information about the scene, like the colour, position and type of object.
Note that when viewing the text, if you scan the words a line at a time, left to right, the fact that most of the scene is indistinct does not prevent you from reading.
You may find that you use these rings of information to scan the world. Even where an item is not within the most detailed part of the view, there is still information to be picked out. If we add to this the ability of our short term memory to keep track of information not immediately in view, we can build up a more useful picture of the world mentally, even if the instantaneous snapshot of sense data available might not provide all of the information.
A discussion and animations showing how the mind can take very limited sense information and determine meaning from the world is in the Simple Vision post.
Experiment: colour and detail limits
As another experiment, find a set of four or five differently coloured pens or other small objects that are the same shape as each other but distinctly different colours. Put them behind your back and shuffle them around. While staring straight in front of you pick one pen, and bring it slowly from way behind, around the side, with your arm at full stretch. There is probably a point at which you can have a pretty good guess at the colour, but you won’t be certain. Keep bringing it further to the front of your view, and try not to move your focus off a spot straight ahead. The weird thing is that after you reach a point where you are certain of the colour, if you move the object back to a less colour sensitive part of the view, you can still see the colour. This is just a clue that your brain has effectively labelled the object as being that colour using information it already has. More than what you sense alone, your mind uses information it believes to be true about the world, and adds this to what you are seeing in real time. So if you had a red pen, even after your sense of the world cannot detect either a pen shape or the colour red, your mind has labelled that something in that vicinity is pen-like and red. This predictive and top-down approach to vision is a topic for another day. This can be fooled and there have been experiments where colours, objects or even people can be switched without us noticing. Conjurers and sales techniques can exploit these limits to create illusions or beliefs.

Saccades

We have mentioned a couple of times the effect of your eyes shifting their direction of focus, either deliberately as you check out some area of interest in your field of view, perhaps a bright colour, a flashing light, a sense of movement. Your eye does this to read words on a page or screen, or involuntarily as your mind attempts to scan the scene to take in information, spot threats and opportunities, or novelty in the world. If you tried the experiments and illustrations, you probably noticed how hard it was to resist the urge to shift your focus to something that moved or that I asked you to be aware of. It is deeply wired into us to follow movement, threats, objects of interest, and to scan in order to take in detail over a wider area. It’s primal, and our conscious mind does not always get consulted before our eyes move (though of course we can decide to direct our gaze as well).
The involuntary movements are called saccades. The fascinating thing is that while our eyes are moving, we do not take in any sight information from our eyes. The retina keep on reacting to light, but the brain at some point, ignores the feed. Why is this? The movements are really fast (equivalent to 200 up to 900 degrees per second, depending on the range and urgency) and it seems likely that our eye and brain simply cannot process and make sense of this information in real time. It would give us noise rather than useful information. Once the eye is at rest in its new position, the information is much more meaningful.
How to use this illustration
This illustration shows an exaggerated simulation of voluntary and involuntary eye movements (saccades).

The first, top panel shows you where in the scene your retina is focused. below this we can see the uninterrupted stream received from the retina, including the blurring effect during rapid movements. The main panel shows the effect if the feed from the retina is ignored (shown as a black disk) during movements, but the steady images are combined with the (rapidly decaying) information about the whole scene. In the end panel you can switch scenes.

This virtual eye makes these (slightly exaggerated) saccades, or movements in focus, without our control. We can deliberately move our focus but our eye will carry on checking out other areas of the scene. In the lower view, we see everything, including the blurred and confusing image during the rapid movements. We still see something useful, but a lot of detail is compromised by misleading information gathered during the rapid movement.
In the main window, the feed is effectively blocked during the movement (represented as a black disk during the move). This means we get maximum benefit from the detail we see when the eye is still. This illustration also recreates the kind of decay of information, in that detailed information gained from a previous eye position is rapidly lost. Nonetheless, this reduced detail, plus the up to date current detail in one tiny area gives us a pretty comprehensive view at any one time. Enough to navigate, predict and act confidently in the world, despite the narrow beam of up-to-date detail.
Experiment: Saccades
If you want to see your own saccades, you can do this with a mobile phone.
  • Find a book, or something with lots of text on it.
  • Hold your phone in one hand, switch to the front facing camera and pick video mode on the camera app.
  • Move the phone until your eye (on the same side as the hand holding the phone) is in the frame, but your other eye is not blocked by the phone.
  • Now hold the book in front your other eye.
  • Press record on the phone and now read for a few seconds.
  • Stop recording, and play back the recording.
You will see the tiny movements your eye makes.
A little point to note: you recorded the eye that cannot see the book, and yet this eye moves in perfect accord with the eye scanning the text. It is so hardwired that eyes move together to focus and pick a target, we have almost no conscious control over this behaviour.
Experiment: Saccades 2
Why do we need to use a camera to show our saccades? Surely you can see them yourself in the mirror.

Look in a mirror and try this (it must be a real mirror, and not a mobile phone app)

While you look at your own reflection, shift your gaze from looking at one of your eyes to the other and back again. You will be able to see that your eye has moved, but you cannot see the move. Why not? Because your brain blocks the feed from your eye during the saccade.

Repeat the same experiment stood in front of another person and ask them to switch their gaze from one of your eyes to to the other and back again. Watch their eyes while they do this. You can see the movement in their eyes, that you could not see in your own in the mirror.

Summary

Our eye is incredible, able to detect light at a staggering level of detail and accuracy. And yet it is a little disappointing to learn that this impressive ability covers less than 2 degrees of our view. Even with this limited range, there are over 100million photo receptors in the human eye.The illustrations above give you some idea about how this impressive yet limited instrument can give our brain a view of the world.
It gets even puzzling when you learn that information from those 120 million cells rely on only around 1 million neurons in the optic nerve to carry the sense information on the the brain. Clearly there are some tricks to make this work. We do know that nature can perform compression (just like our digital movie files are squeezed into smaller spaces by analysing what changes across and between frames).
But it is also to do with the fact that different parts of the brain are interested in different aspects of the image, not all of which is relevant all of the time.
As powerful has our mind is, it simply cannot process all of the light information in our view in one go to solve many problems such as spacial awareness and mapping, movement prediction, threat detection, face detection, object identification, light and shade compensation, colour identification and many more, all in real time. In that case why would we have evolved superior eyes that can produce far more than our brain can process? Nature hates waste.
The Science and Other Sources
To come up with these illustrations and observations I have taken ideas from several sources and taken a few artistic liberties in departing from the pure science as well. I have also left out some details in order to keep this one bite sized subject. My aim is to make you question what you think you can see rather than to create perfect working models of a human eye.

If you want to know more, the best reference I have found so far on human vision is Vision and Brain: How We Perceive the World, by James V. Stone, published by MIT Press. It’s heavy in places but it covers many aspects that are useful.

Also interesting is Visual Intelligence: How We Create What We See by Donald Hoffmann.

Some of the most interesting observations about how we don;t experience what we think we do are in The Mind is Flat: The Illusion of Mental Depth and the Improvised Mind, by Nick Chater

Both go far beyond what is captured by the eye, but they give great overviews of how vision works based on what we receive from our eyes, and what we do with it in order to “see” anything.

For some other quick and fascinating facts about the eye, take a look at this XKCD cartoon.

Above I refer to some experiments that look into how far away from the centre of your vision you are unaware of text during reading (even if it changes). This comes from the Nick Chater book (The Mind is Flat), where he references a paper:

The span of the effective stimulus during a fixation in reading by George W. McConkie
Keith Rayner
McConkie, G.W. & Rayner, K. Perception & Psychophysics (1975) 17: 578.
Related Thoughts

TV Screens

I was brought up in the 1970’s. Whenever I see a recording or image of a TV screen from then I am astounded by just how poor the quality of the image was. And yet at the time it was what we had and it was great. Our minds are so good at filling in the detail that is missing, to create continuity.
Now we have displays that are better than the foveal acuity of the retina at normal viewing distances, and that’s across the whole screen. Some virtual reality headsets are taking advantage of the limits of our perception to only render the super detail in the exactly spot our retinas are focused on at the time.

 

The Magic of Paintings.

We are surrounded by images all the time, photography, digital images, paintings, websites and paper documents, and it is almost impossible for us to imagine a time when there was no depiction of the world that was not captured purely by our minds. The world is dynamic, and (as we have seen) we can only pick out tiny elements of it at any time. The first photo-realistic, or representative paintings must have been truly mind blowing. The fact that a scene in the world can be frozen in time, in full detail from corner to corner, maybe even scenes that the viewer could never be able to see first hand from distant places, is incredible. The viewer could take their time to scan inch by inch, every part of the picture, stand close, up, step back, and drink in this frozen moment in time. We tend to take this ability for granted, especially now we can take, exchange, and find images of just about anything taken anywhere.
The reason photo’s are amazing is because it captures a moment, and represents an experience that might never happen again. It is a super power that lets us use our limited detail to scan in a rich moment in time, and consider the elements of the image, that a constantly shifting world does not allow.

 

An Artist’s Retina

Chuck Close is an artist whose early works reproduced in paint, ultra close-up detailed portraits of people on a huge scale. His painting transcended the image as a whole and yet from viewing distance you were hard pressed to believe that these were paintings rather than photos. Chuck went on to develop a unique style of rendering his portraits using a form of colour/light reproduction for what, at a distance, were still extremely faithful portraits, but close up are deconstructed colour mixes.
Take a look. It shows how an artist’s understanding of, and experimentation with vision can be even more enlightening and provocative than a scientist’s. Images are details from the official Chuck Close web site (WARNING: this site contains some images of nudity)

The Images used in the illustrations

The image of monkeys is the authors own. The text for the illustrations is from a thought posted by David McRaney, author and podcaster of the amazing You Are Not So Smart podcast.

Other Experiences
It is very easy for me to start a discussion like this, and get stuck in my own perspective. The point of Remented is that it explores the mind and the brain from the human experience. The experiences I chose may not represent your own.

An obvious flaw here is that my focus in this article has been about how we see; so what if you cannot see at all, or you do not see in the way that many others do?

The human brain devotes a huge amount of space and ability to vision, in what is called the visual cortex. It is one of the unique features of the human brain. But it is a bit misleading to call it visual. A lot of it is to do with spacial reasoning, or understanding relationships between things, and the properties of objects. It turns out that those projections of the world and predictions about objects, where they are, what they mean (to you, eg. food, danger) are active in almost the same way with our other senses. If someone is blind or visually impaired, their cortex responds in similar ways to when a sighted person sees something. Even as a sighted person, you can shut your eyes and be able to tell what kind of space you are in from the way that  sounds reverberate, and what kinds of sounds you hear. You can tell if you are inside or outside, how large the space is is, whether you are alone, or with a few or many people. If your world is without vision altogether you will be even better at this. We use these spacial reasoning abilities even for abstract ideas that don’t originate from any sense. When you imagine the relationships between people you know, your family, how something works, and when you make plans by imagining a sequence of tasks, you are using these same pieces of brain machinery, and mental powers.

Just think for a moment how movies, animated cartoons, computers/phones, and more recently virtual and augmented reality deliberately fool our senses in order to make our minds project something that is not there, as though it is. They work because designers and programmers are learning how to plant compelling ideas in your mind, and trigger the senses in a way that enhances those ideas.

You may have been born with exceptional attention to detail, or amazing spacial reasoning that makes you great at solving certain kinds of problems, but poor at noticing subtle signals of people’s behaviour around you. It is unlikely that your eyes work very differently, but how your mind processes, prioritises, and adds meaning to the view can be very different for the person standing right next to you, looking at the same scene.

There are things that can go wrong in brain mechanics. So for example, certain areas of the brain can become damaged (maybe as a result of a stroke or injury), and prevent a person from being able to process their perfectly working sense images from the eye into meaningful objects, or faces. These can be distressing and debilitating. Sometimes the plasticity of the brain (its ability to reconfigure its connections) can compensate for such damage, but often it cannot.

Our beliefs about the world can have a huge affect on what we see in a scene. It can affect what we decide is important for us to place our limited focus on, so that we may miss other details, and we may assess something as being benign, while the person next to us sees something threatening. Conditions and diseases, such as dementia, can cause sufferers to hold a different set of beliefs about where they are, who they are with, what year it is, and thus have a very different projection on the same scene as you are seeing.

I am here visiting some of the reasons why any individuals experience of vision could be wildly different from mine, but still concluding that the ideas illustrated above are relevant, and maybe even explain why one person’s experience of a scene can be so very different from another’s.