We Know It When We See It by Richard Masland: What You Didn’t Know About What You See | EXCERPT

In We Know It When We See It, pioneering neuroscientist Richard Masland covers everything from what happens when light hits your retina, to how artificial intelligence comes into play when trying to understand the neurobiology of vision. It is a profound yet accessible investigation into how our bodies make sense of the world.

Read an excerpt to learn what you didn’t know about what you see:

This is a book about how we see. Thinkers have pondered vision for a long time, but most of their ideas were, by modern standards, naive: the eye is, in fact, something like a camera, but there is a whole lot more to vision than that. It may seem natural and simple that we can recognize the face of a friend—so much so that the ancients didn’t even identify it as a problem—but there is actually nothing simple about it. To truly understand vision, you have to understand more than just how our eyes work. You also must understand how our brains make sense of the outside world.

Paradoxically, brains are pretty slow; neurons and their synapses work millions of times more slowly than modern computers. Yet they beat computers at many perceptual tasks. You are able to recognize your child among the crowd on the playground in milliseconds. How does your brain do it? How does it take a blunt stimulus—a patch of light, a vibration in the air, a change of pressure on the skin—and give it meaning? We have only glimpses of the ways, but what we have learned is fascinating.

I have been a neuroscientist since I was twenty-five—before the discipline of neuroscience officially existed—and I care as much about it now as I did then. I’ve watched our understanding evolve, and I’ve participated in the work myself. The basic narrative of this book is “how vision works”—from the retina to the highest visual centers deep in the temporal lobe. But I also want to let you follow the scientific journey, to see how basic neurobiology—not the talk-show kind—looks from beside the laboratory bench. So I’ll mix in some scenes from the lab, and sketch some of the players.

We’ll go through vision step by step. You’ll hear that the world you see is not the world that actually exists: it has been broken into fragments by your retina and sent to your brain in separate channels, each telling the brain its specific little thing about the image. You’ll learn how this recoding is accomplished by neurons in your retina, and why. We’ll follow these signals into the brain, where they build our perceptions.

The brain holds many mysteries, but an important insight is that much of the brain works not by fixed point-to-point connections, like the telephone system, but by means of swarms of neurons interconnected, like a spiderweb, into nerve nets. These days, nerve nets are often associated with computers, but in fact they were thought up a half century ago by a far-seeing Canadian neuroscientist, Donald Hebb. A few years later the idea was co-opted by computer scientists. During the next decades nerve nets moved in and out of fashion, but better computers eventually allowed computer scientists to create the field of machine learning, better known as artificial intelligence. They showed that computer nerve nets can learn to perform dramatic feats, leading neuroscientists to look again at nerve nets in the brain. So today we have a remarkable alliance between neurobiology and computer science, each field informing the other.

Do brains use nerve nets to interpret the world? Does the brain work by “machine learning”? The answer seems to be yes—and brains do it a whole lot better than computers. To be sure, computers dazzle with certain of their feats—not just playing chess, but learning other, more complex tasks. Generally speaking, though, AI computers are one-trick ponies. And even the simplest require lots of hardware, with a concomitant need for lots of energy. In contrast, our little brains can do a multitude of tasks and use less energy than a night-time reading light. Seen that way, computers are very bad brains, and a search is on to make them more brain-like.

The key to machine learning, as imagined long ago by Hebb, is that a nerve net connected by fixed wiring cannot do very much. Key is that the synapses that connect the neurons of a nerve net (or the simulated “neurons” of a computer) are modifiable by experience. This plasticity is a general rule in the brain—not just in sensory systems. It helps the brain recover from injury, and allows it to allocate extra brain resources to tasks that are particularly important. In vision, the nerve nets of the brain can learn to anticipate the identity of an object in the world—to supplement the raw information coming from the retina with its knowledge of images it has seen before. Boiled down, this means that much of perception is not just a fixed response to the visual scene but is learned. The brain’s nerve nets recognize certain combinations of features when they see them.

Where does this lead in our search for understanding the actual experience of perception, thinking, emotion? We don’t have a detailed answer, but we can see, far in the distance, how the final answer may look. Known, verifiable science can take us to an entry point. I will take us part of the way, to the seam where sensory experience turns into perception and thought.

Finally, where are “you” in all this? It’s easy enough to talk about the brain as we see it from the outside, but where is the inner person that we imagine to be looking out through our eyes? There we can barely begin—and we run inexorably into the nature of consciousness, the self. We’ll go there at the very end, with no answer but an attempt to see the problem more clearly.