Elon Musk recently announced that Neuralink’s next project will be a “Blindsight” cortical implant to restore vision: “The resolution will be low at first, like early Nintendo graphics, but eventually it can exceed normal human vision.”
Unfortunately, this claim is based on the misconception that neurons in the brain are like pixels on a screen. It’s no wonder engineers often assume that “more pixels means better vision.” After all, that’s how monitors and phone screens work.
In our newly published research, we created a computational model of human vision to simulate what vision an extremely high-resolution cortical implant might provide. A cat movie with a resolution of 45,000 pixels is sharp and clear. A movie created using a simplified version of the model with 45,000 cortical electrodes, each stimulating a single neuron, still has a recognizable cat, but much of the detail of the scene is lost.
The reason the movie created by the electrodes is so blurry is that neurons in the human visual cortex do not represent tiny dots or pixels. Instead, each neuron has a specific receptive field, which is the location and pattern that a visual stimulus must have in order for that neuron to fire. Electrically stimulating a single neuron produces a blob whose appearance is determined by that neuron’s receptive field. The smallest electrode—the electrode that stimulates a single neuron—will produce a blob about the width of your pinky finger when held at arm’s length.
Consider what happens when you look at a single star in the night sky. Each point in space is represented by thousands of neurons with overlapping receptive fields. A small point of light, such as a star, causes a complex firing pattern among all these neurons.
To produce the visual experience of seeing a single star through cortical stimulation, you need to reproduce a pattern of neural responses similar to the pattern that would be produced by natural vision.
To do this, you’ll obviously need thousands of electrodes. But you’ll also need to replicate the exact pattern of neuronal responses, which requires knowing the receptive field of each neuron. Our simulations show that it’s not enough to know where each neuron’s receptive field is in space – if you don’t also know the direction and size of each receptive field, then the star turns into a blurry mess.
So even a single star—a single bright pixel—produces an extremely complex neural response in the visual cortex. Imagine the even more complex pattern of cortical stimulation required to accurately reproduce natural vision.
Some scientists have suggested that it might be possible to produce natural vision by stimulating just the right combination of electrodes. Unfortunately, no one has yet proposed a sensible way to determine the receptive field of each neuron in a given blind patient. Without this information, there is no way to see stars. Vision from cortical implants will remain grainy and inaccurate regardless of the number of electrodes.
Vision restoration is not a simple engineering problem. Predicting what kind of vision a device will provide requires understanding how the technology interacts with the complexities of the human brain.
How we created our virtual patients
In our work as computational neuroscientists, we develop simulations that predict the perceptual experiences of patients trying to regain their vision.
We previously built a model to predict the perceptual experience of patients with retinal implants. We simulated the neurophysiological architecture of the region of the brain involved in the early stages of visual processing to create a virtual patient to predict what cortical implant patients would see. Our model approximates how receptive fields increase from central to peripheral vision and the fact that each neuron has its own unique receptive field.
Our model successfully predicted data describing the perceptual experience of participants in a wide range of studies on cortical stimulation in humans. After verifying that our model could predict the available data, we used it to make predictions about the quality of vision that potential future cortical implants could produce.
Models like ours are an example of virtual prototyping, which involves using computer systems to improve product design. These models can facilitate new technology development and evaluate device performance. Our work also suggests they could provide more realistic expectations about what kind of vision bionic eyes can provide.
First of all, do no harm
In our nearly 20 years of research on bionic eyes, we’ve seen that the complexity of the human brain overwhelms companies. When these devices fail, patients pay the cost, left with orphan technology in their eyes or brains.
The Food and Drug Administration could require vision-saving technology companies to develop failure plans that minimize harm to patients when the technologies stop working. Possibilities include requiring companies that implant neuroelectronic devices in patients to enter into technology escrow agreements and carry insurance to ensure continued medical care and technology support in the event they go bankrupt.
If cortical implants could achieve anything close to the resolution of our simulations, that would still be a feat worth celebrating. The granular and imperfect vision would be life-changing for thousands of people who currently suffer from incurable blindness. But this is a time for cautious optimism rather than blind optimism.
This article is republished from The Conversation, a nonprofit, independent news organization that brings you facts and trusted analysis to help you understand our complex world. By Ione Fine University of Washington and Geoffrey Boynton, University of Washington
Read more:
Ione Fine is receiving Grant R01EY014645 from the NIH National Eye Institute
Geoffrey Boynton receives EY R01 EY014645 grant funding from the National Institutes of Health