Reconstruction of Images in the Mind's Eye Brings Us One Step Closer to Recording Dreams
Neuroscientist Jack Gallant can read your mind, or at least reconstruct the images you are seeing in your mind's eye.
Gallant has built a "brain decoder," or a machine that can read the images in your mind and then project them. In order to perform this feat, he shows the subject a video, and then an fMRI machine measures their brain patterns. These patterns are then converted by an algorithm into composite images that are created from a database of thousands of Youtube videos.
A video of the reconstructed images:
[Credit: Jack Gallant]
While his device received plenty of attention from media outlets, Gallant only intended it to be a preliminary demonstration of the direction in which his research was heading. "It was one of the coolest things we ever did," he said, "but it's not science." He is still hard at work researching the visual information system in the brain, and he believes that an actual "brain decoder" would essentially be a model of a brain. "It just so happens that if you build a really good model of the brain, then that turns out to be the best possible decoder."
This same technology can potentially be used to record dreams, but the process is more involved. The most significant obstacle standing in the way of recording dreams, or using this technology on a more commonplace basis in general, is the lack of confirmation that it is working correctly. In Gallant's experiment, they knew exactly which images were shown to the subjects, so they could easily gauge the accuracy of the reconstructed images. But in the case of dreams, Japanese researchers worked backwards in a sense, and relied on the subjects' own accounts of the images they saw in their dream state. They would then correlate these images with the appropriate signals in order to train the algorithm, and this allowed them to identify up to 60% of the images seen by the dreamers. But since the machine needs to be trained to each individual, this is a far cry from a universal "brain decoder."
Gallant responded to the common fear that this technology could be used for insidious purposes, such as the government reading ordinary citizens' minds on a regular basis: "I actually agree that you should be afraid, but you don't have to be afraid for another 50 years."
Gallant asserts that it will take half a century in order to fix the other two obstacles towards regular use: portability and the weakness of brain signals. As of now, your mind can only be "read" when sitting in an fMRI machine, which are huge, expensive, and immobile. Furthermore, as a result of the relative weakness of brain signals and the poor resolution of the fMRI machine, the reconstructed images are blurry at best. "Until somebody comes up with a method for measuring brain activity better than we can today there won't be many portable brain-decoding devices that will be built for a general use," said Gallant.
But, he still believes that it will most likely happen eventually, in which case the NSA privacy violations would be the least of our problems. The government won't need to record your online activity or listen to your phone calls if they can read your mind.