Posts Tagged ‘augmented reality’

What if we could edit the sensations we feel; paste in our brain pictures that we never saw, cut out unwanted pain or insert non-existent scents into memory?

UC Berkeley neuroscientists are building the equipment to do just that, using holographic projection into the brain to activate or suppress dozens and ultimately thousands of neurons at once, hundreds of times each second, copying real patterns of brain activity to fool the brain into thinking it has felt, seen or sensed something.

The goal is to read neural activity constantly and decide, based on the activity, which sets of neurons to activate to simulate the pattern and rhythm of an actual brain response, so as to replace lost sensations after peripheral nerve damage, for example, or control a prosthetic limb.

“This has great potential for neural prostheses, since it has the precision needed for the brain to interpret the pattern of activation. If you can read and write the language of the brain, you can speak to it in its own language and it can interpret the message much better,” said Alan Mardinly, a postdoctoral fellow in the UC Berkeley lab of Hillel Adesnik, an assistant professor of molecular and cell biology. “This is one of the first steps in a long road to develop a technology that could be a virtual brain implant with additional senses or enhanced senses.”

Mardinly is one of three first authors of a paper appearing online April 30 in advance of publication in the journal Nature Neuroscience that describes the holographic brain modulator, which can activate up to 50 neurons at once in a three-dimensional chunk of brain containing several thousand neurons, and repeat that up to 300 times a second with different sets of 50 neurons.

“The ability to talk to the brain has the incredible potential to help compensate for neurological damage caused by degenerative diseases or injury,” said Ehud Isacoff, a UC Berkeley professor of molecular and cell biology and director of the Helen Wills Neuroscience Institute, who was not involved in the research project. “By encoding perceptions into the human cortex, you could allow the blind to see or the paralyzed to feel touch.”

Holographic projection

Each of the 2,000 to 3,000 neurons in the chunk of brain was outfitted with a protein that, when hit by a flash of light, turns the cell on to create a brief spike of activity. One of the key breakthroughs was finding a way to target each cell individually without hitting all at once.

To focus the light onto just the cell body — a target smaller than the width of a human hair — of nearly all cells in a chunk of brain, they turned to computer generated holography, a method of bending and focusing light to form a three-dimensional spatial pattern. The effect is as if a 3D image were floating in space.

In this case, the holographic image was projected into a thin layer of brain tissue at the surface of the cortex, about a tenth of a millimeter thick, though a clear window into the brain.

“The major advance is the ability to control neurons precisely in space and time,” said postdoc Nicolas Pégard, another first author who works both in Adesnik’s lab and the lab of co-author Laura Waller, an associate professor of electrical engineering and computer sciences. “In other words, to shoot the very specific sets of neurons you want to activate and do it at the characteristic scale and the speed at which they normally work.”

The researchers have already tested the prototype in the touch, vision and motor areas of the brains of mice as they walk on a treadmill with their heads immobilized. While they have not noted any behavior changes in the mice when their brain is stimulated, Mardinly said that their brain activity — which is measured in real-time with two-photon imaging of calcium levels in the neurons — shows patterns similar to a response to a sensory stimulus. They’re now training mice so they can detect behavior changes after stimulation.

Prosthetics and brain implants

The area of the brain covered — now a slice one-half millimeter square and one-tenth of a millimeter thick — can be scaled up to read from and write to more neurons in the brain’s outer layer, or cortex, Pégard said. And the laser holography setup could eventually be miniaturized to fit in a backpack a person could haul around.

Mardinly, Pégard and the other first author, postdoc Ian Oldenburg, constructed the holographic brain modulator by making technological advances in a number of areas. Mardinly and Oldenburg, together with Savitha Sridharan, a research associate in the lab, developed better optogenetic switches to insert into cells to turn them on and off. The switches — light-activated ion channels on the cell surface that open briefly when triggered — turn on strongly and then quickly shut off, all in about 3 milliseconds, so they’re ready to be re-stimulated up to 50 or more times per second, consistent with normal firing rates in the cortex.

Pégard developed the holographic projection system using a liquid crystal screen that acts like a holographic negative to sculpt the light from 40W lasers into the desired 3D pattern. The lasers are pulsed in 300 femtosecond-long bursts every microsecond. He, Mardinly, Oldenburg and their colleagues published a paper last year describing the device, which they call 3D-SHOT, for three-dimensional scanless holographic optogenetics with temporal focusing.

“This is the culmination of technologies that researchers have been working on for a while, but have been impossible to put together,” Mardinly said. “We solved numerous technical problems at the same time to bring it all together and finally realize the potential of this technology.”

As they improve their technology, they plan to start capturing real patterns of activity in the cortex in order to learn how to reproduce sensations and perceptions to play back through their holographic system.

Reference:
Mardinly, A. R., Oldenburg, I. A., Pégard, N. C., Sridharan, S., Lyall, E. H., Chesnov, K., . . . Adesnik, H. (2018). Precise multimodal optical control of neural ensemble activity. Nature Neuroscience. doi:10.1038/s41593-018-0139-8

https://www.technologynetworks.com/neuroscience/news/using-holography-to-activate-the-brain-300329?utm_campaign=Newsletter_TN_BreakingScienceNews&utm_source=hs_email&utm_medium=email&utm_content=62560457&_hsenc=p2ANqtz–bJrpQXF2dp2fYgPpEKUOIkhpHxOYZR7Nx-irsQ649T-Ua02wmYTaBOkA9joFtI9BGKIAUb1NoL7-s27Rj9XMPH44XUw&_hsmi=62560457

Advertisements

By Andrea Peterson

Almost everywhere you turn, it seems like people have their eyes glued to smartphone screens playing Pokemon Go. Since its launch last week, the app has quickly become a cultural phenomenon that has fans of all ages hunting around their neighborhoods for collectible digital creatures that appear on players’ screens almost like magic as they explore real-world locations.

But there’s at least one place that would really like to keep Pokemon out: the Holocaust Museum.

The Museum itself, along with many other landmarks, is a “PokeStop” within the game — a place where players can get free in-game items. In fact, there are actually three different PokeStops associated with various parts of the museum.

“Playing the game is not appropriate in the museum, which is a memorial to the victims of Nazism,” Andrew Hollinger, the museum’s communications director, told The Post in an interview. “We are trying to find out if we can get the museum excluded from the game.”

The Holocaust Museum’s plight highlights how apps that layer a digital world on top of the real one, or so-called augmented reality games, can come with unforeseen consequences and raises questions about how much control the physical owner of a space can exert as those two worlds intersect.

One image circulating online appears to show a player encountering an unsettling digital critter inside the museum: a poison gas-type Pokemon called Koffing floating by a sign for the museum’s Helena Rubinstein Auditorium. Clearly, it’s an awkward if not offensive thing to find in a place dedicated to making sure the world never forgets those killed in the gas chambers at Nazi death camps.

The image, which appears to have originated from a now deleted post on the photo-sharing site imgur, might be a hoax: That particular Pokemon didn’t appear nearby when this Post reporter visited the museum Monday afternoon, although the specific Pokemon that appears in each location does vary from time to time. Hollinger said that the museum is aware of and concerned about the potential Koffing appearance.

If a player really did encounter a Koffing inside the museum, it seems unlikely that Niantic Labs— the developers behind the game — would have made it show up there on purpose. But even the specter of that particular Pokemon spawning there raises questions about what sort of controls the game has in place to stop its fantasy world from causing distress in the real world, even if that distress is unintentional.

Niantic did not immediately respond to inquiries about the alleged Koffing sighting or if there was any way to honor the Holocaust Museum’s request to stop Pokemon from popping up inside its building.

Hollinger stressed that the museum is generally pro-technology and encourages visitors to use social media to share how their experiences with the exhibits moved them. “But this game falls very much outside that,” he said. Hollinger added that it seemed disrespectful, especially in more solemn parts of the complex like the Hall of Remembrance.

On Monday afternoon, there were plenty of people inside the museum who seemed to be distracted from its haunting exhibits as they tried to “catch ’em all.” A player even used a lure module, a beacon that attracts Pokemon to a specific PokeStop, on the museum’s marker — making double-headed bird-like creatures dubbed Doduos and rodent-like Rattatas practically swarm on users’ screens.

The player behind the lure, a 30-year-old visiting from North Carolina named Dustin who declined to share his last name with The Post for privacy reasons, was excited to catch a crustacean-like Krabby while waiting in the museum’s lobby with a group of friends to pick up tickets for a scheduled tour of the exhibits.

Although the museum is uncomfortable with its Pokemon infestation, most of the players building up their digital critter collection inside the building at least didn’t seem to mean any disrespect.

“It’s not like we came here to play,” said Angie, a 37-year-old member of Dustin’s huddle who also declined to share her last name for privacy reasons, “But gotta catch ’em all.”

https://www.washingtonpost.com/news/the-switch/wp/2016/07/12/holocaust-museum-to-visitors-please-stop-catching-pokemon-here/?campaign_id=A100&campaign_type=Email