The holodeck: Neural interface

In the last few weeks I have examined a few technologies that could function like a holodeck. Today I’ll examine the potential of the neural interface a hypothetical matrix-style technology.

A neural interface is a device that links directly to and interacts with your brain. Werther via a direct connection like in The Matrix or via a non-invasive method, the neural interface promises to be the ultimate holodeck experience even if it seems a little scary as well.

for a realistic holodeck experience you’d need three things. The first is that the computer understands what the hell you are trying to do. That your brains signal for turning left is actually translated into a left turning movement. with a tried and tested method like EEG this can be done today even if it is a little rudimentary and not quite exact. Luckily the brain can adapt and learn and increase efficiency of EEG based controllers (the brain waves the machine reads become more distinct when using it regularly.

Though EEG has been able to give us basic control of computers they are limited. The EEG only reads general brainwaves but does not have the resolution required for the fine motor control you’d need to play a first person shooter for instance. A better bet might be a fMRI. The fMRI measures the blood flow to the brain. The more blood flows to a certain part the more active it is. The fMRI has the advantage that it can measure the entire brain not just the outer layer. The fMRI also has the potential of being way more accurate then an EEG. Downsides to the fMRI are that it works with magnets and thus needs to be shielded from the rest of the world (unless you want to pry the cutlery from the wall each day.) Also the fMRI is the size of a small room and gets bigger if higher resolutions are required though advances in nanotechnology might decrease the size eventually.

A third method would be by inserting a network of small sensors into the brain capable of reading brain activity on a smaller scale then EEG. Downside to this obviously is that you’d need brain surgery with all the risks of complications. We’d even be able to let brain cells directly interact with the chips if we want to.Don’t worry about needing to plug cables into your spine though. These chips would probably be accessed via a wireless technology.

Second thing you’d need is that the computer output is translated into sensory input again. The easiest way to do this is just using current technology. TV’s, headphones, speakers, vibration in game controllers are all designed to translate computer output into sensory input. Trouble is that even the best of these are not realistic. Even advanced simulators are clearly not real.

Luckily we don’t need our senses to create sensory input because all sensory input is processed in the brain. And in the brain alone can we make sense of what our senses detect. By directly stimulating the brain you can create false sensory input. In effect you create the holodeck within your head. experiences gained this way would be indistinguishable from real experiences (except maybe for the fact you are able to fly of course.) Again there are basically two ways of doing this. Invasive or non-invasive.

The non-invasive methods work by stimulating your brain either via electricity tDCS or magnetism TMS. They work by activating your neurons so that they start firing signals to other neurons. In this way you can trick the senses into seeing, hearing and feeling things that are not actually happening. One thing to worry about though is whether this would create a double image so that you’d not only see the ‘holodeck’ image but also what your eyes actually see. (I imagine that you could get quite sick from a double input quite quickly not unlike seasickness.) The viability of either system as a sensory input device is quite questionable as well. Can we actually fire individual neurons in the right pattern? Can we penetrate the brain deeply enough to create realistic input? Because they are non invasive you need to penetrate the brain from the outside which is quite tricky as you can imagine and currently TMS and tDCS can only stimulate large parts of the brain (in terms of brain tissue even a cubic millimetre is a large part).

The invasive method is quite easy (in theory at least): you just hook a chip to the sensory nerves leading to the brain. When you activate your holodeck you just shut off the real input and replace it with the computer images. In practice it is a little trickier of course. It would require cutting the nerve and attaching the chip to each individual cell. Something which is impossible with current technology.

The third thing you’d need is temporary paralysis. Though this may sound scary it is actually already build into our brains. It’s function is to prevent us from acting out our dreams and thus putting ourselves into dangerous situations. Cases are known in which people attack or have sex with their spouses in their sleep because of a lack of sleep paralysis. The sleep paralysis could easily be activated be tDCS, TMS or by implanting a chip and may even be the easiest to accomplish in our neural interface holodeck.

In conclusion I can say that if a holodeck based on the neural interface will become a reality it is still a long way off. Even if the technology was invented tomorrow to do it practically we’d still lack the knowledge of how the brain actually works to manipulate it so precisely. It sounds scary but would basically be an on demand dream machine. In the case of implants based on wireless technology hacking would be a real concern however.

The holodeck: Exoskeleton

An impression of a personal simulator based on the exoskeleton technology

An impression of a personal simulator based on the exoskeleton technology.

In the coming weeks I will examine a few technologies that could function like a holodeck. Today I’ll examine the potential of the exoskeleton as a holodeck replacement.

The exoskeleton is essentially a robot which you strap to your body. It applications are vast, most noticeably helping you lift heavy loads with ease and people currently in a wheelchair will be able to walk again. It is even predicted that we will all be wearing exoskeletons within the next fifty years. Next to those great promises we can see entertainment applications as well.

If you program the exoskeleton to provide resistance, mount it on a base which can turn on two axis, an arm to simulate list, an awesome sound system and put on 3D/ holographic goggles et voilà you have a personal simulator which fits in a room (see my ‘awesome’ Photoshop impression). For the first time in gaming you will actually feel the weight of the sword in your hand as you slay your enemies, be in the cockpit of your F1 car or at the bridge of the USS Enterprise. The sensors in the exoskeleton would eliminate the need for any other input device. Just grab the sword and you are ready to slash your enemy or take a hold of the steering wheel of your favourite car etc. You can do anything you want as if it was real.

As the systems of the exoskeleton itself get smaller you could add more functionalities to increase the experience. You could for instance add a sense of hot and cold, a sense of touch or a sense of smell. The easiest to incorporate would be hot and cold so it is likely to be added first. Smell is a little harder, because it would require some plumbing to get the smell near your nose. Touch over a large portion of the body is hard to do. It would require a lot of sophisticated output devices, not in the last place because our sense of touch is pretty sophisticated.

This system though being an awesome gaming system would first see military and commercial applications. The military would use it for training soldiers and preparing missions. Commercially it could replace the simulators now used to train pilots and captains. The biggest advantage for this system is that you can change the layout of the flight deck/bridge by loading a different program instead of having to build a new simulator which costs millions. It’s small size is a big advantage as well. Although if you have a larger space you could opt to simulate G-forces making for a more realistic experience. This in turn giving the crew an even better chance of surviving in the event of an emergency.

The biggest problem at the moment is that an exoskeleton is very expensive (although you can hire one for $590 or €460 a month). The technology required is still pretty much in it’s infancy and they are not yet mass produced. Also I do not know of anyone developing a system like this for entertainment purposes at the moment. However, if we really will walk in exoskeletons all day is only a matter of time before somebody will.

The holodeck: Current status.

A simulator for entertainment. U.S. Navy photo by Journalist 1st Class Stephanie Souderlund

A simulator for entertainment. U.S. Navy photo by Journalist 1st Class Stephanie Souderlund

In the coming weeks I will examine a few technologies that could function like a holodeck but first I’ll examine how far we are today.

Lets start with the basics here: What is a holodeck? The concept of the holodeck originates from Star Trek The Next Generation. It is a large room in which reality can be simulated. It is a room which uses a combination of holographic images, teleportation technology, replication technology, tractor beams and force fields to create a lifelike representation of the world (whichever world that might be). The holodeck as described in Star Trek is fiction of course and quite possibly will never be a reality as described. There are however several technologies that will or could basically do the same thing.

First off the personal computer and gaming systems. You might think it is a big step from these to a holodeck but actually a lot of things needed for a holodeck are actually already incorporated in these systems. They render their virtual worlds in 3D, contain information about what are solid objects, how you move over certain terrain, great gaming features and more. Of course a solid object is just solid so a wall and a person will both feel like solid brick but still many information in games is usable for a holodeck. of course the biggest issue is that you cannot enter the world yourself. You will always need to rely on a screen and some kind of input device. (although the Wii, Xbox kinect and Playstation Eye take a few first steps towards eliminating the clumsy (unnatural) controller altogether. On the plus side these systems are cheap and have come a long way in just a few decades.

A step up is a system called the CAVE. It has three or more walls (sometimes including floor and ceiling) on which 3D images are projected. With 3D glasses (similar to the ones for your 3D tv) you get a holographic simulation. By walking around an object you can view it from all sides like an actual holographic image. With new technologies (similar to the aforementioned Xbox Kninect etc.) you are even able to interact with these objects to some degree. The lack of a physical form is a big disadvantage however. To be able to truly interact with an object you need to be able to handle it as well. That is why most video’s of people interacting with virtual objects seems so clumsy, you just cannot get an idea of weight, form and feel of an object. Another big disadvantage of this system is the space you need (it is a room within a room so you need an awful amount of space) and the money a system like this costs.
The last problem is that it is unfit for young children and some people experience headaches when using the system. This is due to the actual technology. The information our eyes gets says an object is somewhere in the room, the actual object is on a screen however and so the eyes shift focus between the screen and where the object is expected to be. This rapid focussing between the two causes the headaches but is also why children shouldn’t use it. Their eyes are still developing and the 3D technology can hurt the development of the eyes.

The best holodeck equivalent  we currently have are the big simulators used to train pilots, ship captains, Formula One drivers or are used in an amusement park as entertainment. They act and feel like the actual thing and by the use of hydraulic pistons simulate movement of the ship, car or plane. The latest version, based in the Netherlands, is even able to simulate gravity (or the lack thereof). The biggest disadvantages of these machines is that they are very large, require a crew to operate (both for maintenance and running the training), cost a lot of money and require you to purchase a new machine every time you want to use it for a different plane/car/boat.