The holodeck: downsides and upsides

Copyright South Park Digital Studios L..L.C.

Copyright South Park Digital Studios L..L.C.

Today the last story in a series about the holodeck and technologies promising us a realistic virtual reality. With the invention of more realistic virtual reality interfaces we will also find that new problems will arise.

When the virtual world cannot be discriminated from the real one addiction will become a big problem. In Star Trek holodeck addiction was indeed a mental illness. They did however underestimated the problem it will become. In Star Trek they argued that because  holodeck isn’t real, people wouldn’t want to spend all their time on it. The problem when a virtual reality system becomes as real as a holodeck is that we cannot distinguish any more between real and fantasy. Though we will know on a cognitive level that it isn’t real all of our senses will tell us otherwise. And to make a long story short, we don’t care about our cognitive objections when it feels so real. Gaming addiction is already a problem with a small percentage of gamers and we don’t even have that realistic virtual reality environments, best you can do now is wear 3D goggles.

You need to remember that a holodeck provides everything you cannot do in the real world and provides the perfect escape. Get bullied at school? Be a super hero! Can’t get a girl? Have twenty! Don’t have an interesting job? How about captain of a star ship? Who wants to come out unless they have to? Would you?

Another potential problem is PTSD or post traumatic stress disorder. PTSD is a mental illness caused by a traumatic event. Many soldiers suffer from PTSD when they come back from a war zone. Seen as shooters are one of the most popular games we might see an increase in people who suffer from PTSD. It might be that shooters become less popular or get more cartoony to offset the realism so people can’t suffer from PTSD. Some protection might lie in the fact that we know it is entertainment but if it becomes to real that won’t help any more.

A realistic virtual reality will mean that people will form relationships with virtual characters. While I will explored the implications of this in this post I do want to talk about sex addiction here. When people start to neglect other parts of their life so that they can have sex or masturbate not even for pleasure any more they can be a sex addict. Though different from a drug addiction the effects can still be devastating with the loss of job, partner, family and friends. A holodeck could be a risk for people who are already at risk for a (sex)addiction since it can supply an endless amount of virtual partners in any number of sexual fantasies.

The addictiveness of the holodeck will have undoubtedly an effect on productiveness of employees who are yearning to get back to their fantasy lives. Other forms of entertainment will suffer as well although these might be incorporated in their fantasy worlds.

There is an upside as well though. We will see less loneliness. A virtual partner and friend is just as good as a real one. We just want other people to notice us, that they are virtual is a distinction our mind does not make. this will also lead to less depression and anxiety. We are invincible in our own fantasies and can take on the world. Our virtual training will help us prepare for the real world outside.

And; even if all else fails there is always a virtual psychologist at hand to help you with your problems.

The holodeck: Neural interface

In the last few weeks I have examined a few technologies that could function like a holodeck. Today I’ll examine the potential of the neural interface a hypothetical matrix-style technology.

A neural interface is a device that links directly to and interacts with your brain. Werther via a direct connection like in The Matrix or via a non-invasive method, the neural interface promises to be the ultimate holodeck experience even if it seems a little scary as well.

for a realistic holodeck experience you’d need three things. The first is that the computer understands what the hell you are trying to do. That your brains signal for turning left is actually translated into a left turning movement. with a tried and tested method like EEG this can be done today even if it is a little rudimentary and not quite exact. Luckily the brain can adapt and learn and increase efficiency of EEG based controllers (the brain waves the machine reads become more distinct when using it regularly.

Though EEG has been able to give us basic control of computers they are limited. The EEG only reads general brainwaves but does not have the resolution required for the fine motor control you’d need to play a first person shooter for instance. A better bet might be a fMRI. The fMRI measures the blood flow to the brain. The more blood flows to a certain part the more active it is. The fMRI has the advantage that it can measure the entire brain not just the outer layer. The fMRI also has the potential of being way more accurate then an EEG. Downsides to the fMRI are that it works with magnets and thus needs to be shielded from the rest of the world (unless you want to pry the cutlery from the wall each day.) Also the fMRI is the size of a small room and gets bigger if higher resolutions are required though advances in nanotechnology might decrease the size eventually.

A third method would be by inserting a network of small sensors into the brain capable of reading brain activity on a smaller scale then EEG. Downside to this obviously is that you’d need brain surgery with all the risks of complications. We’d even be able to let brain cells directly interact with the chips if we want to.Don’t worry about needing to plug cables into your spine though. These chips would probably be accessed via a wireless technology.

Second thing you’d need is that the computer output is translated into sensory input again. The easiest way to do this is just using current technology. TV’s, headphones, speakers, vibration in game controllers are all designed to translate computer output into sensory input. Trouble is that even the best of these are not realistic. Even advanced simulators are clearly not real.

Luckily we don’t need our senses to create sensory input because all sensory input is processed in the brain. And in the brain alone can we make sense of what our senses detect. By directly stimulating the brain you can create false sensory input. In effect you create the holodeck within your head. experiences gained this way would be indistinguishable from real experiences (except maybe for the fact you are able to fly of course.) Again there are basically two ways of doing this. Invasive or non-invasive.

The non-invasive methods work by stimulating your brain either via electricity tDCS or magnetism TMS. They work by activating your neurons so that they start firing signals to other neurons. In this way you can trick the senses into seeing, hearing and feeling things that are not actually happening. One thing to worry about though is whether this would create a double image so that you’d not only see the ‘holodeck’ image but also what your eyes actually see. (I imagine that you could get quite sick from a double input quite quickly not unlike seasickness.) The viability of either system as a sensory input device is quite questionable as well. Can we actually fire individual neurons in the right pattern? Can we penetrate the brain deeply enough to create realistic input? Because they are non invasive you need to penetrate the brain from the outside which is quite tricky as you can imagine and currently TMS and tDCS can only stimulate large parts of the brain (in terms of brain tissue even a cubic millimetre is a large part).

The invasive method is quite easy (in theory at least): you just hook a chip to the sensory nerves leading to the brain. When you activate your holodeck you just shut off the real input and replace it with the computer images. In practice it is a little trickier of course. It would require cutting the nerve and attaching the chip to each individual cell. Something which is impossible with current technology.

The third thing you’d need is temporary paralysis. Though this may sound scary it is actually already build into our brains. It’s function is to prevent us from acting out our dreams and thus putting ourselves into dangerous situations. Cases are known in which people attack or have sex with their spouses in their sleep because of a lack of sleep paralysis. The sleep paralysis could easily be activated be tDCS, TMS or by implanting a chip and may even be the easiest to accomplish in our neural interface holodeck.

In conclusion I can say that if a holodeck based on the neural interface will become a reality it is still a long way off. Even if the technology was invented tomorrow to do it practically we’d still lack the knowledge of how the brain actually works to manipulate it so precisely. It sounds scary but would basically be an on demand dream machine. In the case of implants based on wireless technology hacking would be a real concern however.

The holodeck: Modular Robotics

In the coming weeks I will examine a few technologies that could function like a holodeck. Today I’ll examine the modular robotics.

This is my favourite holodeck replacement because it resembles the original holodeck in Star Trek most closely. Imagine a large room, about 2 stories high. You enter on the first floor, the floor you now stand on is silvery grey, these are robots. Half the room is filled with millions of robots, even smaller then a grain of sand.

The holodeck of Star trek, uses many exotic technologies like forcefields, transporters and replicators to create a realistic fantasy world within a confined space. Though this is great it is also uncertain at best if all the required technologies will ever become a reality. It is way easier to use robotics to do pretty much the same thing, with a little help from holographic projectors maybe.

Modular robotics are like high tech LEGO bricks. Each module is a small computer that has sensors and can connect with other modules. when they interact they essentially become a supercomputer which is able to rearrange itself into complex structures. the modules themselves are responsible for forming into the right objects with the right characteristics (soft or hard, warm or cold, colour, large or small, square or round etc.) while a central computer is responsible for the overall scene that needs to be created (e.g. a house with a bench in front on which a woman sits who is scolding you for being late).

If you walk across a street the scene changes accordingly. What will happen is that on one side of the room object are rapidly constructed and on the other side they are broken down just as fast. The robots get from one side to the other in a way that is not unlike the ocean currents. on ground level the robots move in on direction and underground a torrent of robots moves in the opposite direction effectively keeping you in the middle of the room. far off objects are projected on the walls and/or created with holographic projectors.

Of course the first generations of these blocks aren’t all that great. The modular blocks are not intelligent and need to be assembled by hand to do anything but they will eventually become more powerful and will eventually gain more and more of the functions I described above. When they get a resolution of a centimetre square (about half an inch square) it could get some applications. For instance in the military, allowing for urban warfare training in a large area or an architect showing a house not even build yet. When they get down to one millimetre square (about 1/25 inch square) it will be good enough to have wide scale applications. From designing a production line and training workers to work with that production line to entertainment purposes. When it gets down to the size of sand I think you will have a nearly real virtual reality.

Upkeep is easy, just add a bucket of new modules to replace faulty ones every so often. The faulty ones are detected by the modules around it and kept apart until they can be discarded by the user. Further along the line the faulty ones will be filtered out and repaired or recycled in a special part of the ‘holodeck.’ Which will eliminate upkeep altogether. On the downside: so many robots and computers will require a lot of power. In order to meet the power demand we will need new sustainable sources of power like solar, wind, geothermal or fusion power. Another downside is that it requires a relatively large space.

The holodeck: Exoskeleton

An impression of a personal simulator based on the exoskeleton technology

An impression of a personal simulator based on the exoskeleton technology.

In the coming weeks I will examine a few technologies that could function like a holodeck. Today I’ll examine the potential of the exoskeleton as a holodeck replacement.

The exoskeleton is essentially a robot which you strap to your body. It applications are vast, most noticeably helping you lift heavy loads with ease and people currently in a wheelchair will be able to walk again. It is even predicted that we will all be wearing exoskeletons within the next fifty years. Next to those great promises we can see entertainment applications as well.

If you program the exoskeleton to provide resistance, mount it on a base which can turn on two axis, an arm to simulate list, an awesome sound system and put on 3D/ holographic goggles et voilà you have a personal simulator which fits in a room (see my ‘awesome’ Photoshop impression). For the first time in gaming you will actually feel the weight of the sword in your hand as you slay your enemies, be in the cockpit of your F1 car or at the bridge of the USS Enterprise. The sensors in the exoskeleton would eliminate the need for any other input device. Just grab the sword and you are ready to slash your enemy or take a hold of the steering wheel of your favourite car etc. You can do anything you want as if it was real.

As the systems of the exoskeleton itself get smaller you could add more functionalities to increase the experience. You could for instance add a sense of hot and cold, a sense of touch or a sense of smell. The easiest to incorporate would be hot and cold so it is likely to be added first. Smell is a little harder, because it would require some plumbing to get the smell near your nose. Touch over a large portion of the body is hard to do. It would require a lot of sophisticated output devices, not in the last place because our sense of touch is pretty sophisticated.

This system though being an awesome gaming system would first see military and commercial applications. The military would use it for training soldiers and preparing missions. Commercially it could replace the simulators now used to train pilots and captains. The biggest advantage for this system is that you can change the layout of the flight deck/bridge by loading a different program instead of having to build a new simulator which costs millions. It’s small size is a big advantage as well. Although if you have a larger space you could opt to simulate G-forces making for a more realistic experience. This in turn giving the crew an even better chance of surviving in the event of an emergency.

The biggest problem at the moment is that an exoskeleton is very expensive (although you can hire one for $590 or €460 a month). The technology required is still pretty much in it’s infancy and they are not yet mass produced. Also I do not know of anyone developing a system like this for entertainment purposes at the moment. However, if we really will walk in exoskeletons all day is only a matter of time before somebody will.

The holodeck: Holographic TV

In the coming weeks I will examine a few technologies that could function like a holodeck. Today I’ll talk about the holographic TV [hTV] a technology which is just around the corner.

A true hTV differs from a 3D TV by making the objects appear in the room instead of merely creating an illusion that they are 3D.  When viewing a 3D TV everybody watches the same picture no matter where you are in relation to the screen. A hTV sends out the light in such a way that an object will actually appear to be in the room. your relation to the TV will therefore actually determine what you view. You can compare the difference between viewing a scene in a show box and having a model of the same scene. In the show box you see a scene from a fixed perspective while you can walk around a model.

A hTV has the benefit of not being harmful to the eyes and eye development in children and will not cause headaches and exhaustion associated with traditional 3D TV’s. This is because a hTV actually projects the object into the room so your eyes can focus on an “object” instead of the screen.

Restrictions with native hTV content have to do with the available processing power and bandwidth. You can imagine that in order to display an object in 3D from any given angle requires a massive amount of data. This will mean that in the first generations of the hTV it will track your eye movements and only display the viewing angle you see. This means that the number of viewers is automatically limited to the number of people the hTV can track and display an object for. As the cost of bandwidth and processing power decreases more angles will be added.

Early adopters of the hTV will be of course the military gaining an advantage when they are able to plan missions on a live 3D map. I also see potential for industrial designers and architects allowing clients to view their work without having to produce a physical model. This means they can adapt their work to the clients wish live with the click of a button. Of course the advertisement industry will use the technology to attract attention to their products. Later on theme parks and cinema will install hTV technology to entertain their costumers.

The first hTV native consumer applications will probably come from the gaming industry. Which just eliminates the step which makes their 3D games able to display on a 2D screen. Sports will benefit from hTV technology allowing you to view the pitch from any angle you desire and allowing you to walk around so you can see what the players see. The other early adopters for consumers will be the porn industry which, unlike Hollywood, does not care about artistic value of their production as long as it gets the job done.

Hollywood will need more time however so they can figure out how to get the story across when people can view it from unintended angles. Maybe they will keep to showing their films more like the traditional 3D technology. Where there is just one viewpoint for all viewers, independent of the location of the viewer like a show box having layers of 2D images projected a little bit apart from each other creating the illusion of 3D.

the technology to shoot a film in native hTV is already available to consumers. A hacked xbox kinect camera has been shown to film a room 180 degrees in 3D. two or three of these camera’s could capture (nearly) an entire room allowing you to view an action in that room from any angle.

The holodeck: Current status.

A simulator for entertainment. U.S. Navy photo by Journalist 1st Class Stephanie Souderlund

A simulator for entertainment. U.S. Navy photo by Journalist 1st Class Stephanie Souderlund

In the coming weeks I will examine a few technologies that could function like a holodeck but first I’ll examine how far we are today.

Lets start with the basics here: What is a holodeck? The concept of the holodeck originates from Star Trek The Next Generation. It is a large room in which reality can be simulated. It is a room which uses a combination of holographic images, teleportation technology, replication technology, tractor beams and force fields to create a lifelike representation of the world (whichever world that might be). The holodeck as described in Star Trek is fiction of course and quite possibly will never be a reality as described. There are however several technologies that will or could basically do the same thing.

First off the personal computer and gaming systems. You might think it is a big step from these to a holodeck but actually a lot of things needed for a holodeck are actually already incorporated in these systems. They render their virtual worlds in 3D, contain information about what are solid objects, how you move over certain terrain, great gaming features and more. Of course a solid object is just solid so a wall and a person will both feel like solid brick but still many information in games is usable for a holodeck. of course the biggest issue is that you cannot enter the world yourself. You will always need to rely on a screen and some kind of input device. (although the Wii, Xbox kinect and Playstation Eye take a few first steps towards eliminating the clumsy (unnatural) controller altogether. On the plus side these systems are cheap and have come a long way in just a few decades.

A step up is a system called the CAVE. It has three or more walls (sometimes including floor and ceiling) on which 3D images are projected. With 3D glasses (similar to the ones for your 3D tv) you get a holographic simulation. By walking around an object you can view it from all sides like an actual holographic image. With new technologies (similar to the aforementioned Xbox Kninect etc.) you are even able to interact with these objects to some degree. The lack of a physical form is a big disadvantage however. To be able to truly interact with an object you need to be able to handle it as well. That is why most video’s of people interacting with virtual objects seems so clumsy, you just cannot get an idea of weight, form and feel of an object. Another big disadvantage of this system is the space you need (it is a room within a room so you need an awful amount of space) and the money a system like this costs.
The last problem is that it is unfit for young children and some people experience headaches when using the system. This is due to the actual technology. The information our eyes gets says an object is somewhere in the room, the actual object is on a screen however and so the eyes shift focus between the screen and where the object is expected to be. This rapid focussing between the two causes the headaches but is also why children shouldn’t use it. Their eyes are still developing and the 3D technology can hurt the development of the eyes.

The best holodeck equivalent  we currently have are the big simulators used to train pilots, ship captains, Formula One drivers or are used in an amusement park as entertainment. They act and feel like the actual thing and by the use of hydraulic pistons simulate movement of the ship, car or plane. The latest version, based in the Netherlands, is even able to simulate gravity (or the lack thereof). The biggest disadvantages of these machines is that they are very large, require a crew to operate (both for maintenance and running the training), cost a lot of money and require you to purchase a new machine every time you want to use it for a different plane/car/boat.