The holodeck: Neural interface

In the last few weeks I have examined a few technologies that could function like a holodeck. Today I’ll examine the potential of the neural interface a hypothetical matrix-style technology.

A neural interface is a device that links directly to and interacts with your brain. Werther via a direct connection like in The Matrix or via a non-invasive method, the neural interface promises to be the ultimate holodeck experience even if it seems a little scary as well.

for a realistic holodeck experience you’d need three things. The first is that the computer understands what the hell you are trying to do. That your brains signal for turning left is actually translated into a left turning movement. with a tried and tested method like EEG this can be done today even if it is a little rudimentary and not quite exact. Luckily the brain can adapt and learn and increase efficiency of EEG based controllers (the brain waves the machine reads become more distinct when using it regularly.

Though EEG has been able to give us basic control of computers they are limited. The EEG only reads general brainwaves but does not have the resolution required for the fine motor control you’d need to play a first person shooter for instance. A better bet might be a fMRI. The fMRI measures the blood flow to the brain. The more blood flows to a certain part the more active it is. The fMRI has the advantage that it can measure the entire brain not just the outer layer. The fMRI also has the potential of being way more accurate then an EEG. Downsides to the fMRI are that it works with magnets and thus needs to be shielded from the rest of the world (unless you want to pry the cutlery from the wall each day.) Also the fMRI is the size of a small room and gets bigger if higher resolutions are required though advances in nanotechnology might decrease the size eventually.

A third method would be by inserting a network of small sensors into the brain capable of reading brain activity on a smaller scale then EEG. Downside to this obviously is that you’d need brain surgery with all the risks of complications. We’d even be able to let brain cells directly interact with the chips if we want to.Don’t worry about needing to plug cables into your spine though. These chips would probably be accessed via a wireless technology.

Second thing you’d need is that the computer output is translated into sensory input again. The easiest way to do this is just using current technology. TV’s, headphones, speakers, vibration in game controllers are all designed to translate computer output into sensory input. Trouble is that even the best of these are not realistic. Even advanced simulators are clearly not real.

Luckily we don’t need our senses to create sensory input because all sensory input is processed in the brain. And in the brain alone can we make sense of what our senses detect. By directly stimulating the brain you can create false sensory input. In effect you create the holodeck within your head. experiences gained this way would be indistinguishable from real experiences (except maybe for the fact you are able to fly of course.) Again there are basically two ways of doing this. Invasive or non-invasive.

The non-invasive methods work by stimulating your brain either via electricity tDCS or magnetism TMS. They work by activating your neurons so that they start firing signals to other neurons. In this way you can trick the senses into seeing, hearing and feeling things that are not actually happening. One thing to worry about though is whether this would create a double image so that you’d not only see the ‘holodeck’ image but also what your eyes actually see. (I imagine that you could get quite sick from a double input quite quickly not unlike seasickness.) The viability of either system as a sensory input device is quite questionable as well. Can we actually fire individual neurons in the right pattern? Can we penetrate the brain deeply enough to create realistic input? Because they are non invasive you need to penetrate the brain from the outside which is quite tricky as you can imagine and currently TMS and tDCS can only stimulate large parts of the brain (in terms of brain tissue even a cubic millimetre is a large part).

The invasive method is quite easy (in theory at least): you just hook a chip to the sensory nerves leading to the brain. When you activate your holodeck you just shut off the real input and replace it with the computer images. In practice it is a little trickier of course. It would require cutting the nerve and attaching the chip to each individual cell. Something which is impossible with current technology.

The third thing you’d need is temporary paralysis. Though this may sound scary it is actually already build into our brains. It’s function is to prevent us from acting out our dreams and thus putting ourselves into dangerous situations. Cases are known in which people attack or have sex with their spouses in their sleep because of a lack of sleep paralysis. The sleep paralysis could easily be activated be tDCS, TMS or by implanting a chip and may even be the easiest to accomplish in our neural interface holodeck.

In conclusion I can say that if a holodeck based on the neural interface will become a reality it is still a long way off. Even if the technology was invented tomorrow to do it practically we’d still lack the knowledge of how the brain actually works to manipulate it so precisely. It sounds scary but would basically be an on demand dream machine. In the case of implants based on wireless technology hacking would be a real concern however.

The holodeck: Modular Robotics

In the coming weeks I will examine a few technologies that could function like a holodeck. Today I’ll examine the modular robotics.

This is my favourite holodeck replacement because it resembles the original holodeck in Star Trek most closely. Imagine a large room, about 2 stories high. You enter on the first floor, the floor you now stand on is silvery grey, these are robots. Half the room is filled with millions of robots, even smaller then a grain of sand.

The holodeck of Star trek, uses many exotic technologies like forcefields, transporters and replicators to create a realistic fantasy world within a confined space. Though this is great it is also uncertain at best if all the required technologies will ever become a reality. It is way easier to use robotics to do pretty much the same thing, with a little help from holographic projectors maybe.

Modular robotics are like high tech LEGO bricks. Each module is a small computer that has sensors and can connect with other modules. when they interact they essentially become a supercomputer which is able to rearrange itself into complex structures. the modules themselves are responsible for forming into the right objects with the right characteristics (soft or hard, warm or cold, colour, large or small, square or round etc.) while a central computer is responsible for the overall scene that needs to be created (e.g. a house with a bench in front on which a woman sits who is scolding you for being late).

If you walk across a street the scene changes accordingly. What will happen is that on one side of the room object are rapidly constructed and on the other side they are broken down just as fast. The robots get from one side to the other in a way that is not unlike the ocean currents. on ground level the robots move in on direction and underground a torrent of robots moves in the opposite direction effectively keeping you in the middle of the room. far off objects are projected on the walls and/or created with holographic projectors.

Of course the first generations of these blocks aren’t all that great. The modular blocks are not intelligent and need to be assembled by hand to do anything but they will eventually become more powerful and will eventually gain more and more of the functions I described above. When they get a resolution of a centimetre square (about half an inch square) it could get some applications. For instance in the military, allowing for urban warfare training in a large area or an architect showing a house not even build yet. When they get down to one millimetre square (about 1/25 inch square) it will be good enough to have wide scale applications. From designing a production line and training workers to work with that production line to entertainment purposes. When it gets down to the size of sand I think you will have a nearly real virtual reality.

Upkeep is easy, just add a bucket of new modules to replace faulty ones every so often. The faulty ones are detected by the modules around it and kept apart until they can be discarded by the user. Further along the line the faulty ones will be filtered out and repaired or recycled in a special part of the ‘holodeck.’ Which will eliminate upkeep altogether. On the downside: so many robots and computers will require a lot of power. In order to meet the power demand we will need new sustainable sources of power like solar, wind, geothermal or fusion power. Another downside is that it requires a relatively large space.

The holodeck: Exoskeleton

An impression of a personal simulator based on the exoskeleton technology

An impression of a personal simulator based on the exoskeleton technology.

In the coming weeks I will examine a few technologies that could function like a holodeck. Today I’ll examine the potential of the exoskeleton as a holodeck replacement.

The exoskeleton is essentially a robot which you strap to your body. It applications are vast, most noticeably helping you lift heavy loads with ease and people currently in a wheelchair will be able to walk again. It is even predicted that we will all be wearing exoskeletons within the next fifty years. Next to those great promises we can see entertainment applications as well.

If you program the exoskeleton to provide resistance, mount it on a base which can turn on two axis, an arm to simulate list, an awesome sound system and put on 3D/ holographic goggles et voilà you have a personal simulator which fits in a room (see my ‘awesome’ Photoshop impression). For the first time in gaming you will actually feel the weight of the sword in your hand as you slay your enemies, be in the cockpit of your F1 car or at the bridge of the USS Enterprise. The sensors in the exoskeleton would eliminate the need for any other input device. Just grab the sword and you are ready to slash your enemy or take a hold of the steering wheel of your favourite car etc. You can do anything you want as if it was real.

As the systems of the exoskeleton itself get smaller you could add more functionalities to increase the experience. You could for instance add a sense of hot and cold, a sense of touch or a sense of smell. The easiest to incorporate would be hot and cold so it is likely to be added first. Smell is a little harder, because it would require some plumbing to get the smell near your nose. Touch over a large portion of the body is hard to do. It would require a lot of sophisticated output devices, not in the last place because our sense of touch is pretty sophisticated.

This system though being an awesome gaming system would first see military and commercial applications. The military would use it for training soldiers and preparing missions. Commercially it could replace the simulators now used to train pilots and captains. The biggest advantage for this system is that you can change the layout of the flight deck/bridge by loading a different program instead of having to build a new simulator which costs millions. It’s small size is a big advantage as well. Although if you have a larger space you could opt to simulate G-forces making for a more realistic experience. This in turn giving the crew an even better chance of surviving in the event of an emergency.

The biggest problem at the moment is that an exoskeleton is very expensive (although you can hire one for $590 or €460 a month). The technology required is still pretty much in it’s infancy and they are not yet mass produced. Also I do not know of anyone developing a system like this for entertainment purposes at the moment. However, if we really will walk in exoskeletons all day is only a matter of time before somebody will.

The holodeck: Holographic TV

In the coming weeks I will examine a few technologies that could function like a holodeck. Today I’ll talk about the holographic TV [hTV] a technology which is just around the corner.

A true hTV differs from a 3D TV by making the objects appear in the room instead of merely creating an illusion that they are 3D.  When viewing a 3D TV everybody watches the same picture no matter where you are in relation to the screen. A hTV sends out the light in such a way that an object will actually appear to be in the room. your relation to the TV will therefore actually determine what you view. You can compare the difference between viewing a scene in a show box and having a model of the same scene. In the show box you see a scene from a fixed perspective while you can walk around a model.

A hTV has the benefit of not being harmful to the eyes and eye development in children and will not cause headaches and exhaustion associated with traditional 3D TV’s. This is because a hTV actually projects the object into the room so your eyes can focus on an “object” instead of the screen.

Restrictions with native hTV content have to do with the available processing power and bandwidth. You can imagine that in order to display an object in 3D from any given angle requires a massive amount of data. This will mean that in the first generations of the hTV it will track your eye movements and only display the viewing angle you see. This means that the number of viewers is automatically limited to the number of people the hTV can track and display an object for. As the cost of bandwidth and processing power decreases more angles will be added.

Early adopters of the hTV will be of course the military gaining an advantage when they are able to plan missions on a live 3D map. I also see potential for industrial designers and architects allowing clients to view their work without having to produce a physical model. This means they can adapt their work to the clients wish live with the click of a button. Of course the advertisement industry will use the technology to attract attention to their products. Later on theme parks and cinema will install hTV technology to entertain their costumers.

The first hTV native consumer applications will probably come from the gaming industry. Which just eliminates the step which makes their 3D games able to display on a 2D screen. Sports will benefit from hTV technology allowing you to view the pitch from any angle you desire and allowing you to walk around so you can see what the players see. The other early adopters for consumers will be the porn industry which, unlike Hollywood, does not care about artistic value of their production as long as it gets the job done.

Hollywood will need more time however so they can figure out how to get the story across when people can view it from unintended angles. Maybe they will keep to showing their films more like the traditional 3D technology. Where there is just one viewpoint for all viewers, independent of the location of the viewer like a show box having layers of 2D images projected a little bit apart from each other creating the illusion of 3D.

the technology to shoot a film in native hTV is already available to consumers. A hacked xbox kinect camera has been shown to film a room 180 degrees in 3D. two or three of these camera’s could capture (nearly) an entire room allowing you to view an action in that room from any angle.

The holodeck: Current status.

A simulator for entertainment. U.S. Navy photo by Journalist 1st Class Stephanie Souderlund

A simulator for entertainment. U.S. Navy photo by Journalist 1st Class Stephanie Souderlund

In the coming weeks I will examine a few technologies that could function like a holodeck but first I’ll examine how far we are today.

Lets start with the basics here: What is a holodeck? The concept of the holodeck originates from Star Trek The Next Generation. It is a large room in which reality can be simulated. It is a room which uses a combination of holographic images, teleportation technology, replication technology, tractor beams and force fields to create a lifelike representation of the world (whichever world that might be). The holodeck as described in Star Trek is fiction of course and quite possibly will never be a reality as described. There are however several technologies that will or could basically do the same thing.

First off the personal computer and gaming systems. You might think it is a big step from these to a holodeck but actually a lot of things needed for a holodeck are actually already incorporated in these systems. They render their virtual worlds in 3D, contain information about what are solid objects, how you move over certain terrain, great gaming features and more. Of course a solid object is just solid so a wall and a person will both feel like solid brick but still many information in games is usable for a holodeck. of course the biggest issue is that you cannot enter the world yourself. You will always need to rely on a screen and some kind of input device. (although the Wii, Xbox kinect and Playstation Eye take a few first steps towards eliminating the clumsy (unnatural) controller altogether. On the plus side these systems are cheap and have come a long way in just a few decades.

A step up is a system called the CAVE. It has three or more walls (sometimes including floor and ceiling) on which 3D images are projected. With 3D glasses (similar to the ones for your 3D tv) you get a holographic simulation. By walking around an object you can view it from all sides like an actual holographic image. With new technologies (similar to the aforementioned Xbox Kninect etc.) you are even able to interact with these objects to some degree. The lack of a physical form is a big disadvantage however. To be able to truly interact with an object you need to be able to handle it as well. That is why most video’s of people interacting with virtual objects seems so clumsy, you just cannot get an idea of weight, form and feel of an object. Another big disadvantage of this system is the space you need (it is a room within a room so you need an awful amount of space) and the money a system like this costs.
The last problem is that it is unfit for young children and some people experience headaches when using the system. This is due to the actual technology. The information our eyes gets says an object is somewhere in the room, the actual object is on a screen however and so the eyes shift focus between the screen and where the object is expected to be. This rapid focussing between the two causes the headaches but is also why children shouldn’t use it. Their eyes are still developing and the 3D technology can hurt the development of the eyes.

The best holodeck equivalent  we currently have are the big simulators used to train pilots, ship captains, Formula One drivers or are used in an amusement park as entertainment. They act and feel like the actual thing and by the use of hydraulic pistons simulate movement of the ship, car or plane. The latest version, based in the Netherlands, is even able to simulate gravity (or the lack thereof). The biggest disadvantages of these machines is that they are very large, require a crew to operate (both for maintenance and running the training), cost a lot of money and require you to purchase a new machine every time you want to use it for a different plane/car/boat.

Robot workforce: Education and purpose

Image by woodleywonderworks (CC BY)

As robots become more complex they will replace more and more jobs. In these articles I will examine the implications of increasing unemployment. Today the fourth story out of five.

In 1979 Pink Floyd sang the legendary words “We don’t need no education.” These words might have been prophetic. When there is no more work we will not need the traditional education system any more since it is geared towards preparing you for a job. It is naïve to think there won’t be any education whatsoever however. Humans are naturally inquisitive and in a stimulating environment we want to learn. The basic premise of education will change however. From an education system which centres on what we as a society expect from students we will get an education system which centres on individual qualities and interests.

By a society driven education system I don’t mean individual tastes and preferences have no place in the current education system. Of course you can choose if you want an alpha beta or gamma education and later on you choose your own profession. But accept for those choices society expects a certain standard from you. We expect a basic understanding of math, writing, geography, history etc. and based on what level education you have and what profession you choose we expect certain skills to be more advanced and at least at a minimal professional level.

In an education focussed on the individual there are no minimal requirements. In theory if someone  would not want to learn anything he/she would not have to. But as stated we are naturally inquisitive and want to learn and therefore we will. Schools which focus on individual preferences from children already exist. A good example is the democratic school for instance where kids are free to explore any topic they want as long as they are at school. If a kid wants to fish for half a year straight it can and it will learn anything and everything that has to do with fishing on its own. Teachers are present but don’t teach in the traditional sense. They help the children to find what they are looking for but will not steer them in a certain direction.

Reading and writing skills will come more naturally. Let’s take the kid which wants to learn about fishing. If it wants to get better it will need to read a book about fish. It will teach itself to read with the help of teachers so it can get better at fishing. You will see the child will read much quicker then other kids who are more traditionally schooled because it is interested in the subject matter in stead of having to begin with some abstract uninteresting words. If the kids get certain skills at a professional level is up to them. In our society that could be a problem because if they don’t get there they need welfare benefits and cost money. In the neo communist society it isn’t a problem since nobody needs to work anyway.

After we ‘graduate’ from school we will need to find a purpose in life. Nowadays many people find purpose in their work, caring for the kids etc. But without jobs, the household being cleaned by a robot maid and a robot nanny for the kids we will have to find new purposes in life. We will become an adventure and hobby driven society.  The entertainment industry will boom and we will travel like we have never travelled before. We will try and find purpose in our hobbies trying to be best at whatever we like. Who knows, some might actually become scientists out of hobby. Our primary care will be to be happy. The irony will be that the pressure to be happy will actually cause depression for some people. Those who fail to have a purpose in life will die unhappy, often too soon. Problems with drug abuse and other addictions will become greater.

A story about forgetfulness, numbers and associations

Think about a Beatles song. Got it? Now think of a song from Queen and then Meat Loaf. I’m pretty sure the ones you didn’t think about were: Boys, Keep Yourself Alive and For Crying Out Loud. More likely the songs you thought about were Hey Jude, Bohemian Rhapsody and Paradise By The Dashboard light (based on their place in the Dutch 2012 Top 2000). Chances are you haven’t even heard about the first trio  or you forgot about them a long time ago.

This post is about music (if you haven’t guessed that already). Music nowadays is under attack by pretty much anyone who isn’t from the current generation (and even by some from the current generation). We like to think that music from the 60’s, 70’s, 80’s and 90’s was better. I however am not that sure. We will have to wait at least 20 years before we can even think about determining if music was better back then.

The trouble is when we think of music from bygone eras is that we think of the best songs, the biggest hits, the ones that stuck. The music that wasn’t as good, as I have demonstrated above, we forget about. Music nowadays doesn’t seem as good just because we haven’t had time to filter out the bad songs yet, the one day flies. When listening to MTV (they sometimes actually still show music videos) you’ll get a wide array of music, some of which will make it and some of which we’ll forget about, even if it sold a gazillion copies today.

Another thing you have to keep in your mind is that when talking about bands like The Beatles, Queen, Elvis, Bruce Springsteen and The Rolling Stones is that their career spans decades (and some are still going). In this time they produced many albums and dozens of songs. Artists who came up in recent years don’t have that career yet. They may have an album or two and a dozen songs or so but it will take time to get as many songs as the aforementioned bands have (and many never will.) Simple truth is that if you write a million songs chances are that at least one will be (very) good.

Last thing I need to talk about is the association you have with music. As many have said before me: music is emotion. When you think about music from bygone eras you usually think about music from your youth. This music has a strong emotional attachment for you because your youth is filled with lots of happy memories (usually). Our youth is filled with firsts (first day of school, first crush, first kiss etc.), and firsts are the things we remember most. Firsts are our accomplishments, our success and thus we associate our youth with excitement and happiness. When we grow up the number of firsts dwindles fast (simply because you cannot experience a first kiss for a second time) and so we remember less from it and our association might become less happy with it.

Now some will say that they listen to music from the 70’s and 80’s even though they are a child from the 90’s and that the argument above is irrelevant. We’ll I’m guessing that your parents listen to this music and thus it is music from your youth. And even if that is not the case, not everybody conforms to the majority of course.

All those forces are at play when you think about music from bygone eras so that it seems music back then was better. Just keep that in mind next time you listen to today’s music. I also like to say I do not want to talk down aforementioned bands. Music that survived this long just has got to be good and I’m thankful I can still listen to it today (and still do). This story is about perspective, not about talking trash.