Latency Illness

This is why VR headsets make us sick.

As mentioned in my previous post, Oculus Rift is a Fad, there are physiological reasons that we (as human beings) get sick when using virtual reality headsets for an extended period of time.  I will explain the reasons for this effect herein.

The key factor is head tracking latency.

When discussing this, I used to call it “anti-motion sickness”, and some people have referred to the problem as “simulator sickness”, but I have now chosen to use the term “latency illness“, as that is a more accurate description.

To start with the most basic idea, latency is the amount of delay in a system, and in this case, it means the time between a real-world action (a head movement) and the visual representation of that action on the display.  This is the time it takes for this whole process: HMD sensors detect head movement, the headset delivers that information to the driver, which in turn reports the movement to (or is polled by) the game software, which then assembles the updated scene, which is rendered to the display (generally, at the next refresh), and finally the eyes and brain interpret the image(s).

The above process necessarily takes a not-insignificant amount of time, introducing noticeable latency.  In practical terms, when you turn your head, the display is slightly behind.  On a fixed display (with a normal controller), this can be annoying (and combined with a slower refresh rate, can even make people dizzy), but a delay between a mouse (arm) movement and a screen update becomes a characteristic of the game.  However, when the display is “immersive” and provides your entire vision, without a stable frame of reference, and the input is the actual movement of your head, it introduces issues that trigger (undesirable) physiological responses.

Moving your head necessarily moves the vestibular system in your inner ear, which is responsible for balance, spatial orientation, and movement detection; it is your own personal head tracker.  You (correctly) feel through your vestibular sense that your head has moved, but with the introduction of latency and immersion, your visual sense tells you that your head has not moved (yet).  Your brain has to try to make sense of conflicting inputs, which can lead to dizziness (vertigo, disequilibrium, or lightheadedness) and headaches after a short period of time.  This is a form of motion sickness.

The human brain has evolved over the eons to have a specific response to this disconnect between vestibular and visual senses: nausea.  The reason that we get ill when our senses do not concur is that this is experienced when we have consumed something poisonous.  Our great ancestors (or children 🙂 ) may have ingested poison berries, and nausea is the reaction that increases survival fitness by causing us to vomit, expelling the poison from our system in the fastest manner possible.  Unfortunately, when the cause of the nausea is not from our stomach, vomiting does not provide relief.

You may have noticed the fact that all of the above provides a pretty good description of what happens when somebody gets drunk, and of course, those unpleasant aftereffects are known as a hangover.  In the case of latency illness, the process is similar, and many of the same aftereffects are experienced with extended use.  Note that the brain can adapt to incorrect visual input (see the perceptual experiments of George M. Stratton), but when the problems are caused by lengthy use of a VR headset, there are periods of adjustment both when the headset is donned and again when it is removed.

In my experience, one does not get “VR legs”; limited use (less than 5 minutes at a time) of an HMD is always recommended, even if visual latency seems to less disconcerting.  I believe that this is the fatal flaw in this kind of virtual reality technology.  However, other great (game) programming minds, such as Michael Abrash and John Carmack (despite my swipe in the last post 😉 ), are contemplating how to minimize the latency issues, from a technological standpoint; physiology must still play a role, though.

As I mentioned previously, my personal experience with moderate latency illness came when I was doing final testing of the Doom II driver for the Virtual i-O iGlasses.  I had been working on the drivers for a while, and I had become accustomed to the latency, somewhat.  I also knew, thanks to Mark Long of Zombie, that there was a risk of playing a game in the headset for too long.  Nevertheless, I wanted to do proper final testing of the drivers, so I installed everything on a clean system and started playing Doom II in the iGlasses; I kept playing for quite a while, completing several levels.  I did not feel sick or anything, and the drivers worked, so it was time to ship.  Then I took the headset off…

Boom!  Instant hangover.  I almost instantaneously got a massive headache, felt somewhat dizzy, and became very nauseous.  It was everything I could do just to archive the final drivers and write an accompanying email, and once I was done, I just went to bed and laid there feeling like I was taking my lumps for getting smashed, when all I had done was play a few levels of a video game with a virtual reality headset.  I still felt odd after sleeping for several hours.  While this may have been worth it for the contract payment I later received, it definitely was not worth it solely for the gameplay.

Based on that, and my research since then, I strongly recommend that anybody interested in getting an Oculus Rift or other VR headset consider these issues first and, if/when they purchase, avoid playing any game for more than a few minutes at a time, taking a decent recovery period between sessions.

Oculus Rift is a Fad

It’s All Been Done [repeat 3 times] Before…

There has recently been a great deal of “buzz” surrounding the Oculus Rift, which is a virtual reality headset, or HMD (Head Mounted Display) with positional tracking.  The talk really got started at E3 2012, when John Carmack demonstrated a prototype.  After that, Oculus launched a successful Kickstarter campaign, eventually receiving $2.4 million in funding (after seeking just $250K).

The Kickstarter page, however, is very revealing, describing the Oculus Rift as “the first truly immersive virtual reality headset for video games.”  This is just plain false, and it shows not only a lack of research, but an associated failure to understand the history of VR gaming headsets, which (unfortunately for them) strongly predicts the future.

There were virtual reality headsets readily available back in 1995, 18 years ago!  Then, as now, they were being hyped as a new standard of immersion and a paradigm shift in gaming and [insert preferred hyperbole here].  In particular, I speak of the Virtual i-O iGlasses and the CyberMaxx headset, both of which our company owns (and has in storage, notably not in use), though I recall other competitors; even Nintendo got on the virtual reality bandwagon with its Virtual Boy.

I suppose that this is a good time to provide my credentials for this discussion.  In 1995, I wrote the official game drivers for Virtual i-O, which included native drivers for Doom II (ironically, Carmack’s game) and Dark Forces.  I was also later (1996) contracted by VictorMaxx (the manufacturer) to write game drivers for the CyberMaxx, as well as a virtual mouse driver that was controlled by its head tracking.  Also in 1995, I worked on Locus, the first release from Zombie, and also the first (to my knowledge) retail game that was truly (as the box says) “Engineered for head-mounted displays“.  At the very first E3 in Los Angeles, I had my software being shown in three different booths, and I helped demonstrate the iGlasses myself in two of those places.  I know whereof I speak.

Clearly, as evidenced by two nearly adult headsets collecting dust in storage, the “virtual reality revolution” never took place.  Sure, there were games that supported HMDs, and a brief time when some “location-based entertainment” (a fancy phrase for video arcades) had games which used VR hardware, but most people still played (and play) games with just a controller and a standard display.  This is precisely analogous to the non-existent 3D television revolution touted by those with skin in the game; it never happened, and while the hardware is readily (even fairly inexpensively) available, it did not take off.

There are two main issues with why virtual reality gaming has not become mainstream.  First, using a head tracker and/or other VR hardware is inconvenient.  It requires some preparation, there is a degree of setup, and then players need to wear/use slightly (to significantly) awkward devices.  It is nothing that cannot be done fairly easily, but it requires just enough effort that most will generally not bother.  This is quite the opposite of the current mobile gaming revolution (which is happening) where a player simply picks up a device, touches a (virtual) button, and plays.  There is also no “killer app” for the technology (guillotine simulator notwithstanding), so nothing but novelty to drive sales.

The second issue is that position tracking latency in a head mounted display makes you ill, literally.  Extended use (more than 5 minutes or so) of a head tracker will actually give you symptoms similar to motion sickness, or perhaps a severe hangover, including nausea and headaches.  Apparently, the makers of the Oculus Rift claim that this feeling of seasickness could be overcome once you get your “VR legs”; do not buy that.  There are specific physiological reasons for this reaction, which I will describe in my next post.  For now, let me just tell you that I have never gotten seasick nor experienced any other kind of motion sickness, but using the iGlasses for an extended period, during final Doom II driver testing, caused such an unpleasant experience that for a long time I would start feeling unwell just seeing that game played on a normal screen.  Locus was explicitly designed to have short matches and encourage a break from the headsets between rounds.

So, the Oculus Rift is nothing that has not been done before, and although the vertical resolution for each eye is slightly better than the Virtual i-O device, the iGlasses actually were more immersive, since they included stereo audio on the device, not just video.  That all said, I will admit that the Oculus Rift (like others before it) is a cool device, and I am certainly considering one on that basis.  However, it is still just a fad.

A cool fad, but a fad nonetheless.