the road to the Matrix (or at least a part of it) is paved with head mounted displays

Computers. Programmable devices getting smaller and more powerful each day, disrupting things high and low. They change our way of life, change our sense of reality. Programmers write the code which does that. And the two fundamental challenges of a programmer are to name things and to count things. I am also a programmer and in this article I am trying to count the emerging  illusion-technologies and the name concepts they are making real.

But what is reality?

Let's just boastfully ignore the philosophical considerations, the scientific facts (no one saw the Higgs boson, still, we accept it to be real), religious and superstitious beliefs and the alternative realities.

I would focus on the physical reality or at least at the perception of the physical reality, or at least what that means for the majority for us (oops, I'm color blind). Perception is the key. Our body is equipped with sensors and we use them to sense our physical environment, the physical reality we live in:

•  eyes: light or the lack of it
•  ears: sound and gravity
•  nose: smells
•  skin: heat and pressure
•  tongue: taste, heat and pressure

Why am I narrowing the scope that much? Because all the augmented/virtual/mixed reality hype can be reduced to some tricks to fool a subset of our sensors (no, we do not get the full-featured fake reality experience from the Matrix - not yet).

The HMI Perspective

Starting with the Jacquard loom in 1804, machines needed interfaces through which their masters, the humans could control and operate them. Eliminating abstraction layers, human-machine interfaces (HMIs) evolved from punched cards to key boards, later the joy-stick and the mouse was invented. Then the mouse evolved into the pointing stick, touch-pad and the touch-screen. From this perspective, AR, VR and MR can be viewed as the logical next interface between men and machine. The benefits are imminent: hands-free, portable, non-tethered HMI.

XR (AR, VR, MR)?

Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR) are the hottest buzzword today. But the naming is misleading, as opposed to 3D printing, these technologies add nothing new to our physical reality. They trick our eyes and ears to fool our brain so it accepts virtual objects as real. And due to current technical advancements hacking our perception got better resulting "seamless user experience" and real time interaction with the virtual objects. I'll just call them XR, X standing for any of A, V or M.

As there are many fun and practical applications for XR, the developments are boosted and new devices are expected to swarm the the markets soon. But how are these "realities" different from each other?

VR

Let's start with the so called virtual reality headsets. They create the illusion of an artificial world around us by fooling our eyes based on head tracking data. They can partially fool our ears, as well by spacial sound tricks. But the total visual occlusion of the environment can cause nausea and dizziness. How they achieve that? By knowing where we look the the computers quickly generate (i.e. render) the images to be shown on the tiny screens built into the headset. 

AR

Augmented reality can be considered more real and less dizzy as it creates the illusion of adding virtual objects to our real-real environment. Various devices can accomplish this, like mobile phones, AR headsets, Looking Glass, etc. As the virtual objects are added to the real environment the users do not get dizzy, like with the VR headsets, but the field of view is narrower, so large virtual objects seem to be viewed through a window. 

MR

Now this is a funny beast. At first it seems that Microsoft tried to brand its version of AR as mixed reality (MR). But if we listen to what they say about it it turns out that Microsoft has only labeled a subset of AR as MR: the immersive part. And its HoloLens is the best MR headset so far.

New realities: artificial and extended

Some companies exploring the business opportunities provided by VR/AR/MR refer to their services as extended or artificial reality services, but as one digs deeper it turns out that they have just came off with some marketing name for a package of plain old XR. 

Applications

Beside gaming and porn there are many industries which can benefit from the applications of XR. I found the following to bear the highest potential and to be achievable in the near future.

Telepresence

The 3D version of Skype: you get to see the hologram of a person or an object, live or recorded. I prefer live streaming! Seems to be boring, but I have tried it once with HoloLens, and it is WOW! You have to try it to understand. The camera part was a plain old Kinect. Add virtual objects, document visualization, digital signature and you have the customer support service of the future! 

Holo-Manipulation

Manipulating 3D objects in CAD software is one thing, but the intuitive rotation, scaling, slicing of virtual 3D objects can be way better. Look inside objects, understand complex structures. Architects, engineers, designers can benefit a lot from this!

Remote Assistance, Holo-Assistance

3D Skype again, but this time the focus is on the real stuff. The remote assistant sees a 3D image of a real person or object, can give advices and even highlight parts of that object to the other person. As an example, the remote doctor can see the victim of an accident with a broken leg and can instruct someone to secure that, drawing exact markers on the leg, where to tie down an artery, where to bandage the wound, etc.

Object Recognition

AR and MR rely on accurate spacial mapping of the user's surroundings. This means that objects with specific features can be easily identified, highlighted and labeled to the user. Looking for a very special Lego piece in a huge bucket? Easy-peasy!

Holo-Construction

Visualizing CAD designs got really cool and efficient with XR. The best of it is when a future building is shown in true scale at the exact location it will be built. Nothing can beat that!

TL/DR: emerging XR technologies are fun, we just need to it figure out, how can we use them to get all that plastic out of the oceans...

Disclosure: the author is a Mixed Reality consultant at 360World, an agency specialized in AR and MR solutions.