One common technique to create a ‘mixed reality’ (MR) environment is the use of holographic lens technology. There are various ways of making a holographic lens and the technology is currently in rapid development. In this case study we will be discussing the technology used to achieve this, the physical user experience and specifically it’s use in MR environments.
Introduction
First of all, what is Mixed Reality and how does it differ from Augmented Reality? Schroeder (2008) highlights the need for a well-defined concept of a virtual world, due to the lack of a widely agreed upon definition. Moreover the terms “Virtual Reality” (VR), “Augmented Reality” (AR), and “Mixed Reality” (MR) which are each a subset “Extended Reality” (XR) are often used interchangeably so for this study I will be defining Mixed Reality as the combining of a computational space within the real world where physical and digital elements can interact (Tremosa, 2022) for example seeing a computer rendered Anatomical model that you can interact with (see fig. 1). AR is a much broader term which includes the adding of additional information to the real world (hence ‘augment’), for example the overlaying of textual information about a painting that the user is looking at. AR prioritises the real world and the term is generally reserved for Holographic technology. Both AR & MR differ from VR in that VR occludes the real world, surrounding the user within a computational space.
Holographic Technology
One way of mixing the real with the virtual (in simple terms) is to reflect an image onto a glass panel in front of the viewer (fig. 2) however due the physical size of this solution it has limited applications as a head mounted system. Ideally we want the user to be able to observe the real world while simultaneously seeing rendering content overlaid no mater where they look. This may seem relatively simple however its far from it, and the technology to achieve this is in constant development. Doing this for both eyes produces a 3D image that appears to be within the users line of sight. This technique can be broadly referred to as holographic, and the main component in modern headsets, the Holographic Optical Element (HOE), dates back to 1962 in a paper written by Yuri Denisyuk, a Russian researcher (Denisyuk, 1962). This technology projects an image that is then refracted within a sandwich of glass layers (fig. 3) which feature a Fresnel lens known as a wave guide combiner to then reflect the light towards the users eye which also allows light to pass through from the real world (fig. 4). By combining this with tracking and reality capture technology a 3D environment can be aligned with the real world, creating the desired mixed reality. There are several companies producing products that utilise this technology including the industry leading Magic Leap and Microsoft’s Hololens.
There are however some noticeable artefacts to this technology, the most commonly observed and discussed one is the limited field of view (aka FoV) and chromatic aberrations seen as a visible rainbow distortion especially with bright whites, which is a result of the lens elements not focusing all colours to the same point. I would speculate that these artefacts would be minimised with further development of the technology.
There is one other significant disadvantage to this technology in that by reflecting the light the result is an additive process meaning that the darker the overlaid image the more transparent it becomes as less light [or no light] is being projected (fig. 5). This shortcoming is less noticeable in darker environments or by adding a tinted lens element to increase the contrast however this isn’t always achievable or desirable (esp. if you have to see objects in the real world too).
Applications
As holographic lens technology prioritises the real world, in that we have a (reasonably) undistorted view through the transparent lens elements, it can be deployed in a wide variety of situations that require the user to interact with real elements. This also makes the headset a much safer option in situations that require the wearer to be aware of their surroundings, for example; a building site, while mobile, working with heavy equipment, or perhaps in a hazardous location. A physiological benefit of a holographic headset is that as the real world is clearly visible (as opposed to a VR headset) so the user can easily orientate them self which in turn can greatly reduce if not eliminate motion induced nausea (common in VR headsets).
Benefits of the Holographic Lens | Cons of the Holographic Lens |
|
|
Conclusions and recommendations
Holographic based headsets have a wide variety of applications, in fact when you really start to think about it, there application would benefit everything from visualising recipes, timers, temperatures, orders, and more in a Restaurant kitchen through to displaying biometrics, patient notes, and anatomical details in a medical surgery or perhaps visualising the next step on a construction site, while comparing the built components with three dimensional plans overlaid to confirm there accuracy.
Currently the high cost of the headset and the physical characteristics do limit its uptake especially as a personal device however as the technology evolves I expect it will become more widely used in many different scenarios. When designing for this technology the shortcomings should be considered to produce a successful product, for example elements that use a primary colour will have less chromatic distortion (especially when compared to white) due to the physical characteristics of the wave guide technology. Not relying on visual elements that use black, or treating black as a transparent element. Providing navigational elements to counter the limited FoV, eg an arrow to locate a ‘lost’ object (like a fighter jet HUD). If the application does not rely on an interaction with the real world or that the virtual elements are the main focus, it may be better to consider an VR based solution for better rendering of virtual elements and reduced overall cost.
References
Carter, R. (2020). Why is HoloLens Good for Mixed Reality? XR Today. Retrieved from https://www.xrtoday.com/mixed-reality/why-is-hololens-good-for-mixed-reality/
Schroeder, R. (2008). Defining Virtual Worlds and Virtual Environments. Journal of Virtual Worlds Research, 1(1). Retrieved from https://journals.tdl.org/jvwr/article/view/294
Tremosa, L. (2022). Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR? Interaction Design Foundation. Retrieved from https://www.interaction-design.org/literature/article/beyond-ar-vs-vr-what-is-the-difference-between-ar-vs-mr-vs-vr-vs-xr
Kress, B. & Chatterjee, I. (2020). Waveguide combiners for mixed reality headsets: a nanophotonics design perspective. Nanophotonics, 10(1), 41-74. https://doi.org/10.1515/nanoph-2020-0410
Denisyuk, Y. N. (1962). Photographic reconstruction of the optical properties of an object in its own scattered radiation field. Soviet Physics Doklady, 7, 543.
Wisdantio, W. (2017). Pepper’s Ghost Holographic Projector With Single Reflector V.01.3. Wisnu [W] Wisdantio. https://wisnuwisdantio.com/2017/05/02/peppers-ghost-holographic-projector-with-single-reflector-v-01-3/