FSD, Waveguides, and Focus/Depth Planes – A Fatally Flawed Foundation

While you can’t find what Magic Leap hypes as “photonic light field chip” anywhere in their patent application, they do discuss diffractive (mostly) waveguides. I have previously discussed some of the well-known issues with waveguides. This article started out as one purely on the technical side of waveguides, but in reviewing the patents, I had an epiphany about Magic Leap’s patent behavior regarding optic/hardware. While I have touched on some of this before, hopefully, this article puts the pieces together with more clarity.

Magic Leap Started with three key technical ideas:

1. Fiber Scan Display (FSD)
2. Thin Diffractive Waveguides (for the “sunglasses look”) — What Magic Leap dubs, “Photonic Chips” where the exit diffraction grating imparts the apparent focus.
3. Addressing Vergence/Accommodation Conflict (VAC) using focus/depth planes — What Magic Leap hypes as “Photonic Lightfields.” Magic Leap’s invention was to come up with a way were various waveguide layers would impart different apparent focus distances with exit grating. Nice R&D idea, but a terrible idea in practice,

VAC goes all the way back to their 2013 presentation (slide above) and their early patents such as Fig. 8A from Magic Leap’s ‘253 application on the right (which appears in many 2015 and 2016). You will see drawn what was fiber scanning displays 200, 202, 204, 206 and 208 supporting five focus planes. If the FSD worked, then in theory at least, you could simply route the fiber to the 5 FSD fibers optically route the five images to the five planes.

Their hoped-for solution to VAC with many focus/depth planes was highly dependent on  FSD that was never going to be a practical display technology (see my explanation here). They compounded this mistake by keeping with their “photonic light field” concept that they had to cut back to just two focus/depth planes. From several of their own earlier patents, they show they wanted/needed 5 to 6 depth planes somewhat logarithmically space as shown in Fig. Seven from their ‘495 application (below) to address VAC. In the ‘495 application, they even showed in Fig. 9 (left) a non-waveguide optical system (right) that could support more than two focus/depth planes.

Clinging to “Photonic Chips” And Sacrificing VAC

But to have “photonic chips” (e.g., waveguides) and to use their inventions, the number of depth planes had to be drastically cut from the desired 5 or 6 to just two; one at infinity and one closer. The routing of images to individual stacked waveguides is very optically complex/difficult/expensive, even for just two planes. At the same time, it is not clear that just two focus planes do much good to address VAC as there will still be big gaps/errors. So it seems VAC support was sacrificed to support the original “vision” and marketing story.  

As they had to seek a solution with conventional displays, first DLP and then LCOS, the more they end up looking a lot like Hololens with, as my U.K. friend would say, with a “bag on the side” second focus/depth plane adding cost, optical complexity, and hurting image quality.

One “Waveguide” in the Drawings is 3 Waveguides (For Color)

Unlike the very simplified drawing in Fig 8A above, a full-color waveguide usually consists of 3  physical waveguide layers, one each for red, green, and blue (it can be done with two layers, but it sacrifices image quality). Fig. 11B (left) is from Magic Leap application ‘739.

Even assuming FSD would work, it was, at best naive, for them to think they could support more than two focus planes with each plane adding three waveguides. Each waveguide adds to the cost and lowers the image quality as there are more diffraction gratings for the real and virtual images to pass through. Even with just two focus planes, the user has to look through 6 layers of waveguides shown in Magic Leap’s ‘075 application Fig. 15B shown on the left.

Searching For A Way To Support (Just) Two Focus/Depth Planes

Early on, Magic Leap was hoping  FSD would solve the problem of routing various images to their respective focus plane entrance gratings, but as FSD failed to work out, they had to find a different approach.

Their patents show many different approaches suggesting that this was/is a big problem area where they are still searching for a good solution. I’m going to show just some of the many approaches from their patent applications broken into a few categories.

Dual Primary Separation

In the case of the ‘075 application (Fig. 15B above), they used two sets of color with two different wavelengths of green, two of red, and two of blue LEDs to illuminate an (LCOS) device,  although the patent still mentions FSD as a possibility [correction, it turns out that they use the angle of the various LEDs, and not wavelength, to route the various LEDs to the correct waveguide – this was verified in the iFixit teardown, on August 23rd, 2018]. While wavelength selection may seem reasonable in theory, Dolby 3-D® uses in movie theaters; in practice, there is likely to be a lot of cross-talk between focus planes as the diffraction gratings from the “green” of one plane will affect the green from the other. Among other things the application suggests addition filters to gain more wavelength separation (but this will obviously waste light).

Dual Illumination With Special “Injection Optics

In 2016, I wrote that application 20160327789 seems like the best fit. In the ‘789 patent (right) they start with two sets of LED light sources that are at slightly different locations. The injection optic further separates the images for the two planes so they will go to two physically separate injection gratings.

Spatially Moving the Image To Entrance Ports and Two-Sided Waveguides

Magic Leap application 20170248790 has a large grab bag full of different techniques that use beam splitters and other methods to move the image around to route it to different entrance diffraction gratings. Two of about a dozen approaches are shown in Figs. 3 and 19 (below left).

Another technique that shows up in the Magic Leap patents is the two-sided waveguide with an entrance port on each side (above right).

As all these methods show, they add a lot of optical complexity to the object just to support a second depth plane. And optical complexity is almost always bad for cost and image quality.

My Conclusions

Magic Leap has a considerably patent application history and can be used to map out their direction over time. I’m an engineer that deals with facts, technology, and not marketing hype. All the money and smart people are not going to make what is shown work well or be cost-effective. They made some very naive mistakes from the beginning, in particular, FSD and having the waveguide exit grating impart the focus depth.

Maybe the investors bought off on Magic Leaps “greater vision” and that they would spin out software or tracking technologies as an R&D effort or some other reason that is beyond my knowledge. But everything I can see as an engineer says that Magic Leap’s display hardware is not going to have very good image quality, and it is going to be expensive to make. All the yelling “Moore’s Law” and invoking “Steve Jobs” and saying “Google wouldn’t have invested” does not change what the record shows.