Update Feb 28, 2019

• Updated that Microsoft is using 3:2 aspect ratio
• A report from some that used a Hololen 2 that they saw flicker on the outside of the image(as I predicted).
• Blog Down for 20 Hours thanks to lousy bluehost customer service (see rant at bottom)

Hololens 2 Announcement Feb 24th

As Expected, the Hololens 2 (HL2) was announced at Mobile World Congress (MWC) in Barcelona on Feb 24th, 2019. As I reported last time prior to the announcement, HL2 it is using Laser Beam Scanning for the display device. We also got quite a few details, some good, some not so good, and some “fibs” to put it generously.

The announcement was streamed and is now on YouTube (the key information on the HL2 starts about 23 minutes into the over 1-hour presentation with Alex Kipman). Additionally, for this article, I used some pictures from Microsoft as well as CNET and The Verge articles which are labeled accordingly. I relied primarily on the information from the above sources on the HL2 for my comments below.

Much Improved Human Factors – “Hololens 2 makes the Magic Leap One Look Like a Poorly Designed Child’s Toy”

The best thing I saw was the much improved human factors of the helmet design. It looks to me like they went for functionality over looks or messing up the person’s hair. I suspect there will be a name for hair that gets smooshed down in the front, but it beats having it hurt when you wear it for a long time. Last month I even commented that NReal should adopt the configuration of putting the compute- pack and battery on the back of on some form of a cap which would result in a similar configuration.

I also very much like the flip-up of the optics what work like a Magnifying Glasses Headset. or a welding helmet. It is good to be able to get the display totally out of the way without having to take off the headset and find a safe place to put it.

Keeping the compute-pack and battery on the head means there is no cable to snag or catch. They maintain enough eye relief that most people will be able to wear ordinary glasses a huge plus for realist use cases. The design is relatively open an does not block the users downward or peripheral vision.

Another big factor to me is that the HL2 appears to be about 70% transmissive but this is a very rough estimate. Note in the picture on the right how you can see the person’s eyes, something you can’t do with the Magic Leap One and I have been critical about with the ML1. Not only do humans want to see another person’s eye, but it is a very important safety issue. I don’t think qualifies as ANSI “Transparent” (requires 85% transmissive) from the pictures I have seen so far. And if it did meat ANSI Transparent, I would think Microsoft would say so.

In terms of practical human factors, the Hololens 2 makes the Magic Leap One
(ML1) look like a poorly design child’s toy. It looks to me that the ML1 went for style over substance, where the HL2 focused on utility. The HL2 is vastly more practical than the ML1 including you can wear ordinary glasses, flip-up so you can get the optics out of the way without having to take the headset off, no cord to snag, much better peripheral vision, many times more see-through, as well as automatic IPD adjustment based on pupil tracking and about 1.5X the display brightness (more on these later).

Hololens seems to have more than doubled down on gesture recognition. The first generation’s gesture approach was an ergonomic nightmare that would leave your arm sore and pulling your hair out just to input a long WIFI security code. The videos make it looks like the gesture recognition is much improved, but the proof will be when we get to use it on a regular basis.

HL1 Spec’ed at 500 Nits – Good Not Great

First, it is interesting to see nits (common name for candelas per meter squared or cd/m²) being discussed. Usually, nits are not discussed. As I am fond of saying, “when a spec is not given by the manufacturer, the number is usually not good.” In fact, if you Google “Magic Leap Nits” this blog is currently the first hit because Magic Leap does not like to talk about it. I found that the ML1 outputs comparatively dim 210 nits where the first Hololens output about 350 nits. By way of contrast, Lumus which likes to talk nits claims 7,000 nits (yes more than 10X more than the H2) with their new Vision 1080 design.

I have heard that Hololens went to lasers for brightness but this makes no sense. Lumus is getting 7,000 nits with LCOS, Vuzix Blade is >1,000 nits using DLP with a similar diffractive waveguide, where the HL2 is only 500 nits. One should also be concerned as to the scalability of the brightness for eventual outdoor use (say with the Hololens Military Contract) with laser scanning as I discussed last time.

As the optics become more transparent to see the real world and as you want to be able to work in brighter light, the virtual image has to be brighter to stand out. HL2’s 500 nits reasonably good for indoor use where a computer monitor is typically about 200 nits and a smartphone on max brightness is typically 500 to 600 nits, but it still about an order of magnitude too low for outdoor use where concrete sunny day can be 7,000 to 10,000 nits.

How Can 52 Degrees Be More Than Double 34 Degrees?

In a bit of marketing puffery, Alex Kipman said with a big picture so it was no slip of the tong:

“Today, I’m incredibly proud to announce with Hololens 2 we more than double our field of view while maintaining 47 pixels per degree of sight for Hololens”

Alex Kipman https://youtu.be/c1CZsqwnWtM?t=1719

The CNET article stated that the HL2 Field of View (FOV) was 52 degrees when the original Hololens was about 34 degrees. BTW, when only one number is quoted for the FOV, almost always it is the diagonal, because “bigger is better.” How is 52 more than double 34? Well, as Alex Kipman wrote on twitter that they were talking “Area,” good old marketing square law. It was at best misleading, as it is not the way people commonly talk about FOV (and Alex at least certainly must know it). This was an “unforced error” as they weren’t hiding the actual number elsewhere.

Laser Beam Scanning – Resolution Numbers Not “Adding Up”

This blog has 7.5 year history of explaining laser beam scanning and its pros and (mostly) cons. I have measured the claimed resolution through several generations and the always comes out to be about 1/2 the resolution in each direction that is claimed. I have every reason to believe that the HL2’s LBS will be no different in claiming a higher resolution than it can deliver. In fact, the number fail right off the bat.

The Resolution Math Fails!: 54,000 Cycles Divided by 60 = 900 cycles

Microsoft said in the video (and reported elsewhere) that the fast (horizontal scanning) mirror goes at 54,000 cycles per second. I started working in graphics in 1977 (I designed the Sprite Logic on the TMS9918, the first chip to have “Sprites”) and the first thing I do when I see a “big” number is to divide by 60. For any display without persistence, 60 frames per second is the bare minimum to have flicker free video, and for LBS with no persistence, it should be much higher. Doing this simple math, I get only 900 cycles per 1/60th of a second. Note also that a significant percentage of the lines will have to be blanked out while the vertical scan retraces (old CRT knowledge comes in handy here). Only about 720 cycles are in the “active” display (with the laser on).

But Microsoft said they have a 52-degree FOV and 47 pixels per degree and Alex Kipman said the aspect ratio is “more square” than HL1’s 16:9 (HDTV aspect ratio). Likely the aspect ratio is close to 4:3 like Magic Leap and this works out to the Horizontal FOV being 41.6 degrees and the vertical being 31.2 degrees. Multiplying 31.2 by 47 gives 1466 or extremely close 1440 which is twice 720 (within rounding error on the 47 pixels per degree).

Interlaced Video “Trick”

So how do you get 1440 lines out of 720 scans? You have to count both direction of a “cycle” of a scan scan line. This is what Microvision did with their old single mirror design and it must be what Hololens 2 is doing. But then there is the problem that you will end up with alternating pairs of pixels and then and gap of 2 pixels on the sides. So then Microvision old deign and I believe Hololens 2 uses the old CRT trick of Interlaced display.

On the figure below, I have colorized and annotated a 2011 Microvision patent that explained LBS with interlaced scanning. It shows a very simplified image with only a few scan lines. On field is shown in blue and the other field is show in red. I have also drawn a green grid of square pixel locations to show how the scan lines go through pixels and number the columns C0 to C5 and rows of pixels (R0 through R7). A percentage of the scans are blanked out (shown in gray). Below the figure in dark yellow I have show brightness of the beam has to vary to give uniform brightness (it varies dramatically from the sides to the center).

Because they turn the laser on in both direction of the scan, unlike an old CRT that only turns on in one direction, the result is a zig-zag scan. Following just the blue field, you should see that in Column 5 (C5) crosses through the very top of R0 and the very bottom of R1, but then it crosses the very top of R2. So the “pixels” in C5 are in pairs. The red field has the same effect but shifted down one row. So they are counting on the interlaced fields compensating for each other. From experience with prior Microvision projectors, the effect is that left and right side tend to be blurrier than the middle, but even the middle is not sharp (see for example my study of the Microvision based Celluon projector).

I created the GIF below (click on it to open the gif animation in a new tab) that isolates the two fields from the Microvision based Celluon projector. I shot two photos with a high shutter speed to capture the two fields. The animation shows the center and right side and you should note how the right side moves much more than the center. The colors also jump around in this animation due to the misalignment of the red, green, and blue lasers. It will be interesting to see if the Hololens as this issue as well. “Digitally” aligning the colors is another potential source of resolution loss with LBS.

Hololens 2 Should Have Significant 60Hz flicker – Long Term Use Human Factors Problem

Update Feb 28, 2019: After posting the original article, I receive an email from someone that got to try the HL2 and they reported noticing flicker on the sides of the image.

The older Microvision single mirror design used 60Hz “Interlaced” which means that parts of the field were only refreshed at 30Hz. I know of people that could not be in the same room with the old Microvision projectors as they were very sensitive to flicker. The Hololens 2 is likely using 120Hz interlaced which means parts of the image, particularly on left and right sides are only refreshed at 60Hz. If you look at the figure above, I have drawn two circles and area near the center that is refreshed at 120Hz and another area on the side that is only refreshed at 60Hz. The older people in the audience may remember the push in the late 1980s and 1990s to increase CRT refresh rates from 60Hz non-interlaced to over 75Hz due to flicker (once again old CRT knowledge). An this was with monitors that had phosphors with some persistence, unlike LBS that has none.

Horizontal Resolution

Horizontal resolution affected by the speed of switching of the laser beam and where the beam it located relative to the desired pixel including any distortion in the scanning process. As discussed above, the horizontal time of one pixel goes from relatively long on the left and right side of the scan to the shortest time in the center. The beam drive has to go from potentially off to an analog gray scale value with a surge in current that must be supported by the laser and it’s drive electronics. So far, I have yet to see an LBS projector came close to single pixel modulation in the center of the screen.

Regardless of the ability to control the laser and its intensity, the dual direction interlaced scan as well as other issue with LBS such a beam alignment will decimate the sharpness horizontally as well as vertically.

Expect the Effective Resolution To Be Less Than Half

Fundamentally the scan is not fast enough by about 2X to support 1440 lines or about 31 degrees at 47 pixels per degree. The interlaced and bidirectional scanning trick in my experience (see picture on the left) does not solve the resolution issue and in some ways makes it worse.

On top of the scanning rate issue, then we have the “usual suspects” with any laser scanning system. The two most common are laser speckle and beam alignment.

Laser speckle caused by the coherence of laser light. If you shot a laser directly into the eye, the person won’t see speckle, but that is not what the HL2 is doing. The HL2 likely have a pupil expander (essentially a rear projection screen) and then diffraction surfaces inside the waveguide which are all places where speckle should occur. This will give the picture a grainy appearance such as seen with front projecting laser scanning.

Another issue is the beam alignment of the various colored lasers.Even the most microscopic misalignment in 4 degrees of freedom (horizontal, vertical, pitch, and yaw) causes them to diverge by many “pixels” in the image. In the past Microvision (and others) have used “digital alignment,” either crude or fine. With crude alignment they just pick the closest line, but then you have to factor in the dual direction interlaced scan and the lines being at different angles, and since there are 4 degrees of freedom there it will not be the same everywhere on the screen. With “fine” correction, one color is picked, green is best, and then the red and blue images are re-sampled to match the green. But the re-sampling process means that the red and blue images are effectively blurred.

The bottom line based on my past studies of Microvision’s interlace LBS is that the effective resolution will be less than half of what is claimed.

Why No Microsoft Love for Microvision?

On April 26, 2018, Microvision announced they have shipped samples of a so-called 1440P ( 2560 x 1440) LBS engine. I say “so called” because both Microvision’s WVGA and 720p had less than half the measurable resolution that they claimed. This engine is a perfect fit for what it looks like the HL2 is using. Beyond just the numbers, Reddit User “lichtwellen” posted a comparison between a frame of the HL2 video and a Microvision patent.

In May 2019, Microvision, “Announces New License Agreement with a Leading Technology Company” and stated that they have, “entered into a license agreement with a leading global technology company to allow the licensee to use MicroVision’s display technology to manufacture and sell display-only engines based on MicroVision reference designs. The agreement grants a world wide, exclusive, five-year license to display-only technology. In order to maintain exclusivity, the licensee is required to purchase minimum quantities of MEMS and ASICs from MicroVision.” It is not a 100% certainty that this is referring to Microsoft, but everything so far points to it.

The announcement also claims that “the new scanner operates at 120Hz.” But based on Microvisions track record of fudging on specifications and my past studies of their products, I believe they are doing 120Hz interlace which is a full field rate of only 60Hz.

Maybe it is still coming, but I was expecting some kind of joint announcement with Microvision. All signs are now pointing to Microsoft at least licensing technology from Microvision. I’m left wondering why there was no announcement and who didn’t want it yet?

And Why didn’t Microsoft Just Buy Microvision?

In December 2018, Microvision was trading for less than 60 cents a share and the market cap was less than $60M. If Microsoft was serious about LBS, why didn’t they just buy Microvision? Even at a multiple to the stock price, it would be chump change compared to other acquisitions in this space and Microsoft’s spending on Hololens thus far. Maybe they got too good a deal on the “exclusive license discussed above or maybe this is just a stop gap.

What Happens to the “Extra” Horizontal “Pixels?”- IPD

So if the display supposedly supports 2560 pixels and 47 pixels per degree, then why doesn’t HL2 support 2560/47=54 degrees horizontally and more like a 60-degree FOV? And why did Alex Kipman say the display is “more square” as in closer to 4:3 aspect ratio?

Update 2/28/2019: One thing I missed while getting this together is that Microsoft has said that the HL2 aspect ratio is going to 3:2. This will have a slight effect on the numbers below. Mainly it would seem that they would cover a smaller range of IPD adjustment or they also have some vertical adjustment/reserved pixels.

The answer is that they are using the “extra” horizontal pixels to support interpupillary distance (IPD) adjustment. Microsoft has said that they are going to electronically adjust the IPD. This means they have to have guard-band pixels so they can move the image to adjust for IPD. Nominal IPD is about 63mm. From the pictures, and assuming the waveguide are centered, it roughly works out that the exit grading of the waveguides being about 27mm wide (see picture above right).

Now we have to get into the variability in human IPD see for example here in table 2. To cover the 1% percentile of adult women to 99% percentile men requires a range of 50 to 75mm. Dividing by 2 for the two eyes and they need about 12.5m mm of adjustment. If they back off a bit to 95% of women and men, which is the range from 57mm to 71mm, or a 14mm range needing 7mm of adjustment per eye. BTW, the reason why ML has two sizes is due to IPD and not head size).

So if the exit area is 27mm wide and they support 95% of all adult IPDs, they are going to lose/guard-band at least 25% of the horizontal FOV. If you take 25% off of 2560 you get 1920 pixels wide which gives a 4:3 aspect ratio with 1440 lines.

I have found in ALL cases when a company only quotes a single number for the FOV without specifying horizontal or vertical, the FOV they quote is the diagonal. “Bigger is better” and the diagonal is the bigger than horizontal or vertical.

Then we have the fib that laser scanning scan lines equate to pixel rows (they don’t). Next is the fib they are doing 120Hz. Assuming Microsoft’s claim of 54KHz is true and they are using the Microvision so called (but not really) 1440P device, they are only doing 120Hz INTERLACED which is really 60Hz full field refresh.

Butterfly Waveguide Confirmation

A picture from The Verge article confirms that HL2 has a “butterfly waveguide” as I said was possible last time. If you look carefully at the picture below right (and as pointed to by the four arrows) you can see the faint diagonal outline of the right and left (as seen by the user) intermediate DOEs.

I will be curious to see whether there is some visible seam or gap between the left and right side of the image that the user will notice.

Another interesting point is that Magic Leap supports roughly the same FOV but without requiring the butterfly technique. It should be noted that Hololens supports a much wider range of IPD adjustments with a single headset, which is supporting a significantly wider FOV just some of it is reserved for IPD adjustment. Additionally, the HL2 much more eye relief (distance from the eye to the optics) which makes it harder to support a wider FOV.

1. There is a huge improvement in ergonomics and they are squarely aiming at the industrial market. The HL2 puts ease of wearing at a premium over style. By contrast, the Magic Leap One went for style over substance and achieved neither. I expect that the HL2’s flip-up display concept is going to become common.
2. The resolution numbers simply don’t “add up.” I’m rather surprised, and even a little disappointed, that no one else calls them out on the 54,000 cycles per second mirror after dividing by 60 for a nominal refresh rate. I also think they may have problems turning the laser on and off fast enough near the horizontal center of the display. Assuming there is an HL3, there is a good chance that Microsoft will be looking for a different display technology.
3. I’m a bit baffled that they have did not name Microvision as part of the HLS announcement.

Word of warning on Typos

Lately, Ron Padzensky has been helping me out by reviewing some of the article before they go out. I’m doing a lot of traveling lately and wanted to get this article out ASAP and could not wait to send it by Ron. So there will likely be more than the usual number of typos, but I thought it better to get this out than wait.