Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
It is well known that Microsoft’s Hololens uses two Himax manufactured Field Sequential Color (FSC) LCOS microdisplays. Additionally there are reports, particularly from KGI Securities analyst Ming-Chi Kuo as reported in Business Insider that Magic Leap (ML) is also using Himax’s LCOS. Further supporting this is that of all ML patent applications, ML patent application US 2016/0327789 which uses LCOS best fits the available evidence.
I have now from some additional evidence that ML is likely using LCOS. After discussing this new ML evidence, I will relay some Microsoft Hololens 2nd generation (or lack thereof) rumors.
I came across a bit of a strange patent that seems to confirm that ML is using field sequential color (FSC) LCOS. The patent application US 2016/0241827 was filed in January 2015 just 3 months before the lead inventor, Michael Kass then a ML Fellow, left ML. From what I can tell from their public LinkedIn profiles, Mr. Kass and his fellow inventor both worked on software at ML and not hardware and neither one has any background in hardware.
The patent application is directed towards reducing “Color-Breakup” for color sequential displays and shows an LCOS implementation. The concept they are proposing is at least 15 years old that I know of and it is well known to people in the projection industry that DLP’s projectors had “white segment” color wheels and later with LED illumination. Additionally the way they arranged the LEDs in their diagram above with 3 separate LEDs going to a dichroic mirror is how it is done for front projectors and not for a near eye display. The question I had on finding this application was:
Why are two ML people working on software with only a rudimentary knowledge display design and located in California filing for a patent on an “improvement” for field sequential color?
The only logical answer I could come up with is that they had look through ML prototypes that used an LCOS system and were bothered by seeing color breakup. I’m guessing they were told it was LCOS but did not know how it was designed so they grabbed a LCOS design off the internet (only one for a front projector and not for near eye). They didn’t know the history of FSC projectors using white segments, so they re-invented the 15+ year old concept of adding a “white” period where all the RGB colors are on in order to help reduce color breakup.
For bonus speculation, why did the lead inventor Mr. Kass who had only a month before filing this patent been promoted “Distinguished Fellow” then leave only 3 months after filing the provisional patent? Perhaps, just perhaps, he did not like the color breakup he was seeing (just a guess)?
It should be noted that it has been nearly two years since the provisional application was filed which would give ML time to change. But I doubt they could totally change directions as they would be too far down the road with the rest of the design. At least if, as they claim, they will have a product out “soon.” They might change the type of LCOS device either in resolution or manufacturer but it would seem unlikely that they could totally change the technology.
There was a lot of talk that Hololens, after announcing that Hololens would be focusing first on business applications, would be coming out with a 2nd generation Hololens next year. This sometimes gets conflated with the 2nd generation being a “consumer version.” But apparently the costs to make Hololens are high particularly with the custom waveguides having very low yield.
The recent scuttlebutt is that expected 2nd generation is on hold while Microsoft management figures out what they want to do with Hololens. For those that were hoping for a Consumer edition, the idea of focusing on “enterprise/business” sounds scarily similar to what Google did with Google Glass when if realized it did not have a high volume market. While Microsoft is continuing to expand sales of Hololens for businesses worldwide, one gets the feeling that Microsoft is trying to figure out if Hololens will have the size market anytime soon that is worthy of a company Microsoft’s size.
Update Dec 20, 2016 – I posed a question on the Reddit Hololens subgroup about finding a public source for issues with Himax and Hololens and they pointed to “A component maker suffers as Microsoft develops next-gen HoloLens” by Kevin Parrish on Dec. 14, 2016 in Digital Trends. In the article they cited Himax CEO Jordan Wu stating, “near-term headwinds” due to a “major AR customer’s shift in focus to the development of future-generation devices.” This would seem to imply that the “AR Customer,” of which Hololens is the most notable/likely, is switching from using a 720p to their new 1080p device on a Gen. 2 Hololens.
So there is mounting evidence that ML is using LCOS and the most likely manufacturer is Himax. I have had some people write me that ML switched from Himax but I don’t know how credible their sources may be, so this I would categorize as rumor right now.
Either way, Himax can’t be shipping a lot of LCOS to ML right now. The lack of volume coming out of Hololens also means that there are not big new orders from Microsoft for Himax panels.
Karl, good day.
Interesting question. A lot of bla-bla about AR hardware in the press.
But for which user cases we need to buy holo or ML?
Which problem solving?
Yes, it comes down to what these things are going to be able to really do when you look through them, how much they cost, how you feel when you where them, how much of a burden they are to wear, and (often left out) how are you going to store them. There is a big gap between peoples expectation and what can be done.
Hello Karl
Himax’s WLO can also make waveguides. Do you think Hololens or ML will
adopt that for cost reduction ?
Thanks!
An interesting question but I doubt it unless there was a very special arrangement from Hololens, and ML has said they were making theirs in Florida.
There are a lot of different types of waveguides requiring very different technologies. I know Himax needs a “waveguide” for illumination of its Font-Lit LCOS, but this would be a very different level of waveguide as only illumination is injected into the thin part of the waveguide. The image from the display passes through more like a piece of flat glass. This type of waveguide is much simpler to make than the ones that inject an image into the thin edge.
Hi Karl,
Himax actually manufactures the waveguides of the Hololens. They have the suitable replication technology.
Thanks for the information. I can’t remember having seen that information that anywhere.
Karl, could you please offer your insight on this statement:
In fact, the maximum field of view achievable through holographic wave guides is directly related to the index of refraction (n) of the wave guide’s material, as described in Microsoft patent:https://www.google.com/patents/US20130250430.
Concretely, the maximum balanced field of view for a wave guide made of material with index of refraction n=1.7 is 36.1°, and given that typical optical glass (crown or flint) has somewhere from n=1.5 to n=1.6, this matches FoV estimate of 30° rather well.
Is there any way you see that they could get around this optic limitation?
Does the traditional 45 angle waveguide actually have fov limitation?
In the light of this, flat waveguide is a bit like dead end in terms of fov and it seems quite logical that perhaps microsoft will try the most cost effective solution for their consimer version by combining cheaper traditional 45 degree waveguide with lcos, you get the best of two worlds, so to say.
A few things.First that patent application uses prisms to make the light exit. The Waveguide used in the current Hololens uses diffraction gratings to make the light exit. So this patent is NOT what Hololens is currently using even if the inventors may work for Microsoft.
Lumus has a similar prism exiting waveguide technology to that in the patent application. They currently have a 40 degree FOV and reportedly have a 60 degree FOV version https://techcrunch.com/2016/01/07/lumps-is-showing-a-wide-field-of-view-smart-glasses-prototype/ .I have not personally seen it yet and don’t know if there are any issues, but it would seem Lumus has found a way around the limitation described in the patent.
BTW, ODG is just introducing their R-9/Horizon near eye display with about a 50 degree FOV with an OLED Microdisplay and a roughly 45 degree flat plate combiner.
Thanks and sorry for, well, wrong patent. Nevertheless, would you say principally they could increase FOV flat waveguides used in the hololens and the statement of optical limits of 36 degrees does not quite hold up?
I do wonder however was it really than just mostly keeping the pixels per arc minute the reason they kept the fov as it currently is. Although i have heard that for the very first prototype devices which were shown for first outsiders, there was discussion that some people claimed, that fov was slightly bigger than current developer device, but the device was tethered, which caused some people to speculate that current fov is also perhaps “limited” due to optimal processing power. Although as you mentioned multiple times, the seemingly wider fov could have been in fact due to set up stage, right room, right light, no engineering background, first time amazement effect etc.
I actually just found this news: https://mspoweruser.com/microsoft-aims-solve-hololens-field-view-issue-combining-wave-guide-lightfield-displays-patent/
How viable you think this solution is in terms of cost and technology? How could it efect current LCOS technology being used compared to micro oled etc?
I took a quick look and it looks what happens when none of the rational approaches will work, you start looking at the irrational. In some ways it looks like Magic Leap.
I would bet on higher resolution displays and simpler optics.
Do you think this solution is viable enough to show already in 2nd generation device or it is a bit more long time frame?
If you are talking about the recent M/S patent, I don’t expect to see that for a long time if ever. You can get to a pretty good 50 degree FOV with 1080p technology and have an acceptable angular resolution (~1.7 pixels per arc-minute). Something like that Microsoft patent would make things very complex and much more expensive even if it worked.
Yes, was thinking about the viability of that particular light field display. Thanks for your input. Wish you a great holiday!
Karl the a look at this waveguide patent from WaveOptics
EXIT PUPIL EXPANDING DIFFRACTIVE OPTICAL WAVEGUIDING DEVICE
https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2016020643&recNum=1&maxRec=&office=&prevFilter=&sortOption=&queryString=&tab=PCT+Biblio
Here’s what I believe is the “moonshot” waveguide for the ML FSD
Multiple depth plane three-dimensional display using a wave guide reflector array projector
https://www.google.com/patents/US20140003762
Waveoptics at a high level appears to be similar to what Hololens is doing, but I have not dug into it very deeply (it would take a long time for me). I would like to see what it looks like.
I was aware of the ML patent you referenced. It is one of many “moonshot” patent applications ML has made. I wonder what it would look like if you looked through it if it worked.
Curious what your opinion of what this “wide view” waveguide display coupled to HIMX Front Lit LCoS would be in terms of size.
WaveOptics has working prototypes of their displays and they are scalable for mass production.
https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2016113528&recNum=1&maxRec=&office=&prevFilter=&sortOption=&queryString=&tab=PCTDescription
Here’s the Himax Display patent for the packaged Front Lit that was issued a couple weeks before the Front Lit patent I shared with you…
https://patentscope.wipo.int/search/en/detail.jsf?docId=EP178652536&recNum=10&maxRec=1949&office=&prevFilter=&sortOption=Pub+Date+Desc&queryString=FP%3A%28himax%29&tab=NationalBiblio
Himax Display was also issued a MEMS patent the same week.
MANUFACTURING METHOD THEREOF
https://patentscope.wipo.int/search/en/detail.jsf?docId=US178800595&recNum=8&maxRec=1949&office=&prevFilter=&sortOption=Pub+Date+Desc&queryString=FP%3A%28himax%29&tab=NationalBiblio
They also fast tracked a patent application and received approval. The inventor is Roland Van Gelder who is now working as a Senior Hardware Designer at Apple. He was with Spatial Photonics and Himax Display for many years.
METHOD FOR FORMING ANTI STICTION COATING AND ANTI STICTION COATING THEREOF
https://patentscope.wipo.int/search/en/detail.jsf?docId=US178076293&recNum=14&maxRec=1949&office=&prevFilter=&sortOption=Pub+Date+Desc&queryString=FP%3A%28himax%29&tab=NationalBiblio
I think the Himax Front Lit is an interesting technology for making smaller near eye displays. As I wrote, there is a fair chance that ML might be planning on using them. It gets over some objections to using LCOS over OLED Microdisplays. I have not to date been able to see a serious evaluation of the pro’s and con’s of Front Lit. Usually when you try and compact the optics down, there is some lose in image quality. You can generally get away with more in the illumination path than the image path, but in the case of Front Lit, some of the structures for illumination are also in the image (output) path.
I have no idea what they would be using the anti stiction coating for out of the context of a DLP-like device. Spatial Photonics, a “DLP-like” device, is a blast from the past; I have not heard their name in a long time, since Himax acquired then in 2012. It made me think back then that Himax might go into the DLP clone business as an alternative to LCOS. I haven’t seen anything since they were acquired.
Have you had a chance to read this patent from Himax on their light guide plate to reduce ghosting? It’s part of the front lit technology.
0037] Advantages of the structures described herein include the ability to reduce a thickness of the projection display apparatus by using a light guide plate for illuminating a reflective light modulator in a front lit arrangement. Moreover, the light guide plate can be constructed to include grooves or optical elements that can advantageously eliminate ghost effects, improve display brightness and contrast ratio.
https://patentscope.wipo.int/search/en/detail.jsf?docId=EP173542726&recNum=30&maxRec=1949&office=&prevFilter=&sortOption=Pub+Date+Desc&queryString=FP%3A%28himax%29&tab=PCTDescription
There is only so much I can follow. I would like to see a Himax Front Lit display. They can apply for dozens of patents that MIGHT improve some effects and still have other problems. You really want to see the result to tell if they have something. It also helps to talk to experts that have worked with a technology to understand all the pluses and minuses of the technology. Usually the patents stress how they solve one problem but don’t tell you that they introduce others.
When’s your next trip to the Bay Area? The Front Lit is very real and protected by 3 patents.
I certainly believe Font Lit is real, I’m not sure why you think I don’t. I have multiple people including Himax’s web site telling me that it is for sale. I understand that it will make for a more compact design I also understand that they have a U.S. patent but I have not studied the claims.
What I don’t know is any optical issues good and bad with front lit versus more common optical solutions. Usually when you squeeze things down, there are some negative side effects, but I don’t know.
I’m not sure when I am going back to the Bay Area but I am going to be at CES in a couple of weeks and hope to see a Himax Front Lit system in use there. I would also welcome anyone telling us more about the pro’s and con’s Himax Front Lit’s design.
Karl, I don’t feel that you think it’s not real. I’m just suggesting that there’s some things about the technology that might be overlooked regarding color breakup. As far as the Bay Area, that’s where you would be able to arrange a viewing and I’d be more than happy to facilitate that for you.
Thanks for the offer. If I get out that way, I would be happy to take you up on the offer. Hopefully, I will get to see at least one of the front lit designs at CES.
As a general rule, there is something given up for a more compact design. You often require a single optical element to do two or more different jobs where in a more spread out design there would be different optical elements for each job. Almost invariably doing one of the jobs better comes at the detriment of the other job.
In the case of the Himax Front Lit, the waveguide has to do a lot of different things. I has to both spread and direct illumination light down to the LCOS device and then the image has to pass back through it. Off the top of my head, what I would be looking for is the illumination uniformity and contrast uniformity (LCOS/Polarized light systems are sensitive to angle of light issues) and if there are any ill-effects to the image in passing back through the waveguide.
Minor error:
“Update Dec 20, 2017” and “by Kevin Parrish on Dec. 14, 2017” should refer to 2016, not 2017.
Thanks