Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Starts with Ray-Ban®, Ends Up Like Hololens
Introduction
Just a quick update to my last article this time. I found another “No Moore’s Law” for optics reference that was the subject of my last article, “Saying “Moore’s Law or Apple” Does Not Make AR a Consumer Product.”
Jerry Carollo “No Free Lunch and No Moore’s Law” for Optics
I was looking at some pictures I took at Photonics West 2019, I found some slides from Jerry Carollo of Google’s Daydream VR, which in turn led me to a Youtube Video that included his talk on VR and AR. The video had a slide saying “No Moore’s Law” for optics, which was the subject of my last article, “Saying “Moore’s Law or Apple” Does Not Make AR a Consumer Product.”
Some good quotes from the video, “It’s the optics guy’s fault because they can’t figure out how to make it smaller;” “If you have something like this and you have a lens and a certain focal length and a certain field of view there’s nothing magical you can do to make it small;” “That’s why they all look like bricks on the faces. That’s how I just tell my team all the time I say it’s your job to make this look better;” and “So in optics, there are two things, no free lunch, and no Moore’s Law.”
It looks like Google took Jerry Carollo at his word, because as The Verge on October 15, 2019, reported, “Google is discontinuing the Daydream View VR headset, and the Pixel 4 won’t support Daydream”
“Fix It in Post”
I was on a military base talking to some people there about AR. They said that in addition to talking to technical people, they were going “outside the box” to try and figure out the future of AR by talking to people from Hollywood. My immediate reaction was, the way they do AR in Hollywood is to “fix it in post” with computer graphics compositing. Composting in computer graphics is not bound by the laws of physics.
Everything Ends Up Like Hololens
A slide I have presented in my private talks discusses how “Everything in AR starts out trying to look like Ray-Bans® and ends up looking like Hololens.” I based it on the old Tree Swing Cartoon of product/software development.
While the tree swing cartoon’s point is one of miscommunication, the problem for AR is that people are being asked, sometimes (sometimes not) innocently enough, to violate laws of physics.
Everyone, it seems, starts with Ray-Ban® sunglasses as their starting point. As they try and solve one technical problem after another, the headset grows until it becomes a small helmet, ala Hololens (1 and 2). Even if they use something that looks thin like waveguides, the waveguides have to be protected as they are very fragile. Adding SLAM means having to put cameras that are spread out and have to be mounted somewhere, which makes the headset bigger. Supporting transparency while being bright enough to see the image, requires a bright projector, which in turn means battery power and heat management.
People are now coming to expect the high image quality they get from 4K TVs with high dynamic range and by the fanciful visions that Hollywood comes up with for AR that can only be done in post-production CGI.
Hucksters, like Rony Abovitz, CEO of Magic Leap, are not incombered by too much understanding of the physics involved (thus wiggling fiber display con). He has found enough suckers and executives with big egos to keep funding his AR fantasies, at least for a while longer. To me, it is bad when the hucksters are soaking up much of the big money. Good cons generally have something that gives them at least a thin veil of being believable. In the case of Magic Leap, they moved the processing and battery to a separate pack.
5G Won’t Fix Everything
In my last article, I got several people suggesting that 5G was AR’s answer to Moore’s law. From what I hear this the story Magic Leap is telling to raise money from Telecos around the world. I think they may have missed the point that optics are on a much slower-moving curve. The “move the computing to the battery pack” argument as morphed into the mythical “5G will fix everything.” Assuming the 5G cells are very close (the speed of light is ~1 foot or ~1/3rd of a meter per nanosecond) and every cell has a computer with enough processing to support all the users near it, at best, it only removes part of the computing burden and battery power off of the AR headset. It is a bit like squeezing a balloon in that other problems become worse.
The headset will now have a massive transmission burden and will have to have better back end display processing (ex., ARM’s Mali-D77) to adjust for “motion to photon” time lags. Even taking all the processing burdens away, there will still be the optical challenges to get a high-resolution image with the desired FOV presented to the eye.
Targeted Applications Are the Most Promising
One way to avoid the feature creep that ends up with Hololens-like helmets is to better understand and target specific applications rather than trying to be all things to all people and hoping for a miracle to solve everything.
I recall seeing a variation of the Tree Swing cartoon, that added what the sales department presented… it showed an entire amusement park around the tree swing.
All the developers of AR have the same problem: they have not identified exactly what specific problem their AR product addresses, so the product has to do “everything” while looking like Ray-Bans. Since there is no identified unmet need, there can be no value proposition, and therefore a requirement develops that every quality metric has to be made wonderful, including style and beauty. Thus, the product is never optimized to solve a specific unmet need, and costs can’t be brought down because trade-offs are impossible (nothing can be sacrificed). The finished device may do many things, but none of them well, and may fail completely on style and beauty. Yet, if a single unmet need were to be specified, the optical engineers could state the constraints the laws of optics require, and the product manager could decide whether to go forward or not without the Ray-Ban look. AR will succeed when a company identifies an unmet need and satisfies it at a price commensurate with that need, with ergonomics that are good enough to make the device usable. The mass market requires style and beauty along with satisfying an unmet need, consequently the present opportunities for AR must not be in the mass market.
Yes, and they are trying to be a superset of VR starting with chasing FOV. Getting a wide FOV was easy for VR starting from cell-phone-sized displays and not needing to be see-through. As I have written, some people are obsessed with FOV to the exclusion of everything else, like it is the only spec that matters.
Then you have the issues with SLAM and need for the sensors to spread out around the head for detection.
Have you tried DigiLens glasses an OEM is coming out with next year for $200 that doesn’t need a darkened lens to see in bright sunlight like Hololens2 does? I tried them on at Stanford University…Very impressed and for $200
The image quality is far from the best I have seen. We will have to see how well they are accepted.
I can’t wait for something like nreal glasses for $300 …I mean it doesn’t look perfect for the average consumers …it’s not scalable as far as fov variable focus and opacity and it’s not that great use case with sucking your phone battery(although u can use it with a pc) …but something like that can be a pretty excellent 1st gen somewhat popular descent consumer headset …but it will need someone like facebook dedicated to ar/vr to keep pushing it
Nreal is IMO a better starting point for experimenting with AR than say ML2 or HL1/2. The image quality of the display is very good, but the real world is darkened by about 70%. The bigger question for AR, in general, is what applications are going to drive volume. There are industrial/enterprise uses, but most of these don’t translate to the consumer.
I think we all know what kind of content drives consumer adaptation of new technologies…
correct. as we now see coming to a head with facebooks problems. Interfacing social activities for most “regular” people will not provide them an acceptable social return for the cost.( fashion and self identification as a new factor as well) of a product.
The biggest problem is that not a single damn AR company has seriously invested in software, and they’re expecting all of this stuff to magically come out of an incredibly small and relatively amateur/enthusiast level developer pool.
If Magic Leap actually made useful apps we wouldn’t be talking about how crappy certain aspects of the hardware are. But they’ve made no use case whatsoever for their product, and with less than 500 DAUs a month there’s NO reason for me as a developer to invest my time.
Err, less than 500 DAUs — not per month — (and aside from ML employees, that’s a very generous number)
I assume the acronym DAU stands for Daily Active Users.
Look they “only” have about 1,800 employees or about 3 employees per DAU (using your number).
I think the bigger problem is that they didn’t know what they were doing but managed to raise a lot of money. They pretty much scrapped their initial ideas and tried to buy a solution with the money they raised. It’s “jumping out of an airplane and knitting a parachute on the way down.” Magic Leap One is a horrible design with many flaws. I have problems with Hololen 1 and 2 but they are vastly better “products” that at least can serve some market.
True Dat
Looks like Microsoft is on right track by focusing on enterprise applications! This way platform gets funding to mature and still remain relevant in market.
I have a lot of problems with Hololens in the longer run, but at least they built a much more practical product that has applications. The biggest thing Hololens addresses well is eye relief. This is key to making it possible to be used with typical eyeglasses and thus shared between people. At least Hololens is good for something, even if the market is not huge (Hololens 1 sold only 50K units in it first two years according to Microsoft).
Magic Leap went all Hollywood and built a device that is very hard to use and can’t be shared easily. The image quality is very poor and they block out much of the outside light. Then ended up finding a void between VR and enterprise AR.
[…] More info (Karl Guttag’s article 1) More info (Karl Guttag’s article 2) […]
This is a great piece, thank you. I think there is equal hype (maybe more) around mobileAR, and there are really even fewer use cases for it. We read about new mobile AR campaigns and apps every day it seems, but we NEVER hear about usage numbers (so you can guess how they are). There has never really been a mobile AR hit app, period. Yet the hype and investment are off the charts and we keep hearing “its gonna happen” yet see zero consumer desire for it. You can argue the same for headsets. This piece is talking about mainly the technical restrictions. But you do touch upon the use cases (or the comments did) and really, is there ANY evidence that consumers want a AR headset to replace their phones, even if it were RayBan size?
“AR for Consumers” may be a non-starter (at least in the near term) for regulatory reasons: Scanning of the local physical environ is necessary for AR and no-one wants their information about their kids/family stored on some company’s servers.
Great article. In response to Michael Levine, in the case of evidence surrounding consumer demand for AR headsets instead of smartphones, there may be very little if anything at all. But we could argue that there was no demand for the reinvention of the mobile phone prior to 2007. I think like the smartphone, if an AR headset can enhance our lives, make things quicker, easier, and open new opportunities then the consumer will adopt it.
Cell phone advancement was hardly at a stall point in 2007. There were two main trains of thought as I remember it. You had the “smarter phones” like Blackberry, Samsung Blackjack (sort of a copy of Blackberry, hence the name), and many others. Then you had the group of “smaller is better” phones like Motorola’s Razor and many phones from Nokia, Erickson, and others. There were many companies working on “projector phones” to give a bigger display. What Apple figured out is that by using a touchscreen (only), they could make the “bar type” (Blackberry) phone with about 3X the display area, thus making things like web browsing much more practical. Having a “soft” mutatable keyboard proved to be a better option than say one with tactile feedback (several companies tried various forms of flip-out keyboards).
Going to AR from an iPhone is a MUCH bigger problem than going from say a Blackberry to an iPhone even though it does not seem that way on first consideration. The “physics” is much harder and there are massive human factor issues (size, weight, fit, input, different vision correction, etc.). Then there is the problem of what you do with an AR headset when you don’t want it (look at the cases for AR headsets today). There is a market for headsets designed for specific purposes with available technology, but the broader consumer market is a much bigger problem.
Agree with Karl’s last post. The iPhone was a less of a leap for many reasons as others have stated, not to mention the fact it was still a screen. It introduced touch to an extent, but the iPod really paved the way for that, at least in terms of Apple products. An AR Consumer headset, one that will replace people’s mobile phone, is many years away. Not only do we need better headset hardware, we will also need 5G, and more compelling use cases. The old adage (I am gonna paraphrase) but new tech cant be 2X or 3X as good as what came before it … it needs to be 10X as good to get people to take notice and truly change a paradigm.
[…] think the following figure from Starts with Ray-Ban®, Ends Up Like Hololens, summarizes what tends to happen with AR headsets. The expectations are for something like Ray-Ban […]