304 North Cardinal St.
Dorchester Center, MA 02124
304 North Cardinal St.
Dorchester Center, MA 02124
This article started a few weeks ago as-as simple mentioning of all the aftermarket Heads-Up Displays (HUD). I saw at CES. But as I kept writing, it turned into more of lessons learned and (primarily) optical design issues on aftermarket HUDs.
There were many aftermarket Heads Up Displays (HUD) for automobiles as well HUDs for motorcycles at CES 2018. I saw some but missed a few in the vortex that is CES. I have a bit of a history of aftermarket HUDs having been the CTO and Co-Founder of Navdy, so I took particular interest in these products. BTW, Navdy appears to be out of business, and I left over three years ago as of this writing.
While at Navdy, I came up with the “HUD on the dash with a spherical combiner” (above left) configuration that has been much-copied. The list of near direct copies and derivative products include Aker ROAV, Carrobot & Hudway, iScout, WayRay, Lumens, Exploride, Kivic. For more on my work in early aftermarket HUD displays, see the Appendix at the end.
In 2016, the average age of cars on the road in the U.S.A. was 11.6 years old and getting longer. Vehicles are simply outliving the technology. People find HUDs are desirable and companies are trying to fill the void.
Automakers get to the design their dashes and bury their HUDs inside the dash and can put optics in places to help manage sunlight. The aftermarket HUDs have to live with the dashboards which vary tremendously in shape, height relative to the driver, and depth to the windshield. The aftermarket HUDs have their optics exposed to the sun sitting on the dash.
The first requirement of a HUD should be to do no harm with the driver’s visions under all realistic driving conditions from full sun to nighttime. The first issue to not block the driver’s view of the road when mounted. Some of the combiner/lens block more than 50% of the light right from the start. Many of the products fail even in easy conditions due to the amount of light they block, and most if not all, with have severe problems in full sun. Blocking vision also goes to display content in that if the image is bright enough to be visible, then it will be bright enough to block the driver’s vision if a large solid is displayed.
Because a HUD is meant to mix both the farther view of the road with the display image, so a driver does not have to refocus, they are supposed to optically move the focus of the image into the user’s far vision. Most use a curved semi-mirror for this purpose, one of them (WayRay) uses refractory (lens) optics, and one (Hudway) does not change the focus.
For an image to be visible, it has to stand out against the real world. With a HUD, the background is whatever you are looking at, and that could be a sunlit concrete road that is sending over 10,000 nits toward the eye. Without exceptional sunlight management, the displays will be washed out. Other than the Lumen’s Micro-LED prototype, I’m not sure if any of the other products are going to be visible in bright sunlight.
As a reference point for those not familiar with display brightness, a typical modern cell phone display has roughly 500 to 700 nits (technically known as candelas-per-square-meter or cd/m2, but everyone hates typing that out) which is a measurement of light in a given direction. A built-in automotive HUD typically supports over 15,000 nits to the eye after bouncing off the windshield or a separate combiner to compete with bright sunlight. Commonly, the windshield or combiner reflects less than 20% of the light towards the eye which means the display needs to output 70,000 to 100,000 nits or over 100 times brighter than a cell phone.
For reference, if sunlight was illuminating a white piece of matt white (non-shiny) paper but without glare (specular, mirror-like reflection), it would give off about 30,000 nits. Sunlight on concrete is going to (non-glare) give off about 12,000 nits and an asphalt road about 3,000 nits. A bright blue sky is about 10,000 nits and looking directly into the sun is about 1.6 billion nits (don’t do that). For decent readability, but very poor color saturation, you would like to have at least 2:1 contrast.
So if you are looking into a blue sky through a clear windshield on a sunny day but not into the sun, you need about 10,000 nits to add to the 10,000 nits of the sky to have 2:1 contrast. In a color balanced “white” image, image green provides about 68%, red 28%, and blue only 4% of the perceived light. So you need to have even more total “white” brightness to be able to read colors. BTW, blue is almost invisible in a HUD, and so cyan (blue plus green) is often used to represent blue. Looking at these numbers and the various lighting issues, you can see why major HUD manufacturers want at least 15,000 nits from the display going toward the eye.
At night, the ambient can be nearly zero nits, and the human eye becomes very sensitive to light. The SMPTE minimum standard for a movie theater is 14 Foot Lamberts = ~48nits and movie theaters showing 3-D movies are often less than half of that. You want a HUD to be able to go to less than 50 nits to not blind the driver at night. You should have about a 15,000/50 = 300:1 difference in brightness from day to night a dimming ratio which can be difficult to support.
All of the aftermarket HUDs have a separate “combiner” that acts to reflect the display image and combine it with the real world. Most built-in HUDs use the windshield as a combiner, but this requires locating the optics inside the dash to project the image light at the correct angle to reflect the light towards the driver’s eye. It also requires a special windshield to prevent a double image from the laminated glass.
With typical aftermarket combiners, the more they reflect the display’s light, the more they block the real world. If they reflect X% of the display’s light, then typically they pass less than 1-X% of the real-word. Most use a spherically curved combiner that both magnifies the display image and changes the focus to make the image seem further away. The amount of magnification and focus change is a function of the radius of curvature of the combiner, the distance from the combiner to the imaging device, and the distance of the combiner to the viewer’s eye.
Most states don’t allow you to tint your windshield at all except at the top. Manufacturers without enough brightness “cheat” by blocking the light coming in and reflecting more of the display light to get some contrast and readability. At some point, you have to ask, “is this a see-through display?” Blocking up to about 20% to 30% of the light is barely noticeable. By 50%, the light blocking is noticeable, and by 75% it looks like dark sunglasses.
Sitting on the dash in direct sunlight causes one of the hardest while less obvious optical issues. Built-in HUDs have “light trap” sun blocking devices that that keep the Sun from washing out the display images. But with a dash mount HUD, the display is exposed to the sun. At Navdy, used an extremely high gain screen in an orientation that would direct sunlight away from the combiner while at the same time directing more of the display’s light toward it (see Appendix below). I didn’t see any sunlight trapping devices in any of the aftermarket HUDs I saw at CES or have looked at in online videos and pictures.
I have seen aftermarket so-called “HUDs” that just use a cell phone have combiners that reflect over 70% of the light while blocking 70% of the light ahead. They are not bright enough to be used in sunlight, and they block so much light at night that you can’t see through them so what is the point. The user would be better off with a simple phone holder. It’s not clear how most aftermarket HUDs are much better.
In the daytime use the key factors for a HUD are the display nits directed at the combiner, the amount of sunlight the display’s “back” reflects into the combiner, and the ratio of light the combiner reflects versus passes. Not having a bright enough display, means you have to block more of the real-world like to make the image readable in daylight.
At night, just about any display has plenty of light, and the critical factors become the black level of the screen and the transparency of the combiner. Often high brightness displays will have lower contrast, and this can be a problem at night. If the combiner is blocking a lot of light to support daytime contrast, then at night it will block too much light to see through (so much for being a “HUD”).
Many companies are using wifi mirroring so they can then use the popular navigation apps like Waze, Google Maps, and Apple maps for navigation including traffic. The big problem is that none of these apps have their content designed for a HUD. Most importantly they usually display an a non-black background. On a HUD, you display “clear” by displaying black. If the background is bright enough to be visible, then it blocks the view through the HUD defeating the whole idea of being transparent. Beyond the background, the content of the display information should be readily interpreted against the real-world.
Beyond navigation, displaying content on a light background defeats the whole concept. With a HUD that is bright enough to be usable, it will be bright enough block the user’s view through the combiner (duh?).
When I was designing Navdy, I was particularly concerned that it could be adjusted so that the drive would not have the combiner in the driver’s critical vision of the road ahead while driving. Even being 70% transmissive and with good light rejection, there would be situations at day or night where you don’t want it blocking the user’s critical view of the road.
Worst yet, I see companies using these to display navigation directions and other content not designed for a HUD so that the display is mostly light and thus blocking the view through the combiner. With a HUD that is bright enough to be usable, it will be bright enough block the user’s view through the combiner (duh?).
These aftermarket HUDs should not block the “critical view” of the road any more than a non-transparent display. I’m appalled by demo setups and videos that show people looking directly through the HUD at the road ahead.
There are a whole set of other issues that I am not going to address except where there are obvious problems. “Fitment” for example refers to how things fit on a vehicle and would require trying these out on various cars and various possible driver eye height. It is a very big problem as dashboard design and people vary significantly. There are issues with how steady the image will be when the car is driving. It would be a major effort to evaluate even a single HUD.
Lumens LED (of Korea) may have had the most technically interesting HUD display prototypes because they used Micro-LED microdisplays, not the HUDs themselves. Each prototype used three microdisplay devices, one each for red, green, and blue that are then optically combined producing a color image.
Lumens said the three panels combined output 100,000 nits, that’s more than 150 times brighter than a typical cell phone. It is the level of brightness you need to use a combiner or windshield that is 80% transparent and deliver about 15,000 nits to the eye. It is the only aftermarket HUD as CES I saw that could claim to have enough brightness.
One of the two HUD prototypes (above left) uses a projection lens after the LEDs to project an image onto a screen. The image on the screen is magnified and has its focus changed by the spherical semi-mirror combiner.
The smaller, and more interesting of the two designs (right), uses the output of the combined Micro-LEDs directly as the output to the combiner with no projection lens or screen. The combiner magnifies and reflects the direct image of the Micro-LEDs.
I’m not sure that using Micro-LEDs in either of this configurations is the best use of the technology, but it great to see a company showing Micro-LED microdisplays in a place as public as CES. I can’t see aftermarket HUDs as the best first use for Micro-LEDs and Lumens gave no pricing, but I suspect these were expensive lab prototypes.
Hudy’s new “Hudly Wireless, has gone to about the opposite end of the technical spectrum of display technology. They have a simple, high brightness LCD with a system that wirelessly communicates with a cell phone and uses its software. Hudly claims “full [wifi] wireless mirroring” and applications like Waze and Google Maps will work according to Hudly.
Hudly is currently running an IndieGoGo campaign at $219 plus shipping (delivery slated for April 2018) that is extending a Kickstarter that recently ended. At this point, they have raised $302,055 between the two campaigns (disclaimer: this is not a recommendation to buy the product nor do I have any business relationship).
The mirroring of the phone via wifi and using the phone as the input/control device is a much better idea that replicating so much of the phone as was done at Navdy. While the various HUD companies often provide their proprietary navigation apps; they can’t compete on things like routing and traffic with the major companies. But as I noted in “On a HUD Clear = Black” above, the popular navigation apps don’t have HUD modes with back = clear backgrounds.
Form factor wise, the LCD can go flat on a dash which is better than the ones like Navdy and Lumen with the display image on edge that you have to look over. But this also exposes the display to direct sunlight that will reflect back onto the combiner and gray out the forward view.
While the functionality may be good and the large 6″ diagonal LCD is nice, I quibble more about the display technology. The reason why other companies, including the automakers, have not used a large bright LCD is that it does not perform well as a HUD. First, let’s look at the Hudly Wireless specifications below:
Specs (From the Hudly Kickstarter Page)
The temperature spec means you must remove it very cold weather or on a summer day. A car’s dash will easily get above 140 degrees less than 100 degrees in sunlight. The dash area acts as a solar oven with the windshield and dark dash. Hudly makes a point that it is easy to remove from the car’s dash (which will be necessary for summer).
While the resolution of 800×480 may seem a bit low by today’s standards, it is ample for typical HUD applications where the graphics must be kept simple and easily seen (high-resolution graphics are wasted on a HUD). The problem may be that popular navigation apps will try and display higher resolution content rather than simplifying it for HUD.
I contacted Hudly about the cd/m² on their Kickstarter, and they say they are now getting 5,000-6,000 cd/m². While about ten times as bright for a cell phone, it is still not enough for a HUD on a sunny day. Using Hudly’s data, it is at best going to deliver 6,000 x 65% = 3,900 nits to the eye, and even with blocking 65% of the daylight, the image is going to be pretty faded. You can see the image fades in some of the Hudley videos that do not appear to be shot in worst case conditions.
The lack of transparency will be a problem seeing through at night. It is an unhappy compromise due to the lack of display brightness for daytime use.
I have a problem with Hudly Wireless’s Kickstarter page (with my bold emphasis) claim:
“Hudly uses optical collimation to display directions in a similar focus plane as the road, elevating your reaction time and situational awareness since there is no longer a need to refocus your eyes for directions.”
Unlike other HUDs (including Hudly’s earlier product) that use a curved combiner or other optics to shift the focus, nothing was evident on the Hudly Wireless.
In the photo on the left below, the display is in-focus and the background out of focus. In the photo on the right, the background is in-focus while the image on the HUD is out of focus. If the unit were doing as Hudly claims, the HUD display in the righthand photo should be in-focus.
Additionally, if you look at the direct view of the LCD in the left-hand photo, if the light coming off the display were highly collimated as claimed, the display should be very dark. If collimated, the light from the LCD should almost all be directed at the combiner and almost none toward the camera.
As a HUD display, it will probably work OK much of the daytime but will perform poorly in direct sunlight or at night. The bigger LCD both good regarding seeing the image but bad regarding mounting the unit and blocking forward vision. Using external apps is a double-edged sword as they are not designed for HUD use and using a custom Hud application means having poorer routing and traffic information.
Hudway, originally from Russia, and now hailing as a California company is rebranding Carrobot of China HUD products (as I have traced the lineage). I don’t know if there are software differences, but the Carrobot C2 is the Hudway Drive, and the Carrobot C2 Lite is the Hudway Cast regarding hardware. Hudway in their videos and web pages try to suggest they designed the hardware. At CES 2018, HUDway was showing their branded Hudway Cast. Following the lineage of Hudway and Carrobot demonstrates the changing aftermarket HUD strategies.
Hudway started with (IMO absolutely worthless) the Hudway Glass, one of perhaps a dozen or more semi-mirror with a cell phone products that were capitalizing on the after-market HUD craze that sold for about $49. The had a slightly curved 40% transmissive (60% reflective) device on which you sat your smartphone (see right). In this configuration, the curve semi-mirror is mostly just there for show/marketing purposes as it has minimal focus changing effects (mostly it just distorts the phone’s image). As said previously, a phone typically outputs only about 500 nits which after the 60/40 mirror means only about 300 nits if the phone is on full brightness will make to the eye.
While 300 nits with a HUD will just work on an overcast day (that they show, see above right), it is useless on a day when the sun it out. And being 40% transmissive means, it is too dark to see anything that does not have lights on it at night. It is not sufficiently see-though, and if the sun comes out, the image will disappear.
Carrobot of China was the first company I know of to try to copy what we did at Navdy. They apparently took the Navdy presale information and copied it for their design. Navdy took so long to finish the product that Carrobot shipped their product well before Navdy. Their product looked more like Navdy’s presale prototype (right) than Navdy’s final product (top of this article).
Carrobot also copied most of Navdy’s feature set including integrating GPS, voice recognition, and the essentially worthless gesture recognition. Inn Carrobot’s comparison table for the C2 generation, they state “Hand Gesture Control (Not Recommended”).
For their second Carrobot switched from using a DLP projector like Navdy to a TFT LCD with LED backlight used as used in built-in automotive HUDs. Carrobot even put out (left) a very nice exploded view (copied in Hudway’s presale campaign for their “Drive”).
While they moved away from DLP projection to a very bright LCD, they kept most of the features and the general configuration of Navdy. They even added a circular steering wheel controller that looks very much like what Navdy put in their final product (since gesture recognition was worthless).
There are pros and cons to using an LCD versus DLP with a screen for the display. Carrobot has no brightness spec’s anywhere, and when critical specs are missing, they are almost always not favorable to the product. Based on my experience, I doubt that there is enough light getting through the LCD, combiner, and back to the eye.
Kivic of Korea has a similar LCD based design, and they specify “17,000 Nits Direct Backlit BLU” (which has to get through a lot to get to the eye). Typically less than 10% of the backlight will make it through the LCD polarization, color filtering, and light-blocked by the LCD structure. The combiner is then on the order of 70% transmissive and 30% reflective. So the Kivic has less than 17,000 x 10% x 30% = >510 nits making it to the eye or about as bright as a cell phone. But a cell phones image is on a black background where a HUD is on top of the real world, so 510 nits are not nearly enough (by a factor of about 30 times).
It was getting to the desired 15,000 nits, which caused me when designing Navdy to use a DLP projector. Built-in LCD HUD requires huge heat sync (much larger than the copper one in the Carrobot C2 design above). The DLP approach was massively more efficient regarding watts-in-to-nits-displayed. Efficiency is a reason why the new “AR HUD” by Continental is using a DLP for the larger area part of the HUD. Lumens (see above) indicates that Micro-LEDs will be more efficient than the DLP someday.
With the tricks I was able to do with the DLP’s screen, I had an exceptionally good light rejection of sunlight. A built-in HUD has ways to baffle and trap the sunlight, but in the Carrobot C2 (and later C2 Lite) the LCD is very exposed. I would expect, have not had the chance to test, that sunlight reflecting off the LCD would block the view through the combiner in some sunlight conditions.
With the Carroabot C2 Lite, they stripped out all the duplicated hardware from the cell phone (GPS, Gyros, sensors) and gesture recognition. Something I thought Navdy should have done in the first place to both lower the cost of design and manufacturing and to get the product out much sooner (let’s just say I lost that discussion). In this way (same with Hudly), the cell phone acts at the computer, GPS, and input/touchpad. The HUD becomes a display. I think this makes infinitely better sense as the HUD capability can improve with each advancement in cell phones. The cell phone makers can add features much more inexpensively than a lower volume HUD manufacturer, and from a user’s perspective, all the hardware in the cell phone is “free” (they already paid for it).
By stripping out the electronics replicated by the smartphone, the Lite/Cast becomes less expensive to make. Almost more importantly, it lets the companies leverage existing popular navigation applications. But using existing software has its downside.
But as the still from Hudway’s video demonstrated (right), there is an unhappy compromise in using content not designed for a HUD. First, if the display is bright enough to make the image stand out, then it blocks the view of the road. Second, much of the content is too small to be readable.
None of the videos and pictures I could find on Carrobot or Hudway demonstrated the use of the C2/Drive or C2-Lite/Cast in challenging sunlight conditions. Driving in the direction of the sun might expose issues with sunlight rejection. Bright light on concrete would show whether it is bright enough (which I doubt).
One HUD I wanted to see with my own eyes at CES but missed by WayRay Navion. I have tried to make up for this mistake by reviewing the information available online. IMO WayRay is the most different and ambitious while being ridiculous as an aftermarket HUD. According to their September 2015 Video, their 5th generation prototype was nearly projection ready.
To begin with, WayRay is huge, and the base will block the view of users on a high percentage of dashes and driver positions. It is massive and too tall. The holographic film combiner is huge, and you have to look through which will likely be both state law and serious safety problem. The feature set of the HUD means that it is going to be so expensive (WayRay won’t give a price) to make that it will be prohibitively expensive.
WayRay appears to have worked the problem backward from their holographic mirror film technology. While making a big deal about using a holographic film, I don’t see it having a significant optical advantage in this application. As best I can tell, it is just doing the optical work of an inexpensive partial mirror, but with perhaps more transparency. At the worst, it is going cause issues when in direct sunlight. I found a hint of this problem in some of the WayRay videos. If you look at the frame capture on the right, you can see where sunlight from the windows on the side cause diffraction “rainbow” on the screen (see the arrow coming from window and rainbow colors inside the red oval). Additionally, you can see in the still frame on the right a milky appearance caused by stray light. These are the problems caused by a holographic film, and I would be concerned that in the worst case lighting condition, the driver’s view would be impaired both in bright sun and by headlights from oncoming cars at night.
Because their film is acting as a flat mirror, WayRay is using a large refractive lens (see left) that magnifies and moves the focus of the display. I don’t know whether they are using a projector or flat panel (either one would fit in the huge base. I would expect this large lens to be problematical regarding causing serious reflections back to the combiner when hit by the sun at the right angle.
As I see it, the WayRay fails in its based operation at a HUD. WayRay goes on to integrate pretty much everything that everyone else has talked about doing their products plus adding driver assist and AR features. Based on what I see as a HUD and WayRay’s history, I have a hard time believing they will deliver on these features at all, no less at a price people can afford.
As wrote in the introduction, I was CTO and Co-Founder of Navdy, and I figure out how to design a dash mounted HUD with over 15,000 nits that would not be washed out even in bright sunlight. Navdy appears to have gone out of business, likely due to costing too much to build and market compared to its selling price. Navdy duplicated much of the smartphone’s hardware (and cost) plus adding what turned out to be extremely limited gesture recognition, rather than leveraging the phone and adding a HUD display without all the “bells and whistles.”
When I started with Navdy in 2013 (and left three years ago), there were already some aftermarket HUD products on the market, but they were expensive, with poor image quality, and were horrible to install. Most notably Pioneer had two designs which both mounted on the visor and JVC attached to the rearview mirror (left).
Pioneer’s first design used a laser beam scanning engine with a rear projector screen (also known as a “pupil expander” or “EPE” in the context of a HUD). The projection screen consisted of a sandwich of plastics with textured surfaces that appear to have been there to reduce the laser speckle. Whatever the reason, the resultant image was fuzzy and noisy. The laser projector engine was much more expensive and had no optical benefits over DLP or LCOS engines after projection onto the screen. It also had a poor dimming capability for night use (the laser shut off rather than dimming). It required the user to remove their sun visor and to drill holes in the car to mount. The product with its custom electronics cost over $3,700 or more than 10X the price we were targeting.
In 2013 Pioneer tried again with a lower cost and easier to install DLP based HUD priced just under $1,000. This version clamped to the visor and disabled it with the HUD as installed. Since the whole system mounted below the visor, it tended to block user’s view out the car, and the screen was uncomfortably close to the driver’s face. I could not see the road when I installed it on my car. This design switch from rear to front projection. The screen was a sandwich of a film and plastic that cause the image to took ruff. The unit was still massive compared to what would fit on the dash if you simply flipped it over.
Also in 2013, JVC Kenwood introduced a rear projection aftermarket HUD using a Syndiant LCOS microdisplay with a rear projection screen and a spherical combiner. They mounted the unit to the rearview mirror. This mounting method was not stable, and it blocked much of the forward view of the driver (see left). The system with electronics was on sale for the equivalent of ~$2,500.
I felt that both the Pioneer Visor Mount and the JVC mirror mount were horrible and even dangerous. I decided that the HUD must mount on the dash similar to aftermarket GPS navigation systems. This configuration would make it easier to install and more importantly, not block the user’s view out of the vehicle.
I saw a lot of promise to further compact Pioneer’s front projection configuration, but there was one big problem. In Pioneer’s visor orientation roof of the car, blocks the sun from the screen but if you flipped it over, the sunlight and put it on the dashboard, the sunlight would overpower and wash out the image on the screen. So the task became to find screen materials that would direct light from the sun away from the combiner while directing light from the projector toward the combiner. By using very high gain screen materials (some that looked like the frosted side of aluminum foil), the sunlight would bounce away from the combiner while at the same time, the projector light would be even brighter and more efficient.
The Pioneer design was also far too big and bulky to fit on the dash. Then there was the issue that some dashboards were higher than others, so I was always seeking to make the design, particularly the screen that would stick, up as low as possible.
I hacked together the first working Navdy prototype from an AAXA DLP projector and the combiner off a JVC unit and an AAXA DLP projector that I took apart and added an extra lens to so, it could focus at a short throw distance. As shown in the picture (right) I used a “screen” made out of the frosted side of some aluminum foil. Later prototypes used the Pioneer laser HUD combiner which gave better image quality until Navdy could get custom combiners made.
I found I could modify the focus mechanism on the AAXA projector and eliminate the extra “short focus” lens. The result what that inexpensively hacked together prototype was brighter, with vastly better sunlight rejection, and with much better image quality than the Pioneer and JVC HUDs. And, it was all done for very little development money.
Eventually, I found some better (than aluminum foil) high gain screen technology, and was able to deliver over 15,000 nits to the eye (necessary to have the display visible when driving into the afternoon sun). The combiner was about 70% transparent (30% reflective of the projector light) which meant we needed over 50,000 nits coming off the projection screen. The screen’s high gain meant the collimated light from the projector was only slightly diffused (for pupil/eye-box expansion). The net result was a system that was optically very efficient, bright while rejecting sunlight to give good contrast both in the day and night.
I have learned a lot about optics and HUDs in the last three years since leaving Navdy and would design a HUD display (aftermarket or built into a car) very differently today.