Introduction

In part 1, I wrote that I was planning on covering optics and display companies at CES and the SPIE AR/VR/MR conferences in 2024 in part 2 of the video I made with Jason McDowall in this article. However, as I started filling in extra information on the various companies, the article was getting long, so I broke the optics and displays into two separate articles.

In addition to optics companies, I will also be touching on eye track with Tobii, who is doing both optics and eye tracking, and Zinn Labs.

Subscription Options Coming to KGOnTech

Many companies, including other news outlets and individuals, benefit from this blog indirectly through education or directly via the exposure it gives to large and small companies. Many, if not most, MR industry insiders read this blog worldwide based on my conference interactions. I want to keep the main blog free and not filled with advertising while still reporting on large and small companies. To make financial sense of all this and pay some people to help me, I’m in the process of setting up subscription services for companies and planning on (paid) webinars for individuals. If you or your company might be interested, please email subscriptions@kgontech.com.

Outline of the Video and Additional Information

Below is an outline of the second hour of the video, as well as additional comments and links to more information. The times in blue on the left of each subsection below link to the time in the YouTube video discussing a given company.

0:00 Waveguides and Slim Optics

0:03 Schott and Lumus

Schott AG is one of the world’s biggest makers of precision glass. In 2020, Schott entered into a strategic partnership with Lumus, and at AR/VR/MR 2024 and 2023, Lumus was prominently featured in the Schott booth. While Schott also makes the glass for diffractive waveguides, the diffraction gratings are usually left to another company. In the case of the Lumus Reflective waveguides, Schott makes the glass and has developed high-volume waveguide manufacturing processes.

Lumus waveguides consistently have significantly higher optical efficiency (for a given FOV), better color uniformity, better transparency, higher resolution, and less front projection (“eye glow”) than any diffractive waveguide. Originally, Lumus had 1-D pupil-expanding waveguides, whereas diffractive waveguides were 2-D pupil-expanding. The 1-D expanding waveguides required a large projection engine in the non-expanding direction, thus making the projection optics bigger and heavier. In 2021, Lumus first demonstrated their 2-D expanding Maximus prototype waveguides with excellent image quality, 2K by 2K resolution, and 50° FOV. With 2-D expansion, projection image optics could be much smaller. Lumus has continued to advance its Reflective 2D expanding waveguide technology with the “Z-Lens.” Lumus says that variants of this technology could support more than a 70-degree FOV.

Waveguides depend on “total internal reflection” (TIR). For this TIR to work, diffractive waveguides and earlier Lumus waveguides require an “air gap” between the waveguide surface and any other surfaces, including “push-pull” lenses, for moving the waveguide’s apparent focus distance and vision correction. These air gaps can be hard to maintain and source unwanted reflections. Lumus Z-Lens can be embedded in optics with no air gap (and the first waveguide to make this claim) due to the shallower angles of the TIR reflections.

While Lumus waveguides are better than any diffractive waveguide in almost every image quality and performance metric, their big questions have always revolved around volume manufacturing and cost. Schott thinks that the Lumus waveguides can be manufactured in high volume at a reasonable cost.

Over the last ten years, I have seen significant improvements in almost every aspect of diffractive waveguides from many companies (for example, my articles on DigiLens and Dispelix). Diffractive waveguides are easier, less expensive, and much easier to customize. Multiple companies have diffraction waveguide design tools, and there are multiple fabrication companies.

As I point out in the video, many MR applications don’t need the highest image quality or resolution; they need “good enough” for the application. Many MR applications only need simple graphics and small amounts of text. Many applications only require limited colors, such as red=bad, green=good, yellow=caution, and white or cyan for everything else. While others can get away with monochrome (say green-only). For example, many military displays, including night vision, are often monochrome (green or white), and most aviation HUDs are green-only.

I often say there is a difference between being “paid to use” and ” paying for” a headset. By this, I mean that someone is paid to use the headset to help them be more effective in their job, whereas a consumer would be paying for the headset.

For more on Lumus’s 2-D expanding waveguides:

• 2021 Exclusive: Lumus Maximus 2K x 2K Per Eye, >3000 Nits, 50° FOV with Through-the-Optics Pictures

For more on Schott and Lumus’s newer Z-Lens at AR/VR/MR 2023:

• DigiLens, Lumus, Vuzix, Oppo, & Avegant Optical AR (CES & AR/VR/MR 2023 Pt. 8)

For more on green-only (MicroLED headsets) and full-color MicroLEDs through diffractive and Lumus reflective waveguides, see:

• MicroLEDs with Waveguides (CES & AR/VR/MR 2023 Pt. 7)

4:58 Fourier (Metasurface)

Fourier is developing metasurface technology to reflect and redirect light from a projector in the temple area of AR glasses to the eye. If a simple mirror-type coating were placed on the lens, projected light from the temple would bounce off at an angle that would miss the eye.

Multiple companies have previously created holographic Optical Elements (HOEs) for a similar optical function. Luminit developed the HOE used with North Focals, and TruLife Optics has developed similar elements (both Luminit and TruLife’s HOEs are discussed in my AWE 2022 video with Brad Lynch).

Fourier’s metasurface (and HOEs) can act not only as a tilted flat mirror but also as a tilted curved mirror with “optical power” to change magnification and focus. At least in theory (I have not seen it, and Fourier is still in development), the single metasurface would be simpler, compact, and have better optical efficiency than birdbath optics (e.g., Xreal and many others) and lower cost and with much better optical efficiency than waveguides. But while the potential benefits are large, I have yet to see a HOE (or metasurface) with great image quality. Will there, for example, be color uniformity, stray light capture, and front projection (“eye glow”) issues as seen with diffractive waveguides?

Laser beam scanning with direct temple projection, such as North Focals (see below left), uses a Hologram embedded or on the surface of a lens to redirect the light. This has been a common configuration for the lower resolution, small FOV, and very small eyebox Laser Beam Scanning (LBS) glasses shown by many companies, including North, Intel, and Bosch. Alternatively, LCOS, DLP, MicroLED, and laser beam scanning projectors have used waveguides to redirect the light and increase the eyebox size (the eyebox is the range of movement of the eye relative to the glasses where the whole image can be seen).

Avegant (above right), Lumus, Vuzix, Digilens, Oppo, and many others have demonstrated that with waveguides with DLP, LCOS, and MicroLEDs in very small form factors as HOEs and Metasufaces (see DigiLens, Lumus, Vuzix, Oppo, & Avegant Optical AR (CES & AR/VR/MR 2023 Pt. 8). Still, waveguides are much lower in efficiency, so much so that the use of MicroOLED is impractical with waveguides. In contrast, using MicroOLED displays is possible with HOEs and Fourier’s metalenses. There are also potential differences in how prescription lenses could be supported.

As discussed above, holographic mirrors can also be used to form the equivalent of a curved mirror that is also tilted. The large CREAL3D prototype (below left) shows the two spherical semi-mirrors. CREAL3D planes to replace these physical mirrors with a flat HOE (below right).

Fourier metalens would perform the same optical function as the HOE. We will have to wait and see the image quality and whether there are significant drawbacks with either HOEs or metalenses. My expectation is that both metalenses and HOEs will have similar issues as diffraction gratings.

Some related articles and videos on small form factor optics and Videos.

• North’s Focals Laser Beam Scanning AR Glasses – “Color Intel Vaunt”
• Intel AR Glasses Follow-Up
• AWE 2021 Part 2: Laser Scanning – Oqmented, Dispelix, and ST Micro
• DigiLens, Lumus, Vuzix, Oppo, & Avegant Optical AR (CES & AR/VR/MR 2023 Pt. 8
• Intel AR “Fixer-Upper” For Sale? Only $350M ???
• 06:49 Innovative LCOS Optics (Digilens, Avegant, Lumus, Magic Leap)
• 14:40 Avegant (Small LCOS Engine)

6:23 Morphonics

Morphontonics has developed methods for making waveguides and similar diffractive structures on large sheets of glass. They can make many small diffractive waveguides at a time or fewer large optical devices. In addition to waveguides, Morphotonics makes a light guide structure for the Leia Lightfield monitor and tablet.

Morphotonics presentation at AR/VR/MR 2023 can be found here: Video of Morphotonics AR/VR/MR 2023 presentation.

10:33 Cellid (Wave Guides)

Cellid is a relatively new entrant in waveguide making. I have seen their devices for several years. As discussed in the video, Cellid has been continually improving its waveguides. However, at least at present, it still seems to be behind the leading diffractive waveguide companies in terms of color uniformity, FOV, and front projection (“eye glow).

11:47 LetinAR

Several companies are using LetinAR’s PinTilt optics in the AR glasses. At CES, JorJin was showing their J8L prototypes in the LetinAR booth. Nimo (as discussed in Mixed Reality at CES and the AR/VR/MR 2024 Video (Part 1 – Headset Companies) was showing their LentinAR-based glasses in their own booth. Sharp featured their LentinAR glasses in their booth but didn’t mention they were based on LetinAR optics.

LetinAR’s optics were also used in an AT&T football helmet display application for the deaf (upper left below).

LetinAR originally developed “pin mirror” optics, which I first covered in 2018 (see CES 2018 in the listings below). The pin-mirror technology has evolved into their current “PinTilt” technology.

While LetinAR has several variations of the PinTilt, the “B-Type” (right) is the one I see being used. They use an OLED microdisplay as the display device. The image light from the OLED makes a TIR (total internal reflection) bounce off the outside surface into a collimating/focusing mirror and then back up through a series of pupil-replicating slats. The pupil replication slats enable the eye to move around and support a larger FOV.

As I discussed in the video, the image quality is much better than with the Pin-Mirrors, but gaps can be seen if your eye is not perfectly placed relative to the slats. Additionally, with the display off, the view can be slightly distorted, which can likely be improved in the manufacturing process. LetinAR also let me know that they are working on other improvements.

LetinAR’s PinTilt is much more optically efficient than diffractive or even Lumus-type reflective waveguides, as evidenced by its use of micro-OLEDs rather than much brighter LCOS, DLP, or micro-LEDs. At the same time, they offer a form factor that is close to waveguides.

Some other articles and videos covering LetinAR:

• CES 2023 Video about LetinAR pin mirrors with Brad Lynch (with picture from CES 2020)
• CES 2019 – AR and other Interesting Display Technology
• CES 2018 (Part 1 – AR Overview)

13:57 Tooz

Tooz was originally spun out of Zeiss Group in 2018, but in March 2023, they returned to become part of Zeiss. Zeiss is an optical giant founded in 1846 but is probably most famous to Americans as the company making the inserts for the Apple Vision Pro.

Tooz’s “Curved Waveguide” works differently than diffractive and Lumus-type reflective waveguides, which require the image to be collimated, use many more TIR light bounces, and have pupil replication. Strictly speaking, none of these are”waveguides,” but the diffractive and Lumus-type devices are what most people in the industry call waveguides.

The Tooz device molds optics and a focusing mirror to move the focus of the display device, which currently can be either a Micro-OLED or, more recently, (green only) Micro-LED. The image light then makes a few TIR bounces before hitting a Fresnel semi-mirror, which directs the light toward the user’s eye (above right). The location of the Fresnel semi-mirror, and thus the image, is not centered in the user’s field of view but slightly off to one side. It is made for a monocular (single-eye) display. The FOV is relatively small with 11- and 15-degree designs.

Tooz’s Curved Waveguide is aimed at data snacking. It has a small FOV and a Monocular display off the side. The company emphasizes the integration of prescription optics and the small and lightweight design, which is optically much more efficient than other waveguides.

Tooz jointly announced just before the AR/VR/MR conference that they were working with North Ocean Photonics to develop push-pull optics to go with waveguides. Tooz, in their AR/VR/MR 2024 presentation, discussed how they were trying to be the prescription optics provider for both their curved waveguides and what they call planar waveguides. One of their slides demonstrated the thickness issue with putting a push/pull set of lenses around a flat waveguide. The lenses need to be thicker to “inscribe” the waveguide due to their curvature (below right).

19:08 Oorym

Oorym is a small startup founded by Yaakov Amitai, a founder and former CTO of Lumus. Oorym has a “waveguide” with many more TIR bounces than Tooz’s design but many less than diffractive and Lumus waveguides. They use a Fresnel light redirecting element. It does not require collimated light and is much more efficient than other waveguides. They can support more than a 50-degree FOV. It is thicker and more diffractive, and Lumus waveguides are in the same order as the thickness of LetinAR. Oorym is also developing a non-head-mounted Heads-Up Display (HUD) device.

Oorym

21:57 Gixel

Gixel’s technology has to be among the most “different” I have seen in a long time. The concept is to have a MicroLED “bar” display with only a single or a few rows of pixels in one direction and with the full horizontal resolution in the other. The “rows” may have full-color pixels or a series of 3 single-color row arrays. Then, a series of pupil-replicating slats rotate to scan the bar/row image vertically synchronously with a time-sequential change of the row display. In this way, the slats scan row display forms a whole image to the eye (and combines the colors if there are separate displays for each color).

They didn’t have a full working prototype, but they did have the rotating slats working.

My first impression is that it has a Steampunk feel to the design. I can see a lot of issues with the rotating slats, their speed and vibration, the time-sequential display, and a mirage of other potential issues. But still, it wins for the sheer audacity of the approach.

23:42 Meta Research (Time Sequential Fixed Foveated Display) & Varjo

Meta Research presented the concept of a time-sequence fixed-foveated display using single pancake optics. The basic idea is that pancake optics work by making two passes through some of the refractive and mirror optics, which magnifies the display. In a normal pancake, quarter waveplates change the light’s polarization and affect the two passes. A (pixel-less) liquid crystal shutter can act as a switchable quarter waveplate. This way, the display light will make one or two passes through part of the optics to cause two different magnifications. By time sequencing the display with the LC shutter’s switching, both a lower angular resolution but larger image and a higher angular resolution but smaller “foveated” display will be seen by the eye time sequentially.

This basically happens with a single set of optics and a single display, which is what Varjo was doing with their “fixed foveated display,” which used two display devices, optics, and a combining beam splitter.

I like to warn people that when a research group from a big company presents a concept like this to all their competitors at a conference like AR/VR/MR, it is definitely NOT what they are doing in a product.

Fixed (and Eye Tracking) Foveated Displays

In 2017, Varjo was focused on its foveated display technology. Their first prototype had a “fixed foveated display,” meaning the central high-resolution region didn’t move. Varjo claimed they would soon have the foveated display tracking the eye, but as far as I know, they never solved the problem.

It turns out that tracking the eye and moving the display is a seemingly impossible problem to solve with the eye’s saccadic movement, even with exceptional eye tracking. As I like to say, “While eye tracking may know where the eye is pointing, you don’t know what the eye has seen.” Originally, researchers thought that human vision fully blanks with saccadic movement, but later research suggests that it only semi-blanks out vision with movement. Combined with the fact that what a human “sees” is basically a composite of multiple eye positions, making a foveated display that tracks the eye is exceedingly difficult, if not impossible. The problem with artifacts due to eye movement, such as field sequential color breakup, they will tend to appear as flashes that are distracting.

We are seven years since Varjo told me they were close to solving the eye-tracking foveated display. Varjo figured out that about 90% of the benefit of a moving foveated display could be realized with a fixed foveated display near the center of the FOV. They may also have realized that solving the problems with a moving foveated display was more difficult than they thought. Regardless, Varjo has pivoted from being a “foveated display company” to a “high-resolution VR/MR company” aimed primarily at enterprise applications. Pixel sizes and resolution of display devices improved to the point where it is now better to use a higher resolution display than to combine two displays optically.

Eyeway Vision Foveated Display (and Meta)

In 2021, I visited Eyeway Vision, which also worked on foveated displays using dual laser scanning displays per eye. After an acquisition by Meta fell through, Eyeway Vision went bankrupt. Eyeway Vision had a fixed foveated display and sophisticated eye tracking, but it went bankrupt before solving the moving foveated display.

Eyeway Vision’s founder, Boris Greenburg, has recently joined VoxelSensors, and VoxelSensors is looking at using their technology for eye/gaze tracking and SLAM (see Zinn Labs later)

Foveated Display (ex., Varjo) vs. Foveated Rending (ex., Apple Vision Pro)

I want to be clear between foveated rendering, where the display is fixed, and just the level of detail in the rendering changes based on eye tracking, from a foveated display, where a high-resolution sub-display is inset within a lower resolution display. Foveated rendering such as the Apple Vision Pro or Meta Quest Pro is possible, although today’s implementations have problems. However, it may be impossible to have a successful eye-tracking foveated display.

For more on this blog’s coverage of Foveated Displays, see:

• 2017 Varjo Foveated Display (Part 1)
• 2017 Varjo Foveated Display Part 2 – Region Sizes
• 2018 Varjo Foveated Display and Passthrough AR at AWE 2018
• 2021 Exclusive: EyeWay Vision Part 1: Foveated Laser Scanning Display
• 2021 Exclusive: EyeWay Vision Part 2: Through the Optics Pictures
• 2021 EyeWay Vision Part 3: Analysis

32:05 Magic Leap (Mostly Human Factors)

At AR/VR/MR 2024, Magic Leap gave a presentation that mostly discussed human factors. They discussed some issues they encountered when developing the Magic Leap One, including fitting a headset to a range of human faces (below right). I thought the presentation should have been titled “Why the Apple Vision Pro is having so many problems with fitting.”

In 2016, This Blog Caught Magic Leap’s Misleading Video

In showing Magic Leap’s history, they showed a prototype headset that used birdbath optics (above left). Back in 2016, Magic Leap released a video that stated, “Shot directly through Magic Leap technology . . . without the use of special effects or compositing.” I noted at the time that this left a lot of legal wiggle room and that it might not be the same “technology” they would use in the final product, and this turned out to be the case. I surmised that the video used OLED technology. It’s also clear from the video that it was not shot through a waveguide. It appears likely that the video was shot using an OLED through birdbath optics, not with the Waveguide Optics and LCOS display that the Magic Leap One eventually used.

In 2019, Magic Leap sued (and lost to) Nreal (now Xreal), which developed an AR headset using birdbath optics and an OLED display. Below are links to the 2016 article analyzing the Magic Leap deceptive video and my 2020 follow-up article:

• 2016 Magic Leap – The Display Technology Used in their Videos
• 2020 Magic Leap Starts to Auger-In with ~3.7X the Money as Theranos – I Tried to Warn Everyone

36:45 NewSight Reality (Not Really “Transparent” MicroLED)

Sorry for being so blunt, but NewSight Reality’s “transparent” MicroLED concept does not and will not ever work. The basic concept is to put optics over small arrays of LEDs, and similar to pupil replication, the person will see an image. It is the same “physics” as MojoVision’s contact display (which I consider a scam). In fact, NewSight’s prototype has nine MojoVision displays on a substrate (below center)

The fundamental problem is that to get a display of any resolution, plus the optics, the “little dots” are so big that they, combined with diffraction, cause a blurry set of gray dots in a person’s vision. Additionally, the pupil replication effect ends up with a series of circles where you can see the image.

38:55 Other Optics and Eye Tracking

The next section is on other optics and eye tracking. Thanks to Tobii being involved in both, they sort of tie this section together.

39:01 AddOptics

AddOptics developed a 3-D-printed optical mold process. It was founded by former Luxexcel employees (Luxexcel was subsequently acquired by Meta in 2022).

I covered AddOptics last year in CES 2023 (Part 3)—AddOptics Custom Optics. The big addition in 2024 was that they showed their ability to make push-pull optics for sandwiching a waveguide. They showed they could support waveguides that required an air gap or not. As far as I am aware, most, if not all, diffractive waveguides require an air gap. The only waveguide I know of that claims they don’t need an air gap is the newer Lumus reflective-based waveguide (discussed in a previous article). Still, I have not heard of whether AddOptics is working with Lumus or one of Lumus’s customers.

Luxexcel had developed a process to directly 3-D print optics without the need for any resurfacing. This means they need to print very fine layers very precisely, lens by lens. While it means each lens it custom can be custom fit, it also seems to be an expensive process compared to the way prescription lenses are made today. By making “low run” 3-D printed molds (something that Luxexcel could also do), AddOptics would have a lower cost per unit and a faster approach. It would require having a stock of molds, but it would not require a prohibitive number of molds to support most combinations of diopter and cylinder (astigmatism) correction.

42:12 Tobii

Tobii, founded in 2001, has long been known for its eye-tracking technology. Tobii was looking to embed LED illuminators in lenses and was working with Interglass. When Interglass (founded in 2004) went bankrupt in 2020, Tobii hired the key technical team members from Integlass. Meta Materials (not to be confused with Meta, formerly Facebook) acquired the assets of Interglass and is also making a similar technology.

The Interglass/Tobii/Meta-Materials process uses many glass molds to support variations of diopter and cylinder adjustments for prescriptions. The glass molds are injected with UV-cured plastic resin, which, after curing, forms lens blanks/rounds. When molding, the molds can be rotated to set the cylinder angle. The round lens blanks can then be cut by conventional lens fitting equipment.

At 2023’s AR/VR/MR, Tobii demonstrated (left two pictures below) how their lenses were non-birefringent, which is important when working with polarized light-based optics (e.g., Pancake Optics, which Tobii says they can make) and displays (LCDs and LCOS). Tobii has videos on its website that show the lens-making and electronic integrating process (below right).

43:44 Zinn (and VoxelSensors)

Zinn Labs uses a Prophesee event-based camera sensor (Zinn and Prophesee announcement). The Prophesee event camera sensor was jointly developed with Sony. Zinn uses Prophesee’s 320×320 6.3μm pixel BSI (BackSide Illuminated) event-based sensor in a 1/5” optical format.

Event camera pixels work like the human eye in detecting changes rather than the absolute value of each pixel. The pixels are much more complex than a conventional camera sensor, with photodiodes and comparators integrated into each pixel using Sony’s BSI process. Rather than scanning out the pixel value at a frame rate, each pixel reports when it changes significantly (more details can be found in the Prophesee white paper – free, but you have to give an email address). The advantage of the event camera in image recognition is that it tends to filter out/ignore everything that is not changing.

Zinn Labs has developed algorithms that then take the output from the event camera and turn it into where the eye is gazing (for more information, see here).

VoxelSensors (and Zinn Labs)

VoxelSensors has a very different type of event sensor called a “SPAES (Single Photon Active Event Sensor)” that could be used for eye/gaze tracking. Quoting from VoxelSensors:

VoxelSensors leverages its distinctive SPAES (Single Photon Active Event Sensor) technology, allowing the integration of multimodal perception sensors, such as innovative hand and gaze tracking and SLAM, with high precision, low power consumption, and low latency. Fusing these key modalities will enable the development of next-gen XR systems.

As discussed earlier, VoxelSensors also recently hired Eyeway Vision found Boris Greenberg, who has extensive experience in eye/gaze tracking.

VoxelSensors’s SPAES uses a laser scanner to scan the area of interest in a narrow-band infrared laser (where the Prophesee event camera would use IR LED flood illumination) and then detect the laser scanner’s return to the area of interest. With narrow-band filtering to filter out all but the laser’s wavelength, the SPAES is designed to be extremely sensitive (they claim as little as a single photon) to the laser’s return. Like the Prophesee event camera, the VoxelSensors’s SPAES returns the pixel location when an event occurs.

While the VoxelSensor’s pixel is more complex than a traditional sensor, it seems simpler than Prophesee’s event camera pixel, but then VoxelSensor requires scanning lasers versus LED. Both are using event sensors to reduce the computational load. I have no idea at this point which will be better at eye tracking.

VoxelSensors with one or more sets of laser scanners and sensors can detect in three dimensions, which is obviously useful for SLAM but might also have advantages for eye tracking.

For more on Voxel Sensors my 2023 CES article: CES 2023 (4) – VoxelSensors 3D Perception, Fast and Accurate.

44:13 Lumotive (LCOS-Based Laser Scanning for LiDAR)

Lumotive has a technology that uses LCOS devices to scan a laser beam. Today, LiDAR systems use a motor-driven rotating prism or a MEMs mirror to scan a laser beam, resulting in a fixed scanning process. The Lumotive method will let them dynamically adjust and change the scanning pattern.

46:03 GreenLight Optics

I’ve known Green Light Optics since its founding in 2009 and have worked with them to help me with several optical designs over the years. Greenlight can design and manufacture optics and is located in Cincinnati, Ohio. I ran into GreenLight at the Photonics West exhibit following the AR/VR/MR conference. I thought it would be helpful for other companies that might need optical design and manufacturing to mention them.

Quoting GreenLights website:

“Greenlight Optics is an optical systems engineering and manufacturing company specializing in projection displays, LED and laser illumination, imaging systems, plastic optics, and the integration of optics with electrical and mechanical systems.”

Next Time – Display Devices and Test and Measurement Companies

In the next part of this series will on CES and AR/VR/MR 2023, I plan to cover display devices and a few test and measurement companies.