304 North Cardinal St.
Dorchester Center, MA 02124
304 North Cardinal St.
Dorchester Center, MA 02124
It was nice to get out and actually look through AR and VR displays at AWE 2021. I started to write about everything I saw at AWE, but to me Tilt-5 was such a standout that I decided to start with an article about my Tilt -5 experience and later follow up with several articles on other things I saw.
I enjoyed Tilt-5 the most of everything I saw at AWE. It was surprising because I had known of Tilt-5 and its predecessor castAR for many years but had not seen it. Unfortunately, I had dismissed its use of a retro-reflective screen as a desperate act and hokey, and with castAR failing in 2017, I figured that was it. While I admired CEO Jeri Ellsworth’s tenacity, I wondered she was a bit of a Don Quixote with a loyal following.
But then I finally got to see the Tilt-5, and to me, it was instantly magical, and I personally ordered a unit the same day. I have seen a lot of AR and VR technology, and I don’t get this feeling too often. Tilt-5 is not the be-all and end-all of AR, nor are they trying to be, but it seems to be good at what Tilt-5 is trying to accomplish, a game system. I’m left in this article to start to explain why it felt so magical.
In walking, the exhibit floor of AWE was tucked near the back, and I rounded a corner to find a crowd of people around a small booth. I saw the game board and realized that this must be Tilt-5 before I saw the sign (below left is the first view I saw). Before I got to try it on, people were giggling and going, “oh wow,” and then I put the headset on, and I was joining them (I would suggest watching Norman Chan’s Tested Hands-On with Tilt-5 for a similar, but much longer, reaction).
Tilt-5 turns the AR concepTilt-5 turns the AR concept a bit inside-out using a retro-reflective (glass beaded) screen. What Tilt-5’s CEO Jeri Ellsworth calls “AR Somewhere.” I’m told all the time you have to “see it to believe it,” and 99 out of 100 times, the experience is underwhelming. But with Tilt-5, you really do need to see it. Consider the following:
What makes it “magical” is that everything seems to work the way it should in a way I have not seen in any other AR device. Below is a through one lens picture (a set of short through the (one) lens video clips is here). Of course, the single picture below cannot convey the feeling of binocular stereo depth. The retro-reflective game board appears to be a portal into another world.
Below is a series of short video clips shot through the lens taken by Tilt-5. The moving camera helps convey somewhat the 3-D effect (better seen with both eyes):
Tilt-5 was demonstrating at AWE that the virtual work could be extended beyond the Tilt-5 demonstrated at AWE that the virtual work could be extended beyond the gameboard provided some tracking markers were in view (see right). Retro-reflective fabric is relatively inexpensive (about $10/square yard on Amazon), making for expanding the Tilt-5’s virtual world.
This blog primarily concentrates on displays and optics, not the other aspect of AR/VR/MR products. But for completeness, I would like to mention a few other features and technologies in the Tilt-5.
The Tilt-5 comes with a control stick wand with 6DOF tracking based on IR tracking and inertial sensors, and it also has buttons that connect to the glasses via Bluetooth.
The headset includes dual speakers, and a microphone, and inertial measurement. The most interesting processing hardware is an Intel Movidius vision processor for real-time 3-D image warping locally in the headset. This headset supports quick 3-D motion feedback with head movements without waiting for the host to redraw the whole image.
The glasses have two 8 megapixel IR cameras working at different IR wavelengths. One is for tracking the gameboard and wand, and the second camera tracks what Tilt-5 calls “Tangible Objects.” Tangible objects could include game pieces and cards, some of which may have bar code identifiers on them.
It should be understood that Tilt-5 is a startup aiming at being a game system with a consumer price point, and the device has also had a long gestation period. I want to remind people that the Atari 2600 game system was introduced in 1977 for $189.96 or ~$877 in 2021 dollars.
The resolution is a modest 1280 by 720p resolution using a Citizen FineDevice field sequential color FLCOS with a custom projector with unique optical characteristics by Tilt-5. The effects of the field sequential color are evident but not that bad and sometimes show up worse in videos than they appear in real life. There is a lot of opportunities to keep improving the system.
There is some scintillation/grain from a beaded screen, and Ellsworth discussed how it could be improved with better (but perhaps more expensive) screen technology. Tilt-5 is not a technology designed for photographs or replacing a computer monitor, and the technology works best as a table-top display.
To some, the headset looks geeky and toy-like, but they want it to look like a game/toy system. I could see people putting “skins” on the headset or perhaps using retro-reflective tape for a dynamic skin.
If there is one thing I don’t like, it is wired to the host phone, tablet, or PC. I would like to see this wirelessly connected with a rechargeable headset. Cords are a snag hazard both for people and for damaging equipment and the cords themselves. Ellsworth says that the data rate to the headset is not excessive due to the processing in the headset, which would be possible in the future.
I am also not a fan of supporting the headset’s weight (about 95 grams) with just the nose and temples, and I would rather it had an adjustable headband. They could then put some weight, including a battery for wireless operation, in a pack on the back for better weight distribution (similar to the Hololens 2 – right).
I want to briefly explain some of the physics behind Tilt-5 as I understand it below.
The optics of the Tilt-5 is elegantly simple. The optics of the Tilt-5 is elegantly simple. With a combination of polarizers and quart waveplates, light for both eyes is selectively steered out to the retro-reflective-screen, and the only light to the specific eye is let through when it returns.
The LCOS projector outputs linearly polarized light that is rotated/retarded by a first-quarter waveplate on the projector’s output to a second waveplate that further rotates/retards the light so an angled linear polarizer will reflect the light. After reflecting off the polarizer, the light will be retarded again by waveplate 2 on its way to the retro-screen. The retro screen will not affect the polarization and will reflect directly back toward the glasses. The light is then retarded again by waveplate 2, which will cause the light to pass through the linear polarizer. The reason for the multiple passes through the waveplate is to set the light up to reflect the first time and then pass after reflecting off the screen.
The Tilt-5 will use different waveplates for the left and right eye so that the light is polarized in the opposite direction when it hits the screen. Each polarizer will only pass light for the given eye, thus supporting stereo vision.
The real world only has to make one pass through a single polarizer. A typical linear polarizer will block about 60% of the incoming unpolarized light and thus it will be about 40% transparent. This is better than most birdbath designs (such as Nreal and others) that block about 75% of the real-world light. Because of the waveplate arrangement, only about 25% of the projector’s light will be lost in the reflective and transmissive pass of the polarizer.
Something very different from almost every other AR headset is no “combiner” per se. Both the virtual light and the real-world light follow the same path to the eye. With a combiner, usually, the virtual path is reflected, and the real-world is passed/combined.
The last optical element must be as big as the FOV is at the distance it is from the eye. The size of the image projected on the screen sets the size of the FOV, and it can be as big as 110-degrees from the projector.
On a normal white screen, light becomes highly and spread out over a 180-degree hemisphere, and only the tiniest percent of light will make it back to any one person. A retro-reflective screen sends most of the incoming light back in the direction of the light source over a very narrow-angle. A typical retro-screen is covered in tiny round glass balls glued to a gray background.
With the round glass beads (below left), light is refracted on entering and will hit the back at an angle such that about 20-30% of the total light will be reflected off the back from the bead. The light reflected off the back will be refracted by the side opposite to which it entered, and parallel to the direction it entered the bead. Since the projected image is coming from the direction of the eye, a high percentage of the light will go toward the eye. The microphotograph below is a picture glass bead from a retro-reflective fabric (similar in principle to what Tilt-5 is using). Importantly, the refraction and reflection process of beads will (largely) preserve the polarization of the light.
The three photos below show the effect of lighting the retro-reflective fabric from above, behind and from the direction of the camera.
I don’t know for sure, but I think there are two things at play with the Tilt-5 that help with VAC. The first is that the projector acts like a very high f-number light source, sort of like a pin-hole, and in turn, it may be easier for the eye to focus.
From the point of view of focusing, the light will appear to come from the screen. In Tilt-5’s normal table-top use, the screen is physically horizontal, and the user is looking down at it at an angle. Things appearing to be above and below the table/screen surface will still focus at the screen’s distance and not at their virtual position. But the difference between how far above or below they appear will be relative to the eye to screen distance will typically not be enough to cause a VAC problem.
Because of the retro-screen, the light projected from the Tilt-5 is comparatively dim and polarized. When encountering typical real-world surfaces like hands or other non-reflective objects, the projected light will be scattered and largely depolarized. Thus almost non of the projected light will be seen via the polarized lenses on the Tilt-5.
Shadows, or the lack thereof, is another curiosity of the Tilt-5. Normally with projectors, if you put your hand in the way, it will cast a shadow. But, because the projected light is coming from the eye’s direction, the same hand or another object blocking light that would cast a shadow keeps the user from seeing the shadow.
Because of the retro-screen, each player only will see the light from their headset. Thus multiple players only see their virtual content individually while and real-world content such as playing cards and playing pieces are seen by everyone.
Ellsworth talks a lot about working with many games developers and claims many writing games that take advantage of the Tilt-5. She has a background in working on consumer game products and knows the critical importance of having many people developing for a platform. The games end will sell the system. As far as I can see, the Tilt-5 is the only AR platform well-aimed at tabletop games, particularly with most other companies retreating to “enterprise” markets.
Tilt-5 has a long history and is the story of tenacity and belief of Tilt-5’s CEO, Jeri Ellsworth. If you don’t know her story, I recommend listing the AR Show Podcast interview or searching for the many articles and videos about her journey. Ellsworth invented the retro-reflective AR concept at Valve and then took it out in 2013 as Cast-AR. When Cast-AR then folded, Ellsworth bought back the IP and founded Tilt-5. I can only admire how she kept going, and it is inspiring. Knowing the story and seeing the results, I can’t help rooting for Tilt-5.
I don’t know why it has been such a long struggle for Tilt-5 to raise money. But on the surface, the Valley VC and corporate funding entities, with all their talk of wanting to back women-led companies, should be embarrassed that they showered the likes of Magic Leap billions of dollars. At the same time, Tilt-5 had to go begging on Kickstarter.
On this blog, I “grade” products by how they fulfill their intended purpose. Tilt-5 is not trying to be all things to all people but rather a fun interactive game experience. It is doing a great overall job on that scoring, and the experience is magical and affordable for the intended audience.
Maybe I am jaded by many years of overhyped products, and I came in with low expectations for the Tilt-5. But I came away feeling like a kid with a great new toy which is how this type of product should make the user feel.
I’m also a bit frustrated that it seems that too much money has been poor on some overhype products while others get the crumbs. While I love Tilt-5 for what it is, I think it could be even better with more investment.