Introduction – “Kickstarter Quality Fakes” and a Bad Concept

Last week Amazon announced several (not so) new concepts. My attention was grabbed by all the fake images associated with the Amazon Glow. The Glow incorporates a small “down projector” (projects down on a surface). I then got curious as to what projector technology Amazon might be using (and will give my best guess). There are so many ways in which the Glow down projector is a flawed concept, and it only has gotten worse with age.

All of Amazon’s still images scream “fake,” such as the one below. They are what I like to call “Kickstarter Quality Fakes.” Projectors can only add light, so how can anything projected be darker than the white screen? Some of the video sequences look like they just used “cleaver scene lighting” to make the projection look better, but several others show signs of being faked. Also, it is fun to play CSI (Crime Scene Investigation) to figure out how the different ways they faked the images and videos.

While the fake content grabbed my attention, there are serious problems with the overall concept. I’m also going to show some pictures I took of similar concepts through the years using Laser Beam Scanning (Microvision), LCOS (Syndiant), and DLP (Texas Instruments) display technologies.

Pico Projector Connections to AR

Augmented reality (AR) headsets share some of the technology and issues with the Glow. Small microdisplay devices such as DLP, LCOS, and Laser Beam Scanning (LBS) that were originally developed for pico projectors to go into cell phones in the period from about 2006 to 2015 have found their way into many of today’s AR headsets.

Pico projectors in cell phones failed so spectacularly that few today know that almost every major phone brand had a serious R&D effort with embedded pico-projectors. Only a few of these efforts, most notably Samsung, made it to the market with their Galaxy Beam (2010) and Beam 2 (2012).

AR has a similar technical issues as front projectors with faded objects because the virtual image competes with ambient light. Thus AR projectors must be much brighter than VR displays. VR headsets are also leveraging flat-panel technology that was developed for mass-market cell phones.

I There are many lessons from my pic-projector in cell phones experience that color my thoughts on AR. These include:

• Just because the R&D people at many major companies think a concept makes sense does not mean it will be a mass market product.
• Fighting ambient light can be a losing proposition.
• You can’t compete on price with small flat panel displays.
• Consider the “use model” and whether it is realistic.
• How will the input work, and how well?
• Portability is defined as whether it fits in your pocket when not in use.

Why They Faked the Pictures and Videos

I wrote about why the cell phone projector concept failed in 2013 in “Whatever happened to pico projectors embedding in phones?” The main reasons are the impracticality of overcoming ambient light and the widespread availability of low-cost touch screen LCDs (aka, Tablets). Today, even fairly bright conference room projectors have largely gone away in favor of large screen TVs due to people not wanting to sit in the dark to see the screen clearly. The exceptions being movie theaters and large conference/meeting rooms needing screens over 100-inches where people are willing to sit in the dark or dimly lit rooms.

As I like to say, “a flat panel display brings its black with it.” Flat panels, when off, are black and absorb ambient light. A screen for a projector reflects ambient light as well as reflects the light from the projector. To have well-saturated colors, one wants the projector to have about 10x the brightness of the ambient light.

Even if the projector could be bright enough, watching a projected image 10x brighter than a well-lit room would be uncomfortable. So to make it look like the concept makes sense, the producers appear to feel it necessary to fake the pictures and videos even in rooms that were not well lit.

They may give some excuse that the camera works differently from the eye, which is true. But in real life, the images are going to be very washed out in normal room lighting

Bad User Interface – No Right Way to Do the Wrong Thing

The next big problem is that a down shooting projector touch screen has a terrible user interface compared to a touch screen tablet. The big problems:

• The user’s hand and arms block the image so they can’t see things, including other things to press. As the user’s hand/finger approaches the screen, they start blocking what they want to touch.
• No tactile feedback and will get false actuation unless one hunt and pecks. Not at all kid-friendly.
• Limited or no multi-touch. You can’t touch anything in a shadow.
• The resolution/accuracy is likely poor which limits the playability.
• There is no good place to rest hands and arms. With a phone or a tablet, one can rest their hands and arms on the side. A child has to hover their hands (a torcher), rest them in their lap, or rest them spread-eagle them on either side of the map.
• The video suggests that they may try supporting gestures like turning a page. I’m highly doubtful that this will turn out well if they have it, given all the other issues.

And the above assumes that the optical touch detection works well, and it can detect when a user has touched or not.

Missing Specs – Some Estimates

My first My first law of missing specs: “Missing specs that should be there are almost always not good.” When I look for the spec of the projector (copied below), all I see is the size of the projection and that it is “touch-sensitive” (really optical touch).

There is no resolution for the projector (but they give one for the LCD), so I assume it is a low-resolution pico-projector given the size and price. Also, there is no brightness as normally measured in lumens. As I will discuss later, it looks to me that the resolution is about 640×360 pixels, also known as nHD (one-ninth-HD).

From the video sequences that are not heavily faked and knowing the size of the projection mat, it looks to me like the brightness is about 50 lumens. The projected area on the mat is about 0.1 square meters. For an ideal matte-white screen (a rough assumption for the plastic mat), the formula for Nits (CD/m²) is Lumens/(PI x area). Assuming the mat is 90% ideal, it works out that the projected image is about 140 nits. I originally worked backward from the apparent brightness of the image to get the lumens of the projector.

A 140-nit image may seem almost as bright as a typical computer monitor, but a monitor absorbs ambient light where a screen reflects it. A typical well-lit room will have about 300 LUX (lumens per square meter) or 30 lumens illuminating the 0.1m² mat. Thus, with good room lighting, the contrast will be about 50/30 or ~1.66:1, which is pretty poor. A moderately lit room will have between 100 and 200 LUX, which will push the contrast to between 5:1 and 2.5:1, which seems to be what is shown in some of Amazon’s videos.

Best Guess of the Projector Type

Three main candidate projector types are DLP, LCOS, and Laser Beam Scanning (LBS). The videos give some clues about the projector technology, and I will discuss each of these more later.

It is definitely not LBS. LBS always has telltale scanning artifacts when captured by videos. Also, there is not of the speckle one would see with front projected laser scanning.

It could be LCOS, but I would expect to see a higher resolution. I would also expect to see some color flickering captured by the camera with a low-cost LCOS to meet the system price point.

By process of elimination and given the resolution, brightness, size, and cost, my best guess is it is a TI-DLP nHD projector. The characteristics line up well with the DLP nHD EVM design pictured above right. The nHD EVM even has the 100% offset projection necessary to support a rectilinear image projected way from the pedestal. For more on “offset projection,” see the slide below from a TI video (with my edits in red).

I’m not saying TI’s nHD EVM is that exact optical design, but something like it would seem to fit well.

Amazon Glow Obvious Fakes

AlAll of the still pictures on the Amazon website (current link as I write this article) are blatantly fake. It is most obvious when comparing them to similarly staged scenes in Amazon’s corresponding video (link to CNET’s capture on YouTube).

Take the example below of the boy pointing below. He is in a well-lit room in the still photo, and the fake “projected” content is all darker than the screen/mat. They even bothered to photo-edit in fake light rays. They show a projection onto the boy’s arms and hands that, among other problems, does not line up correctly, and then they put a fake shadow under the boy’s arms while there may be a real shadow under his right pointer finger.

It looks like they posed the boy with the projector projecting in the darkened room and took one picture. Then they turned the lights on, put a printout of the image on the table, and took a second picture with the boy in a similar pose. They then photo-edited the two images together.

A big tell that they likely put a printout on the table is a trapezoidal-shaped shadow on the mat of the upright LCD unit from a light behind the unit. On the crop on the right, I have enhanced the shadow to make it more obvious. If they had photo-edited the original image without printing it, which was my first thought, it is unlikely that they would have inserted the shadow of the LCD unit.

Looking back at the picture of the girl shown in the introduction and enhancing the shadows (below), one can be seen that the lighting is more from behind the girl, and she is casting shadows toward the projector. If this were a real projected image, there would be shadows in the blocked image on the other side of her hand.

Less Obvious Fakes and Post Processing

Even a beginner should be able to tell that the images above were fake. But they went to Even a beginner should be able to tell that the images above were fake. But they made some effort to make it look real in some other video sequences look better with significant post-production.

There are two similar video clips that are being faked in different ways. The first sequence loops repeated on Amazon’s Glow page but you can also currently download the MP4 file (current link to video here) of a boy looking at a Toy Story page (hereafter “Toy Story video”). There is the same child in a similar setup in the main Amazon video on YouTube (link to the start of the sequence here) looking at a page about monsters (hereafter “monster video”).

I noticed that the projected image in the Toy Story video looks much better than any other video sequence Amazon has put out. The image is clearly “live,” as you can see the reflection on the LCD. But I also noticed that the child in the video had been edited during post-production. The first clue what the glowing nature of his hand and the lack of shadow when he touches the page and Buzz flies away. But the most obvious editing occurs later as they did a sloppy job of compositing his fingers as he turned the page. Look at the sequence from the five frames captured below and how the fingers keep changing shape.

For further proof that the child was not there when the Toy Story video was filmed, there is no reflection of the child’s hand on the LCD (below left – you will have to click on the image to see the full-size image). Also, if you click on the image below, you will be able to see that the text is darker and without the red tint of the border (look at the “B” in “Buzz”). Since projectors can only add light, it is a clear sign that something is being faked.

In both the images above, several lines of a few letters of text were copied repeatedly to get a rough idea of the number of text lines. The text looks like it would require about 14 to 16 lines/pixels per character height, including spacing (based on experience). The net result is an estimate of the vertical resolution being about 320 to 360 pixels tall. nHD is the most common format with that number of lines, which points to it being an nHD display.

The Monster Video has been faked more simply. It looks like they did some simple masking of the projected area and then improved the contrast and color in post-production. Once again, a tell in the image is that the text is darker than the border. You can also compare it (below right) to a later frame in the same video where they have not played with the contrast and color in post-production (below left)

On the right is a comparison of the Toy Story page’s text to the Monster page. The Toy Story page is much cleaner/better.

I have not figured how they created the Toy Story projected image, but they clearly put some extra effort into it as it looks better than the Monster image. I guess that they shot it in different lighting and then edited it in along with the child.

Down-Projecting is an Old Concept that Has Not Aged Well

Back in 2007, when I was CTO at Syndiant and started working on pico-projectors for cell phones, one of my first ideas was to down-project onto a surface (right-center). In 2010, Light Blue Optics (LBO) showed a down-shooting projector using laser with holograms on phase-type LCOS (far right). The Light Blue’s projector supported a touch interface. Unfortunately, while extremely innovative and an amazing technical achievement with (real) holograms, LBO was out of business about two years later.

CES 2019 and 2020 Down Projectors

Occasionally since 2010, we have seen a resurgence of down-projectors. Most famously, there were multiple wrist projector scams which I wrote about in 2016 and again in 2018.

Sony was at CES 2017 with an Android touch screen down-projector that could also be used as a short-throw wall projector (here is a video showing it working). It was launched as a product for $1,587 in 2017 and has not been heard from since.

Microvision has probably been the longest-lasting promoter of the down projector touch screen concept, starting with development kits in 2017 and following up with full-blown design concepts at CES 2019 and 2020. Microvision used their LBS projector technology, but others have also shown the DLP and LCOS concept at CES 2019 and 2020.  I wrote about the Microvision,  Bosch (using DLP), and ASU (Syndiant’s LCOS) projectors at CES 2019.

Microvision was back at CES 2020 with a more refined design. They also had a “virtual private housing” (VPH) design that would detect if something was blocking or reflecting the laser scan that would dim the scan in that area to allow for a brighter projection while staying within laser eye safety regulations. At CES 2020, Microvision said they had at least one customer for the concept, but since then, nothing has emerged. Hachi was demonstrating a DLP touch screen projector that doubled as a short-throw front projector (here is a video demonstrating both functions).

Conclusion

It seems that every few years since, someone reinvents the down-projector on a touch surface concept. But in 2007, the same year as many phone companies were working on pico-projector designs, Apple introduced the iPhone. By removing the keyboard using the area for a touch screen display, the iPhone more than double the display area of the Blackberry and similar smartphones. Then smartphone displays got bigger, and their massive volumes drove down the cost of flat-panel touch screens. Then in 2012, Apple introduced the iPad, which led to driving down the cost of larger touch screens.

As stated earlier, flat panels absorb ambient light, and thus display does not have to fight it. Furthermore, there is now over a decade of experience in using flat-panel touch screens. In normal room lighting, a flat panel touch screen beats a down projector in almost every way, and it will work without having to resort to turning off overhead lights and fake images. As they say, “those that don’t learn from history are doomed to repeat it.

While I have primarily addressed Glow’s use of a “down projector,” I would question the whole product concept even if they substituted the projector for a flat panel. It is very hard for me to believe that the use cases shown in the video are very realistic, but other articles on Amazon Glow have addressed this issue.

Appendix – Video Sequences With Different Display Technologies Captured by iPhone

While taking pictures with my iPhone of various down-projectors at CES 2020, the iPhone was in “Live Photo mode,” which records a 1.5-second video. These little clips show some of the artifacts that a camera can capture with various technologies.

The artifacts will vary with different camera sensors and even different settings with the same camera. For example, in the Microvision clip below, there is just a rolling retrace line visible in the first clip and a lot of flicker in the second clip. With DLP and LCOS, I have seen videos that will not show any artifacts. But, I have never seen a video of an LBS projector that didn’t have scanning artifacts of some kind. Thus, I could quickly rule out LBS as being an option for the Amazon Glow.

With the Hachi DLP projector below, I did not see any obvious artifact. You can still get rolling color artifacts in videos of DLP with some cameras and setting combinations, but they are less so with the use of fast-switching LEDs (compared to old color wheel technology).  

Finally, we have Syndiant’s LCOS projector demonstration. In this, we can see the colors rolling due to field-sequential color. Once again, different cameras and camera settings will show this effect differently or not at all, but cameras are more likely the catch color ripples with LCOS than DLP. Generally, the older lower resolution and lower cost LCOS devices will have more color ripple than the newer LCOS designs. The lack of color ripple in the Amazon Glow videos tends to rule out LCOS as the technology used.