Apple Vision Pro (Part 2) – Hardware Issues


This part will primarily cover the hardware and related human physical and visual issues with the Apple Vision Pro (AVP). In Part 3, I intend to discuss my issues with the applications Apple has shown for the AVP. In many cases, I won’t be able to say that the AVP will definitely cause problems for most people, but I can see and report on many features and implementation issues and explain why they may cause problems.

It is important to note that there is a wide variation between humans in their susceptibility and discomfort with visual issues. All display technologies are based on an illusion, and different people have different issues with various imperfections in the illusions. Some people may be able to adapt to some ill effects, whereas others can’t or won’t. This article points out problems I see with the hardware that might not be readily apparent in a shot demo based on over 40 years of working with graphics and display devices. I can’t always say there will be problems, but some things concern me.

The Appendix has some “cleanup/corrections” on Part 1 of this series on the Apple Vision Pro (AVP).

Demos are a “Magic Show” and “Sizzle Reels”

Things a 30-minute demo won’t show

I’m constantly telling people that “Demos are Magic Shows,” what you see has been carefully selected not to show any problems and only what they want you to see. Additionally is impossible to find all the human factor physical and optical issues in the cumulative ~30-minute demo sessions at WWDC. Each session was further broken into short “Sizzle Reels” of various potential applications.

The experience that people can tolerate and enjoy with a short theme park ride or movie clip might make them sick if they endure it for more than a few minutes. In recent history, we have seen how 3-D movies reappeared, migrated to home TVs, and later disappeared after the novelty wore off and people discovered the limitations and downsides of longer-term use.

It will take months of studies with large populations as it is well known that problems with the human visual perception of display technologies vary widely from person to person. Maybe Apple has done some of these studies, but they have not released them. There are some things that Apple looks like they are doing wrong from a human and visual factors perspective (nothing is perfect), but how severe the effects will be on humans will vary from person to person. I will try to point out things I see that Apple is doing that may cause issues and claims that may be “incomplete” and gloss over problems.

Low Processing Lag Time and High Frame Rate are Necessary but not Sufficient to Solve Visual Issues

Apple employed a trick that gets the observer to focus on one aspect of a problem that is a known issue and where they think they do well. Quoting from the WWDC 2023 video at ~1:51:34:

In other head-worn systems, latency between sensors and displays can contribute to motion discomfort. R1 virtually eliminates lag, streaming new images to the displays within 12 milliseconds. That’s eight times faster than the blink of an eye!

I will give him credit for saying that the delay “can contribute” rather than saying it is the whole cause. But they were also very selective with the wording “streaming new images to the displays within 12 milliseconds,” which is only a part of the “motion to photon” latency problem. They didn’t discuss the camera or display latency. Assuming the camera and display are both at 90Hz frame rates and are working one frame at a time, this would roughly triple the total latency, and there may be other buffering delays not mentioned. We then have any errors that will occur.

The statement, “That’s eight times faster than the blink of an eye!” is pure marketing fluff as it does not tell you if it is fast enough.

In some applications, even 12 milliseconds could be marginal. Some very low latency systems process scan lines from the camera to the display with near zero latency rather than frames to reduce the motion-photon-time. But this scan line processing becomes even more difficult when you add virtual content and requires special cameras and displays that work line by line synchronously. Even systems that work on scan lines rather than frames may not be fast enough for intensive applications. Specifically, this issue is well-known in the area of night vision. The US and other militaries still prefer monochrome (green or b&w) photomultiplier tubes in Enhanced Night Vision Goggles (ENVG) over cameras with displays. They still use the photomultiplier tubes (improved 1940s-era technology) and not semiconductor cameras because the troops find even the slightest delay disorienting.

Granted, troops making military maneuvers outdoors for long periods may be an extreme case, but at least in this application, it shows that even the slightest delay causes issues. What is unknown is who, what applications, and which activities might have problems with the level of delays and tracking errors associated with the AVP.

The militaries also use photomultiplier tubes because they still work with less light (just starlight) than the best semiconductor sensors. But I have been told by night vision experts that the delay is the biggest issue.

Poor location of main cameras relative to the user’s eye due to the Eyesight Display

The proper location of the cameras would be coaxial with the user’s two eyes. Still, as seen in the figure (right), the Main Cameras and all the other cameras and sensors are in fixed locations well below the eyes, which is not optimal, as will be discussed. This is very different than other passthrough headsets, where the passthrough cameras are roughly located in front of the eyes.

It appears the main cameras and all the other sensors are so low down relative to the eyes to be out of the way of the “Eyesight Display.” The Eyesight display (right) has a glass cover that contributes a lot of weight to the headset. I hear the glass cover is also causing some calibration problems with the various cameras and sensors, as there is variation in the glass, and its placement varies from unit to unit. The glass cover also contributes significant weight to the headset while inhibiting heat from escaping on top of the power/heat caused by the display itself.

It seems Apple wanted the Eyesight Display so much that they were willing to hurt significantly other design aspects.

Centering correctly for the human visual system

The importance of centering the (actual or “virtual”) camera with the user’s eye for long-term comfort was a major point made by mixed reality (optical and passthrough) headset user and advocate Steve Mann in his March 2013 IEEE Spectrum article, “What I’ve learned from 35 years of wearing computerized eyewear“. Quoting from the article, “The slight misalignment seemed unimportant at the time, but it produced some strange and unpleasant results. And those troubling effects persisted long after I took the gear off. That’s because my brain had adjusted to an unnatural view, so it took a while to readjust to normal vision.” 

I don’t know if or how well Apple has corrected the misalignment with “virtual cameras” (transforming the image to match what the eye should see) as Meta attempted (poorly) with the MQP. Still, they seem to have made the problem much more difficult by locating the cameras so far away from the center of the eyes.

Visual Coordination and depth perception

Having the cameras and sensors in poor locations would make visual depth sensing and coordination more difficult and less accurate, particularly at short distances. Any error will be relatively magnified as things like one’s hands get close to the eyes. In the extreme case, I don’t see how it would work if the user’s hands were near and above the eyes.

The demos indicated using some level of depth perception in the video (stills below) were contrived/simple. I have not heard any demos stressing coordinated hand movement with a real object. Any offset error in the virtual camera location might cause coordination problems. Nobody may know or have serious problems with a short demo, particularly if they don’t do anything close up, but I am curious about what will happen with prolonged use.

Vergence Accommodation Confict or Variable Focus

There must be on the order of a thousand papers and articles on the issue of vergence-accommodation conflict (VAC). Everyone in the AR/VR and 3-D movie industries knows about the problem. The 3-D stereo effect is caused by having a different view for each eye which causes the eyes to rotate and “verge,” but the muscles in the eye will adjust focus, “accommodate,” based on what it takes to focus. If the perceived distances are different, it causes discomfort, referred to as VAC.

Figure From: Kieran Carnegie, Taehyun Rhee, “Reducing Visual Discomfort with HMDs Using Dynamic Depth of Field,” IEEE Computer Graphics & Applications, Sept.-Oct. 2015, doi:10.1109/MCG.2015.98

Like most other VR headsets, the AVP most likely has a fixed focus at about 2 meters (+/- 0.5m). From multiple developer reports, Apple seems to be telling developers to put things further away from the eyes. Two meters is a good compromise distance for video games where things are on walls or further away. VAC is more of a problem when things get inside 1m, such as when the user works with their hands, which can be 0.5m or less away.

When there is a known problem with many papers on the subject and no products solving it, it usually means there aren’t good solutions. The Magic Leap 1 tried a dual-focus waveguide solution but at the expense of image quality and cost and abandoned it on Magic Leap 2. Meta regularly presents papers and videos about their attempts to address VAC, including Half Dome 1, 2, and 3, focus surfaces, and a new paper using varifocal at Siggraph in August 2023.

There are two main approaches to VAC; one involves trying to solve for focus everywhere, including light fields, computational holograms, or simultaneous focus planes (ex. CREAL3D, VividQ, & Lightspace3D), and the other uses eye tracking to control varifocal optics. Each requires more processing, hardware complexity, and a loss of absolute image quality. But just because the problem is hard does not make it disappear.

From bits and pieces I have heard from developers at WWDC 2023, it sounds like Apple is trying to nudge developers to make objects/screens bigger but with more virtual distance. In essence, to design the interfaces to reduce the VAC issue from close-up objects.

Real-World Monitors are typically less than 0.5m away

Consider a virtual computer monitor placed 2m away; it won’t behave like a real-world monitor less than 1/2 meter away. You can blow up the monitor to have the text be the same size, but if working properly in the virtual space, the text and other content won’t vary in size the same way when you lean in, no less being able to point at something with your finger. Many subtle things you do with a close-up monitor won’t work with a virtual, far-away large monitor. If you make the virtual monitor act like it is the size and distance of a real-world monitor, you have a VAC problem.

I know some people have suggested using large TVs from further away as computer monitor to relax the eyes, but I have not seen this happening much in practice. I suspect it does not work very well. I have also seen “Ice Bucket challenges,” where people have worn a VR headset as a computer monitor for a week or month, but I have yet to see anyone say they got rid of their monitors at the end of the experiment. Granted, the AVP has more resolution and better motion sensing and tracking than other VR headsets, but these may be necessary but not sufficient. I don’t see a Virtual workspace as efficient for business applications compared to using one or more monitors (I am open to seeing studies that could prove otherwise).

A related point that I plan to discuss in more detail in Part 3 is that there have been near-eye “glasses” for TVs (such as Sony Glasstron) and computer use for the last ~30 years. Yet, I have never seen one used on an airplane, train, or office in all these years. It is not that the displays didn’t work or were too expensive for an air traveler (who will spend $350 on noise-canceling earphones) and had a sufficient resolution for at least watching movies. But 100% of people decide to use a much smaller (effective) image; there must be a reason

The inconvenient real world with infinite focus distances and eye saccades

VAC is only one of many image generation issues I put in the class of “things not working right,” causing problems for the human visual system. The real world is also “inconvenient” because it has infinite focus distances, and objects can be any distance from the user.

The human eye works very differently from a camera or display device. The eye jumps around in “saccades,” that semi-blank vision between movements. Where the eye looks is a combination of voluntary and involuntary movement and varies if one is reading or looking, for example, at a face. Only the center of vision has a significant resolution and color differentiation, and a sort of variable resolution “snapshot” is taken at each saccade. The human visual system then pieces together what a person “sees” from a combination of objective things captured by each saccade and subjective information (eyewitnesses can be highly unreliable). Sometimes the human vision pieces together some display illusions “wrong,” and the person sees an artifact; often, it is just a flash of something the eye is not meant to see.

Even with great eye tracking, a computer system might know where the eye is pointing, but it does not know what was “seen” by the human visual system. So here we have the human eye taking these “snapshots,” and the virtual image presented does not change quite the way the real world does. There is a risk that the human visual system will know something is wrong at a conscious (you see an artifact that may flash, for example) or unconscious level (over time, you get a headache). And once again, everybody is different in what visual problems most affect them.

Safety and Peripheral Vision

Anyone who has put on a VR headset from a major manufacturer gets bombarded with messages at power-up to make sure they are in a safe place. Most have some form of electronic “boundaries” to warn you when you are straying from your safe zone. As VR evangelist Bradley Lynch told me, the issue is known as “VR to the ER,” for when an enthusiastic VR user accidentally meets a real-world object.

I should add that the warnings and virtual boundaries with VR headsets are probably more of a “lawyer thing” than true safety. As I’m fond of saying, “No virtual boundary is small enough to keep you safe or large enough not to be annoying.”

Those in human visual factors say (to the effect), “Your peripheral vision is there to keep you from being eaten by the tigers,” translated to the modern world, it keeps you from getting hit by cars and running into things in your house. Human vision and anatomy (how your neck wants to bend) are biased in favor of looking down. The saying goes, there are many more dangerous things on the ground than in the air.

Peripheral vision has very low resolution and almost no sense of color, but it is very motion and flicker-sensitive. It lets you sense things you don’t consciously see to make you turn your head to see them before you run into them. The two charts on the right illustrate a typical person’s human vision for the Hololens 2 and the AVP. The lightest gray areas are for the individual right and left eye; the central rounded triangular mid-gray area is where the eye has binocular overlap, and you have stereo/depth vision. The near-black areas are where the headset blocks your vision. The green area shows the display’s FOV.

Battery Cable and No Keep-Alive Battery

What is concerning from a safety perspective is that with the AVP, essentially all peripheral vision is lost, even if the display is in full passthrough mode with no content. It is one thing to have a demo in a safe demo room with “handlers/wranglers,” as Apple did at the WWDC; it is another thing to let people loose in a real house or workplace.

Almost as a topper on safety, the AVP has the battery on an external cable which is a snag hazard. By all reports, the AVP does not have a small “keep-alive” battery built into the headset if the battery is accidentally disconnected or deliberately swapped (this seems like an oversight). So if the cable gets pulled, the user is completely blinded; you better hope it doesn’t happen at the wrong time. Another saying I have is, “There is no release strength on a breakaway cable that is weak enough to keep you safe that is strong enough not to release when you don’t want it to break.

Question, which is worse?:

A) To have the pull force so high that you risk pulling the head into something dangerous, or

B) To have the cord pull out needlessly blinding the person so they trip or run into something

This makes me wonder what warnings, if any, will occur with the AVP.

Mechanical Ergonomics

When it comes to the physical design of the headset, it appears that Apple strongly favored style over functionality. Even from largely favorable reviewers, there were many complaints about physical comfort being a problem.

Terrible Weight Distribution

About 90% of the weight of the AVP appears to be in front of the eyes, making the unit very front-heavy. The AVP’s “solution” is to clamp the headset to the face with the “Light Seal” face adapter applying pressure to the face. Many users with just half-hour wear periods discussed the unit’s weight and pressure on the face. Wall Street Journal reporter Joanne Stern discussed the problem and even showed how it left red marks on her face. Apple was making the excuse that they only had limited face adapters and that better adapters would fix or improve the problem. There is no way a better Light Seal shape will fix the problem with so much weight sitting beyond the eyes and without any overhead support.

Estimation of the battery size and weight

Experience VR users that tried on the AVP report that they think the AVP headset weighs at least 450 grams, with some thinking it might be over 500 grams. Based on the battery cable size, I think it weighs about 60 grams pulling asymmetrically on the headset. Based on a similar size but slightly differently shaped battery, the AVP’s battery is about 200 grams. While a detachable battery gives options for larger batteries or a direct power connection, it only saves about 200-60 = 140 grams of weight on the head in the current configuration.

Many test users commented on their being an over-the-head strap, and one was shown in the videos (see lower right above). Still, this strap shown is very far behind the unit’s center of gravity and will do little to take the weight off the front that could help reduce the clamping force required against the face. This is basic physics 101.

I have seen reports that several strap types will be available, including ones made out of leather. I expect there will have to be front-to-back straps built-in to relieve pressure on the user’s face.

I thought they could clip a battery back with a shorter cable to the back of the headset, similar to the Meta Quest Pro and Hololens 2 (below), but this won’t work as the back headband is flexible and thus will not transfer the force to help balance the front. Perhaps Apple or 3rd parties will develop a different back headband without as much flexibility, incorporating a battery to help counterbalance the front. Of course, all this talk of straps will be problematic with some hairstyles (ex., right) where neither a front-to-back nor side-to-side strap will work.

Meta Quest Pro is 722 grams (including a ~20Wh battery), and Hololens 2 is 566 grams (including a ~62Wh battery). Even with the forehead pad, the Hololens 2 comes with a front-to-back strap (not shown in the picture above), and the Meta Quest Pro needs one if worn for prolonged periods (and there are multiple aftermarket straps). Even most VR headsets lighter than the AVP with face seals have overhead straps.

If Apple integrated the battery into the back headband, they would only add about 200 grams or a net 140 grams, subtracting out the weight of the cable. This would place the AVP between the Meta Quest Pro and Hololens 2 in weight.

Apple denies physics and the shape of human heads to think they won’t need better support than they have shown for the AVP. I don’t think the net 140 grams of a battery is the difference between needing head straps.


I see Many of the problems with the AVP because doing Passthrough AR well is very hard and because of trade-offs and compromises they made between features and looks. I think Apple made some significant compromises to support the Eyesight feature that even many fans of the technology say Eyesight will have Uncanny Valley problems with people.

As I wrote in Part 1, the AVP blows away the Meta Quest Pro (MQP) and has a vastly improved passthrough. The MQP is obsolete by comparison. Still, I am not convinced it is good enough for long-term use. There are also a lot of basic safety issues.

Next time, I plan to explore more about the applications Apple presented and whether they are realistic regarding hardware support and human factors.

Appendix: Some Cleanup on Part 1

I had made some size comparisons and estimated that the AVP’s battery was about 35Wh to 50Wh, and then I found that someone had leaked (falsely) 36Wh, so I figured that must be it. But not a big difference, as other reports now estimate the battery at about 37Wh. My main point is that the power was higher than some reported, and my power estimate seems close to correct.

All the pre- and post-announcement rumors suggested that the AVP uses pancake optics. I jumped to an erroneous conclusion from the WWDC 2023 video that they made it look like it was aspheric refractive. In watching the flurry of reports and concentrating on the applications, I missed circling back to check on this assumption. It turns out that Apple’s June 5th news release states, “This technological breakthrough, combined with custom catadioptric lenses that enable incredible sharpness and clarity . . . ” Catadioptric means a combination of refractive and reflective optical elements, which included pancake optics. Apple recently bought Limbak, an optics design company known for catadioptric designs, including those used in Lynx (which are catadioptric, but not pancake optics, and not what the AVP uses). They also had what they called “super pancake” designs. Apple eschews using any word used by other companies as they avoided saying MR, XR, AR, VR, and Metaverse, and we can add to that list “pancake optics.”

From Limbak’s Website: Left shows their 2 -element “Super-Pancake,” and the middle two show Lynx’s optics.
Karl Guttag
Karl Guttag
Articles: 243


  1. I’m happy to see your focus on VAC. It seems to be something that is conveniently ignored or handwaved away/trivialized every time there’s a new phase of VR hype (another of which we’re perhaps entering now). And there are also many misunderstandings of the issue. It’s somewhat surprising to see Apple push a product out anyway in spite of this (though perhaps not surprising that many on their design team didn’t want to launch anything VR related yet)

    Along similar lines, I’m curious what you think of the significance of gaze contingent ocular parallax rendering in modern VR (or rather, the absence of it). There is some recent-ish Stanford Computational Imaging Lab research that has found it actually accounts for a significant percentage of the discomfort people experience in VR (as in almost half of what was previously attributed to VAC). I’m curious if you’ve heard anything else along these lines, as it would at least seem to be a more tractable problem than addressing VAC. As with the other advances, I doubt it would in any way counteract the need for a VAC solution, but in VR we’ll take anything that makes these things less of miniature torture chambers

    • On the VAC front Apple does have the advantage that while it can do 3D, the primary use case for this thing is to put a bunch of iPads around your head. Apple can just lock the surface the user will mostly focus on at the accommodative distance, and they have a user base that’s used to being told “you can’t adjust that because we know better than you”.

  2. Have you considered the properties of Norwegian developed Tlens (tunable lens, with autofocus within 1 ms) could be the missing link, and the reason why AVP is abel to take the giant step in MR/AR?

  3. My monitor is about 80cm away, moving it closer makes it feel uncomfortable.
    As far as I know, ergonomics guides state that 50cm to 1m is the ideal distance for a monitor.

    • You have a reasonable point in terms of what ergonomic guides say. I was on the short end. I would suggest that at 80cm you are further away that most people.

      I know that various ergonomic guides say 50cm to 100cm, but the long end is pretty unrealistic unless you have a very large monitor with a low pixel density and/or only work with low resolution content. I think most people sit near the short end (50cm) and then lean in occasionally (to 30cm or less) if there is something they want to examine closely. 100cm might be recommended by ergonomic theorist and might be better for your eyes, but most people end up between 50cm and 65cm (see for example: when they combine multiple realistic factors. Most people end up where they can touch the monitor screen and they could not do that if they are 100cm away. If you sit at say 50cm away, you can do a “2x zoom” by leaning in a little, something people do almost reflexively.

      When sitting, a “typical” person’s arm’s bent in relaxed typing position, you finger tips are about 40-60cm away. The display on laptop then ends up at about 40 to 60cm away. When you read a book, it is closer than a laptop screen.

      There is a far point that with a VR headset that the eyes accommodation would be more relaxed position and might cause less eye strain. But then you get back to other ergonomic factors such as how the virtual screen behaves when you move your head/eyes.

  4. I’m happy Google decided to share this post with me. I now have a new favorite blog! You bring up a lot of issues I, a non-XR enthusiast who had an Oculus Rift for a bit before my cats peed on it years ago, had never considered.

    I’m skeptical of this headset, but I also think Apple is targeting specific use cases that aren’t as susceptible to the problems people bring up.

    1) I agree that people scoffing at the price are missing the big picture. A lot of it feels like performative Apple hate in an industry that needs to grow up and move past that school-yard crap.

    The thing comes out to ~$75/mo if you amortize the cost over 4 years. That’s about 5% of the median US rent of $1700/mo. Plenty of people will be able to afford it.

    2) I suspect a lot of the concern and memes around the “uncanny valley” of the eye display and video chat avatar are blowing a minor gripe out of proportion.

    My guess is that it makes a big difference when people know a digital character is being actively controlled by a real human. It’s probably won’t be as jarring as when an NPC in a videogame states blankly while they speak.

    I also think the “uncanny valley” concept itself is overblown. The Northern Express is the target of many uncanny valley memes, but it still grossed $314 Million. People just don’t care as much as internet culture would have you believe. I bet after 1-2 video conferences with an avatar, people will get used to the “unsettling” feeling and get over it.

    There’s also lots of obvious paths forward in terms of improvement. Epic’s “metahumans” come to mind.

    The only physical hurdles I can see are that: part of your face will always be obscured, so the facial capture system will always be blind to possible expressions like a furrowed brow and a device sitting on your face will always distort facial muscle movements and expressions compared to an unencumbered face. However, I could see those problems being corrected for by AI: Apple could record bunch of a bunch of actors performing facial expressions without the headset on then with the headset on and train a model to correct for mask distortion and fill in the obscured bits of the face.

    3) Lag and VAC seem like a problems that might limit the use cases without being a total deal-breaker. Lag could also possibly be mitigated with clever tricks.

    I don’t think many people are going to be walking around town using this thing. The social stigma alone will probably limit that use case. Apple showed off pretty immobile and private use cases: workstation, video conferencing, consuming content at home or on an train/airplane. Lag could be a problem in some of those, but the workstation is the only one of those uses where I see VAC being a relevant issue.

    I think Apple could, again, turn to machine learning to predict saccades and other movement to provide very low lag rendering.

    One trick I’ve seen used is to use image interpolation to largely decouple the refresh rate from the rendering pipeline. This works similarly to video compression where you have an image frame (I-frame) followed by movement frames (p-frames) that simply describe how the pixels move and change over time. As long as you don’t go too long between image frames, the effect is practically imperceptible. The rendering pipeline could spit out, say, N < 90 frames per second, but the system interpolates the missing frames according to tracked movement and predictions to spit out a guaranteed 90 fps. There may be subtle artifacts, but it might be worth it to combat motion sickness.

    4) I think the eye screen is largely being underrated by a lot of reviews. Lots of the barriers to XR are social in nature and I think that justifies some of the compromise Apple engineers made.

    If Apple's reprojection software isn't sufficient to make up for the sub-optimal camera placement, I wonder if a second pair of upper cameras would help and/or if they could use pass-through optics like a periscope to capture images from the center of the eye while leaving the front screen largely unobscured.

    I don't think the front screen has to have extremely good specs to achieve a descent facsimile to eye-contact. Adafruit sells a kit featuring a pair of 240×240 TFT LCDs to add animated eyes to Halloween decorations and it's kinda surprising how convincing the effect is. I wonder if Apple could get away with embedding image sensors directly in the screen if only to provide a "correction signal" for the reprojection algorithm.

    Those are all my thoughts for now.

    • I’m not very invested in this and it’s n=1, but I read a thing and it’s “for large values of one”.

      Ian Hamilton is a mostly VR but with an eye to XR enthusiast, to the extent that he cohosts a regular video XR news show in which he and cohost David Heaney are in VR interacting via (cartoon, UV-avoidant) codec avatars. There’s probably someone in one of these gigacorp labs who’s done it more, but among members of the public these guys might have actually done this thing Apple’s selling more than anyone else in the world. And they’re well accustomed to putting up with the artifacts and oddities of VR.
      “Apple’s demo, meanwhile, featured a live FaceTime call with another person also wearing a Vision Pro and he conveyed the idea that we could collaborate on a FreeForm whiteboard together in this format. Their “Persona” had been scanned by the headset, Apple said, and it felt like the quality level fell squarely between the two demos from Meta last year. This was a representation of a person I’d feel uncomfortable talking to after a few minutes, and it certainly didn’t feel like they were there with me while constrained within a floating window.

      I’m starting here with Personas because this was the only part of Apple’s demo where I felt disappointed. While the highest quality version of Meta’s avatar tech impressed on PC, it was nowhere to be found in standalone on Quest Pro. Apple, meanwhile, is planning it as a core aspect of its first-generation Vision Pro and showed it to me directly in a standalone spatial computer.

      Put another way, both Meta and Apple are reaching a bit into the future with this particular piece of their technologies and neither hyper-real visage is both perfectly believable and easily created.”

  5. Good read. I hope we get an opportunity to test them for a longer session soon, instead of short tech demo. The project’s been in works for years so surely Apple tried its best to circumvent various VR issues for its flagship product, curious about the result. (typo ‘confict’)

  6. Super interesting reading as usual! And a nice follow up to your recent talk at AWE. I’ve tried using a range consumer HMDs for work tasks from time to time (brower-based email, docs, etc.) and it was always a struggle.

    During your AWE talk, and in part 1 of this series of posts, you mentioned the need at for at least 40 pixels per degree for the replacement monitor use case. Do you have supporting published reference for that? Or perhaps that based on experience trying a number of devices over the years?

  7. Am I the only every day VR user who doesn’t seem to feel vergence accomodation conflict discomfort at all? I’ve seen this discussed very frequently and a subject study for dozens of papers. In theory, I can see there’s a potential for a problem. In practice, this doesn’t seem to affect me at all.

    Am I just a special snowflake? Talking with other people about display tech, it’s true we do differ wildly in how we perceive them (and one could infer we differ to the same degree with just about any form of visual perception). I’d rather use a 120Hz 1080p screen with trash colors and contrast for work than a 1440p 60Hz with good colors.

    The thing that does bother me at times with visual perception in VR is that my vision matches head movement, but not eye movement. A problem named “gaze contingent ocular parallax”, I believe. It is annoying, not properly a source of discomfort however. The thing that is sure to get me to remove the headset within seconds is lag though.

    • People’s visual perception is more varied that most people realize. There are both physical/measurable differences such as IPD, the cornea, and the number of photoceptors of each color in the eye. But much trickier to understand are all the visual processing differences between humans. I have known people that can’t see sequential color breakup unless it is severe that then are extremely sensitive to flicker (to the point makes them almost instantly sick). Some adverse visual problems can be “trained out” with exposure. I know when I first when the UK back in 1979, I could see the flicker from the 50Hz refresh, but it more or less went away after a few weeks; at least the conscious flicker, even flicker you don’t “see” is still known to cause eye strain issues).

      In your case, you were probably able to get your visual system to adapt to the Vergence-Accommodation Conflict (VAC). The content also has a lot to do with VAC. Some companies design around the issue by making things appear to more than 1 meter away so as to limit the problems with VAC. Frame rate is another big issue for people depending on content. Frame rate also gets into the motion-to-photon issue in that slower frame rates equal more delay.

      Gaze contingent ocular parallax is one of many things I would classify under “your vision system senses that things are working right.” As I have written, “eye tracking can know where the eye is looking, but not what the person sees.” Saccadic eye movement is asynchronous to the cameras and video output. The headset will known where your eyes were pointing and the eyes could be someplace else by the time the image is generated. It could be that these “secondary,” less obvious, issue could be the most problematic has companies try and support passthrough MR.

      Its not clear that broad market will be willing to deal with the issues and adapt or are physically able to tolerate the problems. It is also not clear that all the problems are “solvable” for a mass market. We also don’t know if there are long term and long exposure problems with VR.

  8. I truly appreciate hearing your educated perspective on technology. I just can’t get enough.

    Any thoughts on the external display?
    Other than the universally lauded opinion that it’s silly.
    I was thinking of available technologies involved.
    Flexible OLED to accommodate the curved design? That would be a first for Apple.
    Basic lenticular overlay or something like the “Diffractive Lightfield Backlighting” IP from HP spinoff Leia Inc
    Normally Apple just acquires smaller companies instead of licensing tech.

    Keep up the great work.
    Can’t wait to hear more from you.

Leave a Reply

%d bloggers like this: