Apple Reality is real

Meta CEO Mark Zuckerberg told us that, in the future, we’ll hold our business meetings in virtual spaces and be represented there by avatars.

He was correct about that. However, it seems like Apple is developing a compelling vision for avatar-based business meetings. New details have emerged about Apple’s augmented reality (AR), and it looks like something that could actually transform how professionals communicate with one another.

A quick caveat: The platforms we’re talking about have not appeared even in beta and so they cannot be actually compared. All we have to rely on is company statements, leaks, reporting, sleuthing, speculation — and common sense.

Let’s start with the sleuthing.

Apple is apparently working on an operating system called “realityOS,” which Apple sometimes abbreviates as “rOS.” We know this because references to “realityOS” and “rOS” have been discovered in pre-release iOS 13 builds, a GitHub repository, and even in App Store upload logs. A realityOS simulator is also mentioned in the GitHub repository, presumably for developers.

Apple uses “Reality” in the trademarked branding for two AR developer tools named “RealityKit” and “Reality Composer.”

Apple names its operating system after the hardware platforms that they are associated with. This is an example:

Apple iPhone: iphoneOS (now abbreviated to iOS).

Apple Watch: watchOS

Apple iPad: ipadOS

Apple Mac: macOS

Apple TV: tvOS

So:

Apple Reality: OS X

All clues lead to my belief that Apple’s mixed reality platform will be called Apple Reality. That would be both a reference to augmented, virtual and mixed “reality,” and also to late Apple founder Steve Jobs’ famous “reality distortion field.”

I think this is a pretty good guess, and so for the remainder of this column I’ll refer to Apple’s forthcoming AR platform as “Apple Reality.”

Reality is your reality

A combination of good reporting and speculation suggests that Apple will launch a headset that can be used both for AR and VR (possibly next to 2024). Apple will concentrate on AR applications, even though the hardware will work with VR. Apple’s obsession with AR is evident in company statements, patents, acquisitions and product launches. However, it seems to be somewhat indifferent about VR.

Apple’s first headset will do AR like an iPhone does, but through stereoscopic goggles. AR is achieved by the iPhone’s camera capturing real-time video and superimposing virtual objects on that video. With Apple’s Reality glasses, you’ll be able to see the world around you, but only on screens through video.

Apple is also reportedly working on a more advanced product — more like regular glasses — that will superimpose AR virtual objects onto your natural field of view.

One year and half ago, I had predicted that. Apple would use Memojis — Apple’s cartoonish representation of users, currently used for iMessage and other platforms — as avatars for virtual meetings.

Mark Gurman, Bloomberg reporter (who has exceptional anonymous sources within Apple and its partners) is now available. supports my prediction by asserting that Memojis “could be central” to the experience of using a mixed-reality future version of FaceTime.

What’s great about Memojis is that the avatar conveys the represented user’s real-time facial expressions, head tilts, gestures and other non-verbal communication, while speaking in the user’s voice. Many users find it more attractive and convenient than Zoom-like video chats, which can leave users feeling uncomfortable, exposed and tired. (It’s called Zoom fatigue.)

Apple’s virtual meeting technology was originally created by a DreamWorks Animation company called Spaces. The company originally set out to create a consumer experience technology to help people interact with virtual objects. Later, the company moved to VR conferencing technology. Spaces was acquired by Apple in August 2020.

Spaces’ methods and technologies integrated into a future FaceTime over Reality glasses would show a first-person view of participants in a circle or arranged around a virtual table, with all accessing shared virtual resources like white boards, 3D models, floating charts, and other virtual objects.

The key to this experience is that each participant in the meeting will be able see their fellow participants as holograms within their own space. This is different from a virtual meeting room. This reduces mental fatigue from VR because the environment you see matches what you actually see. It also means you wouldn’t knock your coffee over while gesturing with your hand, because you could see your cup sitting on the desk.

To communicate with the person on your left, turn your body to the right and make eyecontact with the Memoji. You’d hear their voice coming from your right, too. Person A and person B may be making eye contact, but person C would see both Memojis looking at one another.

While today’s Memojis capture non-verbal cues using a camera, the Reality headset would use both cameras and other sensors that could convey those cues more finely, according to Apple patents.

FaceTime would make virtual meetings a lot more fun, as early adopters could become 3D Memojis models of themselves during the same call as later adopters. They would then appear as themselves on video. Business meetings could involve people on iPhones using FaceTime for a Zoomlike experience and in the same meeting people using Apple Reality having the meetings in the “metaverse” (for lack of a better term). Apple Reality users will see their fellow Reality users as 3D avatars, and older-school FaceTime users on floating rectangles showing video of each participant.

Gurman stated that SharePlay would also be important, allowing business users to share documents and presentations during virtual meetings. Apple Reality is a compelling collaboration tool thanks to 3D AR + SharePlay.

Gurman says that the Reality headset will have external cameras that can capture hand movements. This will allow Memojis, who are able to display real-time hand gestures but also allow users to use virtual keyboards and share virtual whiteboards.

Many of Apple’s recent additions to iOS may in fact be designed to prepare the world and prime developers for realityOS and the Reality platform — the U1 chip I told you about in 2019, ARKit, spatial audio with dynamic head tracking, and others.

Apple may believe, as I do, that AR will someday rise to displace the smartphone as the world’s most dominant and ubiquitous computing platform. It wants to be a leader in that future.

Meta, a company that was formerly Facebook, allows users to immerse themselves in VR 24/7. Apple, however, lets users drop into AR meetings and then exit Reality to return to reality.

Meta is wrong. Apple is right about what people want. Zuckerberg’s vision is that people will live and work in the VR metaverse all day. Reality will be preferred by business people, in reality.