Apple intends to bring new Continuity features to its upcoming mixed reality headset that will make switching between devices and virtual workspaces a seamless experience, if a new patent is anything to go by.
For anyone unfamiliar with the term, Continuity is how Apple describes all the ways that the devices in its ecosystem communicate interchangeably, allowing users to switch between them without interrupting what they are doing.
Handoff for instance enables you to start working on one device, then switch to another nearby device and pick up where you left off in the same app. Other examples of Continuity features are AirPlay to Mac, Sidecar, Universal Control, and Continuity Camera.
Last week, the European Patent Office published a patent application from Apple titled “Multi-Device Continuity for use with Extended Reality (XR) Systems,” in which Apple offers some examples of how it envisions how Handoff-like interoperability will work between an XR headset and other Apple devices.
Headset wearer looks at iPhone display
In one example, Apple describes a scenario in which a headset wearer looks over at an email on an iPhone screen, whereupon a virtual replica of the Mail app’s interface gets overlaid on the iPhone display. With a hand gesture or switch of gaze, the user then transfers the email to a larger virtual display suspended in their environment and continues to draft it via detection of their finger movements by the headset cameras.
In another example, while a song is playing in a media app on their iPhone, the headset user gestures or looks at a HomePod in the same room, and by doing so, transfers music playback to the smart speaker in a continuous, uninterrupted fashion, without physically approaching the speaker. “This handoff logic can be via a direct peer-to-peer connection and/or facilitated by a cloud server,” note the patent authors.
Headset draws virtual overlay of iPhone screen content
Various other scenarios are envisaged that show Apple’s XR headset similarly “managing continuous transfer of control between other devices in the system responsive to three-dimensional location-based user inputs, and/or… one or more of the other devices and the device itself.”
Apple also describes another implementation where, rather than switching from a physical display to a virtual one, the headset augments a desktop Mac by positioning “accessory windows” close to yet outside of the boundaries of the Mac’s monitor screen into an “extended reality environment.”
How far Apple will initially push Continuity with its rumored headset is unknown, but some of the examples in the patent are likely to provide a good general idea of what the company has been working towards achieving.
In terms of what we do know about the headset, it will not need an iPhone to function and will be able to be used on its own. To that end, it will run “xrOS,” a new operating system designed specifically for the AR/VR experience. xrOS will include iOS apps like Safari, Photos, Messages, Maps, Apple TV+, Apple Music, Podcasts, and Calendar, as well as a FaceTime app customized for the headset.
Headset wearer moves iPhone content to extended reality environment
There will be no wearable control device for the headset, with Apple instead relying on hand gestures that are detected by the myriad cameras on the device. Typing, for example, will be done using an “in-air” method through eye movements and hand gestures.
Apple is still planning to unveil the first version of the headset, likely called “Reality Pro,” at WWDC in June this year, with the device shipping toward the end of 2023 at the earliest. For more on what to expect from the headset, we have a dedicated AR/VR roundup that aggregates all of the rumors that we’ve heard so far.
(Via Patently Apple.)