Use iPhone 15 Pro to capture video in 3D
Beginning with iOS 17.2 last week, iPhone 15 Pro and Pro Max can now capture “spatial video” for 3D playback on the upcoming Apple Vision Pro (and presumably other 3D devices). if you have an iPhone 15 Pro I recommend updating your software as soon as possible so that your holiday memories this year can be captured in the new "spatial" 3D format, which is reported to make your videos far more emotional because it feels like you are there.
If you’ve been unimpressed with 3D TV or 3D theaters that uses polarized glasses, this is an entirely different ball game. Each eye gets a dedicated screen so you do not have the fading or flickering problem of other 3D systems. It's like a Fisher Price View Master, but digital.
To enable the option:
- Open the Settings app on iPhone 15 Pro or iPhone 15 Pro Max
- Tap Settings > Camera > Formats
- Turn the slider "on" for Spatial Video for Apple Vision Pro
To record with Spatial Video:
- Open the Camera app
- Swipe left or right to switch to Video mode
- Hold the phone horizontally
- Tap the Apple Vision Pro headset icon near the lower right of your image—yellow means it's enabled, grey is disabled
- Tap the record button to start recording
What is spatial video? Simply, it’s what you or I might call 3D video. Apple has avoided calling it 3D video as that implies that you can walk around the subject in virtual reality. Spatial videos have a fixed point of view.
Are spatial videos recorded in a proprietary format? Nope, they are recorded in a standard video format called MV-HEVC (Multi-View High Efficiency Video Codec) that can be played back on any 3D device if the player supports it. As of now, Apple Vision Pro is one of the first devices to support this new format, but I suspect other devices such as Oculus will support it soon.
Can I watch spatial videos in 2D? Yes, any software that plays HEVC/H.265 video will simply ignore the data for the second eye. This includes basically any software that has been updated in the past six years or so.
How does it work? iPhone Pro Max uses two of the cameras to see the difference between two points of view. However, because the cameras are so close, the difference is not nearly as much as what human eyes see, so artificial intelligence is used to make assumptions and compensate, which is why it’s only available on the iPhone models that have the latest processor.
What are the drawbacks and limitations of recording in spatial video? While there are several things to be aware of, it may be worth it for the impressive 3D effect.
- Spatial videos consume about double the storage
- Spatial videos are limited to 1080p at 30fps, but I suspect future iPhones may allow something higher
- You cannot zoom in or out
- Spatial videos must be recorded horizontally instead of vertically because of how the cameras are oriented; hold the phone sideways like a TV, not upright
- You need to record in good lighting so that the AI can understand what it is seeing
- Recording should be done with as little motion as possible, especially when recording indoors, to help the AI understand what the background is and to keep the viewer from being disoriented
- You need to remember to choose spatial video each time you record
No iPhone 15 Pro? Rumor is that next year's standard iPhone 16 will adopt this feature. And of course, Apple Vision Pro itself will be able to record in 3D. Perhaps even at better quality because it doesn't need to use AI manipulation since the cameras are properly distanced apart. But we won't know that for sure until Apple releases the headset in "early 2024".
Featured image courtesy Apple, Inc.