3D photography has never taken off in the same way as the two-dimensional art form, but the Apple Vision Pro and iPhone are aiming to change that. If they succeed, it could change the way we capture and experience memories.
That lofty goal is far from guaranteed. After all, Apple’s introduction of the Vision Pro (its “first 3D camera”) was simultaneously the most interesting and creepy part of a launch that often felt like a Black Mirror outtake.
The long list of meme-worthy Vision Pro launch moments means it’s easy to forget that Apple announced a device that captures “spatial photos and videos in 3D”. Considering the company’s last camera (outside the iPhone) was the Apple QuickTake in 1994, that’s a pretty big deal.
On the other hand, the Vision Pro is a camera you have to wear on your face – and Apple’s almost comical demo led to widespread mockery, even from those who helped create it. This doesn’t mean 3D photos and video are again doomed to being fringe novelties. It just means the Apple Vision Pro needs a device that can capture those ‘spatial’ memories, while it focuses on the playback experience.
Those 3D cameras will almost certainly be the iPhone and iPad. And while this will all likely come too soon for the iPhone 15, the iPhone 16 – or perhaps next year’s iPad Pro – could add the necessary sensors and software to Apple’s existing Lidar scanner to become the spatial cameras that the Apple Vision Pro desperately needs. If they do, then 3D photos and video could well become the next big smartphone camera feature.
Memory palace
Apple is far from the only company that’s been exploring the possibilities of capturing memories in three dimensions. The idea itself goes all the way back to the first stereoscopes in the 1830s, but more recently we’ve seen the invite-only Wist (below) promise to let you shoot “immersive memories” on your phone that can then be re-lived on headsets like the Meta Quest 2.
An adjacent trend is the rise of apps like Polycam, which is the most popular 3D scanning app for iOS and Android. Polycam’s founder, Chris Heinrich, told us that the Apple Vision Pro’s spatial photos are an exciting moment and that current phone cameras will need to evolve to shoot them.
“Regular images and photos appear very flat in 3D, and do not live up to the potential of the hardware compared to 3D or stereoscopic content,’ Heinrich says. But the Vision Pro’s three-dimensional snaps are also different from full 3D scans. “The spatial photos and videos that they [Apple] announced are not proper 3D models, but rather stereoscopic photos which produce a 3D effect, and perhaps allow for some small head movement,” he adds.
Still, while spatial photos and 3D scanning apps are different beasts, Polycam thinks the two will likely co-exist in many apps. “We anticipate that Apple will let third-party developers import spatial photos and videos from the Photos app like they do regular photos and videos,” Heinrich adds. “And if this is the case, we will likely support importing, viewing, and editing spatial photos and videos with Polycam”, he adds.
Future gazing
For many, the idea of capturing 3D memories will sound like a gimmick. After all, 3D TVs – which the Apple Vision Pro is arguably the natural successor to – were one of tech’s biggest flops. And ten years ago, stereo cameras like the Fujifilm FinePix Real 3D W1 similarly failed to leave a cultural mark.
But even before the arrival of the Vision Pro, a growing number of experts were hailing 3D snapping as the next biggest thing since the arrival of digital photography. And now that Apple has entered the game, the question is how quickly might it finally take off?
The obvious conclusion from the Vision Pro launch was that the iPhone 15, tipped to land in September, would surely be able to shoot spatial photos and video for Apple’s headset. But it might not be quite that simple, according to Polycam’s Chris Heinrich.
He says that spatial photos “may require adding yet another camera to the iPhone that is spaced further (about the distance between our eyes) from the other cameras” and that this “would have cost and design tradeoffs that probably won’t justify adding it just yet”. Instead, it looks more likely that 2024 will be the year for spatial memories to take off. “My guess is that they [Apple] will release the headset first, and if spatial photos and videos are a hit, they will consider adding it on the iPhone 16 or later”, he adds.
What isn’t in question is that the iPhone and iPad are far more suited to shooting spatial photos and videos than the Vision Pro itself. “I think it makes sense for Apple to support capturing spatial photos and videos on the iPhone because, firstly, capturing photos from a hand-held device is much more natural than capturing from a headset and, secondly, it plays into their strengths of owning a multi-device platform”, he concludes.
Minority Report
Polycam isn’t the only app that’s excited about the potential for 3D photos and videos, following the arrival of the Vision Pro. The team behind Halide, still widely considered to be the best camera app around, said on Twitter after WWDC 2023 that “it’s a great time for us to be a camera app on iPhone”, but they also agree that the iPhone 15 is likely too soon for phone-based spatial photos.
“Current iPhones can’t create proper spatial captures the way the headset [Apple Vision Pro] can,” Halide said in a separate Tweet. “We’ll need a significant improvement in terms of processing power and sensor fidelity and quality to make that work” they added.
Given the rate of change on phone cameras – and the relative plateauing of 2D camera features – the iPhone 16 seems like a decent bet to be Apple’s first phone with stereoscopic lenses for capturing 3D photos and video.
But alongside the technical challenge of making that happen, Apple will also need to work hard on getting the average person as excited about ‘spatial’ memories as iOS developers are. As our hands-on Apple Vision Pro review concluded following a demo of the feature, “it was alike a postcard from Minority Report but instead of pre-cogs, we have a past-cog Vision Pro letting us relive moments like never before”.
That definitely sounds like a fun sci-fi experience – but as the last decade of VR headsets has taught us, there’s a big difference between making science fiction a reality and turning that into mainstream success.