Spatial Video production techniques for the Apple Vision Pro Headset
The Apple Vision Pro headset promises to introduce the world to spatial computing. But, we're particularly excited about the potential for a new form of video production. Spatial video has the potential to combine 3D video, 360-degree video, and Apple's high standards of quality to create something new and special. This Channel will share insights, news, and experiments as we learn how to shoot, edit, and publish video content for spatial headsets. We will share video production techniques we learn as we create our own spatial video projects.
Пікірлер
As someone who started with VR Headset in 2014 with the Oculus DK2, then HTC Vive, Rift-S, PSVR1, HP Reverb G2, PSVR2 and currently Quest 3 I was always a big Fan of VR. Regarding AVP. I'am still searching for my Use case. Seems PC VR Gaming is not a thing. Regarding Large multiple Monitors. I already use a 55 inch OLED for my PC which costed me $1200. This is enough for me. Watch VR Movies in better Quality. There is unfortunately no consumer camera out which is capable of making VR Videos in a quality matching even Quest 3's resolution. The Insta Evo I'am currently using is by far not good enough for this. 99% of all VR Videos on KZread out there are just plain bad. I think Apple already saw this problem which was the reason there was no KZread App on AVP at the beginning as the danger was high people search there for VR videos and only found thousands of crappy VR Videos in 480p and 15 fps which may let them believe this is the fault of their VR Headset :) One of the big mistakes made by Meta ... I'am pretty sure, people would have been impressed in exactly the same way with AVP Videos or this Dinosaur Demo shown on Quest 3 as they were on Apple Vision Pro. But if you buy a Quest 3 and get through the setup process, nothing impressive is ever shown to you. Everything looks mostly cheap comic like style like from a 10 year old mobile game. Their environments and Avatars also look cheap. On AVP there is this virtual cinema where you could watch a lot of 3D Movies, even some never released on 3D Blu Ray. Meta never thought about this obvious thing. There is no (legal) way to play 3D Movies on a Meta Quest 3. Why?
The Qoocam EGO is a 3D stereoscopic camera ($300) that can shoot 1080p 60fps video and 12k 3D stills. Its compatible with the Apple Vision Pro
You can shoot these videos also for the Quest 2 and 3 and several other VR headsets. The Vision Pro is not the only headset, you know?
Hello which bluetooth device did you used to make it detected and if the latency is too high for movies or just for gaming ?
A big thanks for this video! Just purchased the Insta360 X4 last week. I've been searching for the simplest way to convert my video to 360VR. Voila! The directions in your video worked perfectly! Thank yoy❣️👍🙂
This was a nice review of the QooCam Ego, with lots of helpful tips. I have a slightly odd question, asking about the ability of the Ego to PLAY BACK video that's been recorded on the iPhone. Because the Ego is such a bargain price compared to the AVP and the Quest3 (which can both play back iPhone spatial video, of course), it would be fun, maybe even useful, for many, if the Ego could do playback.
I'm going to say that probably won't work. The Ego records videos in the side-by-side 3D format. Apple Spatial videos are a different format. I tried using the iphone app "Spatialify" to convert an iPhone spatial video to the side-by-side format. I put that file on an SD card, but the Ego camera would not see it. Maybe somebody could come up with a way to do it, but it didn't work for me. Even if it did, I don't think this would be an enjoyable way to watch your spatial videos. The 3D viewer on the Ego camera is meant to help you frame your shots or confirm how your clips look during shooting. It's not really a system to watch those videos. You wouldn't watch a movie on a camera's viewfinder.
@@SpatialInsights Thanks for your quick and insightful comment. I've sent a similar question to the folks at Kandao to see what they say, and I'll post here anything that's useful. I'm trying to do some 3D/spatial things for medical education (surgical or surgical-adjacent procedures). Watching such a procedure for 2-10 minutes, and comparing the benefits of 3D vs 2D is part of what we're exploring. Using an AVP or Quest3 device is more expensive/awkward than using the simple Kandao QooCam EGO device. The EGO won't be neaarly as good, but it might be a question about whether it's "good enough" and better than 2D.
@@michaellougee987 I may have a solution for you. If your goal is to simply view the spatial 3D videos, a VR phone headset may be a better option. They're a small headset that you can insert a phone into, and they're much cheaper than a full VR device as it's only a plastic holder and lens. This will also be a much better screen quality than the EGO can provide. If your goal is to capture 3D spatial video, I also believe an iPhone would be better for the use you have mentioned, rather than the EGO. As said in the video, the eye spacing is important here, as if you are concentrating on close, small objects, as you might do in a surgical video, the lenses must be closer together, otherwise a cross-eye effect will be produced as your eyes cannot put together the two videos with such different angles of a small subject.
@@kieranallen1982 Thanks for the reminder about the small plastic (or cardboard!) headsets for holding a phone, where the phone which is inserted into is able to play Left+Right images, which then appear in a "stereo/3D" view. Of course it's not as "gee whiz" a 3D view as the AVP or Quest3 present, but it does have a 3D quality. Also, FYI, I've been able to use my favorite editor, Camtasia, to place annotations (titles, arrows, circles, etc) onto the spatial/3D recordings, by editing the Left+Right video from Spatialify, and placing those annotations in exactly the same locations on both the Left and Right images. When later viewed in the Quest 3 or AVP (or any other 3D viewer), those annotations appear "on top" of the 3D video. In general, it's great to see people describe their successes (and problems!) in creating real-world uses for spatial/3D videos. Those videos are nice for reliving family occasions (birthday cake video that's part of Apple's AVP in-store demo), but there should be lots of other purposes for spatial/3D.
This was really helpful! Thank you
I've discovered your videos after going on a Vision Pro KZread video binge after ordering my VP which releases here in Germany in 2 weeks. Ever since the iPhone 15 Pro got that 3D video update I started taking spatial videos (I even uploaded a concert video in 3D here on YT) and have been watching them on my Quest 3 by converting them on my phone to FSBS videos which I then tranfer to my Q3. The process is pretty straightforward. Important to note is that the quality and 3D effect was rather disappointing until I discovered an in the app lab called 4XVR Player, which really makes a huge difference for clarity and the 3D effect, the difference is night and day. I'm really curious how this compares to the Vision Pro, I'll see in roughly 2 weeks... Btw. you have great screen presence, you seem like a natural in front of the camera. Great job!
I’ve had very good success improving the quality of 360 videos taken using Insta360 cameras (X4, X3, OneRS 1”) by using Topaz Video AI. You do have to buy the software and it does take a lot of processing time, but the results are really stunning. I’m surprised more KZread reviewers have not covered how this software can produce results that look like they came from a much more expensive camera.
Thanks for sharing. I just started playing around spatial video and watch them on Meta Quest 3. I feel like the spatial video's only advantage at this moment, is that we can shoot it with an iPhone, a device that's already in everyone's hand. Otherwise, if I am a professional content creator, I would just go for VR180 and shoot with a dedicated camera. Spatial video is just like a "lite-version" of VR180, with less FOV.
Thanks for this video. The work flow you described is what I use. However once complete I upload them to my Meta Quest 3 for viewing. Can't afford an AV Pro, but the Quest does a very good job. The work flow is to have Oculus app on the iphone and then find the Spatial video edited on the iphone in Gallery in the app and then upload it wirelessly to the Quest 3. View in the Files app.
Very good explanation of the editing workflow for apple's spatial video... BUT a simple question. Have you found it possible to use Premiere Pro to do things like add titles and credits, visual annotations (eg an arrow), etc onto the spatial video, and then have those modifications play back correctly in the AVP device? I'm thinking of educational videos, for medical education, for which arrows, circles, etc could be used to illustrate anatomical landmarks during a surgery (as one example). I don't think this is possible yet, but I'd be very happy if I'm proven wrong. THANKS for any comments.
I have experimented with this but controlling depth is difficult unless you have a second 3D monitor to know where on the Z axis the graphic will go.
Too bad the Qoocam requires Apple silicon. I am one of those odd animals who has an AVP but no modern Mac’s.
This is exactly what I’ve been looking for to answer some questions. I am trying to raise some money to get the Apple Vision Pro, the Meta Quest 3, and the Insta360 X4 so I can make 360 videos of the Amazon (where I live) and use them in the States on both devices. It’s not too surprising that the Vision Pro is the most expensive part of the entire package, but I’m looking at having to spend money to make money. Quick question, do you think there is any quality difference between a finished 360 video on the Apple Vision Pro and the Meta Quest 3? Do you have any experience with the Quest 3 to be able to compare them?
If you publish your VR videos on KZread, there is a quality difference between Quest and Apple vision. Full 8K playback of VR videos does work on KZread VR on quest. At the moment, there is no native KZread VR app on Apple Vision. There are 3rd party apps like Moon Player that can play KZread VR, but the experience is not easy. And it does not support 8K playback of VR from KZread. I don’t have the details on hand, but my understanding is that even if KZread made a native VR app for Apple Vision, there are technical limitations and it still would not be able to support 8K VR playback. Apple did announce a partnership with Vimeo at WWDC yesterday. I’m anxious to see if that will offer high quality VR publishing and playback. Also - please be cautious about investing in VR production with the Insta360 X4. The quality from the X4. Is really very low. I’m hopeful that there will be an accessible camera with a little better quality to come in the future.
thanks for the video, very well researched and put together. A quick question here, have you managed to try any of those immersive video workflows (not spatial). I'm blown away by apple's demo immersive videos, and want to create my own. But i would like to know a proven workflow (say from Canon's dual fisheye lens to vision pro) before making an investment in those cameras. thanks!
I have not done that. My mission with this channel is to talk about production methods that are affordable and accessible. I think there’s a lot of potential and I’m hopeful that more cameras are coming soon. The Canon dual fisheye kit is exciting. But it’s a $6000 - $8000 investment. And from what I’ve seen there’s no way to share those VR 180 videos on KZread in a way that people can easily view them on the Apple Vision at full quality. At WWDC yesterday, Apple mentioned a new VR camera coming from Blackmagic, which will probably be another big and expensive option. More info coming later this year. I’m more excited about their announced partnership with Vimeo, which may give us the sharing platform we need.
@@SpatialInsights thanks for the reply! Agree that both the EOS and Blackmagic are too expensive for an average consumer. Good news is Canon just announced a more affordable one, R7 (~US$1300) with a new apsc VR lens (around ~1300). Looking forward to this new option
@@dlep2004 Are you referring to the Canon lens that was announced at WWDC yesterday? According to the information I’ve seen, that is not used to shoot VR. It’s designed to shoot “Spatial” video. Apple’s format for 3D video. Not 180 or 360 VR. The Blackmagic camera is for VR.
@@SpatialInsights that, but also another 3.9mm dual lens (looks like the 5mm something two years ago, different from the one from the keynote), is for VR, 3D 180degree (transformed from 144 degree originally from the lens), for apsc camera.
@@dlep2004 Yes! Wow. I just saw that last night. That is very interesting. I'd really like to try that.
Question: can you view the spatial video in immersive mode when you play them on the headset?
Qoocam ego seems to be a good cheap option
Check with the porn industry. They have huge amount of immersive video content. Ask them what camera they're using. Their standard file is mkv for the immersive video with either file naming conversion or metadata. I think Windows apps like Power Director has VR support. Turn flat videos into immersive video but I don't think you get the 3D effect.
Can you use this device to upload spatial videos to the quest 3 and if so how do they compare to the spatial videos on the avp?
hi, i have some Iphone 15 pro max spatial shaky video because i didn't have a stabilizer. Do you know if it's possible to stablize the 3d sbs video in post ? btw thanks so much for your videos ! (i had a qoocam ego but it died the first day as i tried to use it underwater (1m).
Any recommendations on cameras that can shoot 4K+ spatial videos? It'd be great to get more immersion and a clearer video. Thanks for the tips!
If you're using an iPhone 15 Pro to record 3D videos, you're doing it wrong. Recording 3D videos with the Apple Vision Pro offers greater depth which blows the iPhone 15 Pro out of the water.
Sennheiser Momentum 4 connects but no sound. It even connects in the Bluetooth menu, not accessibility. Any suggestions?
You make a lot of great observations about the Insta360 X4. However, like 100% of such videos I've seen, you only comment on what you have made. There are much higher-quality results made with the Insta360 X4 viewable on KZread. I don't have a VR headset so it would be nice for someone to search for the very best-quality videos and comment on them instead of their own videos. That person could try to analyze how those results were obtained. The best results that I've seen are usually where the camera is motionless on a selfie stick (or moving very slowly) and there is plenty of sunlight. Lots of colors are nice, and things to look at that aren't too distant (as you mentioned). Another issue is how they are viewed. It typically takes a few days for 8K video to appear on KZread (your video is not now available in 8K), people don't manually set their viewer to 8K when it is available, and KZread seems to by default zoom in the picture a little when the video begins. From the beginning the viewer needs to zoom out completely and set their device to 8k.
Hi, I am a happy owner of 360 insta cameras (X3 and OneRS 1 inch) and also of Meta Quest 3. I did several tests but 360 degree videos not look very well in VR headsets: I think a solution could be to transform 360 2d videos in VR180 to double the resolution do you have a workflow for this process?
That would not double the resolution. If you shoot 360 and master it down to 180, you cut the field of view in half and throw away the content from half of your video. There’s no resolution gain. I suspect that the resolution limit has to do with the processing power and storage speed on these cameras. So, if a camera maker wanted, they could make a dedicated 180 camera that records at 8K from a single lens. That would give you full 8K for that 180 field of view. But the current insta360 cameras do not offer that.
I would be interested in learning how to create and edit spacial videos and posting them on KZread. I understand that viewing these on the Vision Pro is not ideal at this time but this expereince will only get better going forward. In addition other VR headsets will be able to view them on their KZread apps.
You can shoot immersive 180 video, with a pair of gopros. NLE programs with ai models have excellent upscaling and HDR conversion these days.
Interestingly, I had never seen those compatibility details. I have and old Apple wireless keyboard and trackpad, but I couldn’t get them to connect. I see there that they explicitly say that the old devices with removable batteries are not supported. I think that’s pretty bizarre and unfortunate. I wonder if the lack of mouse support is due to the way you hold a mouse and it might trigger false hand gestures. Just guessing. I do prefer a mouse over trackpads. I am working on a device that is controlled over Bluetooth Low Energy. Fortunately, that works very well.
Thank you very informative. I would love to see a video for filming with the APV headset itself and also for sharing spatial videos on KZread. Thanks!
Thank you so much. My new Sony XM5s connect through the work around in the accessibility settings.
Your videos are really informative. Thanks for creating this channel and posting them.
500 subscribers! Congratulations 🙌🏽
Thanks for your tips. I have some questions though 1. Does it record in 4K? 2. What about stabilisation when recording handheld? 3. Will it work good for close-ups, for example: food on a plate, or flowers. 4. What about in a low light situation, can it handle it well?
I bought this camera back in early March for a couple of trips I was taking. My review is as follows. I would give it a 7 out of 10. The biggest drawback it has is that it takes so long to turn on and be ready to shoot. I’ve counted it out to 20 seconds. Switching between modes and doing anything takes a little bit of time as well. I take images in raw (DNG) format so I have ultimate editing power in Lightroom, the movies come out as they are though. Whenever it takes a good picture, it’s outstanding, but often times the shutter is slow and it gets a little bit of blur, or it focuses on the wrong thing and you have such a shallow depth of field. So you have to be very careful to get your subject and focus And have the exposure by a set to where the shutter will be fast enough. I’ve also noticed that the images look far better in 3D than the movies do. The movies are OK, but they just don’t have the same wow effect as the images do. The biggest plus to having this over the iPhone, if I am to understand correctly, is that the iPhone 15 Pro does not take 3D images, but 3D movies only. If they can work on their speed issues, I think this would be a winner. You can’t complain about the price though!
Nice one. I am considering buying this camera too. What's your opinion about stabilisation when videos were recorded handheld? What about lighting in low light as I would definitely like to capture Christmas dinner and also food close-up.
@@halimrahman unless you’re using it on a gimbal, the video is pretty shaky. It does have the “in-camera” stabilization feature that you can run on a movie after you take it. That works pretty well, but it does crop into it a little bit. As far as low-light situations, that’s the most challenging part because you have to hold it very still in order to have a sharp picture. I usually bring down the exposure bias to have a quicker shutter speed.
thanks a lot, could you compare more the quality difference between this camera, the iphone 15 pro and the AVP ?
I would like to see a video on /that/
Oh my gosh.
Thank you for this very thorough and clear video. I’m a long time filmmaker, but just started dabbling in spatial videos today, and can’t wait to dive in!
PTGUI is a tool that I use to crop 360 photos into native Apple Vision Pro panoramas. It’s expensive but a one time purchase as opposed to an Adobe subscription.
Shooting with the 3D ViewFinder attached to the Qoo Cam EGO is helpful for discovering and framing shots. The lenses being eye width apart is also ideal. The iPhone 15 Pro post processing and notifications that better lighting is needed or to move back when I’m frequently too close to the subject is also really helpful.
I am saving this until I buy an AVP and start editing.
I have a Fuji Finepix W3 pocket 3D camera. Over the years I have shot lots of 3D stills and videos that I’d like to view on the Apple Vision Pro. This stills are in MPO (multi-picture object format). The videos are AVI. I’d love to find a way to convert my stills and videos from the Fiji W3 so I can view them on the Vision Pro. If you know how to do it, I’d love to see a video about that.
I have that camera, too. I used to love ordering lenticular prints from Fuji. I powered it on recently to see if I could get anything useful out of it. I remember the video quality was very low and I think there was a proprietary file format. Since the video quality would be so low compared to the iPhone or the Ego camera, it wasn't worth it to me to investigate further.
@@SpatialInsights i have one too ;-) but don't use it anymore either...
I see the "windowed" spatial videos as being previews and once you're sure you have the right one you go into full immersion. I agree with you that the Immersive Videos are incredible. Really good stuff.
Not bad. But, with more time, I've found that widescreen spatial videos shot on the iPhone look pretty bad in that immersive mode. But square spatial videos shot on the Apple Vision itself actually do look pretty good in the immersive mode. We have more info in another video - "3D Spatial Video - Easy Tips for Apple Vision Content Creation"
I don't like the square format for videos
With some more time, I've found that the square format spatial videos do look pretty good in the Immersive viewing mode. It might be worth trying. We have more information in another video "3D Spatial Video - Easy Tips for Apple Vision Content Creation".
The blurred view is really terrible.
great tips. this is not for AVP but I have done vertical panoramas. works great for a waterfall like Multnomah Falls in Oregon.
Yes! Apple never said anything about shooting panoramas vertically (As far as I know). But it works great. I don't think it does anything for the Apple Vision, but it's still great.
Thanks for this video. My 7.1 Pioneer stereo actually connected thanks to that work around you showed in the accessibility settings..
You're lucky. I wish my receiver would connect.
These are some great tips! Looking forward to experimenting with these. Thank you!
Thanks! I've had a lot of fund with panoramas.
Hi Thanks for a good channel. A couple of weeks ago I tried the Apple Vision Pro for the fist time and are about to pick up my on in a couple of weeks, when I visit Chicago! I have been researching a bit about spacial and immersive spacial videos, and found your channel. I also found a LinkedIn Learn (from the Lynda time, 2016) course, by Mark W. Gray, called "360 Video Workflow" and the last bonus section is about making immersive spacial content. this "lack", for natural reasons, the last step into the Apple Vision Pro, I hope you will find the gem on that one two. I'm a developer and are looking into developing to the Vision Pro, but I haven't yet found the right project. Anyway, Thanks for you channel
These are really great tips! Thanks so much for this. I am absolutely going to start employing all these ideas. I too was amazed at how immersive the simple panorama photos were on the AVP. So happy I took so many over the years. But these new tips are going to be very handy for all my future panorama photos. Thanks again!
Yeah. All the panoramas already in your library are there, ready to go. That's nice.