Duncan Shepherd: 21 Tips for Editing 360 Video

- Okay so were really just gonna focus on the things I've learned in VR because there's plenty of mistakes I've made and as you can tell from the title I'm trying to kind of encourage everybody else to avoid them. So I'm gonna rip through this stuff fairly quickly. Firstly, frame rate. VR often you can produce things at 60 frames a second or you can do them at 30 frames a second. Obviously a 60 frame a second file is twice the size, so decide early on whether you really need a 60 frames a second output. As long as the VR headset or the viewing system is updating at 120 hertz, you get a nice, smooth panning effect and you probably don't need to go all the way to 60 frames a second. That's one thing.

 Project dimensions. We're doing a lot of work with the Google Jump platform at the moment. That's a 57, 60 frame and stereo so that's 57 60 by 57 60. That's a pretty big size frame. But you can half that when you're working in your edit down to 28 80. To make your rendering times whilst your in the offline a lot faster. And that means you can get your cuts approved by people because you're not spending hours doing renders. So that's a little tip there, and then in final cut you can actually upscale it back at the end of the project back to 57 60 if you're fine with live roles.

Okay, next one. Proxy workflow. Same thing with the Google Jump system because the file size is so big you are inevitably gonna have to be proxying inside Final Cut or certainly that is what I find to the most easy way to work. In fact, what normally happens, Google will give you lower quality footage you can start off with but you will be eventually switching it to the high quality stuff. And they only provide the option to give you the entire take all the time. So the way I deal with that is I put everything inside compound clips then when I need to online something I just drop the high quality footage into my same compound clips, upscale the project to 57 60 and press export and then go home. Look at it in the morning.

We'll fix in post. In VR, no you won't. Basically, it's really really hard, especially in stereo. I mean, there's plenty of stuff you can do in post. There's a team in in Jaunt that uses new, There are plenty of other tools that do that do that sort of thing. But you know, if you can solve a problem in a VR shoot without having to fix it in post, great, do that. You know, find some mechanism to avoid, you know, normally the problem is you can see the camera operator.

Well, if that is a problem, find some way to make the camera operator part of the scene. Or you know design your story in a different way. Same thing with the rig removal, the Google, E camera doesn't have a downward facing camera. It completely removes, you know, about 80 percent of the hard work people are doing trying to get rid of drones, trying to get rid of footprints, and tripods and things. Think about that before you shoot it you know do you even need to see the image down at the bottom.

Nobody's looking down there anyway if they are, they're just incorrect. Stereo. Is it really necessary for you to do a job in stereo when you're working in VR or 360? It may well not be. 90 percent of your views will be on Facebook or on Youtube. Those people aren't looking in stereo and the complexities involved in all the post production process in stereo are 4 to 8 times harder. I mean, you won't be able to patch stuff easily. You won't be able to kind of, a lot of the rig removal problems are much more complex in stereo. So have a think about that.

Do I really need to be in a stereo workflow for whatever story you're trying to tell. Or whatever thing you're designing. Jump gamma. Again, the Google Jump system has a proxy if they deliver you lower res proxies of 28 80x28 80 and then the high res which are suppose to be online with. Those high res files have a gamma problem which needs to be corrected when you basically we convert it to pro res to get a higher quality.

Just because in Final Cut it just works so much simpler in pro res but you will need to perform a correction. Which I can't particularly tell you about but somebody else more technical than me does that at Jaunt. But essentially the black levels are off in the high quality footage coming out of the jump platforms, so be aware of that if you're using jump for any deliveries.

Audio panning, in final cut and in premier pro you can reorient the camera view so you can bring the action back to the center of the, you know, the center of the frame. When you're doing that you need to remember when you're going to go out to our sound mixer he's going to need to know those numbers, because otherwise his ambisonic recordings will be off by whatever degrees you've moved the camera by so you know, bear in mind that you if you're doing a lot of panning you're going to have to go back to the cut, copy those numbers down and don't annoy your audio mixer. Because really the reason I put this is, in because my workflow I use an app called X2, I can't remember what it's called now, it basically goes out to pro tools. I think it's called X2proaudio.

Basically that bakes out an AAF file which contains all the audio elements and it throws them into different tracks of people. I recently did a job at Jaunt where the AF file had 97 tracks which annoyed my audio mixer, so try and think about them, their job is already really, really hard if they're audio mixing in spacial audio and they've got a lot on their plate so be kind to them, don't annoy them. Don't annoy your VFX teams, same thing, the workflow turnover has to be simple enough for them to be able to work without constantly having to go back to you write emails, what's the time, what's the impoint, is there a reference, think about that side of things.

Make it easier for them, they have, they're expensive, they're on tight timelines and their jobs already difficult. So do whatever you can to make that slightly simpler for them. Quite often they don't even work with time code, they work with frame counts so you need to be mindful that all the things we take for granted like, EDL's and XML's may not work in their ecosystem.

Mixing stereo mono, there's a paradigm that says you should never mix stereo mono footage, why not? It's actually fine, quite often we will use we will use the stereo, a beautiful looking stereo shot early on in a piece and people will think oh I'm watching stereo and then if we need to go to mono cameras cause it's an action cam or some other kind of reason we're using mono's people stop noticing it, it's quite ok to go from mono to stereo, it doesn't really interfere with the vision too much, so I would recommend doing that. Stereo Titles, when you're working stereo, obviously objects you're filming have depth and some objects can be quite close to you.

If you're putting titles over those objects it's really important to make sure the title appears in front of the closest object, if not you'll get what's called a stereo conflict and it will render the piece sort of unwatchable makes you get a little bit cross eyed but everything, you start to get a lot of double images basically your mind cannot resolve the paradox that it is seeing. Be mindful of that in stereo titles, that's also useful to check those in a headset view. You can't really see if the stereo is really conflicting incorrectly or if there is a stereo conflict without looking at sight stuff in a headset.

Producers best friend, this is the the old plan for simple frame of mind, producers best friend that's the way that I get the information out of final cut and give it as a list to basically the VFX teams. It's a pretty useful piece of kits, but it does have one caveat which is if you're mixing frame rates in VR quite often some of the numbers can be a little bit off so for example if you've got a 30 frame a second time line but you're throwing in 50 frame a second footage or 25 or if you mess with the frame rates it can upset producers best friend so you will need to check if you're using Producers best friend as part of your VFX pipeline or your turn over pipeline. Alex 4D, another platform another friend of mine. He has made some plugins that I find indispensable primarily there's a little masking tool that is very useful in final cut pro so if you are going to be working in VR check out his plugins, he is a very nice chap.

Don't be timid, there's a lot of momentum towards making VR things quite calm and I think quite boring actually in a lot of cases. I come from a commercials and music video background where you were always trying to be exciting and like wow all the time, that's also true in VR. Just cause it's VR don't relax and think oh yeah, it's special enough, it's not, make it really interesting make these things, make them fun as if they were a TV commercial or a music video. Don't pan the camera, you can reorient the camera inside the editorial tools and you can key frame that. Most people don't do that because it does make you feel a bit nauseous, however if you've got an interesting character or actor walking through the frame, as long as there's something that will drag the eyeball you can avoid the nausea problem, it won't work in every case but it can be done, I've seen it done, I've done it myself. 

You can pan the camera in certain conditions but there has to be something for the viewer to be locked into so they don't just feel slightly sick. Check in the HMD, I think I touched on this with the stereo titles, even with 2, 3 years experience I still catch myself out because I haven't been looking in the headset all day long and then I look at the cut and it's a completely different experience, so you know if you have access to the headset, great, use it all the time. If you don't it's becoming much more easy to attach them to max, the prices are coming down. So I would thoroughly recommend if you're really serious about working in VR and immersive reality then get the headset, it will pay for itself.

Deliverables, there is often different kinds of deliverables you might have Youtube, you might have Facebook, you might be delivering to a phone you might be delivering to all sorts of things, so you are gonna have to be on top of your specifications for the deliverables. I can't particularly speak to which ones I have to do all the time because Jaunt has its own platform for basically baking out all those different flavors but there is a website called purple pill where they cover this stuff in quite a lot of detail.

There's plenty of resources out there but you know you will have to know, again your frame rates, your file sizes there's often naming conventions for samsung phones. Things like that so it will be important to kind of check on that before you get stuck into a job and you're on a deadline and you haven't done that, you can get unscrewed up. Colour correction, VR cameras initially using redcam or something like that to do a kind of back to back fish eye thing, the files are normally not heavy weight enough to do really good color correction. We're still waiting for pro res raw camera or some kind of capture system where we can get a much more robust signal out of the cameras to do really beautiful color correction, as opposed to just balancing it out and making it not look, you know yellow or green or whatever it is that's coming out of the jaunt.

So that is a thing to bear in mind if you're used to working with cannon D3's or cannon whatever, any other camera you've got a lot more control over the color correction later. With VR cameras what you're shooting is pretty much what you're gonna get, you don't have so much latitude. Directors. Be nice to them, they're having a hard time in VR, I mean probably they're not really used to it. They are often very overworked, I like directing them they're great fun, usually the end result of any job that I do is almost always, well is always improved by the collaboration between the editor and the director, it's like a marriage you know, you hate each other, you love each other you get on with each other you fight.

And often you spend a lot of time up late at night as well so you sort of see the worst of that. Creativity, I keep telling people this, I think it's worth stressing, and I did touch on it a little bit earlier, it is worth stressing that just explore your craziest ideas if you can in VR. I like to mess around in it, I think it's more of a surreal art form than it is a real art form. I think that it's really exciting to just really, you know put regular footage into VR footage, put in the headset, look at it like it's in a giant IMAX theater.

There's all sorts of things that people can be experimenting with to keep pushing the whole format forwards. So never let anybody say, oh you can't do this in VR you can't do that in VR. You know what you should do, try it, look at it in VR and if you like it, that's great, you're the ultimate arbiter of your films, there's no one else, I mean you've got no other way of judging whether somethings good or bad so, you know, just go for it. Be your own boss. Selecting shots, I got this from really my mentor when I started doing commercials, there's a guy called Pete Goddard, who is a excellent film editor in London.

He said basically look, if you're select, I used to over select everything, I would like, pull the piece I wanted and a bit extra just in case but we're using NNE's now, you can always just go back to, you can match frame back to the clip or you can extend it out or whatever. Just select the part of the shot, that you think is going in the final film, don't select any extra. That's not really selecting, you're sort of filtering. If you do that, you will have select rolls that are a tenth of the size it's fantastic, and then your job from that point onwards is much faster, you've put together a great cut. 

he other thing to remember when you're selecting shots is and I also got this from Pete, so we were talking about commercials but it's equally true for VR. If you find in any project you've got, if you find the best, lets say you've got 3 hours of footage, if you find the best 30 shots, right, if you put your bar so high that you're only selecting 30 shots those are always going to be the best 30 shots.

They make the show out of those shots, because then nobody's going to be saying well have you got a better take on that one? It's a much faster way of working than it is to try and build the story from the script and put in well that's not really a great set or focus or there's some problem with that shot. Just find the 30 best shots, make that into the story and then if you have to modify that to follow the script or do something else then do that, it's really easy in a nail, it's not gonna be slowing you down at all. So, that's it, that's the end of my dissertation on things I buggered up so you don't have to.