Matt Celia: 15 Tips for Making 360 VR Video

- Hello, thank you so much for coming to this. So I tried to compile 15 important tips that every 360 filmmaker should know and the reason why I did this is 'cause I started a company called Light Sail VR a little over two years ago and we've been making a lot of content in the 360 space and a lot of it was terrible and some of it was actually pretty good and I've learned a lot that I wanna share with the community. So I have a lot to get through and very little time so we're gonna get right into it. Have a good story is my number one tip. 

At Light Sail, our mantra is 'story first', we even put a hashtag on it, #storyfirst. And here's what it is for me. A good story is having compelling characters, a conflict, and some kind of emotional resonance, it should make you feel something. That means it's touching, it's important. A camera placed in the middle of a forest might look really cool but it's not a story. So, one, have a good story. That applies to all media and I think you can tell stories in 360 and if you want it to be at that point I will be next door at the Workflow Suite afterwards. 

Two, and this is probably the most important thing that I can impart on you today after my experience which is to educate your clients. VR is not like traditional media at all, right? Traditional commercial spot, everyone knows, you're getting like an MP4 or ProRes file, you have your deliverables, gonna go out to you. They're like broadcast or like the internet but it's all the same, right? It's the same 16x9, 4K whatever file. VR is super confusing right now. The technology limitations are astounding and the distribution is even more confusing. You've got Facebook 360, you've got YouTube, you've got headsets, right? Totally different experience in both. So you need to design and think about how is your audience viewing it. What spatial audio codec are you using? Never mind if you don't even know what spatial audio is. But there's like quad-binaural, there's Ambix, there's the Facebook two bit years type format, there's Mach1, there's Dolby Atmos, and all these different platforms of distribution of course have different encoding requirements, right? Are you making an interactive piece, or are you making a linear piece? A lot of customers come to me, clients, and they're like, "Hey, I got this thing" "and I wanna put this person in the room with you," and I'm like, "That's an AR experience." "Did you mean AR or VR?" They don't know so it's your job to figure out what they want and figure out if they want an interactive experience it has to be an app. And I encourage people to ask who the audience is, ask who your audience is and guide them to success by offering something of value. We are, as post people and as producers in this new field, we are breaking new ground and it is our responsibility to collaborate with our clients, to create great content and something that everyone's really proud of. Hopefully you figure this all out before you shoot. 

Plan everything is my number three point. Take story board photos. I take this little guy. You know, I'm a VR guy. You'll know this 'cause I always have my headset and a 360 camera in my backpack. I take photos of all the locations that we do and I actually story board out as much as I can and I sit there and I look at it and I obsess over blocking and I obsess over all that stuff. This here is a photo of me using little game pieces to figure out blocking on a show. Don't wing it. You can't fix it in post. You know that idiom of like, "Oh, just fix it in post," and we all joke about how people say that? It's even worse in VR, 'cause like, you know, in traditional media you probably might be able to fix it in post. In VR, sometimes you just can't and there's so many dead projects all over the industry that just died because no one thought about this. So have a shot list, have an idea of how everything comes together. I float around Google Earth VR scouting before to make sure our locations are going to be right. Get involved as a post person early 'cause it's creatively important but also early 'cause it's budge-wise important. We're all trying to run a business and we're all trying to, like, not die, so you need to know what you're getting yourself into. 

Four, pay attention to the geometry of your space. VR is story telling through location. Augmented reality is story telling through people. You can define it any way you want but the most important thing in virtual reality is we're transporting you to a new place. That means that the distance your actors and your walls from the camera is very important. The blocking your actors should be more like a play. Watch your stitch lines, right? If you're doing monoscopic material, maybe you want to make sure your character is right in front of the lens. But if you're doing stereoscopic, the sweet spot is usually between two lenses. 

These are all really important things to know because they're going to make your life in post either easier or harder. Speaking of easier or harder, plate shots. Yes, that is a shot of plate. It's really easy to shoot a plate shot. I carry around a GoPro on a stick just like this, and what I do is I move the camera out of the way, and I, boom, I just grab a little quick shot of the ground. Do you know how much time this saves me in post? And how often people don't do this? And how much more expensive it is to get out the tripod when you don't? This is easy. Do it. Have all your crew, for example, when you're shooting a 360 project, don't move the camera, just have them all move to the other side. Or strike the lights and take a clean plate of the room that you can use to paint out lights and crew and stuff. And if you process those in the same way in stitching, it's as easy as making garbage mats. Very simple. Speaking of stitching, don't stitch every singles shot.  Guess what guys, online-offline workflows have totally returned with VR. We're no longer editing in full res all the time. We are basically making quick stitches. I use Mistika, very, very fast piece of software, I highly encourage you check it out. And I figure out what I want to use, and then when I have what I want to use, I just work on the pieces that I need 'cause every single shot is going to require a lot of composite work, a lot of plate work, it's going to be intense. 

Making proxies is my number seven tip because my last project, speak of the devil, was 6K, stereoscopic, each eye, right? So that's, what, 12K? I don't think that math kind of really works. 60 frames a second, it's beefy. It's pretty hard for a lot of machines to do that. I mean you could build a super computer and probably get there, but like a lot of us aren't going to have that ability. So, you know what, both Premier and Final Cut have amazing proxies, proxy support, use it. 

I also use nested sequences to stay organized, that's my eighth tip, because we end up having a lot of layers. In this shot, what you can see here, is that I've got eight channels of audio from my quad-binaural microphone, plus I've got three layers of audio from the lavaliers from my actors. Plus, I have a camera reference audio. And then even though I only have one video track on top, I'll probably have like a fine stitch that'll come in on top of that, or maybe some visual effects, or title passes. And when I'm working in my sequence, what I like to do is stay very clean and very organized, and using nests really helps me do that. So, some tips on that. One, if you're creating a nest, and you're syncing to your audio, make sure your first frame of video starts at zero, not the first frame of audio. This way when you grab your time code, when you're going back to do your fine stitching, it all matches and you don't have to go into the nest and figure out like what was the offset and all that jazz. I also like to put my ISO cameras in my nest just in case I need them for visual effects. So if I'm shooting on a Z Cam S1, it's got four cameras, I'll put all four individual layers on there so that if I need to pull out something special in the stitch, or for visual effects, or some kind of reference, I have them all time-code locked for me. And then I like to, you know, keep it all clean until I send it to the mix. 

Number nine, use rotate sphere to guide your audience. This is probably the most important groundbreaking thing I learned in editing VR 360 media. These circles are from a blog that Jessica Brillhart wrote up called, I think, "In the Blink of a Mind." And it is about her editing 360 video, it's on Medium, check it out. But basically, what you want to do, is you want to think about the attention of your scene in a spherical way. So if I have a character, and the character is walking from left to right, the audience most likely is going to pick up on that character at some point, and they're going to move their head, right? So if I end that shot looking right over here, what I now want to do, and I'm going to cut to the next shot, is make sure that the first frame of my next shot is, boom, right there. So that the incident cuts, they're looking at the most important thing. And then that character may walk over here, and so on and so forth. Since we don't always shoot like that, since we don't always know how we're editing all the pieces together, although you should as much as possible if you planned it correctly, this is why rotating the sphere to match those cut points helps hide the cut in a way. It's like how we match cuts in like 2-D. It's simple. And this is really effective for guiding the attention of your audience, and making sure they don't miss anything. I'll put this out there, if you have a shot in where there's no attention, and it's hard to know where you should look, I would consider rethinking that because I think people, when they watch that, they're going to have a sense of fomo, they're going to have a, "Where should I look?" "What should I do?" So, plan it and use that rotate sphere tool and guide it. 

Number 10, review everything in a headset. If you're going to a headset, even if you're not going to a headset, review in a headset. Let's be honest, 'cause 360 video on a phone, less awesome. 360 video in a headset, super immersive, life-changing for me. And, know this, your field of view is different on Facebook than it is on YouTube. Your field of view on a Daydream is different than Cardboard, different than Rift, different than Gear VR, right? If you want to be a perfectionist with your titles and all that stuff, you're going to need to make sure that you're always reviewing on this, and there's some really cool tools that can enable you to do that. The latest, you know, Final Cut has support for headset. The latest Premier has support for headset. There's Gemma with a rig that'll plug a Daydream phone out of any HDMI. That's pretty awesome. So check that out. Number 11, keep detailed notes in a spreadsheet. A lot of people I know, they use like Shotgun, or I've used Kanbanchi or Trello before, but like here's the basic things, right? I like to name every shot as a visual effect shot. 

So V-effects one, two, three, four, five. 'Cause every shot in VR is a visual effects shot. And then I keep track of the original shot's name, I keep track what stitch sequence it is in so I can find it, the time code in and out of my, that it, you know, kind of like an EDL, right? And then some notes about what stage is it in. Is it in the stitch stage? Is it in the plating stage? Is it color correction? And that'll go back and forth as I keep working on it. Every VR company I've ever done work for, and mine included, has a system that consists of something like this. I use Google Sheets, it works very well. It's really important to stay organized, 'cause there can be a lot of shots. Twelve, color grade with Scratch VR. I'm a Davinci guy, I was really a Davinci guy, and I still am for a lot of projects. But when working in 360, I switch to Scratch VR, and here's why. Because you can review your grade inside a headset as you are making it, which is very powerful, without having to output and load it up and check. And, you can use masks, windows, et cetera, whatever they call them, over the back seam line, right, without getting weird seams. Resolve, I mean maybe they announced it at the show, I haven't been by their booth, but it's not 360 aware, right? So if you're using Blur effects, Sharpen effects, if you're using any sort of power windows, you could get a seam line on that back, and then you're destroying the whole idea of presence, and that is unfortunate. 

So, Scratch VR, the interface is slightly intimidating, I won't lie, but they have awesome tutorials, and I highly recommend it. Thirteen, add spatial audio to your mix. Please do. If you don't know what it is, please research it. Look at this, here, I'm going to give you an example. You go out, you film a whole thing in 360. It's hard, you do all the video work, you're paying attention to the video, the plating, all that jazz. Why would you just show them a 2-D version of that? Like a center crop. The same thing is with audio. Why do you not take advantage of spatial audio? I get it, it's hard. There's a lot of complexity with the distribution on it. But, you know what? We hear in 360 guys. We only see forwards. So this is a really important tool. Facebook Workstation is free, it integrates with Pro Tools. If you don't want to do something like the Facebook Workstation, use a microphone like the Gear 360 eight ball, or even a Zoom H2n does ambisonic, and works like right inside of Premier. And now you can, with the latest update I think like, even monitor the ambisonic audio while you're editing, which is very cool. Use iFFMPEG or myFFMPEG, or FFMPEG if you're like a command line crazy person, which I'm not. 

These are the tools I use to output because they can output H.264 in resolutions above 4K. H.264 is not locked to 4K, that's a big myth, but the encoders that are with, you know, Final Cut and Adobe are. So you can get a little bit above that. You can also do H.265, and I find that their encoder doesn't wash out my colors. So, my final tip is probably the most important. Which is, ask yourself if this is better by being in 360. Right? Why are we making it in VR? Why are telling the story in an immersive way? If you can't answer that question, you should probably just make a 2-D film. It's going to look better, it's going to be easier, and more people are going to be able to watch it. But, if you've got that thing that is so ground breaking inside the headset, and transports you and really moves you, whether it's because of empathy or it's because of fear, or whatever you're trying to elicit emotionally, make it. Make it good because we need better content. That is my 15 tips in 15 minutes. I'm very impressed I got through all that, I'm sorry if I talked fast. I'll be next door at the Workflow Suite. If you want to come chat and hit me up about knowledge, I'm happy to chat. You can email me, matt@lightsailvr.com I'd give you a Twitter but I don't really use Twitter. So, that is the best way to get ahold of me. Thanks so much, we're all working faster together. Appreciate being here.