Marc Bach: The Art of HDR

- Thank you. How are you doing?

- [Audience Member] Good.

- So, we're gonna talk about the art of HDR. And what I want to do with this thing is instead of talking about specifics of how you set a value in there and get inside of the application, I just want to put it out there that you guys can understand what it HDR is and things that you have to think when you are actually doing it instead of showing you numbers and values and things like that. But, the question is that before we get into this thing I need to show you or tell you about specific things for you to understand or to set the base for understanding what we're gonna talk about. So in this case, when HDR started, we just came from what HDR was before in color space that what you can see in this graphic is basically what your eye can see. So this is the total spectrum of what your eye can see. And in this triangle you see here, is just what Rec 709 that it was up to now the standard for broadcast. 

Then, we expanded that to what it was P3. So they added more colors and more saturation for those colors so we had more values basically for that color space. And then we went to what we have right now that is Rec 2020. So that's the bigger triangle that we see right here. But this is just the amount of colors that we see. Just think about it as if when it was black and white you were painting with only two crayons. You had the black and the white. Right? And then we came out with Rec 709 or even before that and we gave you like 20 crayons. And then we gave you 100. And now we're give you 1,000. So, before it was pretty easy to color somebody and to say like this should be looking like this. And that's the image we have. But right now for let's say skin tone you would grab an orange, because you only had one. But now you had 20 oranges to choose, you know? And then you can fine tune that way more. But that's just the amount. 

What happens with the volume of those colors with SDR we had from 0 to 100 nits right now. And then we changed to HDR to be ten thousand nits. That's where the spectrum is of the volume of the colors. So the amount of colors that we saw right before, they can be as bright as ten thousand nits keeping saturation. Then in here, it's the artist trying to get rid of the problem. Before where shooting with all these color spectrums in here with all these contrasts that this is what your eye can see, the camera is capturing that. But then we're reducing all the way down to when we see it in he screen. And then HDR is trying to solve that by trying to keep it's maximum of that information all the way til the end of the pipeline. So the big question is, how do I color HDR? How do I think about what can I do with the image if I haven't seen it before? Like before with HDR you can go and you can pop any movie. You can grab a blu-ray, you can go to Netflix, you can watch something and you can see it right away, right? But with HDR you don't have that many references. Now we start to have but a year ago, when I started doing this thing or a year and a half ago there wasn't any references. 

There was like maybe 20 movies. And there was a big question, how do I color this? Do I have to make it brighter? Do I have to make it darker? Do I have to expand the image to fill the gap all the way from black to white? Or what's the deal with this thing, right? But it was a question all the time in my head, like okay, I'm gonna go bright and then I will go in and make it dark. And then the next day I would come back and then I would change it to the opposite than I did before. But then I realized that you have already seen HDR. Life is HDR. So when you're in real life you're seeing right now in here maybe I'm under a light that's maybe 7,000 nits, 8,000 nits. And you guys in there are like 10 nits, maybe 50, 100? So these are examples of how real images with real values look in real life. So you can see here in the first, biggest image in here you can see that we can go up to 7,000 nits. And in the same image you can go down here where it looks like it would be the darkest part of the image is 300 nits. This is taken with a photo mirror at the right spot of taking the picture. And the bottom one, you can see that the wall of the parking structure, is hitting 17,000 nits. So, how do you represent all this into that container of zero to 10,000 nits? 

How am I gonna map this thing and put it inside of that to try to reproduce or to have a nice balanced image, right? You have to keep in mind a bunch of things when you are doing that. And it's that the human eye can see a total range of 24 stops. But what happens is that we can only see 12 at a time. So your eye is adapting when you are like in my case, you are seeing the image the opposite way that I'm seeing right now, I have these lights in my face I can barely see anything in here. Because my eye is adapting to this amount of light and then I do see through the blacks. But maybe on your side you can see more through your surroundings then I can see because your eye is going up and down those 12 stops and adapting to that. So that's important when you're color correcting for HDR because you have to think that you can go all the way up and make the image real bright and maybe you'll wanna think that bringing the blacks all the way down to the bottom to be pure black is going to be a nice image. But what will happen sometimes is that the glare of the highlights is not gonna allow you to see through those blacks. So maybe you better bring them up and see through the blacks if that's what you want. Keep in mind also that everything that everybody talks about right now is about 1,000 nits. But this spec is for 10,000 nits so, these values everything that we're gonna talk in here is gonna change throughout the next years when we're gonna have you know, brighter screens. And then we're gonna keep bringing it up and up and up. This is a break up of what the values should be. And you can see how many steps and out there, right? We can only see 24 and then 12 at a time. 

So how you map this thing to display it into the screen right? It's easy right now because we only have a 1,000 nits. So pretty much it's like around 12 to 13 stops that we can see at a time. But we're gonna have brighter screens like 2,000, 4,000, I don't know, 8,000 nits at some point. Sony already has one at 10,000 nits there was a demo at, I think it was CES. But you're gonna have to do that. What happens to your eye in nature, right? So you have to think about what do you wanna do with the image, where you wanna bring it, right? And one of the keys is that the human eye adapts faster to light than to darkness. So when you transition from a dark environment to light that's faster when you go the opposite way around. So what happens if I have an image, let's say it's a scary movie and your outside and it's shining and then you wanna go inside and it's dark. Do you want people to see through? Do you want people to not see anything? So then they are scared? Or the opposite, seeing more is gonna make you more scared? So your gonna think how that transition from going from bright to dark is gonna happen. If it's gonna be instant or are you gonna have a couple shots that's gonna bring it down little by little? And then you get to the tone where you wanna be like in the darkness. 

So that's one of the things you have to think about. Coming with that is this, it's like what do you want the viewer to feel? Do you want them that? To feel scared to see, to not see? To feel like blinded by the light? If you go the opposite around, where you go from a dark environment to a bright one. You wanna shock them? In this case for example, if you have an image like that what you wanna do, you wanna feel you're in the concert. Or you wanna feel you're viewing the concert through a screen. So if you wanna be in the concert you would go all the way up with lights. And then you would barely see anything in the shadows. Versus, you would color this to see it. You would try to squeeze all that information inside so people could see through the blacks and people can see all the highlight power. Another thing that you have to think about is how your content is gonna be seen. If the content is gonna be seen in a controlled environment like this one, just imagine that this is a movie theater. Maybe you wanna see you wanna color your whole thing darker. Because your gonna be able to see through the blacks. Or if it's gonna be seen outside, like it's a mobile device or something like that, so you wanna do it the opposite. You wanna bring it up a little bit. So you're gonna see, I'm gonna show you some values and you're gonna understand why I'm saying all this stuff. 

But this is the main thing, when you can control the environment your gonna see the blacks, you're gonna see all that stuff. So maybe you wanna dig deep into the shadows. Or if you know your content is gonna be seen outside, or it's on a mobile device or something like that, you wanna be going higher. You know, so then people are gonna be able to see better. Because you're gonna see the blacks. Another thing is it's interesting to know, for example LCDs and OLEDs display the color, this difference in different ways. So OLEDs can be brighter, no sorry. OLEDs are not as bright as LCDs. But they can display the blacks better. And then LCDs are brighter but the blacks are not as deep. So your doing this. So for example if you wanna buy a TV and you wanna have it in an environment like this you would like to buy maybe an OLED TV. Versus, if you're gonna have a TV that is gonna be outside like, let's say in a coffee shop and it's gonna be out there, maybe you wanna buy an LCD because it's gonna be brighter. So, when you color your content you have to think these things also. So right now the recommendations are this ones. And you're gonna see that up until here, this is almost the same as what we have been doing now. When we are coloring stuff we just skip to mid tones in here. So that would be around five nits. 

Look how low is that compared to the whole scale up to 10,000 nits. And then the blacks, they tell us that black should be around 0.2 nits. And then you have above mid tones that that's where mid tones is where skin tones should be, it should be between this and this so we should be around like 15 nits, something like that. So if you look at the scale, you're actually living everything for the highlights. So you go here, broad highlights, that would be like a sky, the clouds in a sunny day, something like that. You're gonna bring it up to like 200 nits. And then specular of highlights that would be a reflection of like a watch or something like that reflecting the light to the camera, that would go 500 and up. But then we're clipping it at 1,000 nits today. So this is what they recommend. But it's not strict. So if you're coloring or you're planning on color, you can take this as a reference. But it's not strict. You can move it round. Blacks are not blacks all the time. HDR, it was so small that the difference between something being bright or not, it was so small that we had to play with that so you would always try to stretch the contrast all the way to top and bottom. But now we have all this range and then we can make something dark without having to be pure black because the brights are so high that that give you the contrast. So that's the interesting part of HDR is that you're not locked to that small range. It's so big that you can move up and down. Remember what they eye does. 

But remember this is just a recommendation. You can do whatever you want with the image. It doesn't have to be exactly what it said in there. Because at the end of the day, we're doing just art. And it could be any other one. You know this is Da Vinci then we have Picasso in there. Which one is better? It's up to the eyes of the beholder.