Kommer Kleijn, on cinema frame rates
In this second part of our interview with Kommer Kleijn SBC, who recently was given the IMAGO International Honorary Member Award, we will dig a little deeper into the subject matter of higher frame rates for cinema (HFR). If you did not yet had a chance to read the first part of this inteview then you can click on the colored link just below.
In our previous interview you mentioned the downsides of 24fps that viewers have become accustomed to. Can you list those?
KK: The major disadvantage of 24 is that the picture becomes “blurry”, low definition in fact, as soon as there is movement. With even the slightest movement this is enough to make a 4K camera virtually useless. The only way to make it sharper without creating strobe effects is to shoot and screen at a higher frame rate. This issue is not a small imperfection: it is quite an important shortcoming that hinders progress of image quality. Using a 4K camera at 24fps doesn’t really make much sense. It may send a message to the world that you are using the latest technology but in actuality it doesn’t enhance the image quality that much. Moving cinema frames shot at 24fps are rarely above 1.5K, regardless what camera is used. In fact, frames cannot exceed this definition during movement or otherwise unacceptable strobe effects appear. 24 is so faraway from the speed of human vision that the risk of judder at 24fps becomes so high that we must arrange for all images that contain movement to be blurred. This is done practically by exposing each frame for a very long period of time, much longer than any stills photographer would ever do. This ‘solution’ is both simple and yet genius: by exposing for 1/48th of a second, as soon as anything moves, the resolution of the image is automatically scaled down proportionally by motion blur: the more an object or the camera moves, the more the image is blurred. 90% of the 24fps strobe problems are avoided that way, but unfortunately, and this is often forgotten, this also severely limits the resolution of the image itself, because the images are, well, blurred. On the other hand, if you would shorten the exposure time to sharpen the image spite the movement, then the judder becomes unbearable at 24fps, unless both the actor and the camera remain completely still, which obviously makes no sense in cinema. This is also the reason why camera testing on a fixed chart doesn’t make much sense in my view: it fails to measure resolution in the condition the camera will be actually used: with subject and camera in motion. The only solution to this contradiction is higher frame rates. Only that can give us movement and high picture resolution both at the same time.
So the effects of 24fps we have become used to are that the picture becomes low definition as soon as there is movement and also some residual strobing. We often unconsciously associate these artefacts with ‘cinema going’ as the TV medium never had these artefacts. TV has a temporal resolution of 50 from its very inception (60 in the Americas and Japan).
What would your thoughts be about the philosophical debate that the essence of art is not about hyperrealism or that technical achievement is not the purpose of art?
KK: He who can do more, can do less. I’m not trying to prescribe what to do, anyone who prefers 24fps can keep on using it. It is a particular style, which suits certain stories very well. I am simply suggesting that higher frame rates should be allowed to explore higher resolutions, and allow actual progress in image quality for those who wish to experiment that. I have never asked that lower frame rates be forbidden. Similarly, if one wants to make a movie in Super8 blow up, even though most cinemas have 35mm projectors, I don’t have any objection to this. On the contrary, what I would find regrettable is if we would restrict those who wish to film and project in 35 because some believe that Super8 is more dreamy due to its imperfections….. I propose we can choose what is best for our project.
I also note that this debate arises after almost every major technical advancement. For instance, when colour film stock appeared, some black and white DOP’s were opposed to it because it would be too realistic. Also their wonderful creative know-how, enriching black and white images with lighting techniques compensating for the lack of colour, could become redundant. But colour was not only a technical advancement, it actually enables to tell more, provided it is used as a narrative tool. But this required changes in the way of working and that requires some research and learning which also always encounters resistance.
Mankind sometimes seem to be programmed to resist to change no matter the advancement; sound, colour, digital, 3D, higher frame rate, true HDR, time domain information in sound reproduction, there always is resistance at first, often arguing it is too realistic, while I think the fear is caused by it being different. But you cannot make something better without changing it. Amongst DOP’s but also directors, actors, producers, a whole industry resists as each change demands for habits and working methods to be redefined.
Sometimes one has to return to school, which can be unsettling at first, so people invent other problems. Not only is there fear of the new technology but furthermore it needs to be put to use successfully. In order to do that, one has to create a new language, which is not yet written. This is not apparent to everyone. In fact, what HFR offers is not an attempt to perfect an already existing medium, but to add communication tools which can create a new, superior medium. If used properly, it will be stronger than the previous one, but it will also need to be worked differently!
What comes with higher frame rates?
KK: It allows genuinely augmented image definition in film. I am talking about image sharpness, image resolution, rather than motion definition. The latter is also enhanced but is not the main focus here. In fact this development actually changes things quite a lot! Whether you are director or an actor you are well aware these days that while the actor is moving, their facial expressions won’t be the sharpest, even if all they are doing is speaking. Therefore both director and actor approach their work with an awareness that they can afford to be less precise when a shot contains movement. Enhancing the spatial definition during moving shots dramatically boosts how much narration one can tell in the span of a minute. Directors who will manage to integrate this to their workflow open up infinite new possibilities of storytelling. Obviously this is true for action movies but not limited to these. Also with moderate movement, the resolution gain is significant. The consequence is that you have to offer new content in front of the camera. It needs to be valid, serving the narration and bring emotions and beauty. This doesn’t appear on its own. Using a high frame rate camera to shoot exactly like before is a waste. That is as if one picked up the newest color film stock, back when color was the latest innovation, and just shot grey scaled or ugly toned sets and costumes, to complain in the end that color film is awful to look at. When shooting color you need to present/create good color subject matter, useful for the story, in front of that camera. When shooting HFR High Resolution you also need to create appropriate content. This is the difficulty. New camera technology requires a renewed creative process along with it. This is often misunderstood and met with resistance, as many people are reluctant to study new subject matter after they finished school, or change their habits…
In my opinion, image sharpness during movement is the most crucial improvement, but it is not the only improvement with higher frame rates. HFR also allows for innovative camera moves traditionally avoided because they caused strobing. At 24fps the blur created by the long exposure eliminates most of the strobing but not all. There is still a range of camera moves and subject movements that remain ‘forbidden’, as we all know (the 5 / 7 seconds rules). This now goes too. What a relief. A host of new creative possibilities in camera moves becomes available, almost entirely free of restrictions. New forms of cinema language will emerge.Another little known secret is that until now when filming a close up shot, we have to blur the background and we say this is a genuine artistic choice, but actually it is a technical constraint, as it is necessary to avoid strobe artefacts.
But stopping down the aperture does help to increase the depth of field?
KK: True, but actually, the shot will be unusable if we do and end up in the bin. During a tight shot, if your actor moves and the camera also moves to follow him, and the aperture is closed down to keep both the background and the foreground in focus, then the background will strobe so much that it would distract from the actor’s performance. So obviously it is o.k. if the background is just a plain white wall, but as soon as it is more detailed, the shot could become unusable. The only way to solve this at 24 fps is to blur the background, which makes it not an artistic decision but a technical imperative. At higher frame rates there will be no strobing so you can actually choose, you could still blur the background if you want, but then it is a real artistic choice, not an obligation. In Above Us All everything is always in focus. We do not direct the attention of the viewer with the focus, we let him/her choose freely where to look. Additionally the movie is 3D so we provide stereoscopic depth clues, no need for the focus to do that. But even without 3D, do we really need to direct the attention of the viewer towards the talent using focus? If we worry about the patrons looking at the background instead of the main subject, then maybe we should worry about what movie we are making …
To conclude and return to your initial question about the benefits of higher frame rates, one could assume that when the frame rate doubles the exposure time gets divided in half and so we get double the spatial definition. In fact, it is more effective than that, because when shooting at 50fps, one can lower exposure time as much as desired without worrying about strobe effect. So doubling the frame rate can give us 4 times, 8 times, etc more sharpness. As far as we wish and the lighting budget goes. And thus a new artistic tool becomes available with shutter speed. A well-known setting to play with for stills photographers and not available to DOPs because they were restricted to shooting at too low a frame rate. This parameter, shutter speed, will now become available to cinematographers too, as an artistic tool, after being withheld from us for a century!
I wrote my thesis about using vintage glass in the digital era and I already have the feeling that many DOPs find 4k image too detailed, lacking the rounder texture of photographic film which is why they tend to use that kind of glass. What’s your opinion about this?
KK: I think the reason for this use of glass might possibly be something else. 4K can become counterproductive in some cases. If the shot is stationary and the actor moves in the foreground, then the set becomes sharper than the actor, which obviously does not serve storytelling. I believe this is the reason why there is a love of filming with vintage lenses and filters, because at 24fps, 4K only enhances what does not move (the set) and not what does move (actors). The DOP attentive to storytelling has a right to blame the 4K image for being too sharp (the set) distracting from the actor’s performance (suffering from motion blur, not enhanced by 4K). Making the whole picture fuzzy with old lenses is a relief from this issue. In a way this is perfectly legitimate. One could wonder, however, whether it is still useful to pick a 4K camera. What I propose (HFR) is an answer that allows the image as a whole to be sharp. Sets and actors. So that actors remain our main focus, even in 4K.
However it has been argued that the detailed definition of skin texture might reveal the cast’s imperfection as we adopted high definition?
KK: Yes, as mentioned earlier, each significant technological progress does also change the established workflow. Higher frame rate images bring higher definition so indeed one may not get away easily with filming cheap sets or casting actors in their forties to play roles in their twenties. Inevitably there are consequences. Everything appears more detailed, so more relevant contents and care must be put in front of the new camera.
Are there downsides to shooting at a higher frame rate?
KK: One downside we can see is the infamous “Love of artefacts”. The work does not display the previously mentioned anomalies from the past that we have become attached to. So we need to provide value in return. But creating this new value requires new work methods, so film makers need to adapt which is also a problem. Both these problems are temporary though. We should also mention that it generates a bit more data. Honestly this is both true and not true at the same time. If you think about it, 4K does not add much in terms of definition at 24 fps so you can also use higher frame rates whilst going back to a smaller resolution in order to limit the amount of data. In the end shooting at 50fps multiplies the data by two whilst 4K multiplies it by four. But 50fps/2K actually looks sharper than 4K/24. So if one is scared of costs increase then we might be temporarily better off using 2K. Although there are people who are not scared of data: Ang Lee shot Billy Lynn’s Halftime Walk at 120fps in 4K 3D, which means 240 4K frames to process per second (40 times the normal data rate) which is obviously quite something. But ultimately it was all downscaled to 24fps before distribution. He and Peter Jackson also committed the error of recreating a similar motion blur as traditional 24 fps. This is also a reason why their HFR movies do not display much visible enhancement.
It seems to me that “The Hobbit” was shot at 48fps. In your case you mainly spoke of 50 and 60. Is there a particular reason?
KK: The benefits that come with high frame rates are already achieved at 48fps. Visually there is not much of a difference between 48, 50 or 60. All three can give good results. But 48 works on the digital cinema projector only. 48fps does not work on DVD, neither does it on Blu-Ray or broadcast TV. This will not please your producer. So for example, all “home entertainment” versions of ‘Hobbit’ are 24fps.… In contrast, 50 and 60 are not only good in cinemas, they also work on DVD, Blu-Ray, and on TV … so they have practical and economical advantages. 48 has been defended as allowing for easy 24 fps reductions. But if you actually use the advantages of the higher rates in storytelling, then a 24 fps version will not be possible at all. So the only supposed advantage of 48fps over 50 or 60, is actually moot.
From what I understand high frame rates are achievable in camera and from cinema projectors, and even at home, so what else needs to be transformed, apart from use and practice?
KK: Nothing more technically! All done, although it is recent as it has only been for one or two years that everything is set. In fact five years of international militant activism were needed for higher frame rates to be included in the DCP standard, and then the same amount of time for it to be implemented in the mastering and projecting tools. Then we also had to wait for 35mm film projectors to disappear, as they are limited to 24 or 25. But only recently have all three factors become commonplace. What needs to happen now is for film-makers to do their part. To jump onto new discoveries and impactful experiences, as revolutionary as the advent of color.
More about Kommer Kleijn can be found on http://www.kommer.com