Re-framing the question: HFR is OK…Now can we finally do 30 fps?
The rate of 24 frames per second is insufficient for a high-quality motion imaging system. Period. There, I said it. I may as well start with my cards on the table. It's a painfully clear fact that 24 fps is inadequate for representing the motion of any faster movement than two people sitting and talking to each other. Most people probably have no idea of the trouble filmmakers have to go to in order to keep the illusion of motion from breaking down. Part of the art of a cinematographer is learning how fast you can move the camera, and how fast objects can move within the scene without exceeding the limitation of a succession of 24 still frames each second.
Directors such as James Cameron and Peter Jackson are definitely on the right track for wanting to improve the theatrical experience with an increased frame rate. But they're fixated only on improving the appearance of 3D. To this writer, at least, it's not entirely clear that High Frame Rate (HFR) is the right change to be made to movies to improve them—not 2D movies, and perhaps not even 3D movies. I'll explain why below.
But before I proceed, I think we need a working definition of what frame rate values comprise a “high frame rate.” There don't seem to be industry guidelines in this area. For the purposes of this article I'm going to call HFR any frame rate above about 40 frames per second. I'll call 24-25 fps “standard” frame rate (25 fps is a common motion picture rate in Europe and some other areas). Anything between “standard” and “high” frame rate, that is, between 25 and 40 frames per second I'll refer to as an “intermediate” frame rate (IFR).
Another thing to clarify is that higher frame rates aren't a new idea. Various people and companies have used increased frame rates for movies since the very inception of movies. The first viewer device that used flexible celluloid film was Edison's Kinetoscope in the 1890s. It showed its images at 30-40 fps. The rate of 24 frames/sec. was chosen around 1930 when projectors started to become motor-driven. Prior to that time, movies were all hand-cranked and there was no official standard for the frame rate. A specific value for the speed had to be selected in order to use motors. The 24 fps rate was determined by a survey and represented the average rate projectionists were hand-cranking movies at that time.
Within a few years after the 24 fps rate was selected for movies, professionals began to realize that particular rate wasn't high enough to show proper motion. Flicker from the projector shutter was also an apparent problem. It's no coincidence that the slang term for movies has been “flicks” for several decades.
Quite a bit of research was subsequently done by Kodak engineers and others on the relationship between flash frequency, image brightness, and the appearance of flicker. As a result of such research, several attempts were made to increase the frame rate for new motion picture processes. In the 1950s, Fred Waller's Cinerama process started out at 26 fps, and Todd-AO a few years later was originally 30 fps. These rates would be in the “IFR” category defined above. Around 1980, Doug Trumbull's Showscan process launched using a frame rate of 60 fps. Imax Corporation experimented with 48 fps for a film for the 1992 Seville World’s Fair, Momentum. So we see that the HFR concept really isn't new either; the only thing new is the HFR acronym/buzzword used to market the concept.
Soon after their debuts, Cinerama and Todd-AO both bowed to the pressure to be compatible with “standard” movie theatres and changed to 24 fps. Those rates, as well as 48 and 60 fps rates, can always be used as “special-venue” formats, but there was never a wide release of any feature-length movie using the latter two HFR formats prior to The Hobbit in 2012. Even that apparently wasn't as wide a release as originally planned; only a fraction of the showings used the 48 fps version. So although HFR really isn't new, it hasn't actually crossed over yet from a special-venue type of experience to become a mainstream one either.
However, the onset of HFR movie releases has revealed something fairly important: Other frame rates besides 24 fps are supported by the current digital projection technology. Most engineers in our industry were aware of this, other people perhaps not. The emergence of HFR has driven the point home: The 80-year shackles represented by the entrenched 24 fps rate and mechanical projectors are now broken! So this is a great time to reopen the question of a faster frame rate as a new standard for movies. I'm talking here about an improvement to all feature films, not just big 3D blockbusters.
HFR is being used on 3D movies to address a very specific problem resulting from left eye and right eye images having a time lag between them. A substantially higher frame rate reduces the lag and makes the 3D look better. Of course, in the process it also improves the appearance of motion and reduces flicker, although much more than really necessary from that standpoint.
That being said, as a first approach to improving movies in theatres one might consider simply adopting HFR for every movie, both 2D and 3D. However, many critics weren't complimentary toward the look of The Hobbit in HFR. HFR adds a deeply immersive, hyper-realism feeling to the image. Is this a desirable look for narrative, dramatic feature films? Most of the critics thought not. They dubbed it the “soap opera” look. A similar thing was found with Cinerama more than 60 years ago: The deeply immersive experience seems to work well for travelogues and documentaries, but somehow doesn't look correct to the eye for telling fictional stories.
When the viewer is presented with motion images at 48 frames/sec. or higher, the image takes on a feeling of immediacy, an impression that the action is taking place “live.” The screen seems to melt away and the sensation by the audience is the feeling they're looking through the screen and seeing reality taking place behind it. If your goal is an ultra-realistic imaging system, HFR is great! I loved Showscan—but as a special-venue process, a “novelty.” That type of deeply immersive ultra-realism may be desirable for a “4D” or ride-film type of experience. There may even be an occasional movie plot where having noticeably more intense imagery would make sense, perhaps a movie about a TV show, or someone having vivid dreams, where that hyper-realism effect might contribute to the plot. But the fact is, many people don't seem to be comfortable with the look of HFR for a standard narrative movie.
Too high a frame rate seems to conflict with the willing suspension of disbelief. There may be a correlation here with the “uncanny valley” effect that animators talk about, where the image isn't quite good enough to be perceived as actual reality, but is too good to be accepted by the brain as fake. An uncomfortable feeling results from the conflict. This is why I created the distinction between IFR and HFR categories earlier. IFR frame rates represent an improvement, but a milder one that stays below that threshold of visual discomfort.
We could spend a lot of thought and time agonizing over why this hyper-realistic look has this effect, whether it's something inherent about the human visual system, something about the movie medium, or perhaps merely a cultural conditioning that we'll adapt to in time...but I'm not sure the “why” is important. I'd rather focus on how to react to it. In the future, HFR may or may not become the de facto standard for 3D…time will tell. However, HFR is clearly not desirable for 2D movies.
So if 24 fps is insufficient and HFR is too high to use for every movie, what are we left with? The only practical answer is something in that IFR category defined earlier. The digital projection infrastructure will support many different frame rates. Conceivably, new movies could use a variety of rates between 24 and 40 frames/sec. On the other hand, there's a lot to be said for standardization; as there are already many different types of Digital Cinema Packages (DCPs) being distributed to theatres, it would be a benefit to the industry to try to constrain the number of variants to a manageable set. The data also must be stored and transported using practical, standardized interfaces. And although it may not be a concern to the exhibitor, the finished movie is going to have to be compatible with a DVD and Blu-ray release at some point. So there are some constraints on what frame rates can easily be used in practice.
Although 24 fps is just barely adequate to convey the illusion of motion, the good news is even a minor increase from that can provide a substantial improvement in image quality. From the standpoint of human perception, there's quite a large difference between 24 frames and even 30 frames/sec., for example.
The industry could embark on tests, demonstrations, experiments with focus groups and the like to determine what frame rate or set of rates would be desirable to use instead of 24 fps—but all this was already done more than two decades ago! There was an industry effort in the 1980s to increase the standard frame rate for all feature films from 24 to 30 fps. It started with a request from the MPAA to the Society of Motion Picture and Television Engineers (SMPTE) to look into this question, and a committee was subsequently formed to evaluate the benefit and the impact of such a change.
There were a variety of good reasons for this change being proposed, the main ones being to improve the appearance of motion and to reduce the amount of visible flicker from the film projector shutter. Another reason was to improve the compatibility of movies shot on film to later video release, eliminating the undesirable artifacts from the “3-2 pulldown” method employed to do the frame rate conversion from film to video.
Prints of existing 30 fps titles such as The Miracle of Todd-AO and Oklahoma! were dusted off and used by the SMPTE committee to demonstrate the improvement possible from the increased frame rate. New demonstration material was also produced which additionally incorporated the benefits of newer film stocks with finer grain. The demonstrations showed quite conclusively that increasing the frame rate 25%, from 24 to 30 fps, delivered a substantial improvement in image quality—more of an improvement than the modest numbers might suggest.
The SMPTE committee report completed in 1988 concluded that the benefits of this change were ample and evident. The drawbacks were few, but problematic. Some filmmakers had doubts about changing movies to 30 fps, and suggested that with the increased frame rate movies might look too much like television imagery (similar to the feelings about HFR). But of course, most television broadcasts in the U.S. show 60 pictures per second, not 30. And anyway, the demonstrations showed that a 30 fps movie projected on a theatre screen still looked like a movie, not like a television picture.
The main problem with making the change from 24 to 30 fps was the same one that had always plagued such proposals: compatibility. In the late 1980s when this change was being considered, there were around a hundred thousand theatre screens around the world, and all of their mechanical film projectors would have had to be converted in order to implement this change. Worse than that, since not all movies would change overnight, the projectors would have to be capable of being shifted back and forth between 24 and 30 fps for some period of time until all movies had made the transition. This was simply too difficult, too expensive, and impractical to implement. So despite fairly broad industry support for the concept, the proposed change was eventually abandoned as not feasible.
That was then, this is now. When the only movie distribution medium was film, trying to implement a different frame rate was a huge challenge. However, technology marches on, mechanical film projectors are essentially gone today, and the compatibility problem is now gone along with them. With the advent of digital projectors, a new frame rate is no challenge at all! It makes absolutely no difference to the projector what the frame rate of the material is. It can even change during the program. For example, trailers may be one frame rate and the feature another rate—it makes no difference to the projector.
Despite movie projectors changing from mechanical to digital, nothing else has changed about the reasons for needing to use a higher frame rate—24 fps is still just as inadequate now as it was in the 1980s.
We now have second and third-generation digital projection systems that arguably produce better images than ever came from a 35mm projector—and maybe even 70mm. Digital cinema rolled out over a decade ago with 1.3 megapixel images, then advanced to two megapixel, and then to eight-plus megapixel. From the standpoint of resolution, the images are now wonderful! But we still show the same number of them per second that we have for 80 years!
In conclusion, it's my firm contention that feature films should adopt a higher frame rate for production and exhibition. That rate could be 30 fps, 40 fps, or another value in the “IFR” range. But it probably shouldn't be higher than that, for the reasons discussed (at least for standard, mainstream movies). For many reasons including existing hardware limitations, standardized interfaces, and home-video release formats, 30 fps represents a value that would be extremely easy for the industry to implement very easily. DVD and Blu-ray standards already support 30 fps content today. Many consumer-level digital cameras even shoot HD video at 30 fps today. I think it's an embarrassment that the frame rate used for commercial motion pictures is lower than offered on consumer-level DSLR cameras! And it's an embarrassment that we still haven't shaken off the historical remnant of 100-year-old mechanical film technology represented by the 24 fps frame rate.
If theatres want to maintain an even higher level of experience than offered by 30 fps, they could put pressure on the studios to support a new custom format that would be unique for movie distribution to theatres. For example, the value of 40 fps is one I’ve already mentioned. (Elsewhere, I've written about photographing movies with a 120Hz capture rate using a method that would allow the extraction of 60 fps, 40 fps, 30 fps or even 24 fps masters from the same original photography.)
But for the immediate future, let us suppose theatres adopted 30 fps for general movies. The 3D blockbusters could still use 48 fps or 60 fps HFR, although I'd posit that in a world of 30 fps cinemas the 48 fps HFR format really doesn't make much sense, since the only reason for choosing that particular number is for better compatibility with a 24 fps release. If we eliminate 48 along with 24 fps, we are then left with just two frame rates to support, 30 and 60 fps. That's a realistic and manageable set of values to deal with.
HFR was created to reduce the artifacts with 3D movies. It does that, but it also creates the undesirable “soap opera” effect. It's also fairly challenging from a technical standpoint—it's pushing the limits of the hardware. If the choice is between 4K resolution images with an IFR rate, or having to reduce to 2K resolution images in order to have an HFR rate, I'd vote for the first option, no question.
It would keep things simpler if we didn't have to deal with a high frame rate for every run-of-the-mill movie. So let's suppose we used 30 fps for 3D also...that won't mitigate the 3D artifacts as well as HFR, but it would do a 25% better job than 24 fps does, and has been doing for the last decade of 3D releases prior to The Hobbit. It might eliminate much of the need for higher frame rates entirely.
Getting back to the original reason for HFR in the first place, the left/right frame-lag problem, there are other technological solutions to the problem besides just boosting the frame rate until it starts to disappear. For example, one solution is dual-projector 3D systems, where both left and right frames may be projected simultaneously and continuously. That projection method eliminates the time-lag problem and eliminates the need for HFR and the unnatural-looking motion that goes along with that. Many theatres are already using two projectors for 3D anyway in order to get the required light level on the screen. Dual-projector 3D at 30 fps would be more than adequate for a premium 3D presentation, it would look amazing! It wouldn't have the objectionable motion artifacts, and it wouldn't have the hyper-real problem either.
The reason we have 4K digital-cinema projectors available today is because theatre owners demanded it. They demanded that the cinema experience had to be better than what consumers could get in their homes with HDTV. That requirement has now been satisfied, at least with regard to the number of pixels per frame, but not with regard to the number of pictures per second. That's the next frontier, the next weak link in the cinema image that should be addressed.
Thirty frames per second is enough of a boost to be a noticeable improvement, but small enough that it wouldn't be jarring like HFR is. It could also potentially represent a stepping stone. Once the audience accepts and becomes used to the mildly improved 30 fps IFR experience, additional increases in the frame rate could come later, to 40 fps or even eventually 60 fps, and it might be better accepted by being gradual. The change from 24 to 30 fps is a 25% increase. A change from 30 to 40 fps would then represent an additional 33% increase from that. And 40 fps to 60 fps would finally be another 50% increase. With apologies to the movie What About Bob?, the key to a successful future acceptance of HFR could very well be “baby steps.”
Fox, Jesse David: “What the Critics Are Saying About The Hobbit's High Frame Rate.” Dec. 14, 2012, Vulture.com. www.vulture.com/2012/12/critics-on-the-hobbits-high-frame-rate.html
Richards, David: Temporal Oversampling: A New Paradigm in Moviemaking. May 1, 2011. Self-published, copies available to interested parties on request.
DiGiulio, Edmund: SMPTE Study Group on 30-Frame Film Rate: Final Committee Report on the Feasibility of Motion Picture Frame-Rate Modification to 30 Frames/sec. J.SMPTE 1988, V97:404-408
David Richards is VP of engineering at Moving iMage Technologies. He spent several years in engineering and management positions at Christie prior to becoming a co-founder of MiT in 2003. He has worked in the cinema industry since 1985. He has been active in the SMPTE for most of his professional life, having served on several engineering committees, chaired conferences, and is the past chair of the SMPTE Hollywood section (’96-’97). He is the author of several papers and articles for various trade publications. He has a background in mechanical, electronic and electrical engineering design.