The Reality of VR: Tech innovators ponder virtual reality and immersion’s role in the theatrical space

Features
Technology

Just as computers, the Internet, digital operations, and spicy tuna on crispy rice were once unthinkable for theatres, so too has Virtual Reality’s (VR) entry been a surprise. While IMAX begins setting up its headset-based VR viewing pod installations in theatre spaces, the groundbreaking, near-VR “whole shebang” presentation of Billy Lynn’s Long Halftime Walk, from Sony/TriStar and two-time Oscar-winning filmmaker Ang Lee, briefly snuck in as a main-attraction feature. (Beyond some screenings for industry groups, the “shebang” version hit only a handful of theatres but had a wide traditional 2D release.)

The positive response to Lee’s venture (a first-ever capture in 4K 3D at 120 frames per second with that same playback except for a few 2K projections) was huge. Variety reported that “the images have an astonishing clarity and physical presence…the format changes everything” and the “extraordinary technology” has the potential to be “revolutionary.” U.K.-based IHS Technology consultant David Hancock commented that, with Billy Lynn, “we’re moving toward an immersion that we haven’t had before…so real with its immersion and immediacy of the images, and the level of detail…triggers our emotional response.”

The film’s high frame rate (HFR) was the most unusual technical aspect and has as one of its most illustrious proponents a Hollywood icon, the two-time Oscar-winning writer/filmmaker/producer and VFX/high-tech guru Douglas Trumbull. He thought Billy Lynnwas “very, very close to virtual reality for the big screen,” but also found what Lee dubbed the “whole shebang” a little less than whole.

Trumbull explains that HFR can slip into a “soap opera” or TV look. In an argument familiar to cinephiles, most movie audiences, he believes, seek instead what he calls the elusive look of “cinema” and they can easily tell the difference between film and television. But increasing the frame rate brings issues related to shuttering, as did use of the two digital projectors which the exclusive engagements required for each showing.

Traditional shuttering delivers the cinema look, which he calls the “terrific aspect” of the standard single 3D Christie Mirage projector he used when he demoed for Lee several years ago his patented Magi process of 4K/3D at 120 as Lee was preparing Billy Lynn. But Lee and Sony went their own way with dual digital projectors for the 3D showings.

The fleeting 120 fps experiment would have been significantly diminished were it not for the 3D, the remarkable sound (whether Dolby Atmos or Surround) and dazzling laser luminance. The film’s brightness, in fact, caught the attention of Richard Welsh, SMPTE education VP-elect and CEO of Sundog Media Toolkit. Said Welsh, “Everyone knows the headline that the native format of Billy Lynn is 120 fps 3D 4K, but one of the details often missed is that it was also graded for 28 fL brightness projection,” tech-speak for the very bright, vibrant image delivered, especially compared to the much lower light levels of 3D. “Needless to say, combined with the high frame rate and 4K/3D, audiences have been blown away.”

It would help the cause if we could give a name to this unprecedented marriage of the very latest tools, equipment and technologies for optimum motion picture capture and presentation. But this tech combo still signals a high mark, if not yet a turning point, in cinema history.

However different (and they are!), both headset VR and Lee’s “whole shebang” deliver that much-valued amenity theatres covet: immersion. Headset-based VR has that in spades and has the head start. As covered in our December 2016 report (“Getting Real: Virtual Reality Experiences Eye the Movie Theatre Environment”), IMAX is betting big on bringing VR installations (pods) for the headset set into multiplex spaces worldwide but outside the auditoriums.

Whether VR or cousins AR (Augmented Reality) and MR (Mixed Reality), the platforms and their technology provide immersions into artificial environments where users explore as if “wandering” inside their headsets.

VR applications are already promising in areas like real estate and education, but for the entertainment crowd (gamers especially), the headset and accouterments can be costly, so it’s no surprise that VR sales have disappointed. A late 2016 Nielsen Media Lab team study noted that despite much excitement, “many VR devices aren’t widely owned.” And the subhead of a Jan. 9, 2017 New York Times article read: “Hailed as the Next Big Thing, a Technology is Hampered by Sticker Shock and Complexity.” VR was even an unexpected disappointment at the recent CES international trade show, where The Wall Street Journal reported, “Alexa Wows, Virtual Reality Underwhelms,” signaling triumph of a different “VR” (Voice Recognition), as in “Alexa, what movies are playing in theatres near me?”

Yet it seems that every day, everywhere, municipalities, universities, film studios, and companies like France’s M2K are joining the noisy headset-based VR party with new divisions or labs. Even new kids like Amazon Studios have pushed into VR content with the hire of former Tribeca Film Festival director Genna Terranova to oversee the effort.

As for VR in theatres, Rob Lister, IMAX chief business development officer, who heads the company’s VR initiative, says installation of its VR Centres in theatres continues as planned (a first look was announced for January) with its unique in-theatre experiment of viewing pods for the “next generation” of entertainment. Mostly original gaming and movie-related content will provide the theatre crowd with “immersive, multi-dimensional VR experiences” at around $10 a session. (Initial offerings include “Star Wars: Trials on Tatooine” and “John Wick Chronicles.”)

Not to duck from marketing potential, IMAX content might be “add-ons” extending main feature presentations playing IMAX’s giant screens. Lister calls the initial IMAX sites “pilot” locations “intended to gauge user appeal and other important information.” Nor ducking from the financial potential, he says that “[The Centres] will provide a way for the masses to get the VR experience and pay us for it.”

At least this kind of VR in theatres is getting a chance. At least in one case, the immersive Billy Lynn didn’t. Longtime arts journalist Jamie Laughlin in her late November Dallas Observer article covered what could have been dubbed “Billy Lynn’s Fiasco in Dallas,” an unintended one-day playdate preceded by no promotion.

In the article and an interview with FJI, she questions why so important a film from a technology standpoint and artistically so from an important filmmaker revered by the art-house crowd was booked into the “out of the way” suburban AMC Village on the Parkway 9, given virtually no promotion or press outreach (she accidentally found a mention on mashable.com), and yanked after only a day.

The equally in-the-dark theatre’s manager, who saw the film, called the experience “fantastic” and “had really put his heart into pulling this thing off and even trying to arrange a Saturday morning screening.”

Laughlin writes that even Dolby people were on hand “the entire previous week to test the equipment and software package, troubleshoot and assure a smooth run for Lee’s work in this groundbreaking, aggressive way.” But after its Friday debut, the film was run out of town, quickly replaced by Fantastic Beasts and Where to Find Them.

Laughlin contacted AMC’s director of corporate communications Ryan Noonan, who responded that the film’s premature exit happened because the theatre received “unexpected access to the new wizard monster movie at the last minute.”

When she pressed him regarding the lack of promotion, he cited the mention on Mashable.com and a small blurb in the Fort Worth Star-Telegram as “proof of national and local coverage.” She then quips to readers, “What, you didn’t see that?”

She also writes that Noonan, addressing the fact that “only a handful of tickets sold all day…described the failure as a ‘supply and demand issue.’”

FJI’s request for a reaction to Ang Lee’s amazing technological feat yielded this response from AMC: “The emerging presentation technologies are intriguing, and we remain open to working with our studio partners to allow guests the opportunity to enjoy the most advanced theatrical experiences available.”

Sony Pictures Entertainment spokesperson Jason Allen, having been repeatedly asked for input from Sony/TriStar regarding this technological milestone which Sony helped make possible, ducked with the response that “[The inquiry] is challenging because the promotional window on this movie [Billy Lynn] has essentially passed and we are now looking to the home-entertainment release.” Oh.

To be fair, Billy Lynn, beyond the lack of properly equipped theatres to showcase it as director Lee desired, had challenges from the get-go: an unwieldy title, a genre identity crisis (war, satire, romance, sports, medical drama, art house, mainstream?), and no stars as leads. It’s a strong, beautifully crafted and performed emotional drama, but try selling that one, marketers!

But might this innovative technology still get off the ground? No investigative muscle need be flexed to know that high costs, bureaucracy, fear of change and just plain fear will get in the way. As Trumbull puts it, “Industry resistance takes many forms—from studios and exhibitors. Any standards change is seen as disruptive and costly.”

Enter SMPTE, the Society of Motion Picture and Television Engineers, the industry’s standards enablers who know things. If the business’ complex, time-consuming transition from analog to digital cinema across the movie spectrum could happen, why not a giant leap to vastly improved movie presentation?

SMPTE past president Peter Ludé, also chief innovation officer at RealD, says “to move forward, the industry, meaning the studios and filmmakers, need to decide what new level of quality is desired, and set new standards that manufacturers could implement.”

About digital cinema’s emergence, he observes that “it was only successful because the studios subsidized the capital expense of digital projectors through the Virtual Print Fee program” and what Hollywood would be saving in print costs, shipments, etc. “But now this program is over, and there will be no further studio subsidies.” This and lack of “shebang” product leaves exhibitors have little incentive to install better systems.

Also slowing the rollout of new technologies are standards issues; it’s hard to budge those in place. Ludé explains that DCI and SMPTE specifications “already define a maximum of 60 fps 3D at 2K resolution,” so most manufacturers aren’t motivated to produce “over-specced” (high-end) equipment “without a clear industry consensus that this is what is desired.”

As happened with Ang Lee going 120 fps, standards issues also bump into “a case of the creative leading the technology,” says Patrick Griffis, SMPTE executive VP-elect and VP of technology in the office of the CTO at Dolby Laboratories. Lee pushing the frame rate so high yet retaining 4K and 3D was “such a dramatic step” that current (DCI) projector specifications had not contemplated them.

And where, Ludé asks rhetorically, is that “steady supply of movies that can exploit the upgraded format”? (At least one is in the pipeline: Lee is going “shebang” again with his next project, Thrilla in Manila, about the controversial 1975 final heavyweight boxing match between Muhammad Ali and Joe Frazier.)

On a hopeful note, SMPTE’S Welsh acknowledges the high costs for theatres to deliver the full shebang (high-end specialized playback servers, special-venue laser projectors and a lot of custom interfacing) but believes “it’s only a matter of time before this becomes commercially viable in a standard theatre.” And Ludé cites HFR as easing the way to the whole shebang’s adoption because it is downward-compatible with traditional 2D and eases artifact and blurring, strobing and judder problems affecting picture quality.

Headset VR confronts bigger problems in terms of any marriage with the big screen, and the nature of storytelling is at its core. Since motion picture stories need to dictate where viewer eyes focus, how does material determined by viewers get edited, even written?

Steven Spielberg echoed this digital-era storytelling “Catch 360°” in a recent Daily Mail piece: “I think we’re moving into a dangerous medium with virtual reality. It gives the viewer a lot of latitude not to take direction from the storytellers but make their own choices of where to look.”

Yet, VR’s biggest obstacle to the big-screen home is that it condemns users to a state of acute isolation and solitude, the opposite of why people go to the multiplex.

However different, both headset-based VR and the “shebang” technology both excel at that most coveted amenity of immersion. Auditoriums already offer a broad array of immersive amenities—enhanced seating, viewing, audio playback—and more and better are inevitable.

The addition of immersive sound, says SMPTE’s Ludé, has been simpler and the trend now boasts more above and around speaker locations and new “object-based” sound technologies like Dolby Atmos, DTS-X and Barco (AuroMax) replacing legacy “channel-based” surround systems.

Screen innovations trying to out-immerse IMAX and PLF (Premium Large Format) are the wide panoramics from Belgium’s Barco with their Barco Escape and Korean-giant CJ CGV’s ScreenX. The former is what Barco sees as delivering “brick and mortar VR.” Its DCI-compliant Barco Escape system comprises three cinema projectors and two additional cinemascope screens, making the three screen panoramic spread a broad canvas for filmmakers to fill, whether for an entire movie or select moments.

ScreenX is similar, featuring the multi-projection solution, the auditorium screen in front flanked by two screens, all providing the wide, panoramic 270-degree viewing experience. 

Barco Escape CEO Todd Hoddick said that the format accommodates both features and short-forms, either features in their entirety or natively-captured select scenes.

Theatre seating, and now even recliners, enhance the immersive experience with motion, vibrations and sensory and effects jolts (hey, it’s raining in here!). Foremost in this area are Québec-based D-BOX Technologies Inc. (which made crossing on a sky-high wire in The Walka little scarier), Torrance, CA-based MediaMation, and Seoul-based CJ 4DPLEX’s 4DX, the latter two leaning more towards theme park-like fun, includes 4D motion chairs, 3D surround sound and high-quality projection. Additionally, environmental effects like wind, rain, fog, lights, bubbles and scents can ramp up the on-screen reality. Capabilities are carefully synced to screen action and seat controls personalize audience experience.

Looking ahead and into the VR realm, D-Box VP Marketing Michel Paquette observes that the ultimate VR-enhanced experiences are about these personalized environments. The trick regarding the theatre experience is in deciding what is more important: gimmicky effects or “a highly believable, unforgettable experience in which you really feel like what you’re seeing is real.”

Beyond seating, sound and sight, Trumbull, driven by “an overriding purpose,” has a broader plan to get moviegoers out of living rooms and into theatres. “TV is eating movie’s lunch,” he says, “and the forces of TV are reducing movie audiences so we need to offer even more spectacular and immersive experiences.”

To this end, he has developed, patented, and built his Magi project, which is “pioneering the future of immersive digital media” and the optimum way to create and present it. His focus is on “the nature of the motion picture experience, which I have been trying hard for years to change,” he says.

Beyond his patented “Magi Moving Image” 3D/4K/120 fps technology is, most intriguingly, the Magi Pod, a clever and sensible re-configuration of the movie auditorium, which is a curved space of greater width and less depth and the logical complement to the giant curved screen in front, meant for “the finest 3D ever seen.” The Pods are designed as either pre-fab structures easily dropped into existing multiplex spaces or standalones to function better than small cinemas.

Trumbull’s Magi project dovetails perfectly with what Bardan (a distributor/integrator for the Caribbean and Latin America) CTO Daniel Benitez told FJI for its latest Technology in Theatres section: “The interesting question long-term is: If we can design our theatres without the constraints of a traditional projection booth, what else can we do that we couldn’t before? As an industry, if we can focus on that, we’ll do fine.”