New York Film Festival’s Convergence program sees the potential of virtual reality


Returning for its sixth edition, the Convergence section of the New York Film Festival spotlights alternate formats that point to the potential for movies in the future.

Running Friday, Sept 29, 3 to 6 p.m., and Saturday and Sunday, Sept. 30 and Oct. 1, from 12 noon to 6 p.m. in the Elinor Bunin Munroe Film Center, the event is free and open to the public. But expect long lines for many of the titles.

Dark Corner Studios drew the most attention during a Sept. 26 preview. With three titles at Convergence that explore horror and virtual reality, Dark Corner has positioned itself as a genre specialist in the VR industry. Guy Shelmerdine directed the newest title, Night Night, receiving its world premiere this weekend.

Shelmerdine also directed two other titles showing at Convergence, Mule and Catatonic. For Night Night, he used a custom camera rig designed in collaboration with Radiant Images.

In conjunction with Convergence, Dark Corner is announcing a new app that will provide access to curated content. The platform currently includes eight immersive experiences.

Also showcasing an app is WITHIN, the Los Angeles-based VR/AR entertainment and technology company. The company is currently giving viewers free access to some 40 curated VR titles through its WITHIN app.

WITHIN is screening two episodes from a five-part series called Look But With Love. Directed by Sharmeen Obaid-Chinoy (a two-time Oscar-winner for the documentary shorts Saving Face and A Girl in the River: The Price of Forgiveness), and produced by SOC Films and Here Be Dragons, the series examines people in Pakistan who are trying to help society—by providing free medical care, for example, or by preserving cultural traditions.

A Story of Women follows women in Nowshera as they train to battle insurgents. Because men are not allowed by custom to enter the houses of strangers, women are the front line of defense against terrorists. Obaid-Chinoy films them testing weapons, rappelling down building façades, and explaining their motives for fighting for their community.

The second episode, A Story of Dance, takes viewers inside rehearsals and performances by a troupe preserving traditional art forms. It's a chance for Obaid-Chinoy to stretch out, to test the possibilities that virtual reality offers.

As executive producer Jonny Ahdout explains, "Sharmeen had a very quick understanding of what it means to formulate creative thinking in virtual reality. We had a couple of conversations early on with Chris Milk, our company founder, about what it means to interview people in VR, where you want to position the camera, how you want the spectator to feel."

Ahdout acknowledges problems with current VR technology. Both moving the camera and editing are difficult. So is directing the viewer within the VR image. "You want to find the balance between giving people the freedom to look around where they want, but also attracting attention to what is taking place," he says. "With subtitles it's particularly difficult, because you have to piece them into certain parts of the screen. Newer technology will allow subtitles to follow wherever you're looking.00

"We use 3D sound a lot to place a sound in the environment where we want you to look," he goes on. "There's also staging, where if you have an action taking place that's moving from one side to the other, people generally follow that. Something we'll do in cutting is rotate scenes. If you have an action that's ending here in scene one, and the next action in scene two is starting over there, we'll rotate the entire scene so that they overlap on top of each other."

Editing in VR is a major hurdle compared to flat filmmaking. "Brains don't accept quick cuts the way that they do in filmmaking," Ahdout contends. "So we use long takes where you can feel really present in that moment. It's difficult, there's no rulebook. We just know we can't do it the way filmmaking does."

Another problem is the length of time viewers can tolerate VR. Somewhere between five and fifteen minutes seems to work best with documentary pieces and linear stories. Ahdout thinks viewers can stay inside interactive pieces for much longer stretches.

Shooting VR introduces a whole new set of problems. The VR rig is like a ball of cameras that point in every direction. "You can move it, but it's a difficult and delicate process," Ahdout says. "You have to imagine the camera as a human head, so you can't do handheld shots where you're running around because viewers will get nauseous. For moving shots you want a constant state of speed, you don't want to accelerate or decelerate too quickly, and you want to be on a Steadicam-type rig that keeps everything very smooth.

"We joke sometimes that it's like setting a bear trap. You start the camera and then you go run behind a tree hoping that you get something good."

Look But With Love is the second documentary series that WITHIN has developed and will distribute on its app starting Oct. 5. Obaid-Chinoy knew the subjects, suggested individual segments and put the series together, and was on the ground in Pakistan with another filmmaker from producing company Here Be Dragons.

"The VR industry as a whole has done a great job exploring documentary," Ahdout says. "But the technology we're developing right now is geared more toward social responsibility, interactivity, finding solutions.

"As a platform, we are looking to curate the best and most immersive storytelling content. The goal right now is to show why VR is worth your attention, especially when so many people don't really know what it is. Right now, content isn't necessarily curated like Netflix, which you know beforehand has a collection of great movies and TV. We're doing the same thing with the WITHIN app. We're building a collection of VR content, so if you're new to VR you can visit our app and get a breadth of immersive experiences to try."

The seams show through in Sanctuaries of Silence, a New York Times Op-Docs production directed by Adam Loften and Emmanuel Vaughan-Lee. A profile of acoustic ecologist Gordon Hempton, the piece follows him as he records sound in the rainforests of Olympic National Park, along the Pacific Coast seashore, and on 46th Street in Manhattan. Like a fancy New York Times Magazine cover story, Sanctuaries of Silence starts with a debatable premise—Hempton's belief that our last refuges of silence are being overtaken—and "proves" it by cherry-picking facts and ignoring whatever doesn't fit his argument.

Contrasting a forest and midtown Manhattan isn't much of an intellectual stretch, but it's worth pointing out that forests and seashores are hardly silent. It's not until the end of the piece that Hempton admits his definition of "silence" is more poetic than scientific. And although much of the photography is pretty, annoying artifacts disrupt almost every scene.

Other VR pieces at this year's Convergence include Virtual Virtual Reality from Tender Claws, a creative collective that produced the Full Motion Video game PRY. Virtual Jockey by Vizor (Finnish visual artists Fthr and Lintu) uses Vizor Patches software and Oculus Rift headsets to create individualized virtual worlds from scratch.

Arilynis an augmented-reality installation that brings paintings to life and turns everyday objects into interactive videos on your smartphone. To experience the piece, users download the Arilyn app and look for symbols that will initiate content.

With Gamescape: The Revenge of Full Motion Video, users explore a new generation of FMV, once a mainstay in games like Space Invaders and Donkey Kong. It was created by the NYU Game Center and curated by Clara Fernandez, a Game Center associate professor who specializes in storytelling. Gamescape unfolds on iPads, and if you're not conversant with old-school gaming, it will likely leave you at a complete loss.

On Sunday evening, the Columbia Digital Storytelling Lab will present De-Escalation Room, a live session that will let viewers become participants in developing an immersive storytelling project.

Saturday night's session will explore a new virtual-production toolset developed by Lucasfilm and Industrial Light & Magic. Rachel Rose, Jose Perez and Nick Rasmussen will be on hand to explain how the toolset can help filmmakers build virtual sets and create virtual storyboards. Viewers will be able to experience the system firsthand, provided they aren't at the end of the line.