NSCAD associate professor, Solomon Nagler (left) and Andrew Hicks (right) won the top design prize at the 2014 Mobile HCI conference held in Toronto. (Photo by Eliot Wright)

It was at Expo ‘67 in Montreal that Czech director Raduz Cincera introduced the world to interactive cinema with his film Kino-automat. Screened in a custom-built theatre, audience members used a basic push-button voting system to decide the next scenes at key moments in the movie.

Since then, movies have shifted from 35mm to digital, from big screens to mobile devices and desktops, opening up new possibilities for moviegoers to interact with film. Rapid advances in interactive media, and the advent of wearable tech, have likewise inspired media artists and designers to rethink not only how we tell stories, but also how we, as viewers, can participate in their unfolding.

At NSCAD University in Halifax, associate professor Solomon Nagler and film grad Andrew Hicks have broken new ground in this area with Narratives, a mobile app that gives users an entirely novel way to both create and experience cinematic works through an exploration of physical space.

Harnessing GPS data on mobile devices, Narratives tells interactive stories based on where and how you travel during the day. On the creation side, the application allows filmmakers to test, assemble, and publish interactive, geo-locative video online, stitching together cinematic stories based on direction, place, and time. Viewers then “discover” these stories on their mobiles. Aided by the app’s interactive compass and map, users expose hidden narratives revealed as they take a stroll through a park, or turn a street corner.

“As you rotate your body, you see a cascade of images scroll across the screen that allows you to see, in each direction, a hint of what video would be triggered if you were to go in that direction,” explains Hicks. “The idea is that if you’re walking through the city, and you take a number of rights and lefts in a particular order, the app is going to reassemble the video clips in a unique fashion. It’s a tool for conceptualizing the way a narrative can be taken off a screen into real space.”

The app can also work in the background, tracking users’ movements throughout the day to create unique storylines they can review at any time. Completed or in-process storylines can be posted online for other users of Narratives to view and compare their own unique narrative journeys.

Unlike other geo-locative cinema, which is triggered by a particular location on a map, Narratives plotlines can unfold according to the twists and turns of the journey, allowing anyone in the world to “traverse” the narrative. It was this fresh approach that helped the research team win the prestigious Design Competition and Future Innovations Award at Mobile HCI 2014 in Toronto (September 23-26, 2014), the top venue for leading research and innovation in mobile and wearable technologies.

“We want these stories to exist anywhere in the world. So, the same story you could experience here, in Halifax, Nova Scotia, would be the exact same experience you could have in Vancouver, or Tokyo – it’s all based on your direction over time,” says Hicks.

“As a filmmaker or storyteller,” adds Solomon, “this universality is of absolute importance in terms of how you’re going to structure your story. You want your film to play anywhere, and not necessarily the city that you live in.”

Though initially focused on filmmaking, the researchers came to realize other industrial uses of Narratives. Health researchers, for example, might use a similar app to encourage walking and physical activity to counter the ill effects of sedentary life. Geo-specific tourism and the design of lesson plans focused on student travel offer other potential uses.

“In the health and education fields, the app’s universality is of paramount importance – that we can transfer to various industrial applications that don’t have that specificity that geo-tagging currently has. That’s what makes our application quite different.”

Beyond locative data, researchers plan to incorporate input from other sources, such as biometric sensors, as ways to navigate and interact with the narratives. The use of wearable technologies, such as interactive watches or glasses, to augment the cinematic experience, is another growing area of interest.

“The question you have as a media artist is: how do you integrate your media with this new technology? How can you use the media to interact with the way data is collected?” asks Solomon.

Funding for the project, which began in 2012, was provided by SSHRC (Social Sciences and Humanities Research Council of Canada) and by GRAND through AD-Node (Art and Design Node), an initiative involving NSCAD University, OCAD University, and Emily Carr University of Art + Design to promote collaboration between artists, designers, and researchers in the GRAND network.

Now in the early stages of commercialization, the researchers are looking for potential partners to develop the app, as well as filmmakers to contribute new video content. If all goes well, they expect to launch Narratives for use on iOS and Android devices in 2016.

Through AD-Node, the two NSCAD researchers worked with computer scientists from Dalhousie University to develop the app’s functionality. For Solomon, initiatives such as AD-Node are important for artists and others in the creative economy to engage with new technologies, and to push the scope of artistic practices.

“We are foremost artists, and as artists, we think of commercialization in a pragmatic sense in that we want to reinvent the tools of our art. Because GRAND gave us this opportunity to develop the technology it really catapulted [the project] in a direction that was quite surprising and very fruitful.”

 

MEDIA CONTACT

Spencer Rose
GRAND NCE
778-370-1004
[email protected]