After observing an audience member’s reactions for just a few minutes, the system is able to predict his or her facial expressions for the rest of the film using a pattern-recognition technique that functions similarly to a recommendation engine; it can generalize the reactions of an entire audience, and measure those reactions against an input that states how viewers “should” be reacting.
“If data analysis of audience response in real time is used to customize content, either as revision or as new unfolding content, it will create a tighter loop between what studios and producers view as successful, marketable content in the context of audience demand,” says Siobhan O’Flynn, a digital storytelling consultant and University of Toronto instructor.
But as O’Flynn points out, “scanning and archiving facial data in theme parks is a whole new level of privacy incursion and erosion.” And needless to say, once Disney brings this out of the theatre and into real life, you can be sure that other brands will be eager to get in on the action, too, measuring our facial gestures and modifying content in real time.
So whereas up to now, our data has been collected to better market toward us — who hasn’t noticed all of their online ads change after they start researching a particular topic in a browser window? — that data could be put to a creative use, changing the shape and execution of a story.
But using this new process, Disney is able to measure the audience’s facial gestures so they can match those reactions to specific scenes — even frames — of a film.
The main objective of Disney’s FVAE technology is to help the company quantitatively gauge an audience’s reaction to one of its films, down the road, it may be used more creatively: such as to change a movie’s ending, based on a viewer’s response to certain scenes.