What is in a number? Thinking about the Participant Index and the recent New York Times article announcing its launch

The New York Times published an article announcing the Participant Index, framing it as a way to determine ‘"Why One Film Inspires Activism and Others Falter"

The Index should be great news for those concerned with, and working in, social impact media. We need tools and measurements to help shape our work, and to report back to funders and supporters. 

But, the idea that a film that scores high or low on the Participant Index (which has clearly been well-researched and carefully constructed) seems to presume that the impact of a film can be measured via one small group of internet-savvy Americans within a relatively short-time frame.  If so, then it risks failing to capture the beautiful complexity of storytelling and social change.  

Yes, there is a utility to mapping the pattern of human response to different types of media, such an index is a welcome addition to the field of media impact assessment.  We need easy to use and affordable ways to capture audience response across multiple platforms and levels of engagement. If the Participant Index provides a shared framework so that lots of folks can reference the same set of numbers, then it is truly a service to the entire field of media makers, social justice organizations and funders working together to advance positive social change. A kind of Nielsen rating for social change media is awfully appealing, it reflects a valid desire for simple, easy to share and understand assessment tools.

But let's not confuse media impact in the form of audience response––especially the highly measurable clicks, shares, likes and even donations–– with long-term impact and real social change.   Measuring the impact of social change media requires multiple tools, often customized to the issue, the type of change that is sought, and the intended audience of change-makers (which may not be the general film-going public.)    

What is lost when the both storytellers and the social sector start working towards quarterly results reporting, echoing mega-corporations earnings reports?  Will outcome measurement trump nuanced understanding of the human experience?  Facing the reality that social change is often excruciatingly slow, will media impact funders focus resources where we they can most easily score high, and achieve a big short-term impact?

For example, Jehane Nouajaim’s The Square captures a specific set of people at a moment in time akin to our own 4th of July.  The long-term impact of the Arab spring is impossible to know, and the white men that signed the Declaration of Independence could not have foreseen the diverse and enduring American democracy of today. More pointedly the film resists judgment tied to social action here in the US: there are few obvious social actions to take in support of democracy and human rights in Egypt.  The majority of Americans don't know Mubarek from Morsi, so simply seeing the film has a value in terms of raising global awareness.  If those viewers were to share their interest via social media, all the better.  But the most important audience for the film is on the streets of Egypt, are they included in the survey? 

Comparing a film about food, about which the vast majority of Americans make daily choices, with The Square, seems a precarious framework. Obviously film subjects that affect us personally are more likely to elicit a response. We know intuitively that stories that affect us emotionally are more likely to generate an action.

To add to the complexity, consider that media both reflects and affects social change.  Consider the following scenario: a wrongly convicted man avoids the death penalty and is set free. That's a victory for human rights.  The action reflects years of work on the part organizations such as the Innocence Project that advocated for change.  That organization and its allies have contributed to social change.   If the story goes unreported, it still happened.   When a film such as Deadline (Chevigny and Johnson, 2004) is shown on national television and raises awareness about shortcomings in our judicial system and the reality of capital punishment, that's media impact.  The film may have empowered activists and advocates that lead to improvements in the justice system, that would be be major media impact and social change. 

In looking at media impact and social impact, we must acknowledge that stories which elicit an audience response in the short term are not necessarily those that will lead to the most significant impact.  Let’s be careful not to focus resources on short-term gains in lieu of long-term tough challenges. 

If indeed the Participant Index accurately reflect audience response to audiovisual media, and it is widely available, it will prove a valuable tool. 

But in the end, we need to recognize that the impact of films such asThe Square may not be felt for decades.  A single numerical figure does not reflect the value of a cultural artifact that may inspire others who dream of democracy, and one that will help ensure that history is not forgotten.

Patricia Finneran
9 July 2014