Here’s what Bluefin Labs is trying to capture for mass media content providers, be they media companies or brands: The word-of-mouth about TV shows and commercials that has migrated to the social media space. It wants to make visible the feedback loop that is very evident in live human communications – be they one-on-one at the water cooler or speaker-to-audience on the lecture circuit – but which has been essentially invisible in the world of mass media for decades.

“The value proposition of broadcast and mass media is that you can magnify your audience and reach,” says Bluefin founder and CEO Deb Roy. “The primary cost is you lose the feedback channel.” Instead of talking with each other or with a speaker, listeners are talked at. And the speakers lose the immediate connection with the listeners that helps them understand if they are communicating effectively.

Providers such as The Nielsen Company have long offered services that track how many people were seated to watch a particular show and something about their age or gender, for instance, but that’s just the tip of the iceberg. “Most people still see mass media as a one-way shotgun into the dark, and they hope to hit as many people as they can and that approach is and not where the future lies,” says Roy.

Where it does lie, he believes, is with using semantic technology to do something with public social media as a new and real-time data stream to build a richer feedback loop for mass media content creators.

Today, of course, brands and media companies can use any of a number of social media listening tools, with the help of human operators to specify and configure keywords and search terms and get back sentiment and other values from the stream.  “We say that rather than do that, if we want to do know all that people are saying in response to something on TV, you want to take the TV content, analyze its semantics, and use that analysis to automatically configure search terms,” says Roy.

Bluefin has satellite dishes pulling in 47 TV networks, with 24/7 video streams. It extracts closed caption content from there, and analyzes it with the help of various TV Guide-level metadata (what shows are on, their characters and topics, sports specialty feeds such as player names and play-by-play info). On the social media end, it dives mainly into Twitter and publicly-available Facebook postings to suss out audience engagement with content, but over time it says it will deepen its scan of the whole sphere. Its TV Genome is the mapping of TV media data to social media commentary data at scale, a process that also has to take into account intuitions about time (since online comments that relate to a show or ad are more likely to occur during defined windows close to when they ran), as well as semantics.

Bluefin as a company resulted from Roy’s work at MIT on developing algorithms that learned to find connections between different modalities of data in order to capture deep semantic structure. The work, as he puts it, “focused on intentionality or aboutness, that words are about things in the world. Bluefin creates a machine that makes that aboutness – that that comment is about that thing on TV, or as far as I can tell this comment is not about anything that aired on TV at all. And when it is about something on TV, it pinpoints accurately what it was,” he says.

Break the Semantic Barrier

What comes up with customers, Roy says, is the issue of making sense out of what otherwise are just raw numbers. It’s nice to know that a show generated thousands of online comments, or that sentiment was up a few percentage points, but the way to understand what that actually means is comparing it to what else is out there – to other shows on the same network or in the same category, for instance. “For each of these questions, to make sense of one data point you have to have the rest of the universe you compare against, and so it’s scale that is required to make sense,” he says.

“It’s really easy to say how many people talked of Glee last week. But if you want an airing by airing alignment you need to get fancy with timing, and to put it in context you more or less need it all,” Roy adds. “I use the term semantic barrier to describe this paradox. When you look at it up close there’s no magic here for humans. I can read a comment and if I’m watching the show I know what someone is talking about. But to get the machine to do that at scale, to put the results in context, you hit the semantic barrier.”

How can what Bluefin does to break that barrier change things for mass media producers? Roy says imagine an advertiser launching a product with a handful of 30-second spots. Now imagine if in real-time that advertiser could see how different audiences respond to different creative spots, and more quickly rotate them according to best fits. “You could do a better job of presenting the right version of a message to the right audience, where rightness is measured by audience response,” he says.

Or consider being in the planning phase and deciding who to reach, how to segment the audience and what communication strategy to use. “Imagine if rather than being able to segment based on age or gender — how it’s done today — imagine that you say I want to find the shows that people who talk a lot about my competition are likely to be watching,” he says. Bluefin can do that because it tracks complete audiences, so it knows who is watching what and hears what they are talking about publicly, “and we put 2 and 2 together,” he says.

Or say you are a TV operator who wants to make it easier for your audience to find things they really want to watch. “Imagine you had an audience response map that as an operator you could use to provide recommendations to help your audience find stuff they will be interested in, stuff that people are watching and talking about,” he says. Social media talk is an indicator that they’re engaged with the content, he says. “So from a media company’s point of view that’s a lot of value there.”

Of course, there are some shows that are popular and may not get as much online buzz – for instance, those that might cater to an older demographic. Roy gets that and so do the advertisers and people in the media industry that he works with. “I say that just like any single data source or way of understanding a complex problem, there are blind spots and so too is the case for any method based on social media. But what we do for the first time is give a reliable way to measure the social impact of TV, and if you care about that you may want to talk to us,” he says.

By the way, he mentions, that blind spot as it relates to social media is that it keeps getting smaller as more people get involved. “The volume and velocity of social media users keeps growing,” Roy says. “So it becomes harder and harder to ignore.”

Bluefin Labs’ platform is now in an exclusive beta phase, with plans to officially launch this summer. Some of the pilot customers, he notes, are combining the data Bluefin produces with Nielsen data and finding it a complementary fit. Could there be a partnership with the likes of Nielsen in the future? Roy will only say that he sees potential for many different partnerships. “We are open and agnostic in term of what our data gets combined with. There are lots of possibilities.”