A new article asks, “How stable is China? What are people discussing and thinking in Pakistan? To answer these sorts of questions, the U.S. government has turned to a rich source: social media… The CIA maintains a social-media tracking center operated out of a nondescript building in a Virginia industrial park. The intelligence analysts at the agency’s Open Source Center, who other agents refer to as ‘vengeful librarians,’ are tasked with sifting through millions of tweets, Facebook messages, online chat logs, and other public data on the World Wide Web to glean insights into the collective moods of regions or groups abroad.”

Johan Bollen spoke about the challenges of social media monitoring: “You have little control over the composition of a sample… Regular surveys are conducted with only 1000 people, but those samples are carefully balanced to provide an accurate cross section of a given society. This is much more difficult to do in these online environments. Sure, the samples are huge — there are 750 million people on Facebook — but no matter how you look at it, it’s still possible that the sample could still be biased.”

According to Bollen, another major challenge “is that sentiment analysis only provides a scrape of potentially useful information. ‘Right now, analysis is very specialized. We’re looking at how people feel about very particular topics,’ says Bollen. ‘There’s a lot room for growth in deeper semantic analysis: not just learning what people feel about something, but what people think about things. There are 250 million people on Twitter….if you could perform even a shallow analysis of people’s opinions about something, their semantic opinions, you can learn a lot from the wisdom of the crowd that could be leveraged.’ Diving deep into the semantics of online communication is the next big challenge for government agencies.”

Read more here.

Image: Courtesy CIA