Thursday 19 Jul 2018 | 06:26 | SYDNEY
Thursday 19 Jul 2018 | 06:26 | SYDNEY

Iran, social media and intelligence


Sam Roggeveen


30 June 2009 10:21

Cross-posted on the ABC's opinion page.

Complaints about the barrage of confusing and contradictory Twitter messages emerging from Iran's upheavals are predictable. We've heard similar complaints for years now about the sheer volume of information available on the internet. How on earth are we supposed to make sense of it all?

Such concerns are both entirely understandable and pretty pointless. There is no solution to the increasing complexity of our information environment, and in a free society, we should not want to reduce the amount of information available to us. All we can do is develop new and better tools for coping with that complexity.

One such tool the public has at its disposal is a community of knowledgeable and well-connected journalists. They follow leads, weigh information and assess the credibility of sources in a concentrated and detailed way that few others can match. And so one of the ironies of the information age is that while the structure of the internet makes it harder to turn a profit by doing quality journalism, the increasing complexity of our information environment creates an ever greater need for it.

It is little appreciated by the public, but the government actually employs hundreds of people to do just this kind of quality journalism. And I'm not talking about the ABC, but about our intelligence community. Just like journalists, intelligence analysts are bombarded with information from countless sources and asked to make sense of it for their 'customers', political decision-makers who need to know what's going on in the world.

Intelligence analysts must assess each piece of information for its accuracy and reliability, and then put it into the context of everything else they know. How does event X fit in with announcement Y? Does person A know person B, and have either ever worked for C?

In the past, such data crunching has focused on information gathered secretly, either by electronic eavesdropping, satellite photography or the use of agents. But increasingly, intelligence agencies are also maintaining a close watch on social media. Events in Iran reinforce the need for such surveillance, though terrorist use of the internet has made it a necessity for some years.

There is software in use for these tasks, including link analysis technology that can be used to process vast amounts of information to determine who knows who. And interestingly, intelligence agencies are not just watching social media but using it to help them with their work. US intelligence agencies started experimenting with wikis some years ago because the normal bottom-up approach to writing assessments so often failed.

The traditional model is for an expert desk analyst to write a first draft, which is then rewritten as it goes up the chain. Wikis and blogs allow for a much more collaborative and less hierarchical approach, and they let intelligence agencies tap into pockets of expertise they may not have known about. Bureaucracies like ours do not encourage specialisation, because being a subject-matter expert limits career options. So public servants will move from one policy area to another, perhaps followed by a stint in the intelligence world, then back to a policy job and so on. At each stage they amass expertise which can go to waste after they move. But if they can continue to contribute through an in-house wiki or blog, that expertise can be captured.

Of course, there are always limitations. These are tools for managing complexity, and although intelligence analysts have access to sources ordinary Australians will never see, no mere spy agency can ever master that complexity. It is tempting to believe that, with enough information, one can predict the future, but there are countless examples of intelligence agencies failing at that task. The answer is not to search for the perfect information source, but to understand that good intelligence, just like good journalism, can only ever help us to comprehend the world we are now in, not predict the one to come.

Photo by Flickr user mfilej, used under a Creative Commons license.