Intelligence agencies turn to crowdsourcing

by Sharon Weinberger, BBC, 10 October, 2012

The upcoming release of the James Bond movie Skyfall, combined with the 50th anniversary of the franchise, has sent fans of the suave spy into overdrive. Speculation about the plot and who will sing the theme tune (Adele),  has been joined by stories that pick apart everything from his ingenious – but scientifically dubious – gadgets to the ins and outs of his sex life.

But now, James Bond fans and wannabe spies alike may have the ultimate outlet for their spy ambitions. Research firm Applied Research Associates, has just launched a website that invites the public—meaning anyone, anywhere—to sign up and try their hand at intelligence forecasting. The website is part of an effort, sponsored by the Intelligence Advanced Research Projects Activity (Iarpa), to understand the potential benefits of so-called crowdsourcing for predicting future events. Crowdsourcing aims to use the “wisdom of crowds” and was popularised by projects like Wikipedia.

Like Darpa, its better-known counterpart in the Pentagon, Iarpa funds far-out research ideas. However, Iarpa works on ideas that could eventually be used by the likes of the Central Intelligence Agency (CIA), rather than the military. “The goal that Iarpa has is to eventually transition this to the intelligence community, and use it for something like the National Intelligence Estimates,” says Jenn Carter, who works on the project.

There’s good reason for Iarpa’s interest in finding new ways to collect useful information: the intelligence community has often been blasted for its failure to forecast critical world events, from the fall of the Soviet Union to the Arab Spring that swept across North Africa and the Middle East.  It was also heavily criticized for its National Intelligence Estimate in 2002, which supported claims that Iraq had weapons of mass destruction.

Those failures raised larger questions about how the intelligence agencies come up with forecasts, which is usually a deliberative process involving a large number of analysts.  The Iarpa project, known officially as Aggregative Contingent Estimation, is looking at whether crowdsourcing can result in more accurate forecasts about future events than those traditional forms of intelligence estimation.

Applied Research Associates actually started the project last year with another website called Forecasting Ace, which had over 2,000 registered contributors making predictions on everything from the future of space exploration to political elections.  On the new website, Global Crowd Intelligence, the company hopes that number will grow substantially by making forecasting more like a game of spy versus spy.

“When we contacted our contributors, they said we should try to make the whole process more fun,” says Dirk Warnaar, the principal investigator for the project.

Indeed, what users wanted, it turned out, was something competitive, so that’s what the company has given them. The new website rewards players who successfully forecast future events by giving them privileged access to certain “missions,” and also allowing them to collect reputation points, which can then be used for online bragging rights. When contributors enter the new site, they start off as junior analysts, but eventually progress to higher levels, allowing them to work on privileged missions.

The game works by allowing the newly minted analysts to choose from any number of “missions” to forecast. The forecasts incorporated in the website run the gamut, from those that obviously might be of theoretical interest to intelligence agencies, like a “government force will gain control of the Somali town of Kismayo before 1 November 2012,” to market predictions, such as the likelihood that Apple will introduce a mini-iPad by a specific date.  Analysts attach a specific percentage to the even, such as 80% probability, and then wager “reputation points” on their forecast. They win points, depending on how close their forecast matches reality. The more points you win, the higher you progress.