This feature article by Jakob Rogstadius appeared in the Huffington Post.
Effective intervention in humanitarian disasters depends largely on gaining a comprehensive understanding of the situation as a whole so that first responders and aid organizations know where scarce resources are needed most. At the same time, disasters are highly complex, with multiple events occurring simultaneously and information that is difficult to verify.
However, the technology-driven information explosion in recent years has enabled disaster-affected citizens to share their awareness of their immediate surroundings with the world through social media like Facebook, Twitter and YouTube. Many disasters now generate a steady flow of reports that, in theory, can provide real-time insight into conditions on the ground. In practice, however, this stream becomes a torrent so rapid that only a fraction of the reports can be consumed.
That’s why we developed CrisisTracker, an open source web platform that collects Twitter updates and automatically detects emerging topics in the stream by grouping together messages based on their similarity. It’s a better way to make use of already available citizen reports for situational awareness, while avoiding information overload, by turning emerging patterns into comprehensive stories.
This simple technique works surprisingly well. An incident commander who has used our system recently provided this feedback as part of our study: “I feel very confident that those reports will come out ahead of CNN and BBC, and that they will have the central nuggets of the who, what, when, where, and why of a situation. For an incident commander, it is the difference between learning something in two to three hours versus six to eight.” That’s time that can save lives.
A geographic information systems expert had more encouragement, adding that “you can see over a period of time where people are moving and how that relates to conflict areas.” “During a food or water shortage, you can almost anticipate where needs are going to be based on what you are seeing,” she said.
The reason this works is that event-based stories, much like a traditional newspaper, have several benefits over message streams. Since messages relating to the same event are brought together, stories often capture events from many complementary angles before any single source fully comprehends the event. In particular, expert analysts working with the system have found that CrisisTracker helps them assess the credibility of rumors remotely and at an early stage. In conflict situations, it allows comparison of reports from the perspective of each side. Being able to tap into local populations for real-time country-wide monitoring can be particularly attractive, as trusted international monitors are often few and far apart, but can quickly be relocated to confirm emerging stories.
The initial version of CrisisTracker was developed to enable online disaster volunteers to annotate important stories with meta-data such as geographic location and topic, to make it easier for an analyst to filter and navigate the content. A system deployment during the Syrian civil war showed that in this way, a pool of 10-100 human curators, each working for 30 minutes per day (or 1-10 people full-time), is enough to keep up with and curate the 400,000 messages that the system collected daily during the war. Since curators work with unique stories rather than redundant messages, the workload is proportional to the scale of the disaster itself rather than the number of people with access to social media.
In prolonged disasters that last for weeks, months or even years, this kind of manual processing still becomes too costly, so our latest version based on ongoing research incorporates artificial intelligence techniques to let the system learn how to annotate messages based on examples provided by humans. If we succeed, which our early results indicate we will, this will drastically reduce the cost of running the system and further improve the speed at which important events can be communicated to an analyst or decision maker.
CrisisTracker was recently selected as a winner of Humanity United and USAID‘s Tech Challenge for Atrocity Prevention. This encouraging award will fuel our continued research on how to understand humanitarian disasters through public citizen communication.
Notes
This article is part of a series produced by The Huffington Post, Humanity United and the United States Agency for International Development (USAID), in recognition of the Tech Challenge for Atrocity Prevention. To see all the other posts in the series, click here. For more information about The Tech Challenge for Atrocity Prevention, click here
Publications
[bibtex key=Rogstadius:IbmJournalOfResearchAndDevelopment:2013,rogstadiusintroduction,rogstadius2011real,rogstadiusimproving,rogstadius2011towards]