Conflict reporting with data: a guide for journalists

For years, Syria’s dark dungeons have functioned as hellholes of torture, starvation and murder. Thousands have vanished into this vast network, never to be heard from again; many emerged maimed and broken.

In a stunning investigation, The New York Times laid bare the sadistic violence and put on record war crimes committed by President Bashar al-Assad’s regime in these halls of horror.

Data on the dead and missing was a driving force behind the story by Anne Barnard, a former Times’ Beirut bureau chief and veteran of covering the armed conflict.

“It is almost impossible to do justice to the depth of Barnard’s reporting and the evil it described,” wrote The New Yorker’s Isaac Chotiner when the story was published in May 2019. During a Q&A with the author, he asked how “astonishingly bleak figures” of torture and murder had been gathered.

Barnard reported that “Nearly 128,000 have never emerged, and are presumed to be either dead or still in custody, according to the Syrian Network for Human Rights, an independent monitoring group that keeps the most rigorous tally. Nearly 14,000 were “killed under torture.”

More data is not necessarily better data. We need to know where it is coming from, what is included and what is not.

Data journalism played a vital role in exposing Syria’s atrocities through the use of advanced digital forensics, geolocation and data visualisation among other high tech tools that improve the accuracy and impact of war reporting.

Newsworthy statistics on conflict are widely available, but here is the challenge: How do journalists identify the right datasets to use? How do they evaluate data sources among the many out there? What should they be looking for?

“More data is not necessarily better data. We need to know where it is coming from, what is included and what is not. Reporters should not take all data as unbiased facts,” said Andreas Foro Tollefsen, senior researcher for the Peace Research Institute, Oslo, a main player in the conflict data field.

Conflict Reporting With Data

Reviewing some of the producers of conflict event data sets and their specialties helps narrow the search. Following are thumbnail sketches of three large-scale data-collection projects often cited in media reports and scholarly studies. Also included are examples of how the media use these data sets to report on armed conflict and how these groups collaborate.

Uppsala Conflict Data Program (UCDP): Data sets on conflict and peacekeeping including peace agreements, intrastate armed conflict, non-state conflict, one-sided violence, and conflict termination. UCDP offers datasets on organised violence and peacemaking, which can be downloaded for free through the UCDP downloads website. Illustrative charts, graphs, and maps are also available.

Armed Conflict Location & Event Dataset (ACLED): Described as disaggregated data collection, analysis and crisis mapping platform. Collects real-time data on the locations, dates, actors, fatalities, and types of all reported political violence and protest events across the globe. Users can explore data with an interactive dashboard. Operates under the slogan, “Bringing clarity to crisis.”

Peace Research Institute, Oslo (PRIO): Explores how conflicts erupt and can be resolved; investigates how different kinds of violence affect people and examines how societies tackle crises. Data projects include collecting conflict start and end dates to aid in the study of the duration of violence and adding figures for yearly combat deaths. Active research projects are listed alphabetically and include dozens of topics.

Statistics showed that nearly 10 times more children are at risk now than three decades ago when the toll was 8.5 million.

The data these groups collect is impressive and, at times, daunting. How does it translate into real-world news reporting? What kind of stories does it help to tell?

“Journalists equipped with data and empirics have a very powerful instrument to enlighten, but also to ask decision-makers the right questions,” said PRIO’s Tollefsen, a self-described human geographer. “Looking at data often challenges narratives or generates inquiries that wouldn't be evident without it.”

For instance, data can show trends, maps, and patterns, highlighting whether violence has gone up or down in a region, where conflict is located, and how this relates to conditions on the ground that impacts civilians, such as migration or refugees. A story based on ACLED data showed how conflict in northern Mozambique displaced over half a million people.

PRIO’s data was used in combination with other sources in a recent Save the Children report, “Weapons of War: Sexual violence against children in conflict.” Statistics showed that nearly 10 times more children are at risk now than three decades ago when the toll was 8.5 million.

Syria, Colombia, Iraq, Somalia, South Sudan, and Yemen were identified as countries where children are at greatest risk of sexual violence, including rape, sexual slavery, forced prostitution and pregnancy, sexual mutilation, and sexual torture at the hands of armed groups, government forces, and law enforcement.

“Data is a powerful tool and challenges our understanding of the world,” said Tollefsen whose work focuses on the use of geospatial data and Geographic Information Systems for research on the cause and consequences of conflict.

In another collaborative effort, PRIO examined trends in Middle East conflicts between 1989 – 2019 and compared them to global trends, using UCDP data. Researchers analysed conflict recurrence, ceasefires, and peace agreements during the same period.

The study found that over the past 10 years, the majority of the world’s deadliest conflicts have been in the Middle East, with Syria being the deadliest. Data also plays a pivotal role in breaking news, such as the impact of COVID-19. As the pandemic spread, data journalism provided vital, reliable information through the use of interactive maps, graphics and charts marking cases and deaths.

The Washington Post used ACLED data to explore: “Does COVID-19 raise the risk of violent conflict?” The number of conflict events was tracked over time to see whether trends changed after the World Health Organization declared a global pandemic in March 2020, or the country declared lockdown.

In the story, the Post defined ACLED as “A database that counts the number of conflict events daily around the world. For 2019 and 2020, ACLED includes more than 100 countries in Africa, Asia, Latin America and Eastern Europe — and tracks three categories of violent conflict: battles, violence against civilians and explosions/remote violence.”

A June 2020 article that appeared in Johannesburg’s Mail and Guardian examined how the pandemic had shifted patterns of conduct in Africa.

“Tracking activity over the past 10 weeks, ACLED found that conflict rates held steady across the continent, but patterns of violence have shifted as armed groups and governments take advantage of the pandemic to make moves on political priorities,” wrote ACLED’s director, Clionadh Raleigh.

A detailed infographic accompanied her story. Finding appropriate sources of conflict data is the first step. Interpreting the information correctly can be challenging. Following are tips from experts on how to analyse and evaluate research for accuracy, reliability and fairness, the hallmarks of quality reporting.

Evaluating conflict data

Harvard University research fellow Kelly M. Greenhill co-authored a book “Sex, Drugs, and Body Counts: The Politics of Numbers in Global Crime and Conflict” exploring the misuse of conflict statistics and showing that miscalculations can have “perverse and counterproductive consequences.”

Getting it wrong might, for instance, help prolong wars, give governments an excuse not to act or muddy evaluations of policy successes or failures in conflict zones.

What can journalists do to get it right? Greenhill provided the following questions that should be routinely asked about crime and conflict data:

1. What are the sources of the numbers?
2. What definitions are the sources employing, and what exactly is being measured?
3. What are the interests of those providing the numbers?
4. What do these actors stand to gain or lose if the statistics in question are – or are not – embraced and accepted?
5. What methodologies were employed in acquiring the numbers?
6. Do potentially competing figures exist, and, if so, what is known about their sources, measurement and methodologies?

“To some degree, the politicisation of statistics is inevitable and unavoidable. But journalists as consumers and disseminators of statistics should be more savvy and credulous in their acquisition and utilisation of these data,” said Greenhill, a political science and international relations professor at Tufts University.

She is concerned journalists could inadvertently serve as amplifiers of politically motivated statistical distortions leading to “counter-productive policy outcomes.”

Greenhill has been a consultant to the World Bank, U.N High Commissioner on Refugees, and an analyst for the U.S. Department of Defense. She is working on a new book exploring the influence of rumours, conspiracy theories, propaganda, so-called "fake news" and other forms of extra-factual information on international politics.

Pay attention to what sources the datasets are using. That makes a really big difference.

Comparing Datasets

Assessing similarities and differences in datasets is another way to evaluate how useful they might be for conflict coverage.

Roudabeh Kishi, ACLED’s director of research and innovation, co-authored a report comparing conflict data from several well-known sources, including the Global Terrorism Database (GTD), Integrated Crisis Early Warning System (ICEWS), the Phoenix event dataset, and Uppsala Conflict Data Programme Georeferenced Event Dataset (UCDP GED).

The stated purpose of the study: “To demonstrate how the collection [of event data] mandates, coding rules, and sourcing methods can result in drastically different information on political violence and interpretations of conflict.” Researchers also looked at human-generated data versus automated.

What should journalists look for? Among indicators on the veracity of data, the report listed:

1. Sourcing: “Extensive sourcing – including from local partners and media in local languages – provides the most thorough and accurate information on political violence and demonstrations, as well as the most accurate presentation of the risks that civilians experience in their homes and communities.”
2. Transparency: “Data sets must be usable if they are to be relied upon for regular analysis and users should be able to access every detail of how conflict data are coded and collected.”
3. Coverage and classification: “Clear, coherent, and correct classification is important for users because conflicts are not homogenous: disorder events differ in their frequency, sequences, and intensity.”

“Some people say,`Well, data is data and that’s it.’ It’s important to remember these are different projects with different methodologies, definitions, and usually based on a specific mandate,” Kishi explained. “Pay attention to what sources the datasets are using. That makes a really big difference.”

She advises beginners to learn simple software such as Tableau, an analytic tool that can help in the creation of maps and charts or filter by actors or by violence. There is a training video on ACLED’s website.

Supplying statistics to media is part of our advocacy,

Kishi provided Tableau training for the Syrian Network for Human Rights(SNHR), one of ACLED’s partners. “They are great. You’ll see them in our data,” Kishi said of the Syrian human rights group that supplies data based on actual counts of reports, rather than extrapolations or estimates.

Since its beginning in 2011, the group has amassed a vast archive of eyewitnesses’ names, contact information, and testimonies, as well as the photos and videos that are meticulously preserved as background for scholarly research, use in news stories or for future war crimes trials.

“Supplying statistics to media is part of our advocacy,” said Fadel Abdul Ghany, the group’s founder and director. He oversees a volunteer corps of 35 mostly in Syria, but also living in Turkey, Jordan and Lebanon where they have taken refuge.

SNHR’s data has been a key source for the U.N. High Commissioner for Human Rights and appears in U.S. State Department reports on Syria. Monthly reports document recent death toll, assassinations and attacks against civilians. In January there was information about the spread of COVID-19.

Fadel’s goal is to keep Syria in the spotlight. “Stories in top media like The New York Times, Washington Post, and CNN draw the attention of others,” said Ghany from his home in Qatar. “We are always ready to help journalists. Send us an email, tell us what you are working on and let us know how we can cooperate.”