What you asked

We all know that data can be manipulated to say anything. With this in mind, you asked:

How do fact-checkers deal with situations where the same data has been used to support different sides of an argument?

Anim van Wyk, Chief Editor, Africa Check: "We’re fond of the quip that some people use statistics 'as a drunken man uses lamp posts - for support rather than illumination'. Depending on what you want to prove, you can cherry-pick data which supports your argument.

An example is different stances on racial transformation in South Africa, or the lack thereof. A member of a leftist political party said in 2015 that 'whites are only 10% of the economically active population but occupy more than 60% of the top management positions'. The head of the Free Market Foundation, a liberal think-tank, then wrote: 'Blacks in top management… doubled.'

Both were right -- but by presenting only a specific slice of the same data source to support their argument."

Dinda Purnamasari, Senior Researcher, Tirto.id: "In our experience, many use the right data, but the context is incorrect. Then, the data becomes incredible.

For example, reports that PT Telkom (state-owned telecommunication company in Indonesia) had provided Corporate Social Responsibility funds of around IDR 100 million to a Mosque and, in comparison, IDR 3.5 billion to a church.

We found that the numbers (IDR100 million and IDR3.5 billion) were right, but the purpose of the funding was incorrect. The 100 million was granted by PT Telkom in 2016 to pay the debt from a mosque renovation process. On the other hand, 3.5 billion was granted to renovate the old church, which also became a cultural heritage site in Nusa Tenggara Barat in 2017.

In this case, again, the context of data becomes an important thing in fact-checking. We must understand the methodology and how the data was gathered or estimated, even by double-checking on the ground, if needed."

This brings us to your next question:

How do you determine the reliability of a data source?

Matt Martino, Online Editor, RMIT ABC Fact Check: "When considering a source, it’s always pertinent to ask: 'what is their agenda?' If their motivations for providing data might influence the data in a partisan way, it’s best to leave it alone. As always, it’s a good idea to consult experts in the field on what is the best source to use in verifying a claim."

Tania Roettger, Head of Fact-Checking Team, Correctiv/EchtJetzt: "When we’re investigating a claim, one task is to understand what exactly a given piece of data is able to tell. We establish how and why it was collected, what it contains and it excludes. Usually, we note the shortcomings of a statistic in the article. Whenever we are uncertain about the evidence we have gathered, we discuss the issue among our team."

What about situations where data on an issue isn’t available?

Samar Halarnkar, Editor, Factchecker.in: "If data are not available -- or independently verified data are not available -- there is only one substitute: Verification through old-fashioned, shoe-leather reporting.

For instance, India’s Prime Minister once claimed that his government had built 425,000 toilets within a year. With no independent verification, this claim was hard to dispute. Obviously, it was impossible to verify that 425,000 new toilets had indeed been built in all of India’s schools. But after sending reporters to conduct random verifications in eight Indian states, it quickly became apparent that the Prime Minister’s claim was -- to put it plainly -- a lie."

What are some good examples of data in fact-checking?

To end, you requested that our fact-checkers share good examples where data has been successfully used to check a claim. Here’s a few they came up with:

• This debunked claim that refugees in Germany sent 4.2 Billion Euros to their home countries in 2016
• Tirto.id’s investigation into a claim by the president of Indonesia that the country’s economic growth ranked third in the world
• A look at Australia’s historical financial accounts by RMIT ABC Fact Check
• A new database, created by Factchecker.in, to provide a missing picture of cow-related violence in India

For more detail on these fact-checks, and the rest of your questions, check out the full interview here.

Our next conversation

And now to a subject that is itself often the focus of contentious reportage: the environment.

How have you used data to report on climate change, pollution, or other environmental issues? What are some common challenges on this beat? Got any advice for overcoming them? Share your wisdom.

Until next time,

Madolyn from the EJC Data team